2024-09-17T15:13:45.190 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-17T15:13:45.205 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T15:13:45.281 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7908934 branch: wip-bharath10-testing-2024-09-17-1348 description: rados/singleton/{all/lost-unfound mon_election/classic msgr-failures/few msgr/async-v2only objectstore/{bluestore-options/write$/{write_random} bluestore/bluestore-comp-lz4} rados supported-random-distro$/{centos_latest}} email: skanta@redhat.com first_in_suite: false job_id: '7908934' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-bharath10-testing-2024-09-17-1348 ceph: conf: global: mon client directed command retry: 5 mon election default strategy: 1 ms bind msgr1: false ms bind msgr2: true ms inject socket failures: 5000 ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon scrub interval: 300 osd: bluestore block size: 96636764160 bluestore compression algorithm: lz4 bluestore compression mode: aggressive bluestore fsck on mount: true bluestore write v2 random: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - \(MON_DOWN\) sha1: 1889a165b5128787ec693450a3d1c5df5d6dc203 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 1889a165b5128787ec693450a3d1c5df5d6dc203 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-bharath10-testing-2024-09-17-1348 sha1: 1889a165b5128787ec693450a3d1c5df5d6dc203 owner: scheduled_skanta@teuthology priority: 99 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.b - mon.c - mgr.x - osd.0 - osd.1 - osd.2 seed: 3421 sha1: 1889a165b5128787ec693450a3d1c5df5d6dc203 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-bharath10-testing-2024-09-17-1348 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_1889a165b5128787ec693450a3d1c5df5d6dc203/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 1889a165b5128787ec693450a3d1c5df5d6dc203 targets: smithi083.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAvbNjtzbmYL9r1bXtRmJfHWFDSNDwRa0d/0piRZ10y21rbvZIgQKJE+Rg8GdhJSjIywb8DAL+msbS9z+N5ECF8= tasks: - install: null - ceph: log-ignorelist: - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(OBJECT_ - \(SLOW_OPS\) - slow request - \(POOL_APP_NOT_ENABLED\) pre-mgr-commands: - sudo ceph config set mgr mgr_pool false --force - lost_unfound: null teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-17_15:01:33 tube: smithi user: skanta verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-17T15:13:45.281 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_1889a165b5128787ec693450a3d1c5df5d6dc203/qa; will attempt to use it 2024-09-17T15:13:45.282 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_1889a165b5128787ec693450a3d1c5df5d6dc203/qa/tasks 2024-09-17T15:13:45.282 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-17T15:13:45.284 INFO:teuthology.task.internal:Checking packages... 2024-09-17T15:13:45.305 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1889a165b5128787ec693450a3d1c5df5d6dc203' 2024-09-17T15:13:45.305 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-17T15:13:45.305 INFO:teuthology.packaging:ref: None 2024-09-17T15:13:45.305 INFO:teuthology.packaging:tag: None 2024-09-17T15:13:45.305 INFO:teuthology.packaging:branch: wip-bharath10-testing-2024-09-17-1348 2024-09-17T15:13:45.306 INFO:teuthology.packaging:sha1: 1889a165b5128787ec693450a3d1c5df5d6dc203 2024-09-17T15:13:45.306 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-bharath10-testing-2024-09-17-1348 2024-09-17T15:13:45.447 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4975.g1889a165 2024-09-17T15:13:45.449 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-17T15:13:45.455 INFO:teuthology.task.internal:no buildpackages task found 2024-09-17T15:13:45.455 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-17T15:13:45.462 INFO:teuthology.task.internal:Saving configuration 2024-09-17T15:13:45.474 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-17T15:13:45.480 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-17T15:13:45.507 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi083.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7908934', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 15:03:35.531337', 'locked_by': 'scheduled_skanta@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAvbNjtzbmYL9r1bXtRmJfHWFDSNDwRa0d/0piRZ10y21rbvZIgQKJE+Rg8GdhJSjIywb8DAL+msbS9z+N5ECF8='} 2024-09-17T15:13:45.507 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-17T15:13:45.513 INFO:teuthology.task.internal:roles: ubuntu@smithi083.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'osd.0', 'osd.1', 'osd.2'] 2024-09-17T15:13:45.513 INFO:teuthology.run_tasks:Running task console_log... 2024-09-17T15:13:45.547 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9f3ab1fbe0>, signals=[15]) 2024-09-17T15:13:45.547 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-17T15:13:45.553 INFO:teuthology.task.internal:Opening connections... 2024-09-17T15:13:45.553 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi083.front.sepia.ceph.com 2024-09-17T15:13:45.554 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T15:13:45.637 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-17T15:13:45.642 DEBUG:teuthology.orchestra.run.smithi083:> uname -m 2024-09-17T15:13:45.728 INFO:teuthology.orchestra.run.smithi083.stdout:x86_64 2024-09-17T15:13:45.728 DEBUG:teuthology.orchestra.run.smithi083:> cat /etc/os-release 2024-09-17T15:13:45.786 INFO:teuthology.orchestra.run.smithi083.stdout:NAME="CentOS Stream" 2024-09-17T15:13:45.786 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION="9" 2024-09-17T15:13:45.786 INFO:teuthology.orchestra.run.smithi083.stdout:ID="centos" 2024-09-17T15:13:45.786 INFO:teuthology.orchestra.run.smithi083.stdout:ID_LIKE="rhel fedora" 2024-09-17T15:13:45.786 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION_ID="9" 2024-09-17T15:13:45.786 INFO:teuthology.orchestra.run.smithi083.stdout:PLATFORM_ID="platform:el9" 2024-09-17T15:13:45.786 INFO:teuthology.orchestra.run.smithi083.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-17T15:13:45.786 INFO:teuthology.orchestra.run.smithi083.stdout:ANSI_COLOR="0;31" 2024-09-17T15:13:45.786 INFO:teuthology.orchestra.run.smithi083.stdout:LOGO="fedora-logo-icon" 2024-09-17T15:13:45.787 INFO:teuthology.orchestra.run.smithi083.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-17T15:13:45.787 INFO:teuthology.orchestra.run.smithi083.stdout:HOME_URL="https://centos.org/" 2024-09-17T15:13:45.787 INFO:teuthology.orchestra.run.smithi083.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-17T15:13:45.787 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-17T15:13:45.787 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-17T15:13:45.787 INFO:teuthology.lock.ops:Updating smithi083.front.sepia.ceph.com on lock server 2024-09-17T15:13:45.809 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-17T15:13:45.818 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-17T15:13:45.823 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-17T15:13:45.823 DEBUG:teuthology.orchestra.run.smithi083:> test '!' -e /home/ubuntu/cephtest 2024-09-17T15:13:45.842 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-17T15:13:45.849 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-17T15:13:45.849 DEBUG:teuthology.orchestra.run.smithi083:> test -z $(ls -A /var/lib/ceph) 2024-09-17T15:13:45.908 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-17T15:13:45.941 INFO:teuthology.run_tasks:Running task kernel... 2024-09-17T15:13:45.958 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-17T15:13:45.958 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-17T15:13:45.958 DEBUG:teuthology.orchestra.run.smithi083:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T15:13:45.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T15:13:45.975 DEBUG:teuthology.orchestra.run.smithi083:> uname -r 2024-09-17T15:13:46.032 INFO:teuthology.orchestra.run.smithi083.stdout:5.14.0-503.el9.x86_64 2024-09-17T15:13:46.033 INFO:teuthology.task.kernel:Running kernel on smithi083: 5.14.0-503.el9.x86_64 2024-09-17T15:13:46.033 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum install -y kernel 2024-09-17T15:13:49.832 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - BaseOS 4.0 MB/s | 8.3 MB 00:02 2024-09-17T15:13:51.825 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - AppStream 22 MB/s | 20 MB 00:00 2024-09-17T15:13:58.269 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - CRB 2.4 MB/s | 6.5 MB 00:02 2024-09-17T15:13:59.709 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - Extras packages 84 kB/s | 19 kB 00:00 2024-09-17T15:14:00.763 INFO:teuthology.orchestra.run.smithi083.stdout:Extra Packages for Enterprise Linux 24 MB/s | 23 MB 00:00 2024-09-17T15:14:06.202 INFO:teuthology.orchestra.run.smithi083.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2024-09-17T15:14:08.125 INFO:teuthology.orchestra.run.smithi083.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-17T15:14:08.126 INFO:teuthology.orchestra.run.smithi083.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-17T15:14:08.219 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-17T15:14:08.235 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:14:08.235 INFO:teuthology.orchestra.run.smithi083.stdout: Package Architecture Version Repository Size 2024-09-17T15:14:08.235 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:14:08.235 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-09-17T15:14:08.235 INFO:teuthology.orchestra.run.smithi083.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-17T15:14:08.236 INFO:teuthology.orchestra.run.smithi083.stdout:Installing dependencies: 2024-09-17T15:14:08.236 INFO:teuthology.orchestra.run.smithi083.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-17T15:14:08.236 INFO:teuthology.orchestra.run.smithi083.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-17T15:14:08.236 INFO:teuthology.orchestra.run.smithi083.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-17T15:14:08.236 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:14:08.236 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-09-17T15:14:08.236 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:14:08.236 INFO:teuthology.orchestra.run.smithi083.stdout:Install 4 Packages 2024-09-17T15:14:08.236 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:14:08.236 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 79 M 2024-09-17T15:14:08.236 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 126 M 2024-09-17T15:14:08.237 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-09-17T15:14:08.437 INFO:teuthology.orchestra.run.smithi083.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 360 kB/s | 36 kB 00:00 2024-09-17T15:14:08.829 INFO:teuthology.orchestra.run.smithi083.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 32 MB/s | 16 MB 00:00 2024-09-17T15:14:09.255 INFO:teuthology.orchestra.run.smithi083.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 38 MB/s | 35 MB 00:00 2024-09-17T15:14:09.522 INFO:teuthology.orchestra.run.smithi083.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 26 MB/s | 29 MB 00:01 2024-09-17T15:14:09.522 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-09-17T15:14:09.522 INFO:teuthology.orchestra.run.smithi083.stdout:Total 61 MB/s | 79 MB 00:01 2024-09-17T15:14:10.210 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-09-17T15:14:10.556 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-09-17T15:14:10.557 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-09-17T15:14:11.486 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-09-17T15:14:11.486 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-09-17T15:14:13.919 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-09-17T15:14:14.558 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-17T15:14:14.641 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T15:14:17.378 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T15:14:17.502 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T15:14:22.328 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T15:14:22.360 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T15:14:27.240 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T15:15:05.502 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T15:15:05.541 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-17T15:15:05.879 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T15:15:05.879 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-17T15:15:05.879 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T15:15:05.879 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T15:15:06.772 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T15:15:06.772 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:15:06.772 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-09-17T15:15:06.772 INFO:teuthology.orchestra.run.smithi083.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-17T15:15:06.772 INFO:teuthology.orchestra.run.smithi083.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-17T15:15:06.772 INFO:teuthology.orchestra.run.smithi083.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-17T15:15:06.772 INFO:teuthology.orchestra.run.smithi083.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-17T15:15:06.772 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:15:06.772 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-17T15:15:07.127 DEBUG:teuthology.orchestra.run.smithi083:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T15:15:07.228 INFO:teuthology.orchestra.run.smithi083.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T15:15:07.228 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T15:15:07.229 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-17T15:15:07.229 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-17T15:15:07.229 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-17T15:15:07.229 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-17T15:15:07.229 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi083.front.sepia.ceph.com, path=None, version=distro) 2024-09-17T15:15:07.229 DEBUG:teuthology.orchestra.run.smithi083:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T15:15:07.288 INFO:teuthology.orchestra.run.smithi083.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T15:15:07.288 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T15:15:07.289 DEBUG:teuthology.orchestra.run.smithi083:> sudo rpm -qi grub2-tools 2024-09-17T15:15:07.324 INFO:teuthology.orchestra.run.smithi083.stdout:Name : grub2-tools 2024-09-17T15:15:07.324 INFO:teuthology.orchestra.run.smithi083.stdout:Epoch : 1 2024-09-17T15:15:07.324 INFO:teuthology.orchestra.run.smithi083.stdout:Version : 2.06 2024-09-17T15:15:07.325 INFO:teuthology.orchestra.run.smithi083.stdout:Release : 82.el9 2024-09-17T15:15:07.325 INFO:teuthology.orchestra.run.smithi083.stdout:Architecture: x86_64 2024-09-17T15:15:07.325 INFO:teuthology.orchestra.run.smithi083.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-17T15:15:07.325 INFO:teuthology.orchestra.run.smithi083.stdout:Group : Unspecified 2024-09-17T15:15:07.325 INFO:teuthology.orchestra.run.smithi083.stdout:Size : 8274670 2024-09-17T15:15:07.325 INFO:teuthology.orchestra.run.smithi083.stdout:License : GPLv3+ 2024-09-17T15:15:07.325 INFO:teuthology.orchestra.run.smithi083.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-17T15:15:07.325 INFO:teuthology.orchestra.run.smithi083.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-17T15:15:07.325 INFO:teuthology.orchestra.run.smithi083.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-17T15:15:07.325 INFO:teuthology.orchestra.run.smithi083.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-17T15:15:07.325 INFO:teuthology.orchestra.run.smithi083.stdout:Packager : builder@centos.org 2024-09-17T15:15:07.325 INFO:teuthology.orchestra.run.smithi083.stdout:Vendor : CentOS 2024-09-17T15:15:07.325 INFO:teuthology.orchestra.run.smithi083.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-17T15:15:07.325 INFO:teuthology.orchestra.run.smithi083.stdout:Summary : Support tools for GRUB. 2024-09-17T15:15:07.326 INFO:teuthology.orchestra.run.smithi083.stdout:Description : 2024-09-17T15:15:07.326 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:15:07.326 INFO:teuthology.orchestra.run.smithi083.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-17T15:15:07.326 INFO:teuthology.orchestra.run.smithi083.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-17T15:15:07.326 INFO:teuthology.orchestra.run.smithi083.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-17T15:15:07.326 INFO:teuthology.orchestra.run.smithi083.stdout:hardware devices. 2024-09-17T15:15:07.326 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:15:07.326 INFO:teuthology.orchestra.run.smithi083.stdout:This subpackage provides tools for support of all platforms. 2024-09-17T15:15:07.327 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-17T15:15:07.327 INFO:teuthology.task.kernel:Updating grub on smithi083 to boot 5.14.0-508.el9.x86_64 2024-09-17T15:15:07.327 DEBUG:teuthology.orchestra.run.smithi083:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-17T15:15:07.990 INFO:teuthology.orchestra.run.smithi083.stderr:Generating grub configuration file ... 2024-09-17T15:15:09.298 INFO:teuthology.orchestra.run.smithi083.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-17T15:15:09.333 INFO:teuthology.orchestra.run.smithi083.stderr:done 2024-09-17T15:15:09.340 DEBUG:teuthology.orchestra.run.smithi083:> mktemp 2024-09-17T15:15:09.356 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/tmp.fnJoP0Ctti 2024-09-17T15:15:09.356 DEBUG:teuthology.orchestra.run.smithi083:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.fnJoP0Ctti 2024-09-17T15:15:09.430 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0666 /tmp/tmp.fnJoP0Ctti 2024-09-17T15:15:09.562 DEBUG:teuthology.orchestra.remote:smithi083:/tmp/tmp.fnJoP0Ctti is 6KB 2024-09-17T15:15:09.610 DEBUG:teuthology.orchestra.run.smithi083:> rm -fr /tmp/tmp.fnJoP0Ctti 2024-09-17T15:15:09.625 DEBUG:teuthology.orchestra.run.smithi083:> sudo /bin/ls /boot/loader/entries || true 2024-09-17T15:15:09.693 INFO:teuthology.orchestra.run.smithi083.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-17T15:15:09.693 INFO:teuthology.orchestra.run.smithi083.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-17T15:15:09.693 INFO:teuthology.orchestra.run.smithi083.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-17T15:15:09.693 INFO:teuthology.orchestra.run.smithi083.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-17T15:15:09.694 DEBUG:teuthology.orchestra.run.smithi083:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-17T15:15:09.854 DEBUG:teuthology.orchestra.run.smithi083:> sudo shutdown -r now 2024-09-17T15:15:39.886 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-17T15:15:39.887 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-09-17T15:15:39.888 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T15:15:58.401 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.83 2024-09-17T15:16:07.411 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-09-17T15:16:07.412 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T15:16:10.497 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.83 2024-09-17T15:16:22.509 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-09-17T15:16:22.510 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T15:16:40.897 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.83 2024-09-17T15:16:55.909 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-09-17T15:16:55.910 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T15:16:56.316 DEBUG:teuthology.orchestra.run.smithi083:> true 2024-09-17T15:16:56.656 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-09-17T15:16:56.657 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-508.el9.x86_64"... 2024-09-17T15:16:56.657 DEBUG:teuthology.orchestra.run.smithi083:> uname -r 2024-09-17T15:16:56.689 INFO:teuthology.orchestra.run.smithi083.stdout:5.14.0-508.el9.x86_64 2024-09-17T15:16:56.689 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-17T15:16:56.689 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T15:16:56.689 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-17T15:16:57.690 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-17T15:16:57.690 DEBUG:teuthology.orchestra.run.smithi083:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-17T15:16:57.831 INFO:teuthology.orchestra.run.smithi083.stdout:ttyS1 2024-09-17T15:16:57.852 DEBUG:teuthology.parallel:result is None 2024-09-17T15:16:57.852 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-17T15:16:57.860 INFO:teuthology.task.internal:Creating test directory... 2024-09-17T15:16:57.860 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T15:16:57.906 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-17T15:16:57.913 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-17T15:16:57.919 INFO:teuthology.task.internal:Creating archive directory... 2024-09-17T15:16:57.920 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T15:16:57.953 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-17T15:16:57.959 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-17T15:16:57.960 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T15:16:58.056 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T15:16:58.081 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T15:16:58.083 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-17T15:16:58.090 INFO:teuthology.task.internal:Configuring sudo... 2024-09-17T15:16:58.090 DEBUG:teuthology.orchestra.run.smithi083:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T15:16:58.148 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-17T15:16:58.157 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-17T15:16:58.157 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T15:16:58.198 DEBUG:teuthology.orchestra.run.smithi083:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T15:16:58.310 DEBUG:teuthology.orchestra.run.smithi083:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T15:16:58.400 DEBUG:teuthology.orchestra.run.smithi083:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T15:16:58.474 DEBUG:teuthology.orchestra.run.smithi083:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T15:16:58.541 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-17T15:16:58.541 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T15:16:58.609 DEBUG:teuthology.orchestra.run.smithi083:> sudo service rsyslog restart 2024-09-17T15:16:58.696 INFO:teuthology.orchestra.run.smithi083.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T15:16:59.174 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-17T15:16:59.182 INFO:teuthology.task.internal:Starting timer... 2024-09-17T15:16:59.182 INFO:teuthology.run_tasks:Running task pcp... 2024-09-17T15:16:59.194 INFO:teuthology.run_tasks:Running task selinux... 2024-09-17T15:16:59.202 DEBUG:teuthology.orchestra.run.smithi083:> sudo service auditd rotate 2024-09-17T15:16:59.304 INFO:teuthology.orchestra.run.smithi083.stdout:Rotating logs: 2024-09-17T15:16:59.305 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-17T15:16:59.306 DEBUG:teuthology.orchestra.run.smithi083:> /usr/sbin/getenforce 2024-09-17T15:16:59.327 INFO:teuthology.orchestra.run.smithi083.stdout:Permissive 2024-09-17T15:16:59.327 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi083.front.sepia.ceph.com': 'permissive'} 2024-09-17T15:16:59.328 DEBUG:teuthology.orchestra.run.smithi083:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T15:16:59.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T15:16:59.393 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-17T15:16:59.393 DEBUG:teuthology.orchestra.run.smithi083:> sudo /usr/sbin/setenforce permissive 2024-09-17T15:16:59.469 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-17T15:16:59.479 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-17T15:16:59.490 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-17T15:16:59.491 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi083.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-17T15:21:24.229 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi083.front.sepia.ceph.com')] 2024-09-17T15:21:24.230 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-09-17T15:21:24.232 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T15:21:24.313 DEBUG:teuthology.orchestra.run.smithi083:> true 2024-09-17T15:21:24.387 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-09-17T15:21:24.387 INFO:teuthology.run_tasks:Running task clock... 2024-09-17T15:21:24.398 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-17T15:21:24.399 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T15:21:24.399 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T15:21:24.455 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-17T15:21:24.469 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-17T15:21:24.494 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: ntpd: command not found 2024-09-17T15:21:24.505 INFO:teuthology.orchestra.run.smithi083.stdout:506 Cannot talk to daemon 2024-09-17T15:21:24.519 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-17T15:21:24.533 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-17T15:21:24.589 INFO:teuthology.orchestra.run.smithi083.stderr:bash: line 1: ntpq: command not found 2024-09-17T15:21:24.592 INFO:teuthology.orchestra.run.smithi083.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T15:21:24.592 INFO:teuthology.orchestra.run.smithi083.stdout:=============================================================================== 2024-09-17T15:21:24.592 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T15:21:24.592 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T15:21:24.592 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T15:21:24.592 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T15:21:24.593 INFO:teuthology.run_tasks:Running task install... 2024-09-17T15:21:24.602 DEBUG:teuthology.task.install:project ceph 2024-09-17T15:21:24.602 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1889a165b5128787ec693450a3d1c5df5d6dc203'}} 2024-09-17T15:21:24.602 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '1889a165b5128787ec693450a3d1c5df5d6dc203'} 2024-09-17T15:21:24.602 INFO:teuthology.task.install:Using flavor: default 2024-09-17T15:21:24.610 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-17T15:21:24.610 INFO:teuthology.task.install:extra packages: [] 2024-09-17T15:21:24.610 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1889a165b5128787ec693450a3d1c5df5d6dc203', 'tag': None, 'wait_for_package': False} 2024-09-17T15:21:24.611 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1889a165b5128787ec693450a3d1c5df5d6dc203 2024-09-17T15:21:24.762 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/centos/9/flavors/default/ 2024-09-17T15:21:24.762 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4975.g1889a165 2024-09-17T15:21:24.879 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-17T15:21:24.880 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-17T15:21:24.880 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-17T15:21:24.906 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-17T15:21:24.906 DEBUG:teuthology.orchestra.run.smithi083:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1889a165b5128787ec693450a3d1c5df5d6dc203/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-17T15:21:24.973 DEBUG:teuthology.orchestra.run.smithi083:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-17T15:21:25.049 DEBUG:teuthology.orchestra.run.smithi083:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-17T15:21:25.113 INFO:teuthology.orchestra.run.smithi083.stdout:check_obsoletes = 1 2024-09-17T15:21:25.114 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum clean all 2024-09-17T15:21:25.351 INFO:teuthology.orchestra.run.smithi083.stdout:45 files removed 2024-09-17T15:21:25.378 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-radosgw 2024-09-17T15:21:26.036 INFO:teuthology.orchestra.run.smithi083.stdout:ceph packages for x86_64 223 kB/s | 84 kB 00:00 2024-09-17T15:21:26.395 INFO:teuthology.orchestra.run.smithi083.stdout:ceph noarch packages 76 kB/s | 24 kB 00:00 2024-09-17T15:21:26.671 INFO:teuthology.orchestra.run.smithi083.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-17T15:21:28.663 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - BaseOS 4.2 MB/s | 8.3 MB 00:01 2024-09-17T15:21:33.513 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - AppStream 5.4 MB/s | 20 MB 00:03 2024-09-17T15:21:38.906 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - CRB 4.0 MB/s | 6.5 MB 00:01 2024-09-17T15:21:40.574 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - Extras packages 58 kB/s | 19 kB 00:00 2024-09-17T15:21:41.378 INFO:teuthology.orchestra.run.smithi083.stdout:Extra Packages for Enterprise Linux 33 MB/s | 23 MB 00:00 2024-09-17T15:21:46.992 INFO:teuthology.orchestra.run.smithi083.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-17T15:21:49.217 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-17T15:21:49.230 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:21:49.230 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repository Size 2024-09-17T15:21:49.231 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:21:49.231 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-09-17T15:21:49.231 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-radosgw x86_64 2:19.3.0-4975.g1889a165.el9 ceph 11 M 2024-09-17T15:21:49.231 INFO:teuthology.orchestra.run.smithi083.stdout:Upgrading: 2024-09-17T15:21:49.231 INFO:teuthology.orchestra.run.smithi083.stdout: librados2 x86_64 2:19.3.0-4975.g1889a165.el9 ceph 3.5 M 2024-09-17T15:21:49.231 INFO:teuthology.orchestra.run.smithi083.stdout: librbd1 x86_64 2:19.3.0-4975.g1889a165.el9 ceph 3.3 M 2024-09-17T15:21:49.231 INFO:teuthology.orchestra.run.smithi083.stdout:Installing dependencies: 2024-09-17T15:21:49.231 INFO:teuthology.orchestra.run.smithi083.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-17T15:21:49.231 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-base x86_64 2:19.3.0-4975.g1889a165.el9 ceph 5.5 M 2024-09-17T15:21:49.231 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-common x86_64 2:19.3.0-4975.g1889a165.el9 ceph 22 M 2024-09-17T15:21:49.231 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-selinux x86_64 2:19.3.0-4975.g1889a165.el9 ceph 25 k 2024-09-17T15:21:49.231 INFO:teuthology.orchestra.run.smithi083.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-17T15:21:49.231 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-17T15:21:49.232 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-17T15:21:49.232 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs2 x86_64 2:19.3.0-4975.g1889a165.el9 ceph 756 k 2024-09-17T15:21:49.232 INFO:teuthology.orchestra.run.smithi083.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-17T15:21:49.232 INFO:teuthology.orchestra.run.smithi083.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-17T15:21:49.232 INFO:teuthology.orchestra.run.smithi083.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-17T15:21:49.232 INFO:teuthology.orchestra.run.smithi083.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-17T15:21:49.232 INFO:teuthology.orchestra.run.smithi083.stdout: libradosstriper1 x86_64 2:19.3.0-4975.g1889a165.el9 ceph 508 k 2024-09-17T15:21:49.232 INFO:teuthology.orchestra.run.smithi083.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-17T15:21:49.232 INFO:teuthology.orchestra.run.smithi083.stdout: librgw2 x86_64 2:19.3.0-4975.g1889a165.el9 ceph 5.5 M 2024-09-17T15:21:49.232 INFO:teuthology.orchestra.run.smithi083.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-17T15:21:49.232 INFO:teuthology.orchestra.run.smithi083.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-17T15:21:49.232 INFO:teuthology.orchestra.run.smithi083.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-17T15:21:49.232 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-argparse x86_64 2:19.3.0-4975.g1889a165.el9 ceph 45 k 2024-09-17T15:21:49.233 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-common x86_64 2:19.3.0-4975.g1889a165.el9 ceph 143 k 2024-09-17T15:21:49.233 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cephfs x86_64 2:19.3.0-4975.g1889a165.el9 ceph 161 k 2024-09-17T15:21:49.233 INFO:teuthology.orchestra.run.smithi083.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-17T15:21:49.233 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rados x86_64 2:19.3.0-4975.g1889a165.el9 ceph 321 k 2024-09-17T15:21:49.233 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rbd x86_64 2:19.3.0-4975.g1889a165.el9 ceph 305 k 2024-09-17T15:21:49.233 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rgw x86_64 2:19.3.0-4975.g1889a165.el9 ceph 99 k 2024-09-17T15:21:49.233 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-17T15:21:49.233 INFO:teuthology.orchestra.run.smithi083.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-17T15:21:49.233 INFO:teuthology.orchestra.run.smithi083.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-17T15:21:49.233 INFO:teuthology.orchestra.run.smithi083.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-17T15:21:49.233 INFO:teuthology.orchestra.run.smithi083.stdout:Installing weak dependencies: 2024-09-17T15:21:49.233 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-17T15:21:49.233 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:21:49.234 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-09-17T15:21:49.234 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:21:49.234 INFO:teuthology.orchestra.run.smithi083.stdout:Install 31 Packages 2024-09-17T15:21:49.234 INFO:teuthology.orchestra.run.smithi083.stdout:Upgrade 2 Packages 2024-09-17T15:21:49.234 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:21:49.234 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 62 M 2024-09-17T15:21:49.234 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-09-17T15:21:50.413 INFO:teuthology.orchestra.run.smithi083.stdout:(1/33): ceph-base-19.3.0-4975.g1889a165.el9.x86 6.1 MB/s | 5.5 MB 00:00 2024-09-17T15:21:50.460 INFO:teuthology.orchestra.run.smithi083.stdout:(2/33): ceph-selinux-19.3.0-4975.g1889a165.el9. 505 kB/s | 25 kB 00:00 2024-09-17T15:21:50.535 INFO:teuthology.orchestra.run.smithi083.stdout:(3/33): libcephfs2-19.3.0-4975.g1889a165.el9.x8 9.8 MB/s | 756 kB 00:00 2024-09-17T15:21:50.627 INFO:teuthology.orchestra.run.smithi083.stdout:(4/33): libradosstriper1-19.3.0-4975.g1889a165. 5.4 MB/s | 508 kB 00:00 2024-09-17T15:21:50.769 INFO:teuthology.orchestra.run.smithi083.stdout:(5/33): ceph-radosgw-19.3.0-4975.g1889a165.el9. 8.6 MB/s | 11 MB 00:01 2024-09-17T15:21:50.811 INFO:teuthology.orchestra.run.smithi083.stdout:(6/33): python3-ceph-argparse-19.3.0-4975.g1889 1.1 MB/s | 45 kB 00:00 2024-09-17T15:21:50.870 INFO:teuthology.orchestra.run.smithi083.stdout:(7/33): python3-ceph-common-19.3.0-4975.g1889a1 2.4 MB/s | 143 kB 00:00 2024-09-17T15:21:50.928 INFO:teuthology.orchestra.run.smithi083.stdout:(8/33): python3-cephfs-19.3.0-4975.g1889a165.el 2.7 MB/s | 161 kB 00:00 2024-09-17T15:21:51.004 INFO:teuthology.orchestra.run.smithi083.stdout:(9/33): python3-rados-19.3.0-4975.g1889a165.el9 4.2 MB/s | 321 kB 00:00 2024-09-17T15:21:51.104 INFO:teuthology.orchestra.run.smithi083.stdout:(10/33): librgw2-19.3.0-4975.g1889a165.el9.x86_ 12 MB/s | 5.5 MB 00:00 2024-09-17T15:21:51.129 INFO:teuthology.orchestra.run.smithi083.stdout:(11/33): python3-rbd-19.3.0-4975.g1889a165.el9. 2.4 MB/s | 305 kB 00:00 2024-09-17T15:21:51.163 INFO:teuthology.orchestra.run.smithi083.stdout:(12/33): python3-rgw-19.3.0-4975.g1889a165.el9. 1.7 MB/s | 99 kB 00:00 2024-09-17T15:21:51.371 INFO:teuthology.orchestra.run.smithi083.stdout:(13/33): ceph-common-19.3.0-4975.g1889a165.el9. 12 MB/s | 22 MB 00:01 2024-09-17T15:21:51.422 INFO:teuthology.orchestra.run.smithi083.stdout:(14/33): boost-program-options-1.75.0-8.el9.x86 366 kB/s | 107 kB 00:00 2024-09-17T15:21:51.506 INFO:teuthology.orchestra.run.smithi083.stdout:(15/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 543 kB/s | 45 kB 00:00 2024-09-17T15:21:51.539 INFO:teuthology.orchestra.run.smithi083.stdout:(16/33): libnbd-1.20.2-2.el9.x86_64.rpm 454 kB/s | 171 kB 00:00 2024-09-17T15:21:51.597 INFO:teuthology.orchestra.run.smithi083.stdout:(17/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 711 kB/s | 160 kB 00:00 2024-09-17T15:21:51.656 INFO:teuthology.orchestra.run.smithi083.stdout:(18/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-17T15:21:51.682 INFO:teuthology.orchestra.run.smithi083.stdout:(19/33): python3-prettytable-0.7.2-27.el9.noarc 505 kB/s | 42 kB 00:00 2024-09-17T15:21:51.715 INFO:teuthology.orchestra.run.smithi083.stdout:(20/33): librdkafka-1.6.1-102.el9.x86_64.rpm 3.1 MB/s | 662 kB 00:00 2024-09-17T15:21:51.740 INFO:teuthology.orchestra.run.smithi083.stdout:(21/33): qatlib-service-24.02.0-1.el9.x86_64.rp 613 kB/s | 36 kB 00:00 2024-09-17T15:21:51.766 INFO:teuthology.orchestra.run.smithi083.stdout:(22/33): qatlib-24.02.0-1.el9.x86_64.rpm 2.0 MB/s | 221 kB 00:00 2024-09-17T15:21:51.799 INFO:teuthology.orchestra.run.smithi083.stdout:(23/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 566 kB/s | 47 kB 00:00 2024-09-17T15:21:51.849 INFO:teuthology.orchestra.run.smithi083.stdout:(24/33): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.8 MB/s | 308 kB 00:00 2024-09-17T15:21:51.875 INFO:teuthology.orchestra.run.smithi083.stdout:(25/33): liboath-2.6.7-2.el9.x86_64.rpm 2.0 MB/s | 49 kB 00:00 2024-09-17T15:21:51.900 INFO:teuthology.orchestra.run.smithi083.stdout:(26/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 285 kB/s | 26 kB 00:00 2024-09-17T15:21:51.925 INFO:teuthology.orchestra.run.smithi083.stdout:(27/33): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-17T15:21:51.959 INFO:teuthology.orchestra.run.smithi083.stdout:(28/33): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-17T15:21:52.001 INFO:teuthology.orchestra.run.smithi083.stdout:(29/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.2 MB/s | 839 kB 00:00 2024-09-17T15:21:52.060 INFO:teuthology.orchestra.run.smithi083.stdout:(30/33): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-17T15:21:52.126 INFO:teuthology.orchestra.run.smithi083.stdout:(31/33): libarrow-9.0.0-11.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-09-17T15:21:52.337 INFO:teuthology.orchestra.run.smithi083.stdout:(32/33): librados2-19.3.0-4975.g1889a165.el9.x8 10 MB/s | 3.5 MB 00:00 2024-09-17T15:21:52.418 INFO:teuthology.orchestra.run.smithi083.stdout:(33/33): librbd1-19.3.0-4975.g1889a165.el9.x86_ 9.1 MB/s | 3.3 MB 00:00 2024-09-17T15:21:52.423 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-09-17T15:21:52.423 INFO:teuthology.orchestra.run.smithi083.stdout:Total 19 MB/s | 62 MB 00:03 2024-09-17T15:21:52.537 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-09-17T15:21:52.574 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-09-17T15:21:52.574 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-09-17T15:21:53.268 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-09-17T15:21:53.270 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-09-17T15:21:54.806 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-09-17T15:21:54.902 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-17T15:21:54.932 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/35 2024-09-17T15:21:54.957 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/35 2024-09-17T15:21:54.980 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-17T15:21:55.008 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-17T15:21:55.207 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-17T15:21:55.260 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-17T15:21:55.591 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/35 2024-09-17T15:21:55.701 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : librados2-2:19.3.0-4975.g1889a165.el9.x86_64 7/35 2024-09-17T15:21:55.793 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librados2-2:19.3.0-4975.g1889a165.el9.x86_64 7/35 2024-09-17T15:21:55.902 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rados-2:19.3.0-4975.g1889a165.el9.x86_64 8/35 2024-09-17T15:21:55.927 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephfs2-2:19.3.0-4975.g1889a165.el9.x86_64 9/35 2024-09-17T15:21:55.990 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libcephfs2-2:19.3.0-4975.g1889a165.el9.x86_64 9/35 2024-09-17T15:21:56.027 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/35 2024-09-17T15:21:56.071 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ceph-argparse-2:19.3.0-4975.g1889a165.el9. 11/35 2024-09-17T15:21:56.145 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cephfs-2:19.3.0-4975.g1889a165.el9.x86_64 12/35 2024-09-17T15:21:56.171 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libradosstriper1-2:19.3.0-4975.g1889a165.el9.x86_6 13/35 2024-09-17T15:21:56.235 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4975.g1889a165.el9.x86_6 13/35 2024-09-17T15:21:56.270 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/35 2024-09-17T15:21:56.381 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/35 2024-09-17T15:21:56.412 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/35 2024-09-17T15:21:56.459 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/35 2024-09-17T15:21:56.486 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/35 2024-09-17T15:21:56.515 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/35 2024-09-17T15:21:56.548 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-17T15:21:56.587 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/35 2024-09-17T15:21:56.937 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-17T15:21:56.978 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : librbd1-2:19.3.0-4975.g1889a165.el9.x86_64 23/35 2024-09-17T15:21:57.070 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librbd1-2:19.3.0-4975.g1889a165.el9.x86_64 23/35 2024-09-17T15:21:57.100 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rbd-2:19.3.0-4975.g1889a165.el9.x86_64 24/35 2024-09-17T15:21:57.135 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-17T15:21:57.625 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-17T15:21:57.656 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librgw2-2:19.3.0-4975.g1889a165.el9.x86_64 27/35 2024-09-17T15:21:57.727 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librgw2-2:19.3.0-4975.g1889a165.el9.x86_64 27/35 2024-09-17T15:21:57.805 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rgw-2:19.3.0-4975.g1889a165.el9.x86_64 28/35 2024-09-17T15:21:57.833 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ceph-common-2:19.3.0-4975.g1889a165.el9.x8 29/35 2024-09-17T15:22:00.108 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-common-2:19.3.0-4975.g1889a165.el9.x86_64 30/35 2024-09-17T15:22:00.146 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-common-2:19.3.0-4975.g1889a165.el9.x86_64 30/35 2024-09-17T15:22:00.707 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-common-2:19.3.0-4975.g1889a165.el9.x86_64 30/35 2024-09-17T15:22:00.750 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-base-2:19.3.0-4975.g1889a165.el9.x86_64 31/35 2024-09-17T15:22:00.840 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-base-2:19.3.0-4975.g1889a165.el9.x86_64 31/35 2024-09-17T15:22:00.840 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-17T15:22:00.840 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-17T15:22:00.841 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:22:00.918 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-selinux-2:19.3.0-4975.g1889a165.el9.x86_64 32/35 2024-09-17T15:22:11.121 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4975.g1889a165.el9.x86_64 32/35 2024-09-17T15:22:11.122 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /sys 2024-09-17T15:22:11.122 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /proc 2024-09-17T15:22:11.122 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /mnt 2024-09-17T15:22:11.122 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /var/tmp 2024-09-17T15:22:11.122 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /home 2024-09-17T15:22:11.122 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /root 2024-09-17T15:22:11.122 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /tmp 2024-09-17T15:22:11.122 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:22:12.119 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-radosgw-2:19.3.0-4975.g1889a165.el9.x86_64 33/35 2024-09-17T15:22:12.146 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4975.g1889a165.el9.x86_64 33/35 2024-09-17T15:22:12.146 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T15:22:12.146 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-17T15:22:12.146 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T15:22:12.146 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T15:22:12.146 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:22:12.149 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-17T15:22:12.238 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-17T15:22:12.239 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-17T15:22:13.435 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-17T15:22:13.435 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-base-2:19.3.0-4975.g1889a165.el9.x86_64 1/35 2024-09-17T15:22:13.435 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-common-2:19.3.0-4975.g1889a165.el9.x86_64 2/35 2024-09-17T15:22:13.435 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-radosgw-2:19.3.0-4975.g1889a165.el9.x86_64 3/35 2024-09-17T15:22:13.435 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-selinux-2:19.3.0-4975.g1889a165.el9.x86_64 4/35 2024-09-17T15:22:13.435 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephfs2-2:19.3.0-4975.g1889a165.el9.x86_64 5/35 2024-09-17T15:22:13.435 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libradosstriper1-2:19.3.0-4975.g1889a165.el9.x86_6 6/35 2024-09-17T15:22:13.435 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librgw2-2:19.3.0-4975.g1889a165.el9.x86_64 7/35 2024-09-17T15:22:13.436 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4975.g1889a165.el9. 8/35 2024-09-17T15:22:13.436 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ceph-common-2:19.3.0-4975.g1889a165.el9.x8 9/35 2024-09-17T15:22:13.436 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cephfs-2:19.3.0-4975.g1889a165.el9.x86_64 10/35 2024-09-17T15:22:13.436 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rados-2:19.3.0-4975.g1889a165.el9.x86_64 11/35 2024-09-17T15:22:13.436 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rbd-2:19.3.0-4975.g1889a165.el9.x86_64 12/35 2024-09-17T15:22:13.436 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rgw-2:19.3.0-4975.g1889a165.el9.x86_64 13/35 2024-09-17T15:22:13.436 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-17T15:22:13.436 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-17T15:22:13.436 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-17T15:22:13.436 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-17T15:22:13.436 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-17T15:22:13.436 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-17T15:22:13.437 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-17T15:22:13.437 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-17T15:22:13.437 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-17T15:22:13.437 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-17T15:22:13.437 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 24/35 2024-09-17T15:22:13.437 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-17T15:22:13.437 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-17T15:22:13.438 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-17T15:22:13.439 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-17T15:22:13.439 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-17T15:22:13.439 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-17T15:22:13.439 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-17T15:22:13.439 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados2-2:19.3.0-4975.g1889a165.el9.x86_64 32/35 2024-09-17T15:22:13.439 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-17T15:22:13.439 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librbd1-2:19.3.0-4975.g1889a165.el9.x86_64 34/35 2024-09-17T15:22:14.313 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-17T15:22:14.313 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:22:14.313 INFO:teuthology.orchestra.run.smithi083.stdout:Upgraded: 2024-09-17T15:22:14.314 INFO:teuthology.orchestra.run.smithi083.stdout: librados2-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T15:22:14.314 INFO:teuthology.orchestra.run.smithi083.stdout: librbd1-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T15:22:14.314 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-09-17T15:22:14.314 INFO:teuthology.orchestra.run.smithi083.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-17T15:22:14.314 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-base-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T15:22:14.314 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-common-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T15:22:14.314 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-radosgw-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T15:22:14.314 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-selinux-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T15:22:14.315 INFO:teuthology.orchestra.run.smithi083.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-17T15:22:14.315 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-17T15:22:14.315 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-17T15:22:14.315 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs2-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T15:22:14.315 INFO:teuthology.orchestra.run.smithi083.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-17T15:22:14.315 INFO:teuthology.orchestra.run.smithi083.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-17T15:22:14.315 INFO:teuthology.orchestra.run.smithi083.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-17T15:22:14.315 INFO:teuthology.orchestra.run.smithi083.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-17T15:22:14.316 INFO:teuthology.orchestra.run.smithi083.stdout: libradosstriper1-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T15:22:14.316 INFO:teuthology.orchestra.run.smithi083.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-17T15:22:14.316 INFO:teuthology.orchestra.run.smithi083.stdout: librgw2-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T15:22:14.316 INFO:teuthology.orchestra.run.smithi083.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-17T15:22:14.316 INFO:teuthology.orchestra.run.smithi083.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-17T15:22:14.316 INFO:teuthology.orchestra.run.smithi083.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-17T15:22:14.317 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-argparse-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T15:22:14.317 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-common-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T15:22:14.317 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cephfs-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T15:22:14.317 INFO:teuthology.orchestra.run.smithi083.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-17T15:22:14.317 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rados-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T15:22:14.317 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rbd-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T15:22:14.318 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rgw-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T15:22:14.318 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-17T15:22:14.318 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-17T15:22:14.318 INFO:teuthology.orchestra.run.smithi083.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-17T15:22:14.318 INFO:teuthology.orchestra.run.smithi083.stdout: re2-1:20211101-20.el9.x86_64 2024-09-17T15:22:14.318 INFO:teuthology.orchestra.run.smithi083.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-17T15:22:14.318 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:22:14.319 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-17T15:22:14.651 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-test 2024-09-17T15:22:15.229 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:00:29 ago on Tue 17 Sep 2024 03:21:46 PM UTC. 2024-09-17T15:22:15.358 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-17T15:22:15.359 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:22:15.359 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repository Size 2024-09-17T15:22:15.359 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:22:15.359 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-09-17T15:22:15.359 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-test x86_64 2:19.3.0-4975.g1889a165.el9 ceph 46 M 2024-09-17T15:22:15.360 INFO:teuthology.orchestra.run.smithi083.stdout:Installing dependencies: 2024-09-17T15:22:15.360 INFO:teuthology.orchestra.run.smithi083.stdout: libcephsqlite x86_64 2:19.3.0-4975.g1889a165.el9 ceph 162 k 2024-09-17T15:22:15.360 INFO:teuthology.orchestra.run.smithi083.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-17T15:22:15.360 INFO:teuthology.orchestra.run.smithi083.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-17T15:22:15.360 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:22:15.360 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-09-17T15:22:15.360 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:22:15.360 INFO:teuthology.orchestra.run.smithi083.stdout:Install 4 Packages 2024-09-17T15:22:15.360 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:22:15.361 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 46 M 2024-09-17T15:22:15.361 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 201 M 2024-09-17T15:22:15.361 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-09-17T15:22:15.505 INFO:teuthology.orchestra.run.smithi083.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 2.5 MB/s | 304 kB 00:00 2024-09-17T15:22:15.547 INFO:teuthology.orchestra.run.smithi083.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-17T15:22:15.789 INFO:teuthology.orchestra.run.smithi083.stdout:(3/4): libcephsqlite-19.3.0-4975.g1889a165.el9. 400 kB/s | 162 kB 00:00 2024-09-17T15:22:20.373 INFO:teuthology.orchestra.run.smithi083.stdout:(4/4): ceph-test-19.3.0-4975.g1889a165.el9.x86_ 9.1 MB/s | 46 MB 00:04 2024-09-17T15:22:20.374 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-09-17T15:22:20.374 INFO:teuthology.orchestra.run.smithi083.stdout:Total 9.2 MB/s | 46 MB 00:05 2024-09-17T15:22:20.402 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-09-17T15:22:20.422 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-09-17T15:22:20.423 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-09-17T15:22:20.801 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-09-17T15:22:20.801 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-09-17T15:22:21.314 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-09-17T15:22:21.389 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-17T15:22:21.436 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-17T15:22:21.457 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephsqlite-2:19.3.0-4975.g1889a165.el9.x86_64 3/4 2024-09-17T15:22:25.880 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4975.g1889a165.el9.x86_64 3/4 2024-09-17T15:22:25.972 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-test-2:19.3.0-4975.g1889a165.el9.x86_64 4/4 2024-09-17T15:22:28.396 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-test-2:19.3.0-4975.g1889a165.el9.x86_64 4/4 2024-09-17T15:22:28.396 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-test-2:19.3.0-4975.g1889a165.el9.x86_64 1/4 2024-09-17T15:22:28.396 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephsqlite-2:19.3.0-4975.g1889a165.el9.x86_64 2/4 2024-09-17T15:22:28.397 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-17T15:22:28.690 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-17T15:22:28.690 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:22:28.690 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-09-17T15:22:28.690 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-test-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T15:22:28.691 INFO:teuthology.orchestra.run.smithi083.stdout: libcephsqlite-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T15:22:28.691 INFO:teuthology.orchestra.run.smithi083.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-17T15:22:28.691 INFO:teuthology.orchestra.run.smithi083.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-17T15:22:28.691 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:22:28.691 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-17T15:22:28.958 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph 2024-09-17T15:22:29.536 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:00:43 ago on Tue 17 Sep 2024 03:21:46 PM UTC. 2024-09-17T15:22:29.681 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-17T15:22:29.683 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:22:29.683 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repository Size 2024-09-17T15:22:29.683 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:22:29.683 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-09-17T15:22:29.684 INFO:teuthology.orchestra.run.smithi083.stdout: ceph x86_64 2:19.3.0-4975.g1889a165.el9 ceph 6.6 k 2024-09-17T15:22:29.684 INFO:teuthology.orchestra.run.smithi083.stdout:Installing dependencies: 2024-09-17T15:22:29.684 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mds x86_64 2:19.3.0-4975.g1889a165.el9 ceph 2.4 M 2024-09-17T15:22:29.684 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr x86_64 2:19.3.0-4975.g1889a165.el9 ceph 1.5 M 2024-09-17T15:22:29.684 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4975.g1889a165.el9 ceph-noarch 291 k 2024-09-17T15:22:29.684 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mon x86_64 2:19.3.0-4975.g1889a165.el9 ceph 4.8 M 2024-09-17T15:22:29.684 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-osd x86_64 2:19.3.0-4975.g1889a165.el9 ceph 17 M 2024-09-17T15:22:29.684 INFO:teuthology.orchestra.run.smithi083.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-17T15:22:29.684 INFO:teuthology.orchestra.run.smithi083.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-17T15:22:29.684 INFO:teuthology.orchestra.run.smithi083.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-17T15:22:29.685 INFO:teuthology.orchestra.run.smithi083.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-17T15:22:29.685 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-17T15:22:29.685 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-17T15:22:29.685 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-17T15:22:29.685 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-17T15:22:29.685 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-17T15:22:29.685 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-17T15:22:29.685 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-collections 2024-09-17T15:22:29.685 INFO:teuthology.orchestra.run.smithi083.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-17T15:22:29.685 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-17T15:22:29.685 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-17T15:22:29.685 INFO:teuthology.orchestra.run.smithi083.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-17T15:22:29.686 INFO:teuthology.orchestra.run.smithi083.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-17T15:22:29.686 INFO:teuthology.orchestra.run.smithi083.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-17T15:22:29.686 INFO:teuthology.orchestra.run.smithi083.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-17T15:22:29.686 INFO:teuthology.orchestra.run.smithi083.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-17T15:22:29.686 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-17T15:22:29.686 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-17T15:22:29.686 INFO:teuthology.orchestra.run.smithi083.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-17T15:22:29.686 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-17T15:22:29.686 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-17T15:22:29.686 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-17T15:22:29.686 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-17T15:22:29.686 INFO:teuthology.orchestra.run.smithi083.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-17T15:22:29.687 INFO:teuthology.orchestra.run.smithi083.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-17T15:22:29.687 INFO:teuthology.orchestra.run.smithi083.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-17T15:22:29.687 INFO:teuthology.orchestra.run.smithi083.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-17T15:22:29.687 INFO:teuthology.orchestra.run.smithi083.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-17T15:22:29.687 INFO:teuthology.orchestra.run.smithi083.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-17T15:22:29.687 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:22:29.687 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-09-17T15:22:29.687 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:22:29.687 INFO:teuthology.orchestra.run.smithi083.stdout:Install 37 Packages 2024-09-17T15:22:29.687 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:22:29.688 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 31 M 2024-09-17T15:22:29.688 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 109 M 2024-09-17T15:22:29.688 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-09-17T15:22:30.882 INFO:teuthology.orchestra.run.smithi083.stdout:(1/37): ceph-19.3.0-4975.g1889a165.el9.x86_64.r 24 kB/s | 6.6 kB 00:00 2024-09-17T15:22:32.751 INFO:teuthology.orchestra.run.smithi083.stdout:(2/37): ceph-mds-19.3.0-4975.g1889a165.el9.x86_ 1.1 MB/s | 2.4 MB 00:02 2024-09-17T15:22:33.349 INFO:teuthology.orchestra.run.smithi083.stdout:(3/37): ceph-mgr-19.3.0-4975.g1889a165.el9.x86_ 570 kB/s | 1.5 MB 00:02 2024-09-17T15:22:33.691 INFO:teuthology.orchestra.run.smithi083.stdout:(4/37): ceph-mgr-modules-core-19.3.0-4975.g1889 852 kB/s | 291 kB 00:00 2024-09-17T15:22:33.932 INFO:teuthology.orchestra.run.smithi083.stdout:(5/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.0 MB/s | 253 kB 00:00 2024-09-17T15:22:34.100 INFO:teuthology.orchestra.run.smithi083.stdout:(6/37): python3-cryptography-36.0.1-4.el9.x86_6 7.4 MB/s | 1.2 MB 00:00 2024-09-17T15:22:34.158 INFO:teuthology.orchestra.run.smithi083.stdout:(7/37): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-17T15:22:34.275 INFO:teuthology.orchestra.run.smithi083.stdout:(8/37): python3-pycparser-2.20-6.el9.noarch.rpm 1.1 MB/s | 135 kB 00:00 2024-09-17T15:22:34.359 INFO:teuthology.orchestra.run.smithi083.stdout:(9/37): python3-requests-2.25.1-8.el9.noarch.rp 1.5 MB/s | 125 kB 00:00 2024-09-17T15:22:34.418 INFO:teuthology.orchestra.run.smithi083.stdout:(10/37): python3-urllib3-1.26.5-6.el9.noarch.rp 3.6 MB/s | 215 kB 00:00 2024-09-17T15:22:34.584 INFO:teuthology.orchestra.run.smithi083.stdout:(11/37): lua-5.4.4-4.el9.x86_64.rpm 1.1 MB/s | 188 kB 00:00 2024-09-17T15:22:34.668 INFO:teuthology.orchestra.run.smithi083.stdout:(12/37): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-09-17T15:22:34.718 INFO:teuthology.orchestra.run.smithi083.stdout:(13/37): python3-markupsafe-1.1.1-12.el9.x86_64 696 kB/s | 35 kB 00:00 2024-09-17T15:22:34.768 INFO:teuthology.orchestra.run.smithi083.stdout:(14/37): python3-packaging-20.9-5.el9.noarch.rp 1.5 MB/s | 77 kB 00:00 2024-09-17T15:22:34.819 INFO:teuthology.orchestra.run.smithi083.stdout:(15/37): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-17T15:22:34.869 INFO:teuthology.orchestra.run.smithi083.stdout:(16/37): python3-toml-0.10.2-6.el9.noarch.rpm 835 kB/s | 42 kB 00:00 2024-09-17T15:22:34.945 INFO:teuthology.orchestra.run.smithi083.stdout:(17/37): lua-devel-5.4.4-4.el9.x86_64.rpm 297 kB/s | 22 kB 00:00 2024-09-17T15:22:34.995 INFO:teuthology.orchestra.run.smithi083.stdout:(18/37): luarocks-3.9.2-5.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-09-17T15:22:35.020 INFO:teuthology.orchestra.run.smithi083.stdout:(19/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 1.7 MB/s | 43 kB 00:00 2024-09-17T15:22:35.049 INFO:teuthology.orchestra.run.smithi083.stdout:(20/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 5.9 MB/s | 172 kB 00:00 2024-09-17T15:22:35.082 INFO:teuthology.orchestra.run.smithi083.stdout:(21/37): python3-cherrypy-18.6.1-2.el9.noarch.r 11 MB/s | 358 kB 00:00 2024-09-17T15:22:35.108 INFO:teuthology.orchestra.run.smithi083.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 426 kB/s | 11 kB 00:00 2024-09-17T15:22:35.133 INFO:teuthology.orchestra.run.smithi083.stdout:(23/37): python3-jaraco-classes-3.2.1-5.el9.noa 708 kB/s | 18 kB 00:00 2024-09-17T15:22:35.158 INFO:teuthology.orchestra.run.smithi083.stdout:(24/37): python3-jaraco-collections-3.0.0-8.el9 934 kB/s | 23 kB 00:00 2024-09-17T15:22:35.183 INFO:teuthology.orchestra.run.smithi083.stdout:(25/37): python3-jaraco-functools-3.5.0-2.el9.n 778 kB/s | 19 kB 00:00 2024-09-17T15:22:35.209 INFO:teuthology.orchestra.run.smithi083.stdout:(26/37): python3-jaraco-text-3.2.0-6.el9.noarch 786 kB/s | 20 kB 00:00 2024-09-17T15:22:35.234 INFO:teuthology.orchestra.run.smithi083.stdout:(27/37): python3-logutils-0.3.5-21.el9.noarch.r 1.8 MB/s | 46 kB 00:00 2024-09-17T15:22:35.259 INFO:teuthology.orchestra.run.smithi083.stdout:(28/37): python3-more-itertools-8.12.0-2.el9.no 3.1 MB/s | 79 kB 00:00 2024-09-17T15:22:35.285 INFO:teuthology.orchestra.run.smithi083.stdout:(29/37): python3-pecan-1.4.2-3.el9.noarch.rpm 11 MB/s | 272 kB 00:00 2024-09-17T15:22:35.310 INFO:teuthology.orchestra.run.smithi083.stdout:(30/37): python3-portend-3.1.0-2.el9.noarch.rpm 658 kB/s | 16 kB 00:00 2024-09-17T15:22:35.335 INFO:teuthology.orchestra.run.smithi083.stdout:(31/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 3.5 MB/s | 90 kB 00:00 2024-09-17T15:22:35.361 INFO:teuthology.orchestra.run.smithi083.stdout:(32/37): python3-tempora-5.0.0-2.el9.noarch.rpm 1.4 MB/s | 36 kB 00:00 2024-09-17T15:22:35.394 INFO:teuthology.orchestra.run.smithi083.stdout:(33/37): python3-webob-1.8.8-2.el9.noarch.rpm 6.7 MB/s | 230 kB 00:00 2024-09-17T15:22:35.428 INFO:teuthology.orchestra.run.smithi083.stdout:(34/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 13 MB/s | 427 kB 00:00 2024-09-17T15:22:35.453 INFO:teuthology.orchestra.run.smithi083.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 804 kB/s | 20 kB 00:00 2024-09-17T15:22:38.728 INFO:teuthology.orchestra.run.smithi083.stdout:(36/37): ceph-mon-19.3.0-4975.g1889a165.el9.x86 625 kB/s | 4.8 MB 00:07 2024-09-17T15:22:40.195 INFO:teuthology.orchestra.run.smithi083.stdout:(37/37): ceph-osd-19.3.0-4975.g1889a165.el9.x86 2.3 MB/s | 17 MB 00:07 2024-09-17T15:22:40.199 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-09-17T15:22:40.199 INFO:teuthology.orchestra.run.smithi083.stdout:Total 2.9 MB/s | 31 MB 00:10 2024-09-17T15:22:40.353 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-09-17T15:22:40.391 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-09-17T15:22:40.391 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-09-17T15:22:40.880 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-09-17T15:22:40.880 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-09-17T15:22:42.222 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-09-17T15:22:42.305 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-17T15:22:42.366 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-17T15:22:42.409 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-17T15:22:42.481 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-17T15:22:42.524 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-17T15:22:42.557 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-17T15:22:42.649 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-17T15:22:42.736 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-17T15:22:42.777 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-17T15:22:42.813 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-17T15:22:42.841 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-17T15:22:42.876 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-17T15:22:42.909 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-17T15:22:42.949 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-17T15:22:42.985 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-17T15:22:43.019 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-17T15:22:43.068 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-17T15:22:43.100 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-17T15:22:43.173 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-17T15:22:43.312 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-17T15:22:43.402 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-17T15:22:43.455 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-17T15:22:43.502 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-17T15:22:43.559 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-17T15:22:43.615 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-17T15:22:43.808 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-17T15:22:43.856 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-17T15:22:43.934 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-17T15:22:44.023 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-17T15:22:44.063 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-17T15:22:44.191 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-17T15:22:44.353 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4975.g1889a165.el9. 32/37 2024-09-17T15:22:44.374 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-2:19.3.0-4975.g1889a165.el9.x86_64 33/37 2024-09-17T15:22:44.400 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4975.g1889a165.el9.x86_64 33/37 2024-09-17T15:22:44.400 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T15:22:44.400 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-17T15:22:44.400 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T15:22:44.400 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T15:22:44.400 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:22:45.875 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-osd-2:19.3.0-4975.g1889a165.el9.x86_64 34/37 2024-09-17T15:22:45.902 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-osd-2:19.3.0-4975.g1889a165.el9.x86_64 34/37 2024-09-17T15:22:45.902 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T15:22:45.902 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-17T15:22:45.902 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T15:22:45.902 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T15:22:45.902 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:22:46.359 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mon-2:19.3.0-4975.g1889a165.el9.x86_64 35/37 2024-09-17T15:22:46.383 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mon-2:19.3.0-4975.g1889a165.el9.x86_64 35/37 2024-09-17T15:22:46.383 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T15:22:46.383 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-17T15:22:46.383 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T15:22:46.383 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T15:22:46.383 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:22:46.610 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mds-2:19.3.0-4975.g1889a165.el9.x86_64 36/37 2024-09-17T15:22:46.634 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mds-2:19.3.0-4975.g1889a165.el9.x86_64 36/37 2024-09-17T15:22:46.634 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T15:22:46.634 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-17T15:22:46.634 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T15:22:46.634 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T15:22:46.634 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:22:46.701 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-2:19.3.0-4975.g1889a165.el9.x86_64 37/37 2024-09-17T15:22:48.466 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-2:19.3.0-4975.g1889a165.el9.x86_64 37/37 2024-09-17T15:22:48.466 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-2:19.3.0-4975.g1889a165.el9.x86_64 1/37 2024-09-17T15:22:48.466 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mds-2:19.3.0-4975.g1889a165.el9.x86_64 2/37 2024-09-17T15:22:48.466 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-2:19.3.0-4975.g1889a165.el9.x86_64 3/37 2024-09-17T15:22:48.466 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mon-2:19.3.0-4975.g1889a165.el9.x86_64 4/37 2024-09-17T15:22:48.466 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-osd-2:19.3.0-4975.g1889a165.el9.x86_64 5/37 2024-09-17T15:22:48.466 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4975.g1889a165.el9. 6/37 2024-09-17T15:22:48.466 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-17T15:22:48.466 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-17T15:22:48.466 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-17T15:22:48.467 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-17T15:22:48.467 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-17T15:22:48.467 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-17T15:22:48.467 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-17T15:22:48.467 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-17T15:22:48.467 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-17T15:22:48.467 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-17T15:22:48.467 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-17T15:22:48.467 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-17T15:22:48.467 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-17T15:22:48.467 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-17T15:22:48.467 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-17T15:22:48.467 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-17T15:22:48.467 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-17T15:22:48.467 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-17T15:22:48.468 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-17T15:22:48.468 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-17T15:22:48.468 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-17T15:22:48.468 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-17T15:22:48.468 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-17T15:22:48.468 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-17T15:22:48.469 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-17T15:22:48.469 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-17T15:22:48.469 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-17T15:22:48.469 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-17T15:22:48.469 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-17T15:22:48.469 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-17T15:22:49.565 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-17T15:22:49.565 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:22:49.565 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-09-17T15:22:49.565 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T15:22:49.565 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mds-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T15:22:49.565 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T15:22:49.565 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-modules-core-2:19.3.0-4975.g1889a165.el9.noarch 2024-09-17T15:22:49.565 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mon-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T15:22:49.566 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-osd-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T15:22:49.566 INFO:teuthology.orchestra.run.smithi083.stdout: lua-5.4.4-4.el9.x86_64 2024-09-17T15:22:49.566 INFO:teuthology.orchestra.run.smithi083.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-17T15:22:49.566 INFO:teuthology.orchestra.run.smithi083.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-17T15:22:49.566 INFO:teuthology.orchestra.run.smithi083.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-17T15:22:49.566 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-17T15:22:49.566 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-17T15:22:49.566 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-17T15:22:49.566 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-17T15:22:49.566 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-17T15:22:49.566 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-17T15:22:49.567 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-17T15:22:49.567 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-17T15:22:49.567 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-17T15:22:49.567 INFO:teuthology.orchestra.run.smithi083.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-17T15:22:49.567 INFO:teuthology.orchestra.run.smithi083.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-17T15:22:49.567 INFO:teuthology.orchestra.run.smithi083.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-17T15:22:49.567 INFO:teuthology.orchestra.run.smithi083.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-17T15:22:49.567 INFO:teuthology.orchestra.run.smithi083.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-17T15:22:49.567 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-17T15:22:49.567 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ply-3.11-14.el9.noarch 2024-09-17T15:22:49.567 INFO:teuthology.orchestra.run.smithi083.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-17T15:22:49.568 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-17T15:22:49.568 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-17T15:22:49.568 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-17T15:22:49.568 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-17T15:22:49.568 INFO:teuthology.orchestra.run.smithi083.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-17T15:22:49.568 INFO:teuthology.orchestra.run.smithi083.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-17T15:22:49.568 INFO:teuthology.orchestra.run.smithi083.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-17T15:22:49.568 INFO:teuthology.orchestra.run.smithi083.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-17T15:22:49.568 INFO:teuthology.orchestra.run.smithi083.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-17T15:22:49.568 INFO:teuthology.orchestra.run.smithi083.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-17T15:22:49.568 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:22:49.569 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-17T15:22:49.934 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-base 2024-09-17T15:22:50.519 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:04 ago on Tue 17 Sep 2024 03:21:46 PM UTC. 2024-09-17T15:22:50.608 INFO:teuthology.orchestra.run.smithi083.stdout:Package ceph-base-2:19.3.0-4975.g1889a165.el9.x86_64 is already installed. 2024-09-17T15:22:50.650 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-17T15:22:50.651 INFO:teuthology.orchestra.run.smithi083.stdout:Nothing to do. 2024-09-17T15:22:50.651 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-17T15:22:50.718 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install cephadm 2024-09-17T15:22:51.300 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:05 ago on Tue 17 Sep 2024 03:21:46 PM UTC. 2024-09-17T15:22:51.430 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-17T15:22:51.430 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:22:51.430 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repository Size 2024-09-17T15:22:51.431 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:22:51.431 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-09-17T15:22:51.431 INFO:teuthology.orchestra.run.smithi083.stdout: cephadm noarch 2:19.3.0-4975.g1889a165.el9 ceph-noarch 784 k 2024-09-17T15:22:51.431 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:22:51.431 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-09-17T15:22:51.431 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:22:51.431 INFO:teuthology.orchestra.run.smithi083.stdout:Install 1 Package 2024-09-17T15:22:51.431 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:22:51.432 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 784 k 2024-09-17T15:22:51.432 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 791 k 2024-09-17T15:22:51.432 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-09-17T15:22:51.694 INFO:teuthology.orchestra.run.smithi083.stdout:cephadm-19.3.0-4975.g1889a165.el9.noarch.rpm 2.9 MB/s | 784 kB 00:00 2024-09-17T15:22:51.694 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-09-17T15:22:51.694 INFO:teuthology.orchestra.run.smithi083.stdout:Total 2.9 MB/s | 784 kB 00:00 2024-09-17T15:22:51.695 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-09-17T15:22:51.700 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-09-17T15:22:51.700 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-09-17T15:22:51.706 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-09-17T15:22:51.706 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-09-17T15:22:51.776 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-09-17T15:22:52.272 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: cephadm-2:19.3.0-4975.g1889a165.el9.noarch 1/1 2024-09-17T15:22:52.356 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephadm-2:19.3.0-4975.g1889a165.el9.noarch 1/1 2024-09-17T15:22:52.713 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: cephadm-2:19.3.0-4975.g1889a165.el9.noarch 1/1 2024-09-17T15:22:52.956 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephadm-2:19.3.0-4975.g1889a165.el9.noarch 1/1 2024-09-17T15:22:52.956 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:22:52.956 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-09-17T15:22:52.957 INFO:teuthology.orchestra.run.smithi083.stdout: cephadm-2:19.3.0-4975.g1889a165.el9.noarch 2024-09-17T15:22:52.957 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:22:52.957 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-17T15:22:53.163 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-immutable-object-cache 2024-09-17T15:22:53.766 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:07 ago on Tue 17 Sep 2024 03:21:46 PM UTC. 2024-09-17T15:22:53.902 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-17T15:22:53.929 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:22:53.929 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repo Size 2024-09-17T15:22:53.929 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:22:53.929 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-09-17T15:22:53.929 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4975.g1889a165.el9 ceph 145 k 2024-09-17T15:22:53.930 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:22:53.930 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-09-17T15:22:53.930 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:22:53.930 INFO:teuthology.orchestra.run.smithi083.stdout:Install 1 Package 2024-09-17T15:22:53.930 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:22:53.930 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 145 k 2024-09-17T15:22:53.930 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 432 k 2024-09-17T15:22:53.930 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-09-17T15:22:54.100 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-immutable-object-cache-19.3.0-4975.g1889a1 738 kB/s | 145 kB 00:00 2024-09-17T15:22:54.101 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-09-17T15:22:54.101 INFO:teuthology.orchestra.run.smithi083.stdout:Total 733 kB/s | 145 kB 00:00 2024-09-17T15:22:54.101 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-09-17T15:22:54.108 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-09-17T15:22:54.108 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-09-17T15:22:54.165 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-09-17T15:22:54.166 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-09-17T15:22:54.310 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-09-17T15:22:54.384 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4975.g1889a165. 1/1 2024-09-17T15:22:54.409 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4975.g1889a165. 1/1 2024-09-17T15:22:54.410 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T15:22:54.410 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-17T15:22:54.410 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:22:55.381 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4975.g1889a165. 1/1 2024-09-17T15:22:55.382 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:22:55.382 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-09-17T15:22:55.382 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-immutable-object-cache-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T15:22:55.382 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:22:55.382 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-17T15:22:55.629 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-mgr 2024-09-17T15:22:56.221 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:10 ago on Tue 17 Sep 2024 03:21:46 PM UTC. 2024-09-17T15:22:56.313 INFO:teuthology.orchestra.run.smithi083.stdout:Package ceph-mgr-2:19.3.0-4975.g1889a165.el9.x86_64 is already installed. 2024-09-17T15:22:56.355 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-17T15:22:56.356 INFO:teuthology.orchestra.run.smithi083.stdout:Nothing to do. 2024-09-17T15:22:56.356 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-17T15:22:56.428 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-mgr-dashboard 2024-09-17T15:22:57.043 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:11 ago on Tue 17 Sep 2024 03:21:46 PM UTC. 2024-09-17T15:22:57.185 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-17T15:22:57.187 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:22:57.187 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repository Size 2024-09-17T15:22:57.187 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:22:57.187 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-09-17T15:22:57.187 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4975.g1889a165.el9 ceph-noarch 14 M 2024-09-17T15:22:57.187 INFO:teuthology.orchestra.run.smithi083.stdout:Installing dependencies: 2024-09-17T15:22:57.187 INFO:teuthology.orchestra.run.smithi083.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-17T15:22:57.187 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4975.g1889a165.el9 ceph-noarch 34 k 2024-09-17T15:22:57.187 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4975.g1889a165.el9 ceph-noarch 16 k 2024-09-17T15:22:57.188 INFO:teuthology.orchestra.run.smithi083.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-17T15:22:57.188 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-17T15:22:57.188 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-17T15:22:57.188 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-17T15:22:57.188 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-17T15:22:57.188 INFO:teuthology.orchestra.run.smithi083.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-17T15:22:57.188 INFO:teuthology.orchestra.run.smithi083.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-17T15:22:57.188 INFO:teuthology.orchestra.run.smithi083.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-17T15:22:57.188 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:22:57.188 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-09-17T15:22:57.188 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:22:57.188 INFO:teuthology.orchestra.run.smithi083.stdout:Install 12 Packages 2024-09-17T15:22:57.188 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:22:57.189 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 19 M 2024-09-17T15:22:57.189 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 159 M 2024-09-17T15:22:57.189 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-09-17T15:22:57.754 INFO:teuthology.orchestra.run.smithi083.stdout:(1/12): ceph-grafana-dashboards-19.3.0-4975.g18 143 kB/s | 34 kB 00:00 2024-09-17T15:22:57.780 INFO:teuthology.orchestra.run.smithi083.stdout:(2/12): ceph-prometheus-alerts-19.3.0-4975.g188 62 kB/s | 16 kB 00:00 2024-09-17T15:22:57.905 INFO:teuthology.orchestra.run.smithi083.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 2.1 MB/s | 269 kB 00:00 2024-09-17T15:22:58.006 INFO:teuthology.orchestra.run.smithi083.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 4.0 MB/s | 1.0 MB 00:00 2024-09-17T15:22:58.055 INFO:teuthology.orchestra.run.smithi083.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 11 MB/s | 548 kB 00:00 2024-09-17T15:22:58.081 INFO:teuthology.orchestra.run.smithi083.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 778 kB/s | 19 kB 00:00 2024-09-17T15:22:58.156 INFO:teuthology.orchestra.run.smithi083.stdout:(7/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 27 MB/s | 2.0 MB 00:00 2024-09-17T15:22:58.181 INFO:teuthology.orchestra.run.smithi083.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 5.6 MB/s | 144 kB 00:00 2024-09-17T15:22:58.215 INFO:teuthology.orchestra.run.smithi083.stdout:(9/12): protobuf-compiler-3.14.0-13.el9.x86_64. 2.7 MB/s | 863 kB 00:00 2024-09-17T15:22:58.241 INFO:teuthology.orchestra.run.smithi083.stdout:(10/12): python3-repoze-lru-0.7-16.el9.noarch.r 523 kB/s | 31 kB 00:00 2024-09-17T15:22:58.282 INFO:teuthology.orchestra.run.smithi083.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 2.8 MB/s | 188 kB 00:00 2024-09-17T15:22:59.416 INFO:teuthology.orchestra.run.smithi083.stdout:(12/12): ceph-mgr-dashboard-19.3.0-4975.g1889a1 7.5 MB/s | 14 MB 00:01 2024-09-17T15:22:59.420 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-09-17T15:22:59.420 INFO:teuthology.orchestra.run.smithi083.stdout:Total 8.6 MB/s | 19 MB 00:02 2024-09-17T15:22:59.469 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-09-17T15:22:59.488 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-09-17T15:22:59.488 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-09-17T15:22:59.777 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-09-17T15:22:59.777 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-09-17T15:23:00.373 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-09-17T15:23:00.458 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-17T15:23:00.493 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-17T15:23:00.540 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-17T15:23:00.570 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-17T15:23:00.677 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-17T15:23:00.835 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-17T15:23:00.923 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-17T15:23:00.967 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-17T15:23:00.996 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-17T15:23:01.028 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4975.g1889a165.el9 10/12 2024-09-17T15:23:04.216 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4975.g1889a165.el 11/12 2024-09-17T15:23:04.274 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4975.g1889a165.el9.noa 12/12 2024-09-17T15:23:06.295 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4975.g1889a165.el9.noa 12/12 2024-09-17T15:23:06.295 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4975.g1889a165.el 1/12 2024-09-17T15:23:06.295 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4975.g1889a165.el9.noa 2/12 2024-09-17T15:23:06.295 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4975.g1889a165.el9 3/12 2024-09-17T15:23:06.295 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-17T15:23:06.295 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-17T15:23:06.295 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-17T15:23:06.295 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-17T15:23:06.295 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-17T15:23:06.295 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-17T15:23:06.295 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-17T15:23:06.296 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-17T15:23:06.621 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-17T15:23:06.621 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:06.621 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-09-17T15:23:06.621 INFO:teuthology.orchestra.run.smithi083.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-17T15:23:06.621 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-grafana-dashboards-2:19.3.0-4975.g1889a165.el9.noarch 2024-09-17T15:23:06.621 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-dashboard-2:19.3.0-4975.g1889a165.el9.noarch 2024-09-17T15:23:06.622 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-prometheus-alerts-2:19.3.0-4975.g1889a165.el9.noarch 2024-09-17T15:23:06.622 INFO:teuthology.orchestra.run.smithi083.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-17T15:23:06.622 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-17T15:23:06.622 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-17T15:23:06.622 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-17T15:23:06.622 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-17T15:23:06.622 INFO:teuthology.orchestra.run.smithi083.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-17T15:23:06.622 INFO:teuthology.orchestra.run.smithi083.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-17T15:23:06.622 INFO:teuthology.orchestra.run.smithi083.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-17T15:23:06.622 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:06.622 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-17T15:23:06.859 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-17T15:23:07.443 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:21 ago on Tue 17 Sep 2024 03:21:46 PM UTC. 2024-09-17T15:23:07.576 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-17T15:23:07.577 INFO:teuthology.orchestra.run.smithi083.stdout:======================================================================================= 2024-09-17T15:23:07.577 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repository Size 2024-09-17T15:23:07.577 INFO:teuthology.orchestra.run.smithi083.stdout:======================================================================================= 2024-09-17T15:23:07.577 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-09-17T15:23:07.577 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4975.g1889a165.el9 ceph-noarch 7.4 M 2024-09-17T15:23:07.577 INFO:teuthology.orchestra.run.smithi083.stdout:Installing dependencies: 2024-09-17T15:23:07.578 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-17T15:23:07.578 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-17T15:23:07.578 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-17T15:23:07.578 INFO:teuthology.orchestra.run.smithi083.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-17T15:23:07.578 INFO:teuthology.orchestra.run.smithi083.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-17T15:23:07.578 INFO:teuthology.orchestra.run.smithi083.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-17T15:23:07.578 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-17T15:23:07.578 INFO:teuthology.orchestra.run.smithi083.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-17T15:23:07.578 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-17T15:23:07.578 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-17T15:23:07.578 INFO:teuthology.orchestra.run.smithi083.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-17T15:23:07.578 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:07.578 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-09-17T15:23:07.579 INFO:teuthology.orchestra.run.smithi083.stdout:======================================================================================= 2024-09-17T15:23:07.579 INFO:teuthology.orchestra.run.smithi083.stdout:Install 12 Packages 2024-09-17T15:23:07.579 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:07.579 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 38 M 2024-09-17T15:23:07.579 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 212 M 2024-09-17T15:23:07.579 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-09-17T15:23:08.040 INFO:teuthology.orchestra.run.smithi083.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.0 MB/s | 191 kB 00:00 2024-09-17T15:23:08.132 INFO:teuthology.orchestra.run.smithi083.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 2.8 MB/s | 801 kB 00:00 2024-09-17T15:23:08.157 INFO:teuthology.orchestra.run.smithi083.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 280 kB/s | 33 kB 00:00 2024-09-17T15:23:08.207 INFO:teuthology.orchestra.run.smithi083.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 365 kB/s | 18 kB 00:00 2024-09-17T15:23:08.266 INFO:teuthology.orchestra.run.smithi083.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 658 kB/s | 38 kB 00:00 2024-09-17T15:23:09.908 INFO:teuthology.orchestra.run.smithi083.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 1.7 MB/s | 3.0 MB 00:01 2024-09-17T15:23:10.092 INFO:teuthology.orchestra.run.smithi083.stdout:(7/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 2.7 MB/s | 4.9 MB 00:01 2024-09-17T15:23:10.117 INFO:teuthology.orchestra.run.smithi083.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.1 MB/s | 245 kB 00:00 2024-09-17T15:23:10.384 INFO:teuthology.orchestra.run.smithi083.stdout:(9/12): ceph-mgr-diskprediction-local-19.3.0-49 2.9 MB/s | 7.4 MB 00:02 2024-09-17T15:23:10.409 INFO:teuthology.orchestra.run.smithi083.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 858 kB/s | 250 kB 00:00 2024-09-17T15:23:12.076 INFO:teuthology.orchestra.run.smithi083.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.6 MB/s | 5.1 MB 00:01 2024-09-17T15:23:14.067 INFO:teuthology.orchestra.run.smithi083.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 4.4 MB/s | 16 MB 00:03 2024-09-17T15:23:14.068 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-09-17T15:23:14.069 INFO:teuthology.orchestra.run.smithi083.stdout:Total 5.9 MB/s | 38 MB 00:06 2024-09-17T15:23:14.425 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-09-17T15:23:14.451 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-09-17T15:23:14.451 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-09-17T15:23:14.877 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-09-17T15:23:14.877 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-09-17T15:23:15.637 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-09-17T15:23:15.715 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-17T15:23:15.757 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-17T15:23:15.849 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-17T15:23:15.901 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-17T15:23:16.085 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-17T15:23:16.116 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-17T15:23:16.193 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-17T15:23:16.849 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-17T15:23:16.962 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-17T15:23:18.234 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T15:23:19.438 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-17T15:23:19.466 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4975.g1889a 12/12 2024-09-17T15:23:22.011 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4975.g1889a 12/12 2024-09-17T15:23:22.011 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4975.g1889a 1/12 2024-09-17T15:23:22.011 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-17T15:23:22.011 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-17T15:23:22.011 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-17T15:23:22.011 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-17T15:23:22.012 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-17T15:23:22.012 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-17T15:23:22.012 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-17T15:23:22.012 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-17T15:23:22.012 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T15:23:22.012 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-17T15:23:22.385 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-17T15:23:22.385 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:22.386 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-09-17T15:23:22.386 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4975.g1889a165.el9.noarch 2024-09-17T15:23:22.386 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-17T15:23:22.386 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-17T15:23:22.386 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-17T15:23:22.386 INFO:teuthology.orchestra.run.smithi083.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-17T15:23:22.386 INFO:teuthology.orchestra.run.smithi083.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-17T15:23:22.386 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-17T15:23:22.386 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-17T15:23:22.386 INFO:teuthology.orchestra.run.smithi083.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-17T15:23:22.386 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-17T15:23:22.386 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-17T15:23:22.386 INFO:teuthology.orchestra.run.smithi083.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-17T15:23:22.387 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:22.387 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-17T15:23:22.817 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-mgr-rook 2024-09-17T15:23:23.405 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:37 ago on Tue 17 Sep 2024 03:21:46 PM UTC. 2024-09-17T15:23:23.540 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-17T15:23:23.542 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:23:23.542 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repository Size 2024-09-17T15:23:23.542 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:23:23.542 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-09-17T15:23:23.542 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-rook noarch 2:19.3.0-4975.g1889a165.el9 ceph-noarch 49 k 2024-09-17T15:23:23.542 INFO:teuthology.orchestra.run.smithi083.stdout:Installing dependencies: 2024-09-17T15:23:23.542 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-17T15:23:23.542 INFO:teuthology.orchestra.run.smithi083.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-17T15:23:23.542 INFO:teuthology.orchestra.run.smithi083.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-17T15:23:23.542 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-17T15:23:23.542 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-17T15:23:23.542 INFO:teuthology.orchestra.run.smithi083.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-17T15:23:23.542 INFO:teuthology.orchestra.run.smithi083.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-17T15:23:23.543 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-17T15:23:23.543 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-17T15:23:23.543 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-17T15:23:23.543 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-17T15:23:23.543 INFO:teuthology.orchestra.run.smithi083.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-17T15:23:23.543 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:23.543 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-09-17T15:23:23.543 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:23:23.543 INFO:teuthology.orchestra.run.smithi083.stdout:Install 13 Packages 2024-09-17T15:23:23.543 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:23.544 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 2.2 M 2024-09-17T15:23:23.544 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 26 M 2024-09-17T15:23:23.544 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-09-17T15:23:24.100 INFO:teuthology.orchestra.run.smithi083.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 297 kB/s | 26 kB 00:00 2024-09-17T15:23:24.125 INFO:teuthology.orchestra.run.smithi083.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 172 kB/s | 19 kB 00:00 2024-09-17T15:23:24.225 INFO:teuthology.orchestra.run.smithi083.stdout:(3/13): ceph-mgr-rook-19.3.0-4975.g1889a165.el9 233 kB/s | 49 kB 00:00 2024-09-17T15:23:24.259 INFO:teuthology.orchestra.run.smithi083.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-09-17T15:23:24.284 INFO:teuthology.orchestra.run.smithi083.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.4 MB/s | 222 kB 00:00 2024-09-17T15:23:24.318 INFO:teuthology.orchestra.run.smithi083.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 914 kB/s | 54 kB 00:00 2024-09-17T15:23:24.351 INFO:teuthology.orchestra.run.smithi083.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 481 kB/s | 32 kB 00:00 2024-09-17T15:23:24.376 INFO:teuthology.orchestra.run.smithi083.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 249 kB/s | 14 kB 00:00 2024-09-17T15:23:24.402 INFO:teuthology.orchestra.run.smithi083.stdout:(9/13): python3-google-auth-2.34.0-1.el9.noarch 4.4 MB/s | 220 kB 00:00 2024-09-17T15:23:24.419 INFO:teuthology.orchestra.run.smithi083.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 3.5 MB/s | 59 kB 00:00 2024-09-17T15:23:24.444 INFO:teuthology.orchestra.run.smithi083.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 3.5 MB/s | 90 kB 00:00 2024-09-17T15:23:24.469 INFO:teuthology.orchestra.run.smithi083.stdout:(12/13): python3-pyasn1-modules-0.4.8-6.el9.noa 1.1 MB/s | 279 kB 00:00 2024-09-17T15:23:24.511 INFO:teuthology.orchestra.run.smithi083.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.6 MB/s | 1.0 MB 00:00 2024-09-17T15:23:24.514 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-09-17T15:23:24.514 INFO:teuthology.orchestra.run.smithi083.stdout:Total 2.3 MB/s | 2.2 MB 00:00 2024-09-17T15:23:24.582 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-09-17T15:23:24.597 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-09-17T15:23:24.597 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-09-17T15:23:24.686 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-09-17T15:23:24.686 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-09-17T15:23:25.195 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-09-17T15:23:25.366 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-17T15:23:25.421 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-17T15:23:25.477 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-17T15:23:25.514 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-17T15:23:25.563 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-17T15:23:25.664 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-17T15:23:25.766 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-17T15:23:25.815 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-17T15:23:26.519 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-17T15:23:26.563 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-17T15:23:26.596 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-17T15:23:26.644 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-17T15:23:26.666 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-rook-2:19.3.0-4975.g1889a165.el9.noarch 13/13 2024-09-17T15:23:27.453 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4975.g1889a165.el9.noarch 13/13 2024-09-17T15:23:27.453 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4975.g1889a165.el9.noarch 1/13 2024-09-17T15:23:27.454 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-17T15:23:27.454 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-17T15:23:27.454 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-17T15:23:27.454 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-17T15:23:27.454 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-17T15:23:27.454 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-17T15:23:27.454 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-17T15:23:27.454 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-17T15:23:27.454 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-17T15:23:27.454 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-17T15:23:27.454 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-17T15:23:27.793 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-17T15:23:27.793 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:27.794 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-09-17T15:23:27.794 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-rook-2:19.3.0-4975.g1889a165.el9.noarch 2024-09-17T15:23:27.794 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-17T15:23:27.794 INFO:teuthology.orchestra.run.smithi083.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-17T15:23:27.794 INFO:teuthology.orchestra.run.smithi083.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-17T15:23:27.794 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-17T15:23:27.794 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-17T15:23:27.794 INFO:teuthology.orchestra.run.smithi083.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-17T15:23:27.794 INFO:teuthology.orchestra.run.smithi083.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-17T15:23:27.794 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-17T15:23:27.794 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-17T15:23:27.794 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-17T15:23:27.794 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-17T15:23:27.795 INFO:teuthology.orchestra.run.smithi083.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-17T15:23:27.795 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:27.795 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-17T15:23:28.045 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-mgr-cephadm 2024-09-17T15:23:28.624 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:42 ago on Tue 17 Sep 2024 03:21:46 PM UTC. 2024-09-17T15:23:28.757 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-17T15:23:28.758 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:23:28.758 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repository Size 2024-09-17T15:23:28.758 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:23:28.758 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-09-17T15:23:28.758 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4975.g1889a165.el9 ceph-noarch 154 k 2024-09-17T15:23:28.758 INFO:teuthology.orchestra.run.smithi083.stdout:Installing dependencies: 2024-09-17T15:23:28.758 INFO:teuthology.orchestra.run.smithi083.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-17T15:23:28.758 INFO:teuthology.orchestra.run.smithi083.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-17T15:23:28.758 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-17T15:23:28.758 INFO:teuthology.orchestra.run.smithi083.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-17T15:23:28.758 INFO:teuthology.orchestra.run.smithi083.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-17T15:23:28.759 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:28.759 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-09-17T15:23:28.759 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:23:28.759 INFO:teuthology.orchestra.run.smithi083.stdout:Install 6 Packages 2024-09-17T15:23:28.759 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:28.759 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 7.0 M 2024-09-17T15:23:28.759 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 33 M 2024-09-17T15:23:28.760 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-09-17T15:23:29.431 INFO:teuthology.orchestra.run.smithi083.stdout:(1/6): ceph-mgr-cephadm-19.3.0-4975.g1889a165.e 402 kB/s | 154 kB 00:00 2024-09-17T15:23:29.489 INFO:teuthology.orchestra.run.smithi083.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 9.2 MB/s | 548 kB 00:00 2024-09-17T15:23:29.515 INFO:teuthology.orchestra.run.smithi083.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.3 MB/s | 58 kB 00:00 2024-09-17T15:23:29.540 INFO:teuthology.orchestra.run.smithi083.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.9 MB/s | 74 kB 00:00 2024-09-17T15:23:29.674 INFO:teuthology.orchestra.run.smithi083.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 396 kB/s | 248 kB 00:00 2024-09-17T15:23:30.590 INFO:teuthology.orchestra.run.smithi083.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 3.9 MB/s | 6.0 MB 00:01 2024-09-17T15:23:30.593 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-09-17T15:23:30.593 INFO:teuthology.orchestra.run.smithi083.stdout:Total 3.8 MB/s | 7.0 MB 00:01 2024-09-17T15:23:30.667 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-09-17T15:23:30.679 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-09-17T15:23:30.679 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-09-17T15:23:30.779 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-09-17T15:23:30.780 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-09-17T15:23:31.099 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-09-17T15:23:31.264 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-17T15:23:31.317 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-17T15:23:31.838 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-17T15:23:31.920 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-17T15:23:31.987 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-17T15:23:32.013 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4975.g1889a165.el9.noarch 6/6 2024-09-17T15:23:32.803 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4975.g1889a165.el9.noarch 6/6 2024-09-17T15:23:32.803 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4975.g1889a165.el9.noarch 1/6 2024-09-17T15:23:32.803 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-17T15:23:32.803 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-17T15:23:32.803 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-17T15:23:32.804 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-17T15:23:33.075 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-17T15:23:33.075 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:33.075 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-09-17T15:23:33.075 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-cephadm-2:19.3.0-4975.g1889a165.el9.noarch 2024-09-17T15:23:33.075 INFO:teuthology.orchestra.run.smithi083.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-17T15:23:33.075 INFO:teuthology.orchestra.run.smithi083.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-17T15:23:33.075 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-17T15:23:33.076 INFO:teuthology.orchestra.run.smithi083.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-17T15:23:33.076 INFO:teuthology.orchestra.run.smithi083.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-17T15:23:33.076 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:33.076 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-17T15:23:33.338 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-fuse 2024-09-17T15:23:33.910 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:47 ago on Tue 17 Sep 2024 03:21:46 PM UTC. 2024-09-17T15:23:34.042 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-17T15:23:34.043 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:23:34.043 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repository Size 2024-09-17T15:23:34.043 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:23:34.043 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-09-17T15:23:34.043 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-fuse x86_64 2:19.3.0-4975.g1889a165.el9 ceph 899 k 2024-09-17T15:23:34.044 INFO:teuthology.orchestra.run.smithi083.stdout:Installing dependencies: 2024-09-17T15:23:34.044 INFO:teuthology.orchestra.run.smithi083.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-17T15:23:34.044 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:34.044 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-09-17T15:23:34.044 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:23:34.044 INFO:teuthology.orchestra.run.smithi083.stdout:Install 2 Packages 2024-09-17T15:23:34.044 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:34.044 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 979 k 2024-09-17T15:23:34.044 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 2.9 M 2024-09-17T15:23:34.044 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-09-17T15:23:34.684 INFO:teuthology.orchestra.run.smithi083.stdout:(1/2): ceph-fuse-19.3.0-4975.g1889a165.el9.x86_ 1.7 MB/s | 899 kB 00:00 2024-09-17T15:23:34.801 INFO:teuthology.orchestra.run.smithi083.stdout:(2/2): fuse-2.9.9-16.el9.x86_64.rpm 125 kB/s | 80 kB 00:00 2024-09-17T15:23:34.802 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-09-17T15:23:34.802 INFO:teuthology.orchestra.run.smithi083.stdout:Total 1.3 MB/s | 979 kB 00:00 2024-09-17T15:23:34.813 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-09-17T15:23:34.835 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-09-17T15:23:34.835 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-09-17T15:23:34.917 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-09-17T15:23:34.917 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-09-17T15:23:35.101 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-09-17T15:23:35.237 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-17T15:23:35.317 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-fuse-2:19.3.0-4975.g1889a165.el9.x86_64 2/2 2024-09-17T15:23:35.998 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-fuse-2:19.3.0-4975.g1889a165.el9.x86_64 2/2 2024-09-17T15:23:35.998 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-fuse-2:19.3.0-4975.g1889a165.el9.x86_64 1/2 2024-09-17T15:23:36.276 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-17T15:23:36.276 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:36.276 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-09-17T15:23:36.276 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-fuse-2:19.3.0-4975.g1889a165.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-17T15:23:36.276 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:36.276 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-17T15:23:36.548 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-volume 2024-09-17T15:23:37.144 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:51 ago on Tue 17 Sep 2024 03:21:46 PM UTC. 2024-09-17T15:23:37.284 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-17T15:23:37.284 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:23:37.284 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repository Size 2024-09-17T15:23:37.284 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:23:37.284 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-09-17T15:23:37.284 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-volume noarch 2:19.3.0-4975.g1889a165.el9 ceph-noarch 285 k 2024-09-17T15:23:37.284 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:37.284 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-09-17T15:23:37.285 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:23:37.285 INFO:teuthology.orchestra.run.smithi083.stdout:Install 1 Package 2024-09-17T15:23:37.285 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:37.285 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 285 k 2024-09-17T15:23:37.285 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 1.3 M 2024-09-17T15:23:37.285 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-09-17T15:23:37.501 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-volume-19.3.0-4975.g1889a165.el9.noarch.rp 1.3 MB/s | 285 kB 00:00 2024-09-17T15:23:37.502 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-09-17T15:23:37.502 INFO:teuthology.orchestra.run.smithi083.stdout:Total 1.3 MB/s | 285 kB 00:00 2024-09-17T15:23:37.502 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-09-17T15:23:37.511 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-09-17T15:23:37.511 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-09-17T15:23:37.542 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-09-17T15:23:37.543 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-09-17T15:23:37.778 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-09-17T15:23:37.850 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-volume-2:19.3.0-4975.g1889a165.el9.noarch 1/1 2024-09-17T15:23:37.866 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-volume-2:19.3.0-4975.g1889a165.el9.noarch 1/1 2024-09-17T15:23:37.866 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T15:23:37.866 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-17T15:23:37.866 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:38.959 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-volume-2:19.3.0-4975.g1889a165.el9.noarch 1/1 2024-09-17T15:23:38.960 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:38.960 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-09-17T15:23:38.960 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-volume-2:19.3.0-4975.g1889a165.el9.noarch 2024-09-17T15:23:38.960 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:38.960 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-17T15:23:39.291 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install librados-devel 2024-09-17T15:23:39.883 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:53 ago on Tue 17 Sep 2024 03:21:46 PM UTC. 2024-09-17T15:23:40.017 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-17T15:23:40.018 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:23:40.018 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repo Size 2024-09-17T15:23:40.018 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:23:40.019 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-09-17T15:23:40.019 INFO:teuthology.orchestra.run.smithi083.stdout: librados-devel x86_64 2:19.3.0-4975.g1889a165.el9 ceph 126 k 2024-09-17T15:23:40.019 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:40.019 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-09-17T15:23:40.019 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:23:40.019 INFO:teuthology.orchestra.run.smithi083.stdout:Install 1 Package 2024-09-17T15:23:40.019 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:40.019 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 126 k 2024-09-17T15:23:40.019 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 456 k 2024-09-17T15:23:40.019 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-09-17T15:23:40.229 INFO:teuthology.orchestra.run.smithi083.stdout:librados-devel-19.3.0-4975.g1889a165.el9.x86_64 604 kB/s | 126 kB 00:00 2024-09-17T15:23:40.230 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-09-17T15:23:40.230 INFO:teuthology.orchestra.run.smithi083.stdout:Total 600 kB/s | 126 kB 00:00 2024-09-17T15:23:40.230 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-09-17T15:23:40.235 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-09-17T15:23:40.235 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-09-17T15:23:40.298 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-09-17T15:23:40.299 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-09-17T15:23:40.458 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-09-17T15:23:40.552 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librados-devel-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T15:23:41.046 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librados-devel-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T15:23:41.299 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados-devel-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T15:23:41.299 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:41.299 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-09-17T15:23:41.299 INFO:teuthology.orchestra.run.smithi083.stdout: librados-devel-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T15:23:41.299 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:41.299 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-17T15:23:41.521 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install libcephfs2 2024-09-17T15:23:42.095 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:56 ago on Tue 17 Sep 2024 03:21:46 PM UTC. 2024-09-17T15:23:42.185 INFO:teuthology.orchestra.run.smithi083.stdout:Package libcephfs2-2:19.3.0-4975.g1889a165.el9.x86_64 is already installed. 2024-09-17T15:23:42.229 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-17T15:23:42.230 INFO:teuthology.orchestra.run.smithi083.stdout:Nothing to do. 2024-09-17T15:23:42.230 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-17T15:23:42.297 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install libcephfs-devel 2024-09-17T15:23:42.876 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:56 ago on Tue 17 Sep 2024 03:21:46 PM UTC. 2024-09-17T15:23:43.008 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-17T15:23:43.008 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:23:43.009 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repo Size 2024-09-17T15:23:43.009 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:23:43.009 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-09-17T15:23:43.009 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs-devel x86_64 2:19.3.0-4975.g1889a165.el9 ceph 32 k 2024-09-17T15:23:43.009 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:43.009 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-09-17T15:23:43.009 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:23:43.009 INFO:teuthology.orchestra.run.smithi083.stdout:Install 1 Package 2024-09-17T15:23:43.009 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:43.009 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 32 k 2024-09-17T15:23:43.009 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 142 k 2024-09-17T15:23:43.010 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-09-17T15:23:43.149 INFO:teuthology.orchestra.run.smithi083.stdout:libcephfs-devel-19.3.0-4975.g1889a165.el9.x86_6 226 kB/s | 32 kB 00:00 2024-09-17T15:23:43.150 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-09-17T15:23:43.150 INFO:teuthology.orchestra.run.smithi083.stdout:Total 224 kB/s | 32 kB 00:00 2024-09-17T15:23:43.150 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-09-17T15:23:43.154 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-09-17T15:23:43.154 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-09-17T15:23:43.166 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-09-17T15:23:43.166 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-09-17T15:23:43.246 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-09-17T15:23:43.347 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephfs-devel-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T15:23:43.694 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T15:23:43.965 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephfs-devel-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T15:23:43.965 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:43.965 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-09-17T15:23:43.965 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs-devel-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T15:23:43.965 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:43.965 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-17T15:23:44.157 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install librados2 2024-09-17T15:23:44.736 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:58 ago on Tue 17 Sep 2024 03:21:46 PM UTC. 2024-09-17T15:23:44.825 INFO:teuthology.orchestra.run.smithi083.stdout:Package librados2-2:19.3.0-4975.g1889a165.el9.x86_64 is already installed. 2024-09-17T15:23:44.869 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-17T15:23:44.870 INFO:teuthology.orchestra.run.smithi083.stdout:Nothing to do. 2024-09-17T15:23:44.870 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-17T15:23:44.934 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install librbd1 2024-09-17T15:23:45.507 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:01:59 ago on Tue 17 Sep 2024 03:21:46 PM UTC. 2024-09-17T15:23:45.597 INFO:teuthology.orchestra.run.smithi083.stdout:Package librbd1-2:19.3.0-4975.g1889a165.el9.x86_64 is already installed. 2024-09-17T15:23:45.639 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-17T15:23:45.640 INFO:teuthology.orchestra.run.smithi083.stdout:Nothing to do. 2024-09-17T15:23:45.640 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-17T15:23:45.700 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install python3-rados 2024-09-17T15:23:46.274 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:02:00 ago on Tue 17 Sep 2024 03:21:46 PM UTC. 2024-09-17T15:23:46.362 INFO:teuthology.orchestra.run.smithi083.stdout:Package python3-rados-2:19.3.0-4975.g1889a165.el9.x86_64 is already installed. 2024-09-17T15:23:46.405 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-17T15:23:46.406 INFO:teuthology.orchestra.run.smithi083.stdout:Nothing to do. 2024-09-17T15:23:46.406 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-17T15:23:46.467 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install python3-rgw 2024-09-17T15:23:47.044 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:02:01 ago on Tue 17 Sep 2024 03:21:46 PM UTC. 2024-09-17T15:23:47.134 INFO:teuthology.orchestra.run.smithi083.stdout:Package python3-rgw-2:19.3.0-4975.g1889a165.el9.x86_64 is already installed. 2024-09-17T15:23:47.178 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-17T15:23:47.179 INFO:teuthology.orchestra.run.smithi083.stdout:Nothing to do. 2024-09-17T15:23:47.179 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-17T15:23:47.242 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install python3-cephfs 2024-09-17T15:23:47.816 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:02:01 ago on Tue 17 Sep 2024 03:21:46 PM UTC. 2024-09-17T15:23:47.905 INFO:teuthology.orchestra.run.smithi083.stdout:Package python3-cephfs-2:19.3.0-4975.g1889a165.el9.x86_64 is already installed. 2024-09-17T15:23:47.949 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-17T15:23:47.950 INFO:teuthology.orchestra.run.smithi083.stdout:Nothing to do. 2024-09-17T15:23:47.950 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-17T15:23:48.009 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install python3-rbd 2024-09-17T15:23:48.592 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:02:02 ago on Tue 17 Sep 2024 03:21:46 PM UTC. 2024-09-17T15:23:48.682 INFO:teuthology.orchestra.run.smithi083.stdout:Package python3-rbd-2:19.3.0-4975.g1889a165.el9.x86_64 is already installed. 2024-09-17T15:23:48.725 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-17T15:23:48.726 INFO:teuthology.orchestra.run.smithi083.stdout:Nothing to do. 2024-09-17T15:23:48.726 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-17T15:23:48.792 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install rbd-fuse 2024-09-17T15:23:49.365 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:02:03 ago on Tue 17 Sep 2024 03:21:46 PM UTC. 2024-09-17T15:23:49.497 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-17T15:23:49.498 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:23:49.498 INFO:teuthology.orchestra.run.smithi083.stdout: Package Architecture Version Repository Size 2024-09-17T15:23:49.498 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:23:49.498 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-09-17T15:23:49.498 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-fuse x86_64 2:19.3.0-4975.g1889a165.el9 ceph 85 k 2024-09-17T15:23:49.498 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:49.498 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-09-17T15:23:49.499 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:23:49.499 INFO:teuthology.orchestra.run.smithi083.stdout:Install 1 Package 2024-09-17T15:23:49.499 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:49.499 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 85 k 2024-09-17T15:23:49.499 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 231 k 2024-09-17T15:23:49.499 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-09-17T15:23:49.695 INFO:teuthology.orchestra.run.smithi083.stdout:rbd-fuse-19.3.0-4975.g1889a165.el9.x86_64.rpm 435 kB/s | 85 kB 00:00 2024-09-17T15:23:49.695 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-09-17T15:23:49.696 INFO:teuthology.orchestra.run.smithi083.stdout:Total 432 kB/s | 85 kB 00:00 2024-09-17T15:23:49.696 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-09-17T15:23:49.702 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-09-17T15:23:49.702 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-09-17T15:23:49.765 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-09-17T15:23:49.765 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-09-17T15:23:49.907 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-09-17T15:23:50.016 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-fuse-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T15:23:50.394 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: rbd-fuse-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T15:23:50.657 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-fuse-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T15:23:50.657 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:50.657 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-09-17T15:23:50.657 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-fuse-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T15:23:50.657 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:50.657 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-17T15:23:50.919 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install rbd-mirror 2024-09-17T15:23:51.507 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:02:05 ago on Tue 17 Sep 2024 03:21:46 PM UTC. 2024-09-17T15:23:51.642 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-17T15:23:51.642 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:23:51.643 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repo Size 2024-09-17T15:23:51.643 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:23:51.643 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-09-17T15:23:51.643 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-mirror x86_64 2:19.3.0-4975.g1889a165.el9 ceph 3.2 M 2024-09-17T15:23:51.643 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:51.643 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-09-17T15:23:51.643 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:23:51.643 INFO:teuthology.orchestra.run.smithi083.stdout:Install 1 Package 2024-09-17T15:23:51.643 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:51.644 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 3.2 M 2024-09-17T15:23:51.644 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 14 M 2024-09-17T15:23:51.644 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-09-17T15:23:52.607 INFO:teuthology.orchestra.run.smithi083.stdout:rbd-mirror-19.3.0-4975.g1889a165.el9.x86_64.rpm 3.3 MB/s | 3.2 MB 00:00 2024-09-17T15:23:52.607 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-09-17T15:23:52.607 INFO:teuthology.orchestra.run.smithi083.stdout:Total 3.3 MB/s | 3.2 MB 00:00 2024-09-17T15:23:52.608 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-09-17T15:23:52.617 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-09-17T15:23:52.617 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-09-17T15:23:52.700 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-09-17T15:23:52.700 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-09-17T15:23:53.145 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-09-17T15:23:53.199 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-mirror-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T15:23:53.223 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T15:23:53.223 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T15:23:53.223 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-17T15:23:53.223 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T15:23:53.224 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T15:23:53.224 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:54.273 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-mirror-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T15:23:54.273 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:54.273 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-09-17T15:23:54.273 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-mirror-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T15:23:54.273 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:54.273 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-17T15:23:54.501 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install rbd-nbd 2024-09-17T15:23:55.101 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:02:09 ago on Tue 17 Sep 2024 03:21:46 PM UTC. 2024-09-17T15:23:55.237 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-09-17T15:23:55.237 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:23:55.237 INFO:teuthology.orchestra.run.smithi083.stdout: Package Architecture Version Repository Size 2024-09-17T15:23:55.237 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:23:55.237 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-09-17T15:23:55.237 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-nbd x86_64 2:19.3.0-4975.g1889a165.el9 ceph 170 k 2024-09-17T15:23:55.238 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:55.238 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-09-17T15:23:55.238 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-09-17T15:23:55.238 INFO:teuthology.orchestra.run.smithi083.stdout:Install 1 Package 2024-09-17T15:23:55.238 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:55.238 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 170 k 2024-09-17T15:23:55.238 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 490 k 2024-09-17T15:23:55.238 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-09-17T15:23:55.469 INFO:teuthology.orchestra.run.smithi083.stdout:rbd-nbd-19.3.0-4975.g1889a165.el9.x86_64.rpm 738 kB/s | 170 kB 00:00 2024-09-17T15:23:55.470 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-09-17T15:23:55.470 INFO:teuthology.orchestra.run.smithi083.stdout:Total 734 kB/s | 170 kB 00:00 2024-09-17T15:23:55.470 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-09-17T15:23:55.477 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-09-17T15:23:55.477 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-09-17T15:23:55.542 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-09-17T15:23:55.543 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-09-17T15:23:55.699 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-09-17T15:23:55.808 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-nbd-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T15:23:56.264 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: rbd-nbd-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T15:23:56.533 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-nbd-2:19.3.0-4975.g1889a165.el9.x86_64 1/1 2024-09-17T15:23:56.533 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:56.533 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-09-17T15:23:56.533 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-nbd-2:19.3.0-4975.g1889a165.el9.x86_64 2024-09-17T15:23:56.533 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:56.533 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-09-17T15:23:56.760 DEBUG:teuthology.parallel:result is None 2024-09-17T15:23:56.761 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1889a165b5128787ec693450a3d1c5df5d6dc203 2024-09-17T15:23:56.911 DEBUG:teuthology.orchestra.run.smithi083:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-17T15:23:56.933 INFO:teuthology.orchestra.run.smithi083.stdout:19.3.0-4975.g1889a165.el9 2024-09-17T15:23:56.934 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4975.g1889a165.el9 2024-09-17T15:23:56.934 INFO:teuthology.task.install:The correct ceph version 19.3.0-4975.g1889a165 is installed. 2024-09-17T15:23:56.935 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-17T15:23:56.936 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-17T15:23:56.936 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-17T15:23:57.000 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-17T15:23:57.000 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-17T15:23:57.000 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/daemon-helper 2024-09-17T15:23:57.068 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-17T15:23:57.132 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-17T15:23:57.132 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-17T15:23:57.132 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-17T15:23:57.195 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-17T15:23:57.260 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-17T15:23:57.260 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-17T15:23:57.260 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/stdin-killer 2024-09-17T15:23:57.325 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-17T15:23:57.388 INFO:teuthology.run_tasks:Running task ceph... 2024-09-17T15:23:57.471 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-17T15:23:57.471 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 777 /var/log/ceph 2024-09-17T15:23:57.495 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-17T15:23:57.496 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-17T15:23:57.566 INFO:tasks.ceph:Creating extra log directories... 2024-09-17T15:23:57.566 DEBUG:teuthology.orchestra.run.smithi083:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-17T15:23:57.639 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-17T15:23:57.639 INFO:tasks.ceph:config {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore write v2 random': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(OBJECT_', '\\(SLOW_OPS\\)', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', '\\(MON_DOWN\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-17T15:23:57.639 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7908934', 'branch': 'wip-bharath10-testing-2024-09-17-1348', 'description': 'rados/singleton/{all/lost-unfound mon_election/classic msgr-failures/few msgr/async-v2only objectstore/{bluestore-options/write$/{write_random} bluestore/bluestore-comp-lz4} rados supported-random-distro$/{centos_latest}}', 'email': 'skanta@redhat.com', 'first_in_suite': False, 'job_id': '7908934', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 3, 'size': 10}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-bharath10-testing-2024-09-17-1348'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore write v2 random': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', '\\(MON_DOWN\\)'], 'sha1': '1889a165b5128787ec693450a3d1c5df5d6dc203'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '1889a165b5128787ec693450a3d1c5df5d6dc203'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-bharath10-testing-2024-09-17-1348', 'sha1': '1889a165b5128787ec693450a3d1c5df5d6dc203'}}, 'owner': 'scheduled_skanta@teuthology', 'priority': 99, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'osd.0', 'osd.1', 'osd.2']], 'seed': 3421, 'sha1': '1889a165b5128787ec693450a3d1c5df5d6dc203', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rados', 'suite_branch': 'wip-bharath10-testing-2024-09-17-1348', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_1889a165b5128787ec693450a3d1c5df5d6dc203/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '1889a165b5128787ec693450a3d1c5df5d6dc203', 'targets': {'smithi083.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAvbNjtzbmYL9r1bXtRmJfHWFDSNDwRa0d/0piRZ10y21rbvZIgQKJE+Rg8GdhJSjIywb8DAL+msbS9z+N5ECF8='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'log-ignorelist': ['objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(OBJECT_', '\\(SLOW_OPS\\)', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', '\\(MON_DOWN\\)'], 'pre-mgr-commands': ['sudo ceph config set mgr mgr_pool false --force'], 'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore write v2 random': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'fs': 'xfs', 'sha1': '1889a165b5128787ec693450a3d1c5df5d6dc203', 'cluster': 'ceph'}}, {'lost_unfound': None}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-17_15:01:33', 'tube': 'smithi', 'user': 'skanta', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-17T15:23:57.639 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-17T15:23:57.687 DEBUG:teuthology.orchestra.run.smithi083:> sudo install -d -m0777 -- /var/run/ceph 2024-09-17T15:23:57.751 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-17T15:23:57.751 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T15:23:57.806 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T15:23:57.807 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_1 2024-09-17T15:23:57.862 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T15:23:57.862 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T15:23:57.863 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-17T15:23:57.863 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T15:23:57.863 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T15:23:57.863 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-09-17 15:23:53.475434751 +0000 2024-09-17T15:23:57.863 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-09-17 15:21:00.910498224 +0000 2024-09-17T15:23:57.863 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-09-17 15:21:00.910498224 +0000 2024-09-17T15:23:57.863 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-09-17 15:21:00.910498224 +0000 2024-09-17T15:23:57.863 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T15:23:57.926 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-09-17T15:23:57.926 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-09-17T15:23:57.926 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000204795 s, 2.5 MB/s 2024-09-17T15:23:57.927 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T15:23:57.984 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_2 2024-09-17T15:23:58.038 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T15:23:58.039 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T15:23:58.039 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-17T15:23:58.039 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T15:23:58.039 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T15:23:58.039 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-09-17 15:23:53.476434726 +0000 2024-09-17T15:23:58.039 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-09-17 15:21:01.259489896 +0000 2024-09-17T15:23:58.039 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-09-17 15:21:01.259489896 +0000 2024-09-17T15:23:58.039 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-09-17 15:21:01.259489896 +0000 2024-09-17T15:23:58.039 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T15:23:58.101 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-09-17T15:23:58.102 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-09-17T15:23:58.102 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000206503 s, 2.5 MB/s 2024-09-17T15:23:58.103 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T15:23:58.159 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_3 2024-09-17T15:23:58.215 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T15:23:58.215 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T15:23:58.215 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-17T15:23:58.215 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T15:23:58.215 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T15:23:58.215 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-09-17 15:23:53.476434726 +0000 2024-09-17T15:23:58.216 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-09-17 15:21:01.597481830 +0000 2024-09-17T15:23:58.216 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-09-17 15:21:01.597481830 +0000 2024-09-17T15:23:58.216 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-09-17 15:21:01.597481830 +0000 2024-09-17T15:23:58.216 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T15:23:58.278 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-09-17T15:23:58.278 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-09-17T15:23:58.278 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.00020364 s, 2.5 MB/s 2024-09-17T15:23:58.279 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T15:23:58.335 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_4 2024-09-17T15:23:58.391 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T15:23:58.391 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T15:23:58.391 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-17T15:23:58.391 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T15:23:58.391 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T15:23:58.391 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-09-17 15:23:53.476434726 +0000 2024-09-17T15:23:58.391 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-09-17 15:21:01.922474075 +0000 2024-09-17T15:23:58.391 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-09-17 15:21:01.922474075 +0000 2024-09-17T15:23:58.391 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-09-17 15:21:01.922474075 +0000 2024-09-17T15:23:58.392 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T15:23:58.453 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-09-17T15:23:58.453 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-09-17T15:23:58.453 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000224236 s, 2.3 MB/s 2024-09-17T15:23:58.455 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T15:23:58.510 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-17T15:23:58.510 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi083.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2024-09-17T15:23:58.511 INFO:tasks.ceph:Generating config... 2024-09-17T15:23:58.512 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-17T15:23:58.512 INFO:tasks.ceph:[global] mon election default strategy = 1 2024-09-17T15:23:58.512 INFO:tasks.ceph:[global] ms bind msgr1 = False 2024-09-17T15:23:58.512 INFO:tasks.ceph:[global] ms bind msgr2 = True 2024-09-17T15:23:58.512 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-17T15:23:58.512 INFO:tasks.ceph:[global] ms type = async 2024-09-17T15:23:58.512 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-17T15:23:58.513 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-17T15:23:58.513 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-17T15:23:58.513 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-17T15:23:58.513 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-17T15:23:58.513 INFO:tasks.ceph:[mon] mon scrub interval = 300 2024-09-17T15:23:58.513 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-17T15:23:58.513 INFO:tasks.ceph:[osd] bluestore compression algorithm = lz4 2024-09-17T15:23:58.513 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-09-17T15:23:58.513 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-17T15:23:58.513 INFO:tasks.ceph:[osd] bluestore write v2 random = True 2024-09-17T15:23:58.513 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2024-09-17T15:23:58.513 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-17T15:23:58.514 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-17T15:23:58.514 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-17T15:23:58.514 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-17T15:23:58.514 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-17T15:23:58.514 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-17T15:23:58.514 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-17T15:23:58.514 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-17T15:23:58.514 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2024-09-17T15:23:58.514 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2024-09-17T15:23:58.514 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-17T15:23:58.514 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2024-09-17T15:23:58.515 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2024-09-17T15:23:58.515 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-17T15:23:58.515 INFO:tasks.ceph:[osd] osd op queue = debug_random 2024-09-17T15:23:58.515 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2024-09-17T15:23:58.515 INFO:tasks.ceph:Setting up mon.a... 2024-09-17T15:23:58.515 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-17T15:23:58.597 INFO:teuthology.orchestra.run.smithi083.stdout:creating /etc/ceph/ceph.keyring 2024-09-17T15:23:58.600 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-17T15:23:58.641 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T15:23:58.706 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.83'), ('mon.b', '[v2:172.21.15.83:3301,v1:172.21.15.83:6790]'), ('mon.c', '[v2:172.21.15.83:3302,v1:172.21.15.83:6791]')] 2024-09-17T15:23:58.706 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.83,[v2:172.21.15.83:3301,v1:172.21.15.83:6790],[v2:172.21.15.83:3302,v1:172.21.15.83:6791]', 'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore write v2 random': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon scrub interval': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}} 2024-09-17T15:23:58.707 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-17T15:23:58.707 DEBUG:teuthology.orchestra.run.smithi083:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-17T15:23:58.761 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.83 --addv b '[v2:172.21.15.83:3301,v1:172.21.15.83:6790]' --addv c '[v2:172.21.15.83:3302,v1:172.21.15.83:6791]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-17T15:23:58.847 INFO:teuthology.orchestra.run.smithi083.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-17T15:23:58.848 INFO:teuthology.orchestra.run.smithi083.stdout:monmaptool: generated fsid 5d1df884-ae2f-44a8-abc8-b077329edada 2024-09-17T15:23:58.848 INFO:teuthology.orchestra.run.smithi083.stdout:setting min_mon_release = quincy 2024-09-17T15:23:58.848 INFO:teuthology.orchestra.run.smithi083.stdout:epoch 0 2024-09-17T15:23:58.848 INFO:teuthology.orchestra.run.smithi083.stdout:fsid 5d1df884-ae2f-44a8-abc8-b077329edada 2024-09-17T15:23:58.848 INFO:teuthology.orchestra.run.smithi083.stdout:last_changed 2024-09-17T15:23:58.848143+0000 2024-09-17T15:23:58.848 INFO:teuthology.orchestra.run.smithi083.stdout:created 2024-09-17T15:23:58.848143+0000 2024-09-17T15:23:58.848 INFO:teuthology.orchestra.run.smithi083.stdout:min_mon_release 17 (quincy) 2024-09-17T15:23:58.848 INFO:teuthology.orchestra.run.smithi083.stdout:election_strategy: 1 2024-09-17T15:23:58.848 INFO:teuthology.orchestra.run.smithi083.stdout:0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-09-17T15:23:58.848 INFO:teuthology.orchestra.run.smithi083.stdout:1: [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] mon.b 2024-09-17T15:23:58.848 INFO:teuthology.orchestra.run.smithi083.stdout:2: [v2:172.21.15.83:3302/0,v1:172.21.15.83:6791/0] mon.c 2024-09-17T15:23:58.849 INFO:teuthology.orchestra.run.smithi083.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-17T15:23:58.850 DEBUG:teuthology.orchestra.run.smithi083:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-17T15:23:58.864 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 5d1df884-ae2f-44a8-abc8-b077329edada... 2024-09-17T15:23:58.865 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T15:23:58.947 INFO:teuthology.orchestra.run.smithi083.stdout:[global] 2024-09-17T15:23:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: chdir = "" 2024-09-17T15:23:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T15:23:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: auth supported = cephx 2024-09-17T15:23:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: filestore xattr use omap = true 2024-09-17T15:23:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: mon clock drift allowed = 1.000 2024-09-17T15:23:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: osd crush chooseleaf type = 0 2024-09-17T15:23:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: auth debug = true 2024-09-17T15:23:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: ms die on old message = true 2024-09-17T15:23:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: ms die on bug = true 2024-09-17T15:23:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T15:23:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: mon pg warn max object skew = 0 2024-09-17T15:23:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T15:23:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T15:23:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: osd pool default size = 2 2024-09-17T15:23:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd allow primary affinity = true 2024-09-17T15:23:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd allow pg remap = true 2024-09-17T15:23:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on legacy crush tunables = false 2024-09-17T15:23:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on crush straw calc version zero = false 2024-09-17T15:23:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on no sortbitwise = false 2024-09-17T15:23:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on osd down out interval zero = false 2024-09-17T15:23:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on too few osds = false 2024-09-17T15:23:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T15:23:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T15:23:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: mon_allow_pool_size_one = true 2024-09-17T15:23:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T15:23:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: osd default data pool replay window = 5 2024-09-17T15:23:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: mon allow pool delete = true 2024-09-17T15:23:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: mon cluster log file level = debug 2024-09-17T15:23:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: debug asserts on shutdown = true 2024-09-17T15:23:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: mon health detail to clog = false 2024-09-17T15:23:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: mon host = "172.21.15.83,[v2:172.21.15.83:3301,v1:172.21.15.83:6790],[v2:172.21.15.83:3302,v1:172.21.15.83:6791]" 2024-09-17T15:23:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: mon client directed command retry = 5 2024-09-17T15:23:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: mon election default strategy = 1 2024-09-17T15:23:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: ms bind msgr1 = False 2024-09-17T15:23:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: ms bind msgr2 = True 2024-09-17T15:23:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: ms inject socket failures = 5000 2024-09-17T15:23:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: ms type = async 2024-09-17T15:23:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: fsid = 5d1df884-ae2f-44a8-abc8-b077329edada 2024-09-17T15:23:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:58.950 INFO:teuthology.orchestra.run.smithi083.stdout:[osd] 2024-09-17T15:23:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: osd journal size = 100 2024-09-17T15:23:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: osd scrub load threshold = 5.0 2024-09-17T15:23:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: osd scrub max interval = 600 2024-09-17T15:23:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: osd mclock profile = high_recovery_ops 2024-09-17T15:23:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: osd recover clone overlap = true 2024-09-17T15:23:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: osd recovery max chunk = 1048576 2024-09-17T15:23:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug shutdown = true 2024-09-17T15:23:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug op order = true 2024-09-17T15:23:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug verify stray on activate = true 2024-09-17T15:23:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug trim objects = true 2024-09-17T15:23:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: osd open classes on start = true 2024-09-17T15:23:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug pg log writeout = true 2024-09-17T15:23:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: osd deep scrub update digest min age = 30 2024-09-17T15:23:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: osd map max advance = 10 2024-09-17T15:23:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: journal zero on create = true 2024-09-17T15:23:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: filestore ondisk finisher threads = 3 2024-09-17T15:23:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: filestore apply finisher threads = 3 2024-09-17T15:23:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: bdev debug aio = true 2024-09-17T15:23:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug misdirected ops = true 2024-09-17T15:23:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: bluestore block size = 96636764160 2024-09-17T15:23:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: bluestore compression algorithm = lz4 2024-09-17T15:23:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: bluestore compression mode = aggressive 2024-09-17T15:23:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: bluestore fsck on mount = True 2024-09-17T15:23:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: bluestore write v2 random = True 2024-09-17T15:23:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: bluestore zero block detection = True 2024-09-17T15:23:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: debug bluefs = 20 2024-09-17T15:23:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: debug bluestore = 20 2024-09-17T15:23:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-09-17T15:23:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: debug osd = 20 2024-09-17T15:23:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: debug rocksdb = 10 2024-09-17T15:23:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T15:23:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd full ratio = 0.9 2024-09-17T15:23:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd nearfull ratio = 0.8 2024-09-17T15:23:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug verify cached snaps = True 2024-09-17T15:23:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug verify missing on start = True 2024-09-17T15:23:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: osd failsafe full ratio = 0.95 2024-09-17T15:23:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: osd mclock override recovery settings = True 2024-09-17T15:23:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: osd objectstore = bluestore 2024-09-17T15:23:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: osd op queue = debug_random 2024-09-17T15:23:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: osd op queue cut off = debug_random 2024-09-17T15:23:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:58.954 INFO:teuthology.orchestra.run.smithi083.stdout:[mgr] 2024-09-17T15:23:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-09-17T15:23:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: debug mgr = 20 2024-09-17T15:23:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: debug mon = 20 2024-09-17T15:23:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: debug auth = 20 2024-09-17T15:23:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min pgs per osd = 4 2024-09-17T15:23:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min bytes per osd = 10 2024-09-17T15:23:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: mgr/telemetry/nag = false 2024-09-17T15:23:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:58.955 INFO:teuthology.orchestra.run.smithi083.stdout:[mon] 2024-09-17T15:23:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-09-17T15:23:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: debug mon = 20 2024-09-17T15:23:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: debug paxos = 20 2024-09-17T15:23:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: debug auth = 20 2024-09-17T15:23:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: mon data avail warn = 5 2024-09-17T15:23:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: mon mgr mkfs grace = 240 2024-09-17T15:23:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min pgs per osd = 4 2024-09-17T15:23:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd reporter subtree level = osd 2024-09-17T15:23:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd prime pg temp = true 2024-09-17T15:23:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min bytes per osd = 10 2024-09-17T15:23:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T15:23:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T15:23:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T15:23:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T15:23:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T15:23:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T15:23:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: # 1m isn't quite enough 2024-09-17T15:23:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: mon_down_mkfs_grace = 2m 2024-09-17T15:23:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_filestore_osds = false 2024-09-17T15:23:58.957 INFO:teuthology.orchestra.run.smithi083.stdout: mon scrub interval = 300 2024-09-17T15:23:58.957 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:23:58.957 INFO:teuthology.orchestra.run.smithi083.stdout:[client] 2024-09-17T15:23:58.957 INFO:teuthology.orchestra.run.smithi083.stdout: rgw cache enabled = true 2024-09-17T15:23:58.957 INFO:teuthology.orchestra.run.smithi083.stdout: rgw enable ops log = true 2024-09-17T15:23:58.957 INFO:teuthology.orchestra.run.smithi083.stdout: rgw enable usage log = true 2024-09-17T15:23:58.957 INFO:teuthology.orchestra.run.smithi083.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T15:23:58.957 INFO:teuthology.orchestra.run.smithi083.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T15:23:58.957 INFO:teuthology.orchestra.run.smithi083.stdout:[mon.a] 2024-09-17T15:23:58.957 INFO:teuthology.orchestra.run.smithi083.stdout:[mon.b] 2024-09-17T15:23:58.957 INFO:teuthology.orchestra.run.smithi083.stdout:[mon.c] 2024-09-17T15:23:58.958 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-17T15:23:58.959 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-17T15:23:59.053 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-17T15:23:59.053 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-17T15:23:59.053 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-17T15:23:59.069 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-17T15:23:59.069 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-17T15:23:59.123 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi083.front.sepia.ceph.com 2024-09-17T15:23:59.124 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-17T15:23:59.124 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-17T15:23:59.124 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T15:23:59.198 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-17T15:23:59.198 DEBUG:teuthology.orchestra.run.smithi083:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-17T15:23:59.254 INFO:tasks.ceph:Setting up mon nodes... 2024-09-17T15:23:59.254 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-17T15:23:59.254 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-17T15:23:59.343 INFO:teuthology.orchestra.run.smithi083.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-17T15:23:59.346 INFO:tasks.ceph:Setting up mds nodes... 2024-09-17T15:23:59.346 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-17T15:23:59.346 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-17T15:23:59.346 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi083.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2024-09-17T15:23:59.346 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-17T15:23:59.370 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-17T15:23:59.370 INFO:tasks.ceph:role: osd.0 2024-09-17T15:23:59.371 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi083.front.sepia.ceph.com 2024-09-17T15:23:59.371 DEBUG:teuthology.orchestra.run.smithi083:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-17T15:23:59.442 INFO:teuthology.orchestra.run.smithi083.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T15:23:59.442 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T15:23:59.442 INFO:teuthology.orchestra.run.smithi083.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T15:23:59.442 INFO:teuthology.orchestra.run.smithi083.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T15:23:59.442 INFO:teuthology.orchestra.run.smithi083.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T15:23:59.442 INFO:teuthology.orchestra.run.smithi083.stdout: = sunit=0 swidth=0 blks 2024-09-17T15:23:59.442 INFO:teuthology.orchestra.run.smithi083.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T15:23:59.442 INFO:teuthology.orchestra.run.smithi083.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T15:23:59.442 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T15:23:59.443 INFO:teuthology.orchestra.run.smithi083.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T15:23:59.948 INFO:teuthology.orchestra.run.smithi083.stdout:Discarding blocks...Done. 2024-09-17T15:23:59.950 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi083.front.sepia.ceph.com -o noatime 2024-09-17T15:23:59.951 DEBUG:teuthology.orchestra.run.smithi083:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-17T15:24:00.059 DEBUG:teuthology.orchestra.run.smithi083:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-17T15:24:00.089 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-17T15:24:00.153 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-17T15:24:00.153 INFO:tasks.ceph:role: osd.1 2024-09-17T15:24:00.153 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi083.front.sepia.ceph.com 2024-09-17T15:24:00.154 DEBUG:teuthology.orchestra.run.smithi083:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-17T15:24:00.225 INFO:teuthology.orchestra.run.smithi083.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T15:24:00.226 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T15:24:00.226 INFO:teuthology.orchestra.run.smithi083.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T15:24:00.226 INFO:teuthology.orchestra.run.smithi083.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T15:24:00.226 INFO:teuthology.orchestra.run.smithi083.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T15:24:00.226 INFO:teuthology.orchestra.run.smithi083.stdout: = sunit=0 swidth=0 blks 2024-09-17T15:24:00.226 INFO:teuthology.orchestra.run.smithi083.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T15:24:00.226 INFO:teuthology.orchestra.run.smithi083.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T15:24:00.226 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T15:24:00.226 INFO:teuthology.orchestra.run.smithi083.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T15:24:00.696 INFO:teuthology.orchestra.run.smithi083.stdout:Discarding blocks...Done. 2024-09-17T15:24:00.698 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi083.front.sepia.ceph.com -o noatime 2024-09-17T15:24:00.698 DEBUG:teuthology.orchestra.run.smithi083:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-17T15:24:00.781 DEBUG:teuthology.orchestra.run.smithi083:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-17T15:24:00.850 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-17T15:24:00.914 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-17T15:24:00.914 INFO:tasks.ceph:role: osd.2 2024-09-17T15:24:00.914 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi083.front.sepia.ceph.com 2024-09-17T15:24:00.914 DEBUG:teuthology.orchestra.run.smithi083:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-17T15:24:00.986 INFO:teuthology.orchestra.run.smithi083.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T15:24:00.986 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T15:24:00.986 INFO:teuthology.orchestra.run.smithi083.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T15:24:00.986 INFO:teuthology.orchestra.run.smithi083.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T15:24:00.986 INFO:teuthology.orchestra.run.smithi083.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T15:24:00.986 INFO:teuthology.orchestra.run.smithi083.stdout: = sunit=0 swidth=0 blks 2024-09-17T15:24:00.986 INFO:teuthology.orchestra.run.smithi083.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T15:24:00.987 INFO:teuthology.orchestra.run.smithi083.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T15:24:00.987 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T15:24:00.987 INFO:teuthology.orchestra.run.smithi083.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T15:24:01.505 INFO:teuthology.orchestra.run.smithi083.stdout:Discarding blocks...Done. 2024-09-17T15:24:01.507 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi083.front.sepia.ceph.com -o noatime 2024-09-17T15:24:01.508 DEBUG:teuthology.orchestra.run.smithi083:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-17T15:24:01.627 DEBUG:teuthology.orchestra.run.smithi083:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-17T15:24:01.659 DEBUG:teuthology.orchestra.run.smithi083:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T15:24:01.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-17T15:24:01.756+0000 7f6855015640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-17T15:24:01.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-17T15:24:01.759+0000 7f6855015640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-17T15:24:01.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-17T15:24:01.759+0000 7f6855015640 -1 bdev(0x5592ec3a5800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-17T15:24:01.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-17T15:24:01.759+0000 7f6855015640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-17T15:24:04.919 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-17T15:24:04.946 DEBUG:teuthology.orchestra.run.smithi083:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T15:24:05.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-17T15:24:05.032+0000 7f215ca44640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-17T15:24:05.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-17T15:24:05.033+0000 7f215ca44640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-17T15:24:05.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-17T15:24:05.033+0000 7f215ca44640 -1 bdev(0x5561ccc01800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-17T15:24:05.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-17T15:24:05.033+0000 7f215ca44640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-17T15:24:08.191 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-17T15:24:08.216 DEBUG:teuthology.orchestra.run.smithi083:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T15:24:08.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-17T15:24:08.302+0000 7fe758c1a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-17T15:24:08.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-17T15:24:08.302+0000 7fe758c1a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-17T15:24:08.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-17T15:24:08.302+0000 7fe758c1a640 -1 bdev(0x55c441471800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-17T15:24:08.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-17T15:24:08.302+0000 7fe758c1a640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-17T15:24:11.466 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-17T15:24:11.496 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-17T15:24:11.496 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-17T15:24:11.496 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-17T15:24:11.564 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-17T15:24:11.564 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-17T15:24:11.631 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-17T15:24:11.631 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-17T15:24:11.697 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-17T15:24:11.698 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-17T15:24:11.763 INFO:tasks.ceph:Adding keys to all mons... 2024-09-17T15:24:11.763 DEBUG:teuthology.orchestra.run.smithi083:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-17T15:24:11.827 INFO:teuthology.orchestra.run.smithi083.stdout:[mgr.x] 2024-09-17T15:24:11.827 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQAPn+lmd+SDFBAALiYylntyrqPQ1X9cVI6odQ== 2024-09-17T15:24:11.827 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.0] 2024-09-17T15:24:11.828 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQARn+lm0/ApLRAA7lShbgxdwP+3kJNuwxFL5w== 2024-09-17T15:24:11.828 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.1] 2024-09-17T15:24:11.828 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQAVn+lm6NwDAhAA0PaJC3xKL2BXnjA0Wkvd+g== 2024-09-17T15:24:11.828 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.2] 2024-09-17T15:24:11.828 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQAYn+lmoK4UEhAAahVtWk3hOXvoi2/rTPeXww== 2024-09-17T15:24:11.829 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T15:24:11.919 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T15:24:11.970 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T15:24:12.021 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T15:24:12.072 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-17T15:24:12.072 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-17T15:24:12.097 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T15:24:12.192 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-17T15:24:12.216 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-17T15:24:12.281 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T15:24:12.377 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-17T15:24:12.401 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-17T15:24:12.466 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T15:24:12.561 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-17T15:24:12.586 DEBUG:teuthology.orchestra.run.smithi083:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-17T15:24:12.641 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-17T15:24:12.642 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-17T15:24:12.642 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-17T15:24:12.684 INFO:tasks.ceph.mon.a:Started 2024-09-17T15:24:12.684 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-17T15:24:12.684 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-17T15:24:12.686 INFO:tasks.ceph.mon.b:Started 2024-09-17T15:24:12.686 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-17T15:24:12.686 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-17T15:24:12.688 INFO:tasks.ceph.mon.c:Started 2024-09-17T15:24:12.688 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph config set mgr mgr_pool false --force 2024-09-17T15:24:23.284 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-17T15:24:23.285 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-17T15:24:23.285 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-17T15:24:23.287 INFO:tasks.ceph.mgr.x:Started 2024-09-17T15:24:23.287 DEBUG:tasks.ceph:set 0 configs 2024-09-17T15:24:23.288 INFO:tasks.ceph:Setting crush tunables to default 2024-09-17T15:24:23.288 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --cluster ceph osd crush tunables default 2024-09-17T15:24:23.465 INFO:teuthology.orchestra.run.smithi083.stderr:adjusted tunables profile to default 2024-09-17T15:24:23.476 INFO:tasks.ceph:check_enable_crimson: False 2024-09-17T15:24:23.476 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-17T15:24:23.477 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-17T15:24:23.477 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-17T15:24:23.501 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-17T15:24:23.501 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-17T15:24:23.566 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-09-17T15:24:23.566 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-17T15:24:23.630 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --cluster ceph osd new 9defe403-224a-4a6c-9184-dcb72bef6bf2 0 2024-09-17T15:24:23.852 INFO:teuthology.orchestra.run.smithi083.stdout:0 2024-09-17T15:24:23.863 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --cluster ceph osd new d7d78e01-f761-444b-98c4-62587b991738 1 2024-09-17T15:24:24.074 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-09-17T15:24:24.085 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --cluster ceph osd new 23c2275c-8731-45f3-97f9-5100b6676f99 2 2024-09-17T15:24:24.262 INFO:teuthology.orchestra.run.smithi083.stdout:2 2024-09-17T15:24:24.273 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-17T15:24:24.274 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-17T15:24:24.275 INFO:tasks.ceph.osd.0:Started 2024-09-17T15:24:24.276 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-17T15:24:24.276 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-17T15:24:24.278 INFO:tasks.ceph.osd.1:Started 2024-09-17T15:24:24.278 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-17T15:24:24.278 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-17T15:24:24.280 INFO:tasks.ceph.osd.2:Started 2024-09-17T15:24:24.281 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T15:24:24.454 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:24:24.455 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":5,"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","created":"2024-09-17T15:24:23.007214+0000","modified":"2024-09-17T15:24:24.259028+0000","last_up_change":"0.000000","last_in_change":"2024-09-17T15:24:24.259028+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"9defe403-224a-4a6c-9184-dcb72bef6bf2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d7d78e01-f761-444b-98c4-62587b991738","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"23c2275c-8731-45f3-97f9-5100b6676f99","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T15:24:24.466 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-17T15:24:24.466 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-17T15:24:24.767 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T15:24:24.925 INFO:teuthology.misc.health.smithi083.stdout: 2024-09-17T15:24:24.925 INFO:teuthology.misc.health.smithi083.stdout:{"epoch":5,"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","created":"2024-09-17T15:24:23.007214+0000","modified":"2024-09-17T15:24:24.259028+0000","last_up_change":"0.000000","last_in_change":"2024-09-17T15:24:24.259028+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"9defe403-224a-4a6c-9184-dcb72bef6bf2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d7d78e01-f761-444b-98c4-62587b991738","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"23c2275c-8731-45f3-97f9-5100b6676f99","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T15:24:24.935 DEBUG:teuthology.misc:0 of 3 OSDs are up 2024-09-17T15:24:25.748 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T15:24:25.747+0000 7f508660d640 -1 Falling back to public interface 2024-09-17T15:24:25.761 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T15:24:25.760+0000 7f2be0c14640 -1 Falling back to public interface 2024-09-17T15:24:26.252 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:24:26.251+0000 7fe091558640 -1 Falling back to public interface 2024-09-17T15:24:28.576 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-09-17T15:24:28.575+0000 7efec5f1b640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-17T15:24:28.576 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-09-17T15:24:28.575+0000 7efec5f1b640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-17T15:24:29.844 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T15:24:29.843+0000 7f2be0c14640 -1 osd.2 0 log_to_monitors true 2024-09-17T15:24:29.846 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T15:24:29.845+0000 7f508660d640 -1 osd.1 0 log_to_monitors true 2024-09-17T15:24:30.530 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:24:30.529+0000 7fe091558640 -1 osd.0 0 log_to_monitors true 2024-09-17T15:24:31.237 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T15:24:31.523 INFO:teuthology.misc.health.smithi083.stdout: 2024-09-17T15:24:31.523 INFO:teuthology.misc.health.smithi083.stdout:{"epoch":6,"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","created":"2024-09-17T15:24:23.007214+0000","modified":"2024-09-17T15:24:30.576839+0000","last_up_change":"0.000000","last_in_change":"2024-09-17T15:24:24.259028+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"9defe403-224a-4a6c-9184-dcb72bef6bf2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d7d78e01-f761-444b-98c4-62587b991738","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"23c2275c-8731-45f3-97f9-5100b6676f99","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T15:24:31.534 DEBUG:teuthology.misc:0 of 3 OSDs are up 2024-09-17T15:24:31.585 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T15:24:31.585+0000 7f2bdc59a640 -1 osd.2 0 waiting for initial osdmap 2024-09-17T15:24:31.589 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:24:31.588+0000 7fe08d4e7640 -1 osd.0 0 waiting for initial osdmap 2024-09-17T15:24:31.591 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T15:24:31.590+0000 7f2bd83c1640 -1 osd.2 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T15:24:31.591 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:24:31.591+0000 7fe08930e640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T15:24:32.139 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T15:24:32.138+0000 7f508201c640 -1 osd.1 0 waiting for initial osdmap 2024-09-17T15:24:32.142 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T15:24:32.141+0000 7f5079e3b640 -1 osd.1 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T15:24:37.837 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T15:24:38.124 INFO:teuthology.misc.health.smithi083.stdout: 2024-09-17T15:24:38.124 INFO:teuthology.misc.health.smithi083.stdout:{"epoch":9,"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","created":"2024-09-17T15:24:23.007214+0000","modified":"2024-09-17T15:24:33.591215+0000","last_up_change":"2024-09-17T15:24:32.585667+0000","last_in_change":"2024-09-17T15:24:24.259028+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"9defe403-224a-4a6c-9184-dcb72bef6bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":335704639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":335704639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6811","nonce":335704639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":335704639}]},"public_addr":"172.21.15.83:6808/335704639","cluster_addr":"172.21.15.83:6809/335704639","heartbeat_back_addr":"172.21.15.83:6811/335704639","heartbeat_front_addr":"172.21.15.83:6810/335704639","state":["exists","up"]},{"osd":1,"uuid":"d7d78e01-f761-444b-98c4-62587b991738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":1258824819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6801","nonce":1258824819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6803","nonce":1258824819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":1258824819}]},"public_addr":"172.21.15.83:6800/1258824819","cluster_addr":"172.21.15.83:6801/1258824819","heartbeat_back_addr":"172.21.15.83:6803/1258824819","heartbeat_front_addr":"172.21.15.83:6802/1258824819","state":["exists","up"]},{"osd":2,"uuid":"23c2275c-8731-45f3-97f9-5100b6676f99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":239948269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6805","nonce":239948269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6807","nonce":239948269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":239948269}]},"public_addr":"172.21.15.83:6804/239948269","cluster_addr":"172.21.15.83:6805/239948269","heartbeat_back_addr":"172.21.15.83:6807/239948269","heartbeat_front_addr":"172.21.15.83:6806/239948269","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T15:24:31.547535+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T15:24:30.867578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T15:24:30.866429+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T15:24:38.135 DEBUG:teuthology.misc:3 of 3 OSDs are up 2024-09-17T15:24:38.135 INFO:tasks.ceph:Creating RBD pool 2024-09-17T15:24:38.136 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-17T15:24:38.609 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'rbd' created 2024-09-17T15:24:38.623 DEBUG:teuthology.orchestra.run.smithi083:> rbd --cluster ceph pool init rbd 2024-09-17T15:24:41.639 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-17T15:24:41.640 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-17T15:24:41.640 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-17T15:24:41.928 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:24:41.940 INFO:teuthology.orchestra.run.smithi083.stdout:[{"version":3,"timestamp":"2024-09-17T15:24:32.139917+0000","name":"","changes":[{"name":"osd.1/osd_mclock_max_capacity_iops_ssd","new_value":"9706.202064"}]}] 2024-09-17T15:24:41.941 INFO:tasks.ceph_manager:config epoch is 3 2024-09-17T15:24:41.941 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-17T15:24:41.941 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-17T15:24:41.941 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-17T15:24:42.272 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:24:42.283 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":4,"flags":0,"active_gid":4106,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":2843179650}]},"active_addr":"172.21.15.83:6812/2843179650","active_change":"2024-09-17T15:24:27.565096+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":1268405840}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":1010618890}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":1134822123}]}]} 2024-09-17T15:24:42.285 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-17T15:24:42.285 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-17T15:24:42.285 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T15:24:42.568 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:24:42.569 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":13,"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","created":"2024-09-17T15:24:23.007214+0000","modified":"2024-09-17T15:24:41.620717+0000","last_up_change":"2024-09-17T15:24:32.585667+0000","last_in_change":"2024-09-17T15:24:24.259028+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-17T15:24:38.423278+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":13,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9defe403-224a-4a6c-9184-dcb72bef6bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":335704639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":335704639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6811","nonce":335704639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":335704639}]},"public_addr":"172.21.15.83:6808/335704639","cluster_addr":"172.21.15.83:6809/335704639","heartbeat_back_addr":"172.21.15.83:6811/335704639","heartbeat_front_addr":"172.21.15.83:6810/335704639","state":["exists","up"]},{"osd":1,"uuid":"d7d78e01-f761-444b-98c4-62587b991738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":1258824819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6801","nonce":1258824819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6803","nonce":1258824819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":1258824819}]},"public_addr":"172.21.15.83:6800/1258824819","cluster_addr":"172.21.15.83:6801/1258824819","heartbeat_back_addr":"172.21.15.83:6803/1258824819","heartbeat_front_addr":"172.21.15.83:6802/1258824819","state":["exists","up"]},{"osd":2,"uuid":"23c2275c-8731-45f3-97f9-5100b6676f99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":239948269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6805","nonce":239948269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6807","nonce":239948269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":239948269}]},"public_addr":"172.21.15.83:6804/239948269","cluster_addr":"172.21.15.83:6805/239948269","heartbeat_back_addr":"172.21.15.83:6807/239948269","heartbeat_front_addr":"172.21.15.83:6806/239948269","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T15:24:31.547535+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T15:24:30.867578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T15:24:30.866429+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T15:24:42.581 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-17T15:24:42.581 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T15:24:42.866 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:24:42.866 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":13,"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","created":"2024-09-17T15:24:23.007214+0000","modified":"2024-09-17T15:24:41.620717+0000","last_up_change":"2024-09-17T15:24:32.585667+0000","last_in_change":"2024-09-17T15:24:24.259028+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-17T15:24:38.423278+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":13,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9defe403-224a-4a6c-9184-dcb72bef6bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":335704639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":335704639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6811","nonce":335704639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":335704639}]},"public_addr":"172.21.15.83:6808/335704639","cluster_addr":"172.21.15.83:6809/335704639","heartbeat_back_addr":"172.21.15.83:6811/335704639","heartbeat_front_addr":"172.21.15.83:6810/335704639","state":["exists","up"]},{"osd":1,"uuid":"d7d78e01-f761-444b-98c4-62587b991738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":1258824819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6801","nonce":1258824819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6803","nonce":1258824819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":1258824819}]},"public_addr":"172.21.15.83:6800/1258824819","cluster_addr":"172.21.15.83:6801/1258824819","heartbeat_back_addr":"172.21.15.83:6803/1258824819","heartbeat_front_addr":"172.21.15.83:6802/1258824819","state":["exists","up"]},{"osd":2,"uuid":"23c2275c-8731-45f3-97f9-5100b6676f99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":239948269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6805","nonce":239948269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6807","nonce":239948269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":239948269}]},"public_addr":"172.21.15.83:6804/239948269","cluster_addr":"172.21.15.83:6805/239948269","heartbeat_back_addr":"172.21.15.83:6807/239948269","heartbeat_front_addr":"172.21.15.83:6806/239948269","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T15:24:31.547535+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T15:24:30.867578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T15:24:30.866429+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T15:24:42.878 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-17T15:24:42.878 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-17T15:24:42.879 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-17T15:24:43.015 INFO:teuthology.orchestra.run.smithi083.stdout:34359738372 2024-09-17T15:24:43.015 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T15:24:43.033 INFO:teuthology.orchestra.run.smithi083.stdout:34359738372 2024-09-17T15:24:43.033 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T15:24:43.034 INFO:teuthology.orchestra.run.smithi083.stdout:34359738372 2024-09-17T15:24:43.034 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T15:24:43.318 INFO:teuthology.orchestra.run.smithi083.stdout:34359738371 2024-09-17T15:24:43.332 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738372 got 34359738371 for osd.1 2024-09-17T15:24:43.372 INFO:teuthology.orchestra.run.smithi083.stdout:34359738371 2024-09-17T15:24:43.373 INFO:teuthology.orchestra.run.smithi083.stdout:34359738371 2024-09-17T15:24:43.386 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738372 got 34359738371 for osd.0 2024-09-17T15:24:43.387 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738372 got 34359738371 for osd.2 2024-09-17T15:24:44.333 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T15:24:44.386 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T15:24:44.387 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T15:24:44.656 INFO:teuthology.orchestra.run.smithi083.stdout:34359738372 2024-09-17T15:24:44.670 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738372 got 34359738372 for osd.1 2024-09-17T15:24:44.670 DEBUG:teuthology.parallel:result is None 2024-09-17T15:24:44.688 INFO:teuthology.orchestra.run.smithi083.stdout:34359738372 2024-09-17T15:24:44.701 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738372 got 34359738372 for osd.0 2024-09-17T15:24:44.701 DEBUG:teuthology.parallel:result is None 2024-09-17T15:24:44.704 INFO:teuthology.orchestra.run.smithi083.stdout:34359738372 2024-09-17T15:24:44.717 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738372 got 34359738372 for osd.2 2024-09-17T15:24:44.717 DEBUG:teuthology.parallel:result is None 2024-09-17T15:24:44.717 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T15:24:44.717 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:24:45.039 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:24:45.039 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:24:45.052 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2024-09-17T15:24:43.572329+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81444,"kb_used_data":344,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283034076,"statfs":{"total":289910292480,"available":289826893824,"internally_reserved":0,"allocated":352256,"data_stored":80495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":82963791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-17T15:24:41.635460+0000","last_change":"2024-09-17T15:24:41.635460+0000","last_active":"2024-09-17T15:24:41.635460+0000","last_peered":"2024-09-17T15:24:41.635460+0000","last_clean":"2024-09-17T15:24:41.635460+0000","last_became_active":"2024-09-17T15:24:39.618458+0000","last_became_peered":"2024-09-17T15:24:39.618458+0000","last_unstale":"2024-09-17T15:24:41.635460+0000","last_undegraded":"2024-09-17T15:24:41.635460+0000","last_fullsized":"2024-09-17T15:24:41.635460+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:35:32.749904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-17T15:24:41.635690+0000","last_change":"2024-09-17T15:24:41.635690+0000","last_active":"2024-09-17T15:24:41.635690+0000","last_peered":"2024-09-17T15:24:41.635690+0000","last_clean":"2024-09-17T15:24:41.635690+0000","last_became_active":"2024-09-17T15:24:39.625082+0000","last_became_peered":"2024-09-17T15:24:39.625082+0000","last_unstale":"2024-09-17T15:24:41.635690+0000","last_undegraded":"2024-09-17T15:24:41.635690+0000","last_fullsized":"2024-09-17T15:24:41.635690+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:51:35.823375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-17T15:24:41.631530+0000","last_change":"2024-09-17T15:24:41.631530+0000","last_active":"2024-09-17T15:24:41.631530+0000","last_peered":"2024-09-17T15:24:41.631530+0000","last_clean":"2024-09-17T15:24:41.631530+0000","last_became_active":"2024-09-17T15:24:39.625570+0000","last_became_peered":"2024-09-17T15:24:39.625570+0000","last_unstale":"2024-09-17T15:24:41.631530+0000","last_undegraded":"2024-09-17T15:24:41.631530+0000","last_fullsized":"2024-09-17T15:24:41.631530+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:57:52.843841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058675600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-17T15:24:41.635548+0000","last_change":"2024-09-17T15:24:41.635548+0000","last_active":"2024-09-17T15:24:41.635548+0000","last_peered":"2024-09-17T15:24:41.635548+0000","last_clean":"2024-09-17T15:24:41.635548+0000","last_became_active":"2024-09-17T15:24:39.625795+0000","last_became_peered":"2024-09-17T15:24:39.625795+0000","last_unstale":"2024-09-17T15:24:41.635548+0000","last_undegraded":"2024-09-17T15:24:41.635548+0000","last_fullsized":"2024-09-17T15:24:41.635548+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:33:20.862791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-17T15:24:41.635202+0000","last_change":"2024-09-17T15:24:41.635202+0000","last_active":"2024-09-17T15:24:41.635202+0000","last_peered":"2024-09-17T15:24:41.635202+0000","last_clean":"2024-09-17T15:24:41.635202+0000","last_became_active":"2024-09-17T15:24:39.618501+0000","last_became_peered":"2024-09-17T15:24:39.618501+0000","last_unstale":"2024-09-17T15:24:41.635202+0000","last_undegraded":"2024-09-17T15:24:41.635202+0000","last_fullsized":"2024-09-17T15:24:41.635202+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:38:44.925808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"13'2","reported_seq":22,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-17T15:24:41.634249+0000","last_change":"2024-09-17T15:24:41.632020+0000","last_active":"2024-09-17T15:24:41.634249+0000","last_peered":"2024-09-17T15:24:41.634249+0000","last_clean":"2024-09-17T15:24:41.634249+0000","last_became_active":"2024-09-17T15:24:39.625863+0000","last_became_peered":"2024-09-17T15:24:39.625863+0000","last_unstale":"2024-09-17T15:24:41.634249+0000","last_undegraded":"2024-09-17T15:24:41.634249+0000","last_fullsized":"2024-09-17T15:24:41.634249+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:59:34.094507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-17T15:24:41.631529+0000","last_change":"2024-09-17T15:24:41.631529+0000","last_active":"2024-09-17T15:24:41.631529+0000","last_peered":"2024-09-17T15:24:41.631529+0000","last_clean":"2024-09-17T15:24:41.631529+0000","last_became_active":"2024-09-17T15:24:39.625085+0000","last_became_peered":"2024-09-17T15:24:39.625085+0000","last_unstale":"2024-09-17T15:24:41.631529+0000","last_undegraded":"2024-09-17T15:24:41.631529+0000","last_fullsized":"2024-09-17T15:24:41.631529+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:33:37.750200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000589027,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-17T15:24:41.635178+0000","last_change":"2024-09-17T15:24:41.635178+0000","last_active":"2024-09-17T15:24:41.635178+0000","last_peered":"2024-09-17T15:24:41.635178+0000","last_clean":"2024-09-17T15:24:41.635178+0000","last_became_active":"2024-09-17T15:24:39.625416+0000","last_became_peered":"2024-09-17T15:24:39.625416+0000","last_unstale":"2024-09-17T15:24:41.635178+0000","last_undegraded":"2024-09-17T15:24:41.635178+0000","last_fullsized":"2024-09-17T15:24:41.635178+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:58:18.531873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738372,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":114688,"data_stored":26819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738372,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27144,"kb_used_data":116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344696,"statfs":{"total":96636764160,"available":96608968704,"internally_reserved":0,"allocated":118784,"data_stored":26838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738372,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":118784,"data_stored":26838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:24:45.053 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:24:45.331 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:24:45.331 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:24:45.343 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2024-09-17T15:24:43.572329+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81444,"kb_used_data":344,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283034076,"statfs":{"total":289910292480,"available":289826893824,"internally_reserved":0,"allocated":352256,"data_stored":80495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":82963791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-17T15:24:41.635460+0000","last_change":"2024-09-17T15:24:41.635460+0000","last_active":"2024-09-17T15:24:41.635460+0000","last_peered":"2024-09-17T15:24:41.635460+0000","last_clean":"2024-09-17T15:24:41.635460+0000","last_became_active":"2024-09-17T15:24:39.618458+0000","last_became_peered":"2024-09-17T15:24:39.618458+0000","last_unstale":"2024-09-17T15:24:41.635460+0000","last_undegraded":"2024-09-17T15:24:41.635460+0000","last_fullsized":"2024-09-17T15:24:41.635460+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:35:32.749904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-17T15:24:41.635690+0000","last_change":"2024-09-17T15:24:41.635690+0000","last_active":"2024-09-17T15:24:41.635690+0000","last_peered":"2024-09-17T15:24:41.635690+0000","last_clean":"2024-09-17T15:24:41.635690+0000","last_became_active":"2024-09-17T15:24:39.625082+0000","last_became_peered":"2024-09-17T15:24:39.625082+0000","last_unstale":"2024-09-17T15:24:41.635690+0000","last_undegraded":"2024-09-17T15:24:41.635690+0000","last_fullsized":"2024-09-17T15:24:41.635690+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:51:35.823375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-17T15:24:41.631530+0000","last_change":"2024-09-17T15:24:41.631530+0000","last_active":"2024-09-17T15:24:41.631530+0000","last_peered":"2024-09-17T15:24:41.631530+0000","last_clean":"2024-09-17T15:24:41.631530+0000","last_became_active":"2024-09-17T15:24:39.625570+0000","last_became_peered":"2024-09-17T15:24:39.625570+0000","last_unstale":"2024-09-17T15:24:41.631530+0000","last_undegraded":"2024-09-17T15:24:41.631530+0000","last_fullsized":"2024-09-17T15:24:41.631530+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:57:52.843841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058675600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-17T15:24:41.635548+0000","last_change":"2024-09-17T15:24:41.635548+0000","last_active":"2024-09-17T15:24:41.635548+0000","last_peered":"2024-09-17T15:24:41.635548+0000","last_clean":"2024-09-17T15:24:41.635548+0000","last_became_active":"2024-09-17T15:24:39.625795+0000","last_became_peered":"2024-09-17T15:24:39.625795+0000","last_unstale":"2024-09-17T15:24:41.635548+0000","last_undegraded":"2024-09-17T15:24:41.635548+0000","last_fullsized":"2024-09-17T15:24:41.635548+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:33:20.862791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-17T15:24:41.635202+0000","last_change":"2024-09-17T15:24:41.635202+0000","last_active":"2024-09-17T15:24:41.635202+0000","last_peered":"2024-09-17T15:24:41.635202+0000","last_clean":"2024-09-17T15:24:41.635202+0000","last_became_active":"2024-09-17T15:24:39.618501+0000","last_became_peered":"2024-09-17T15:24:39.618501+0000","last_unstale":"2024-09-17T15:24:41.635202+0000","last_undegraded":"2024-09-17T15:24:41.635202+0000","last_fullsized":"2024-09-17T15:24:41.635202+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:38:44.925808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"13'2","reported_seq":22,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-17T15:24:41.634249+0000","last_change":"2024-09-17T15:24:41.632020+0000","last_active":"2024-09-17T15:24:41.634249+0000","last_peered":"2024-09-17T15:24:41.634249+0000","last_clean":"2024-09-17T15:24:41.634249+0000","last_became_active":"2024-09-17T15:24:39.625863+0000","last_became_peered":"2024-09-17T15:24:39.625863+0000","last_unstale":"2024-09-17T15:24:41.634249+0000","last_undegraded":"2024-09-17T15:24:41.634249+0000","last_fullsized":"2024-09-17T15:24:41.634249+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:59:34.094507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-17T15:24:41.631529+0000","last_change":"2024-09-17T15:24:41.631529+0000","last_active":"2024-09-17T15:24:41.631529+0000","last_peered":"2024-09-17T15:24:41.631529+0000","last_clean":"2024-09-17T15:24:41.631529+0000","last_became_active":"2024-09-17T15:24:39.625085+0000","last_became_peered":"2024-09-17T15:24:39.625085+0000","last_unstale":"2024-09-17T15:24:41.631529+0000","last_undegraded":"2024-09-17T15:24:41.631529+0000","last_fullsized":"2024-09-17T15:24:41.631529+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:33:37.750200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000589027,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-17T15:24:41.635178+0000","last_change":"2024-09-17T15:24:41.635178+0000","last_active":"2024-09-17T15:24:41.635178+0000","last_peered":"2024-09-17T15:24:41.635178+0000","last_clean":"2024-09-17T15:24:41.635178+0000","last_became_active":"2024-09-17T15:24:39.625416+0000","last_became_peered":"2024-09-17T15:24:39.625416+0000","last_unstale":"2024-09-17T15:24:41.635178+0000","last_undegraded":"2024-09-17T15:24:41.635178+0000","last_fullsized":"2024-09-17T15:24:41.635178+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:58:18.531873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738372,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":114688,"data_stored":26819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738372,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27144,"kb_used_data":116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344696,"statfs":{"total":96636764160,"available":96608968704,"internally_reserved":0,"allocated":118784,"data_stored":26838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738372,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":118784,"data_stored":26838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:24:45.343 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T15:24:45.344 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-17T15:24:45.344 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-17T15:24:45.344 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T15:24:45.667 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:24:45.667 INFO:teuthology.orchestra.run.smithi083.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-17T15:24:45.680 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-17T15:24:45.680 INFO:teuthology.run_tasks:Running task lost_unfound... 2024-09-17T15:24:45.691 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T15:24:45.973 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:24:45.973 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":13,"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","created":"2024-09-17T15:24:23.007214+0000","modified":"2024-09-17T15:24:41.620717+0000","last_up_change":"2024-09-17T15:24:32.585667+0000","last_in_change":"2024-09-17T15:24:24.259028+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-17T15:24:38.423278+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":13,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9defe403-224a-4a6c-9184-dcb72bef6bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":335704639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":335704639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6811","nonce":335704639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":335704639}]},"public_addr":"172.21.15.83:6808/335704639","cluster_addr":"172.21.15.83:6809/335704639","heartbeat_back_addr":"172.21.15.83:6811/335704639","heartbeat_front_addr":"172.21.15.83:6810/335704639","state":["exists","up"]},{"osd":1,"uuid":"d7d78e01-f761-444b-98c4-62587b991738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":1258824819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6801","nonce":1258824819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6803","nonce":1258824819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":1258824819}]},"public_addr":"172.21.15.83:6800/1258824819","cluster_addr":"172.21.15.83:6801/1258824819","heartbeat_back_addr":"172.21.15.83:6803/1258824819","heartbeat_front_addr":"172.21.15.83:6802/1258824819","state":["exists","up"]},{"osd":2,"uuid":"23c2275c-8731-45f3-97f9-5100b6676f99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":239948269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6805","nonce":239948269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6807","nonce":239948269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":239948269}]},"public_addr":"172.21.15.83:6804/239948269","cluster_addr":"172.21.15.83:6805/239948269","heartbeat_back_addr":"172.21.15.83:6807/239948269","heartbeat_front_addr":"172.21.15.83:6806/239948269","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T15:24:31.547535+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T15:24:30.867578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T15:24:30.866429+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T15:24:45.986 INFO:tasks.lost_unfound.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-09-17T15:24:38.423278+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '13', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 13, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T15:24:45.986 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-09-17T15:24:46.271 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 8 2024-09-17T15:24:46.284 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-17T15:24:46.563 INFO:teuthology.orchestra.run.smithi083.stdout:epoch 13 2024-09-17T15:24:46.563 INFO:teuthology.orchestra.run.smithi083.stdout:fsid 5d1df884-ae2f-44a8-abc8-b077329edada 2024-09-17T15:24:46.563 INFO:teuthology.orchestra.run.smithi083.stdout:created 2024-09-17T15:24:23.007214+0000 2024-09-17T15:24:46.563 INFO:teuthology.orchestra.run.smithi083.stdout:modified 2024-09-17T15:24:41.620717+0000 2024-09-17T15:24:46.564 INFO:teuthology.orchestra.run.smithi083.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-17T15:24:46.564 INFO:teuthology.orchestra.run.smithi083.stdout:crush_version 4 2024-09-17T15:24:46.564 INFO:teuthology.orchestra.run.smithi083.stdout:full_ratio 0.95 2024-09-17T15:24:46.564 INFO:teuthology.orchestra.run.smithi083.stdout:backfillfull_ratio 0.9 2024-09-17T15:24:46.564 INFO:teuthology.orchestra.run.smithi083.stdout:nearfull_ratio 0.85 2024-09-17T15:24:46.564 INFO:teuthology.orchestra.run.smithi083.stdout:require_min_compat_client luminous 2024-09-17T15:24:46.564 INFO:teuthology.orchestra.run.smithi083.stdout:min_compat_client jewel 2024-09-17T15:24:46.564 INFO:teuthology.orchestra.run.smithi083.stdout:require_osd_release squid 2024-09-17T15:24:46.564 INFO:teuthology.orchestra.run.smithi083.stdout:stretch_mode_enabled false 2024-09-17T15:24:46.565 INFO:teuthology.orchestra.run.smithi083.stdout:pool 1 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 13 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 1.88 2024-09-17T15:24:46.565 INFO:teuthology.orchestra.run.smithi083.stdout: removed_snaps_queue [2~1] 2024-09-17T15:24:46.565 INFO:teuthology.orchestra.run.smithi083.stdout:max_osd 3 2024-09-17T15:24:46.565 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0 up in weight 1 up_from 8 up_thru 10 down_at 0 last_clean_interval [0,0) v2:172.21.15.83:6808/335704639 v2:172.21.15.83:6809/335704639 exists,up 9defe403-224a-4a6c-9184-dcb72bef6bf2 2024-09-17T15:24:46.565 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1 up in weight 1 up_from 8 up_thru 10 down_at 0 last_clean_interval [0,0) v2:172.21.15.83:6800/1258824819 v2:172.21.15.83:6801/1258824819 exists,up d7d78e01-f761-444b-98c4-62587b991738 2024-09-17T15:24:46.565 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2 up in weight 1 up_from 8 up_thru 10 down_at 0 last_clean_interval [0,0) v2:172.21.15.83:6804/239948269 v2:172.21.15.83:6805/239948269 exists,up 23c2275c-8731-45f3-97f9-5100b6676f99 2024-09-17T15:24:46.575 INFO:tasks.lost_unfound.ceph_manager:['osd.0 up in weight 1 up_from 8 up_thru 10 down_at 0 last_clean_interval [0,0) v2:172.21.15.83:6808/335704639 v2:172.21.15.83:6809/335704639 exists,up 9defe403-224a-4a6c-9184-dcb72bef6bf2', 'osd.1 up in weight 1 up_from 8 up_thru 10 down_at 0 last_clean_interval [0,0) v2:172.21.15.83:6800/1258824819 v2:172.21.15.83:6801/1258824819 exists,up d7d78e01-f761-444b-98c4-62587b991738', 'osd.2 up in weight 1 up_from 8 up_thru 10 down_at 0 last_clean_interval [0,0) v2:172.21.15.83:6804/239948269 v2:172.21.15.83:6805/239948269 exists,up 23c2275c-8731-45f3-97f9-5100b6676f99'] 2024-09-17T15:24:46.576 INFO:tasks.lost_unfound.ceph_manager:waiting for clean 2024-09-17T15:24:46.576 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:24:46.858 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:24:46.858 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:24:46.870 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2024-09-17T15:24:45.572603+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81444,"kb_used_data":344,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283034076,"statfs":{"total":289910292480,"available":289826893824,"internally_reserved":0,"allocated":352256,"data_stored":80495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":82963791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000274"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-17T15:24:41.635460+0000","last_change":"2024-09-17T15:24:41.635460+0000","last_active":"2024-09-17T15:24:41.635460+0000","last_peered":"2024-09-17T15:24:41.635460+0000","last_clean":"2024-09-17T15:24:41.635460+0000","last_became_active":"2024-09-17T15:24:39.618458+0000","last_became_peered":"2024-09-17T15:24:39.618458+0000","last_unstale":"2024-09-17T15:24:41.635460+0000","last_undegraded":"2024-09-17T15:24:41.635460+0000","last_fullsized":"2024-09-17T15:24:41.635460+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:35:32.749904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-17T15:24:41.635690+0000","last_change":"2024-09-17T15:24:41.635690+0000","last_active":"2024-09-17T15:24:41.635690+0000","last_peered":"2024-09-17T15:24:41.635690+0000","last_clean":"2024-09-17T15:24:41.635690+0000","last_became_active":"2024-09-17T15:24:39.625082+0000","last_became_peered":"2024-09-17T15:24:39.625082+0000","last_unstale":"2024-09-17T15:24:41.635690+0000","last_undegraded":"2024-09-17T15:24:41.635690+0000","last_fullsized":"2024-09-17T15:24:41.635690+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:51:35.823375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-17T15:24:41.631530+0000","last_change":"2024-09-17T15:24:41.631530+0000","last_active":"2024-09-17T15:24:41.631530+0000","last_peered":"2024-09-17T15:24:41.631530+0000","last_clean":"2024-09-17T15:24:41.631530+0000","last_became_active":"2024-09-17T15:24:39.625570+0000","last_became_peered":"2024-09-17T15:24:39.625570+0000","last_unstale":"2024-09-17T15:24:41.631530+0000","last_undegraded":"2024-09-17T15:24:41.631530+0000","last_fullsized":"2024-09-17T15:24:41.631530+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:57:52.843841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058675600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-17T15:24:41.635548+0000","last_change":"2024-09-17T15:24:41.635548+0000","last_active":"2024-09-17T15:24:41.635548+0000","last_peered":"2024-09-17T15:24:41.635548+0000","last_clean":"2024-09-17T15:24:41.635548+0000","last_became_active":"2024-09-17T15:24:39.625795+0000","last_became_peered":"2024-09-17T15:24:39.625795+0000","last_unstale":"2024-09-17T15:24:41.635548+0000","last_undegraded":"2024-09-17T15:24:41.635548+0000","last_fullsized":"2024-09-17T15:24:41.635548+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:33:20.862791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-17T15:24:41.635202+0000","last_change":"2024-09-17T15:24:41.635202+0000","last_active":"2024-09-17T15:24:41.635202+0000","last_peered":"2024-09-17T15:24:41.635202+0000","last_clean":"2024-09-17T15:24:41.635202+0000","last_became_active":"2024-09-17T15:24:39.618501+0000","last_became_peered":"2024-09-17T15:24:39.618501+0000","last_unstale":"2024-09-17T15:24:41.635202+0000","last_undegraded":"2024-09-17T15:24:41.635202+0000","last_fullsized":"2024-09-17T15:24:41.635202+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:38:44.925808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"13'2","reported_seq":22,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-17T15:24:41.634249+0000","last_change":"2024-09-17T15:24:41.632020+0000","last_active":"2024-09-17T15:24:41.634249+0000","last_peered":"2024-09-17T15:24:41.634249+0000","last_clean":"2024-09-17T15:24:41.634249+0000","last_became_active":"2024-09-17T15:24:39.625863+0000","last_became_peered":"2024-09-17T15:24:39.625863+0000","last_unstale":"2024-09-17T15:24:41.634249+0000","last_undegraded":"2024-09-17T15:24:41.634249+0000","last_fullsized":"2024-09-17T15:24:41.634249+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:59:34.094507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-17T15:24:41.631529+0000","last_change":"2024-09-17T15:24:41.631529+0000","last_active":"2024-09-17T15:24:41.631529+0000","last_peered":"2024-09-17T15:24:41.631529+0000","last_clean":"2024-09-17T15:24:41.631529+0000","last_became_active":"2024-09-17T15:24:39.625085+0000","last_became_peered":"2024-09-17T15:24:39.625085+0000","last_unstale":"2024-09-17T15:24:41.631529+0000","last_undegraded":"2024-09-17T15:24:41.631529+0000","last_fullsized":"2024-09-17T15:24:41.631529+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:33:37.750200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000589027,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-17T15:24:41.635178+0000","last_change":"2024-09-17T15:24:41.635178+0000","last_active":"2024-09-17T15:24:41.635178+0000","last_peered":"2024-09-17T15:24:41.635178+0000","last_clean":"2024-09-17T15:24:41.635178+0000","last_became_active":"2024-09-17T15:24:39.625416+0000","last_became_peered":"2024-09-17T15:24:39.625416+0000","last_unstale":"2024-09-17T15:24:41.635178+0000","last_undegraded":"2024-09-17T15:24:41.635178+0000","last_fullsized":"2024-09-17T15:24:41.635178+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:58:18.531873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738373,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":114688,"data_stored":26819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738373,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27144,"kb_used_data":116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344696,"statfs":{"total":96636764160,"available":96608968704,"internally_reserved":0,"allocated":118784,"data_stored":26838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738373,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":118784,"data_stored":26838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:24:46.871 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:24:47.148 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:24:47.148 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:24:47.160 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2024-09-17T15:24:45.572603+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81444,"kb_used_data":344,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283034076,"statfs":{"total":289910292480,"available":289826893824,"internally_reserved":0,"allocated":352256,"data_stored":80495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":82963791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000274"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-17T15:24:41.635460+0000","last_change":"2024-09-17T15:24:41.635460+0000","last_active":"2024-09-17T15:24:41.635460+0000","last_peered":"2024-09-17T15:24:41.635460+0000","last_clean":"2024-09-17T15:24:41.635460+0000","last_became_active":"2024-09-17T15:24:39.618458+0000","last_became_peered":"2024-09-17T15:24:39.618458+0000","last_unstale":"2024-09-17T15:24:41.635460+0000","last_undegraded":"2024-09-17T15:24:41.635460+0000","last_fullsized":"2024-09-17T15:24:41.635460+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:35:32.749904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-17T15:24:41.635690+0000","last_change":"2024-09-17T15:24:41.635690+0000","last_active":"2024-09-17T15:24:41.635690+0000","last_peered":"2024-09-17T15:24:41.635690+0000","last_clean":"2024-09-17T15:24:41.635690+0000","last_became_active":"2024-09-17T15:24:39.625082+0000","last_became_peered":"2024-09-17T15:24:39.625082+0000","last_unstale":"2024-09-17T15:24:41.635690+0000","last_undegraded":"2024-09-17T15:24:41.635690+0000","last_fullsized":"2024-09-17T15:24:41.635690+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:51:35.823375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-17T15:24:41.631530+0000","last_change":"2024-09-17T15:24:41.631530+0000","last_active":"2024-09-17T15:24:41.631530+0000","last_peered":"2024-09-17T15:24:41.631530+0000","last_clean":"2024-09-17T15:24:41.631530+0000","last_became_active":"2024-09-17T15:24:39.625570+0000","last_became_peered":"2024-09-17T15:24:39.625570+0000","last_unstale":"2024-09-17T15:24:41.631530+0000","last_undegraded":"2024-09-17T15:24:41.631530+0000","last_fullsized":"2024-09-17T15:24:41.631530+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:57:52.843841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058675600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-17T15:24:41.635548+0000","last_change":"2024-09-17T15:24:41.635548+0000","last_active":"2024-09-17T15:24:41.635548+0000","last_peered":"2024-09-17T15:24:41.635548+0000","last_clean":"2024-09-17T15:24:41.635548+0000","last_became_active":"2024-09-17T15:24:39.625795+0000","last_became_peered":"2024-09-17T15:24:39.625795+0000","last_unstale":"2024-09-17T15:24:41.635548+0000","last_undegraded":"2024-09-17T15:24:41.635548+0000","last_fullsized":"2024-09-17T15:24:41.635548+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:33:20.862791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-17T15:24:41.635202+0000","last_change":"2024-09-17T15:24:41.635202+0000","last_active":"2024-09-17T15:24:41.635202+0000","last_peered":"2024-09-17T15:24:41.635202+0000","last_clean":"2024-09-17T15:24:41.635202+0000","last_became_active":"2024-09-17T15:24:39.618501+0000","last_became_peered":"2024-09-17T15:24:39.618501+0000","last_unstale":"2024-09-17T15:24:41.635202+0000","last_undegraded":"2024-09-17T15:24:41.635202+0000","last_fullsized":"2024-09-17T15:24:41.635202+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:38:44.925808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"13'2","reported_seq":22,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-17T15:24:41.634249+0000","last_change":"2024-09-17T15:24:41.632020+0000","last_active":"2024-09-17T15:24:41.634249+0000","last_peered":"2024-09-17T15:24:41.634249+0000","last_clean":"2024-09-17T15:24:41.634249+0000","last_became_active":"2024-09-17T15:24:39.625863+0000","last_became_peered":"2024-09-17T15:24:39.625863+0000","last_unstale":"2024-09-17T15:24:41.634249+0000","last_undegraded":"2024-09-17T15:24:41.634249+0000","last_fullsized":"2024-09-17T15:24:41.634249+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:59:34.094507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-17T15:24:41.631529+0000","last_change":"2024-09-17T15:24:41.631529+0000","last_active":"2024-09-17T15:24:41.631529+0000","last_peered":"2024-09-17T15:24:41.631529+0000","last_clean":"2024-09-17T15:24:41.631529+0000","last_became_active":"2024-09-17T15:24:39.625085+0000","last_became_peered":"2024-09-17T15:24:39.625085+0000","last_unstale":"2024-09-17T15:24:41.631529+0000","last_undegraded":"2024-09-17T15:24:41.631529+0000","last_fullsized":"2024-09-17T15:24:41.631529+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:33:37.750200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000589027,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-17T15:24:41.635178+0000","last_change":"2024-09-17T15:24:41.635178+0000","last_active":"2024-09-17T15:24:41.635178+0000","last_peered":"2024-09-17T15:24:41.635178+0000","last_clean":"2024-09-17T15:24:41.635178+0000","last_became_active":"2024-09-17T15:24:39.625416+0000","last_became_peered":"2024-09-17T15:24:39.625416+0000","last_unstale":"2024-09-17T15:24:41.635178+0000","last_undegraded":"2024-09-17T15:24:41.635178+0000","last_fullsized":"2024-09-17T15:24:41.635178+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:58:18.531873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738373,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":114688,"data_stored":26819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738373,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27144,"kb_used_data":116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344696,"statfs":{"total":96636764160,"available":96608968704,"internally_reserved":0,"allocated":118784,"data_stored":26838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738373,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":118784,"data_stored":26838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:24:47.160 INFO:tasks.lost_unfound.ceph_manager:clean! 2024-09-17T15:24:47.161 INFO:tasks.lost_unfound.ceph_manager:creating pool_name unfound_pool 2024-09-17T15:24:47.161 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unfound_pool 16 2024-09-17T15:24:47.645 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'unfound_pool' created 2024-09-17T15:24:47.660 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unfound_pool rados --yes-i-really-mean-it || true 2024-09-17T15:24:48.779 INFO:teuthology.orchestra.run.smithi083.stderr:enabled application 'rados' on pool 'unfound_pool' 2024-09-17T15:24:49.796 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-17T15:24:49.796 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T15:24:49.824 INFO:tasks.ceph.osd.2:Stopped 2024-09-17T15:24:49.824 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 2 2024-09-17T15:24:50.165 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-09-17T15:24:50.164+0000 7f514a174640 -1 mon.a@0(leader).osd e16 definitely_dead 0 2024-09-17T15:24:50.795 INFO:teuthology.orchestra.run.smithi083.stderr:marked down osd.2. 2024-09-17T15:24:50.809 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 2 2024-09-17T15:24:51.101 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-09-17T15:24:51.101+0000 7f514a174640 -1 mon.a@0(leader).osd e17 definitely_dead 0 2024-09-17T15:24:51.797 INFO:teuthology.orchestra.run.smithi083.stderr:marked out osd.2. 2024-09-17T15:24:51.814 INFO:tasks.util.rados:rados -p unfound_pool put dummy /etc/fstab 2024-09-17T15:24:51.814 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put dummy /etc/fstab 2024-09-17T15:24:51.865 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-17T15:24:51.866 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-17T15:24:52.015 INFO:teuthology.orchestra.run.smithi083.stdout:34359738375 2024-09-17T15:24:52.015 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T15:24:52.016 INFO:teuthology.orchestra.run.smithi083.stdout:34359738375 2024-09-17T15:24:52.017 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T15:24:52.350 INFO:teuthology.orchestra.run.smithi083.stdout:34359738374 2024-09-17T15:24:52.353 INFO:teuthology.orchestra.run.smithi083.stdout:34359738374 2024-09-17T15:24:52.364 INFO:tasks.lost_unfound.ceph_manager:need seq 34359738375 got 34359738374 for osd.1 2024-09-17T15:24:52.366 INFO:tasks.lost_unfound.ceph_manager:need seq 34359738375 got 34359738374 for osd.0 2024-09-17T15:24:53.365 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T15:24:53.367 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T15:24:53.666 INFO:teuthology.orchestra.run.smithi083.stdout:34359738374 2024-09-17T15:24:53.672 INFO:teuthology.orchestra.run.smithi083.stdout:34359738374 2024-09-17T15:24:53.680 INFO:tasks.lost_unfound.ceph_manager:need seq 34359738375 got 34359738374 for osd.1 2024-09-17T15:24:53.685 INFO:tasks.lost_unfound.ceph_manager:need seq 34359738375 got 34359738374 for osd.0 2024-09-17T15:24:54.681 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T15:24:54.686 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T15:24:54.986 INFO:teuthology.orchestra.run.smithi083.stdout:34359738375 2024-09-17T15:24:54.999 INFO:tasks.lost_unfound.ceph_manager:need seq 34359738375 got 34359738375 for osd.1 2024-09-17T15:24:54.999 DEBUG:teuthology.parallel:result is None 2024-09-17T15:24:55.022 INFO:teuthology.orchestra.run.smithi083.stdout:34359738375 2024-09-17T15:24:55.034 INFO:tasks.lost_unfound.ceph_manager:need seq 34359738375 got 34359738375 for osd.0 2024-09-17T15:24:55.035 DEBUG:teuthology.parallel:result is None 2024-09-17T15:24:55.035 INFO:tasks.lost_unfound.ceph_manager:waiting for recovery to complete 2024-09-17T15:24:55.035 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:24:55.355 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:24:55.356 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:24:55.368 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-09-17T15:24:53.574246+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54372,"kb_used_data":308,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689308,"statfs":{"total":193273528320,"available":193217851392,"internally_reserved":0,"allocated":315392,"data_stored":80721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3190,"internal_metadata":55309194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.796583"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":17,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-17T15:24:51.805518+0000","last_change":"2024-09-17T15:24:47.788372+0000","last_active":"2024-09-17T15:24:51.805518+0000","last_peered":"2024-09-17T15:24:51.805518+0000","last_clean":"2024-09-17T15:24:51.805518+0000","last_became_active":"2024-09-17T15:24:47.788133+0000","last_became_peered":"2024-09-17T15:24:47.788133+0000","last_unstale":"2024-09-17T15:24:51.805518+0000","last_undegraded":"2024-09-17T15:24:51.805518+0000","last_fullsized":"2024-09-17T15:24:51.805518+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:01:11.136214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":4,"reported_epoch":18,"state":"peering","last_fresh":"2024-09-17T15:24:51.805716+0000","last_change":"2024-09-17T15:24:50.795802+0000","last_active":"2024-09-17T15:24:47.637664+0000","last_peered":"2024-09-17T15:24:47.637664+0000","last_clean":"2024-09-17T15:24:47.637664+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T15:24:51.805716+0000","last_undegraded":"2024-09-17T15:24:51.805716+0000","last_fullsized":"2024-09-17T15:24:51.805716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"peering","last_fresh":"2024-09-17T15:24:51.220022+0000","last_change":"2024-09-17T15:24:51.220022+0000","last_active":"2024-09-17T15:24:51.220022+0000","last_peered":"2024-09-17T15:24:48.780361+0000","last_clean":"2024-09-17T15:24:48.780361+0000","last_became_active":"2024-09-17T15:24:47.781427+0000","last_became_peered":"2024-09-17T15:24:47.781427+0000","last_unstale":"2024-09-17T15:24:51.220022+0000","last_undegraded":"2024-09-17T15:24:51.220022+0000","last_fullsized":"2024-09-17T15:24:51.220022+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":4,"reported_epoch":18,"state":"peering","last_fresh":"2024-09-17T15:24:51.805062+0000","last_change":"2024-09-17T15:24:50.794998+0000","last_active":"2024-09-17T15:24:47.637664+0000","last_peered":"2024-09-17T15:24:47.637664+0000","last_clean":"2024-09-17T15:24:47.637664+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T15:24:51.805062+0000","last_undegraded":"2024-09-17T15:24:51.805062+0000","last_fullsized":"2024-09-17T15:24:51.805062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"peering","last_fresh":"2024-09-17T15:24:50.794875+0000","last_change":"2024-09-17T15:24:50.794875+0000","last_active":"2024-09-17T15:24:50.794875+0000","last_peered":"2024-09-17T15:24:49.051686+0000","last_clean":"2024-09-17T15:24:49.051686+0000","last_became_active":"2024-09-17T15:24:47.786656+0000","last_became_peered":"2024-09-17T15:24:47.786656+0000","last_unstale":"2024-09-17T15:24:50.794875+0000","last_undegraded":"2024-09-17T15:24:50.794875+0000","last_fullsized":"2024-09-17T15:24:50.794875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":17,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-17T15:24:51.806317+0000","last_change":"2024-09-17T15:24:47.787187+0000","last_active":"2024-09-17T15:24:51.806317+0000","last_peered":"2024-09-17T15:24:51.806317+0000","last_clean":"2024-09-17T15:24:51.806317+0000","last_became_active":"2024-09-17T15:24:47.786871+0000","last_became_peered":"2024-09-17T15:24:47.786871+0000","last_unstale":"2024-09-17T15:24:51.806317+0000","last_undegraded":"2024-09-17T15:24:51.806317+0000","last_fullsized":"2024-09-17T15:24:51.806317+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:14:42.196510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":17,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-17T15:24:51.806605+0000","last_change":"2024-09-17T15:24:47.787090+0000","last_active":"2024-09-17T15:24:51.806605+0000","last_peered":"2024-09-17T15:24:51.806605+0000","last_clean":"2024-09-17T15:24:51.806605+0000","last_became_active":"2024-09-17T15:24:47.786830+0000","last_became_peered":"2024-09-17T15:24:47.786830+0000","last_unstale":"2024-09-17T15:24:51.806605+0000","last_undegraded":"2024-09-17T15:24:51.806605+0000","last_fullsized":"2024-09-17T15:24:51.806605+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:05:02.468465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"peering","last_fresh":"2024-09-17T15:24:50.795323+0000","last_change":"2024-09-17T15:24:50.795323+0000","last_active":"2024-09-17T15:24:50.795323+0000","last_peered":"2024-09-17T15:24:49.051582+0000","last_clean":"2024-09-17T15:24:49.051582+0000","last_became_active":"2024-09-17T15:24:47.785954+0000","last_became_peered":"2024-09-17T15:24:47.785954+0000","last_unstale":"2024-09-17T15:24:50.795323+0000","last_undegraded":"2024-09-17T15:24:50.795323+0000","last_fullsized":"2024-09-17T15:24:50.795323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":30,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-17T15:24:51.806095+0000","last_change":"2024-09-17T15:24:41.635178+0000","last_active":"2024-09-17T15:24:51.806095+0000","last_peered":"2024-09-17T15:24:51.806095+0000","last_clean":"2024-09-17T15:24:51.806095+0000","last_became_active":"2024-09-17T15:24:39.625416+0000","last_became_peered":"2024-09-17T15:24:39.625416+0000","last_unstale":"2024-09-17T15:24:51.806095+0000","last_undegraded":"2024-09-17T15:24:51.806095+0000","last_fullsized":"2024-09-17T15:24:51.806095+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:58:18.531873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"peering","last_fresh":"2024-09-17T15:24:51.219921+0000","last_change":"2024-09-17T15:24:51.219921+0000","last_active":"2024-09-17T15:24:51.219921+0000","last_peered":"2024-09-17T15:24:48.779347+0000","last_clean":"2024-09-17T15:24:48.779347+0000","last_became_active":"2024-09-17T15:24:47.780095+0000","last_became_peered":"2024-09-17T15:24:47.780095+0000","last_unstale":"2024-09-17T15:24:51.219921+0000","last_undegraded":"2024-09-17T15:24:51.219921+0000","last_fullsized":"2024-09-17T15:24:51.219921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":4,"reported_epoch":18,"state":"peering","last_fresh":"2024-09-17T15:24:51.804132+0000","last_change":"2024-09-17T15:24:50.794874+0000","last_active":"2024-09-17T15:24:38.602774+0000","last_peered":"2024-09-17T15:24:38.602774+0000","last_clean":"2024-09-17T15:24:38.602774+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T15:24:51.804132+0000","last_undegraded":"2024-09-17T15:24:51.804132+0000","last_fullsized":"2024-09-17T15:24:51.804132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-17T15:24:51.803871+0000","last_change":"2024-09-17T15:24:47.786083+0000","last_active":"2024-09-17T15:24:51.803871+0000","last_peered":"2024-09-17T15:24:51.803871+0000","last_clean":"2024-09-17T15:24:51.803871+0000","last_became_active":"2024-09-17T15:24:47.785813+0000","last_became_peered":"2024-09-17T15:24:47.785813+0000","last_unstale":"2024-09-17T15:24:51.803871+0000","last_undegraded":"2024-09-17T15:24:51.803871+0000","last_fullsized":"2024-09-17T15:24:51.803871+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:07:13.003526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":32,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-17T15:24:51.803909+0000","last_change":"2024-09-17T15:24:41.632020+0000","last_active":"2024-09-17T15:24:51.803909+0000","last_peered":"2024-09-17T15:24:51.803909+0000","last_clean":"2024-09-17T15:24:51.803909+0000","last_became_active":"2024-09-17T15:24:39.625863+0000","last_became_peered":"2024-09-17T15:24:39.625863+0000","last_unstale":"2024-09-17T15:24:51.803909+0000","last_undegraded":"2024-09-17T15:24:51.803909+0000","last_fullsized":"2024-09-17T15:24:51.803909+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:59:34.094507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":4,"reported_epoch":18,"state":"peering","last_fresh":"2024-09-17T15:24:51.806497+0000","last_change":"2024-09-17T15:24:51.219903+0000","last_active":"2024-09-17T15:24:47.637664+0000","last_peered":"2024-09-17T15:24:47.637664+0000","last_clean":"2024-09-17T15:24:47.637664+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T15:24:51.806497+0000","last_undegraded":"2024-09-17T15:24:51.806497+0000","last_fullsized":"2024-09-17T15:24:51.806497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":29,"reported_epoch":17,"state":"peering","last_fresh":"2024-09-17T15:24:51.220033+0000","last_change":"2024-09-17T15:24:51.220033+0000","last_active":"2024-09-17T15:24:51.220033+0000","last_peered":"2024-09-17T15:24:48.779453+0000","last_clean":"2024-09-17T15:24:48.779453+0000","last_became_active":"2024-09-17T15:24:39.618501+0000","last_became_peered":"2024-09-17T15:24:39.618501+0000","last_unstale":"2024-09-17T15:24:51.220033+0000","last_undegraded":"2024-09-17T15:24:51.220033+0000","last_fullsized":"2024-09-17T15:24:51.220033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":4,"reported_epoch":18,"state":"peering","last_fresh":"2024-09-17T15:24:51.806440+0000","last_change":"2024-09-17T15:24:51.219810+0000","last_active":"2024-09-17T15:24:47.637664+0000","last_peered":"2024-09-17T15:24:47.637664+0000","last_clean":"2024-09-17T15:24:47.637664+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T15:24:51.806440+0000","last_undegraded":"2024-09-17T15:24:51.806440+0000","last_fullsized":"2024-09-17T15:24:51.806440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-17T15:24:51.807016+0000","last_change":"2024-09-17T15:24:47.788403+0000","last_active":"2024-09-17T15:24:51.807016+0000","last_peered":"2024-09-17T15:24:51.807016+0000","last_clean":"2024-09-17T15:24:51.807016+0000","last_became_active":"2024-09-17T15:24:47.788135+0000","last_became_peered":"2024-09-17T15:24:47.788135+0000","last_unstale":"2024-09-17T15:24:51.807016+0000","last_undegraded":"2024-09-17T15:24:51.807016+0000","last_fullsized":"2024-09-17T15:24:51.807016+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:21:49.137928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":30,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-17T15:24:51.806757+0000","last_change":"2024-09-17T15:24:41.635548+0000","last_active":"2024-09-17T15:24:51.806757+0000","last_peered":"2024-09-17T15:24:51.806757+0000","last_clean":"2024-09-17T15:24:51.806757+0000","last_became_active":"2024-09-17T15:24:39.625795+0000","last_became_peered":"2024-09-17T15:24:39.625795+0000","last_unstale":"2024-09-17T15:24:51.806757+0000","last_undegraded":"2024-09-17T15:24:51.806757+0000","last_fullsized":"2024-09-17T15:24:51.806757+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:33:20.862791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-17T15:24:51.806765+0000","last_change":"2024-09-17T15:24:47.788932+0000","last_active":"2024-09-17T15:24:51.806765+0000","last_peered":"2024-09-17T15:24:51.806765+0000","last_clean":"2024-09-17T15:24:51.806765+0000","last_became_active":"2024-09-17T15:24:47.788675+0000","last_became_peered":"2024-09-17T15:24:47.788675+0000","last_unstale":"2024-09-17T15:24:51.806765+0000","last_undegraded":"2024-09-17T15:24:51.806765+0000","last_fullsized":"2024-09-17T15:24:51.806765+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:12:21.402742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":4,"reported_epoch":18,"state":"peering","last_fresh":"2024-09-17T15:24:51.805692+0000","last_change":"2024-09-17T15:24:50.795772+0000","last_active":"2024-09-17T15:24:38.602774+0000","last_peered":"2024-09-17T15:24:38.602774+0000","last_clean":"2024-09-17T15:24:38.602774+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T15:24:51.805692+0000","last_undegraded":"2024-09-17T15:24:51.805692+0000","last_fullsized":"2024-09-17T15:24:51.805692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-17T15:24:51.806950+0000","last_change":"2024-09-17T15:24:47.789254+0000","last_active":"2024-09-17T15:24:51.806950+0000","last_peered":"2024-09-17T15:24:51.806950+0000","last_clean":"2024-09-17T15:24:51.806950+0000","last_became_active":"2024-09-17T15:24:47.788851+0000","last_became_peered":"2024-09-17T15:24:47.788851+0000","last_unstale":"2024-09-17T15:24:51.806950+0000","last_undegraded":"2024-09-17T15:24:51.806950+0000","last_fullsized":"2024-09-17T15:24:51.806950+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:44:17.715811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":30,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-17T15:24:51.806859+0000","last_change":"2024-09-17T15:24:41.635690+0000","last_active":"2024-09-17T15:24:51.806859+0000","last_peered":"2024-09-17T15:24:51.806859+0000","last_clean":"2024-09-17T15:24:51.806859+0000","last_became_active":"2024-09-17T15:24:39.625082+0000","last_became_peered":"2024-09-17T15:24:39.625082+0000","last_unstale":"2024-09-17T15:24:51.806859+0000","last_undegraded":"2024-09-17T15:24:51.806859+0000","last_fullsized":"2024-09-17T15:24:51.806859+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:51:35.823375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"peering","last_fresh":"2024-09-17T15:24:51.220302+0000","last_change":"2024-09-17T15:24:51.220302+0000","last_active":"2024-09-17T15:24:51.220302+0000","last_peered":"2024-09-17T15:24:48.780356+0000","last_clean":"2024-09-17T15:24:48.780356+0000","last_became_active":"2024-09-17T15:24:39.618458+0000","last_became_peered":"2024-09-17T15:24:39.618458+0000","last_unstale":"2024-09-17T15:24:51.220302+0000","last_undegraded":"2024-09-17T15:24:51.220302+0000","last_fullsized":"2024-09-17T15:24:51.220302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"18'1","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-17T15:24:51.806882+0000","last_change":"2024-09-17T15:24:47.788133+0000","last_active":"2024-09-17T15:24:51.806882+0000","last_peered":"2024-09-17T15:24:51.806882+0000","last_clean":"2024-09-17T15:24:51.806882+0000","last_became_active":"2024-09-17T15:24:47.787836+0000","last_became_peered":"2024-09-17T15:24:47.787836+0000","last_unstale":"2024-09-17T15:24:51.806882+0000","last_undegraded":"2024-09-17T15:24:51.806882+0000","last_fullsized":"2024-09-17T15:24:51.806882+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:53:44.379421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738375,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":163840,"data_stored":41988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738375,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27180,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":151552,"data_stored":38733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:24:55.369 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:24:55.650 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:24:55.651 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:24:55.663 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-09-17T15:24:55.574538+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":42,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54404,"kb_used_data":336,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689276,"statfs":{"total":193273528320,"available":193217818624,"internally_reserved":0,"allocated":344064,"data_stored":89404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3190,"internal_metadata":55309194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001087"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T15:24:52.802973+0000","last_change":"2024-09-17T15:24:47.788372+0000","last_active":"2024-09-17T15:24:52.802973+0000","last_peered":"2024-09-17T15:24:52.802973+0000","last_clean":"2024-09-17T15:24:52.802973+0000","last_became_active":"2024-09-17T15:24:47.788133+0000","last_became_peered":"2024-09-17T15:24:47.788133+0000","last_unstale":"2024-09-17T15:24:52.802973+0000","last_undegraded":"2024-09-17T15:24:52.802973+0000","last_fullsized":"2024-09-17T15:24:52.802973+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:01:11.136214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T15:24:52.807065+0000","last_change":"2024-09-17T15:24:52.806937+0000","last_active":"2024-09-17T15:24:52.807065+0000","last_peered":"2024-09-17T15:24:52.807065+0000","last_clean":"2024-09-17T15:24:52.807065+0000","last_became_active":"2024-09-17T15:24:52.806648+0000","last_became_peered":"2024-09-17T15:24:52.806648+0000","last_unstale":"2024-09-17T15:24:52.807065+0000","last_undegraded":"2024-09-17T15:24:52.807065+0000","last_fullsized":"2024-09-17T15:24:52.807065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:02:24.789157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":24,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T15:24:52.809735+0000","last_change":"2024-09-17T15:24:52.809683+0000","last_active":"2024-09-17T15:24:52.809735+0000","last_peered":"2024-09-17T15:24:52.809735+0000","last_clean":"2024-09-17T15:24:52.809735+0000","last_became_active":"2024-09-17T15:24:52.809439+0000","last_became_peered":"2024-09-17T15:24:52.809439+0000","last_unstale":"2024-09-17T15:24:52.809735+0000","last_undegraded":"2024-09-17T15:24:52.809735+0000","last_fullsized":"2024-09-17T15:24:52.809735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:19:33.600790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T15:24:52.806994+0000","last_change":"2024-09-17T15:24:52.806891+0000","last_active":"2024-09-17T15:24:52.806994+0000","last_peered":"2024-09-17T15:24:52.806994+0000","last_clean":"2024-09-17T15:24:52.806994+0000","last_became_active":"2024-09-17T15:24:52.806588+0000","last_became_peered":"2024-09-17T15:24:52.806588+0000","last_unstale":"2024-09-17T15:24:52.806994+0000","last_undegraded":"2024-09-17T15:24:52.806994+0000","last_fullsized":"2024-09-17T15:24:52.806994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:25:17.682564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":24,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T15:24:52.807039+0000","last_change":"2024-09-17T15:24:52.806941+0000","last_active":"2024-09-17T15:24:52.807039+0000","last_peered":"2024-09-17T15:24:52.807039+0000","last_clean":"2024-09-17T15:24:52.807039+0000","last_became_active":"2024-09-17T15:24:52.806638+0000","last_became_peered":"2024-09-17T15:24:52.806638+0000","last_unstale":"2024-09-17T15:24:52.807039+0000","last_undegraded":"2024-09-17T15:24:52.807039+0000","last_fullsized":"2024-09-17T15:24:52.807039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:46:53.025644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T15:24:52.804409+0000","last_change":"2024-09-17T15:24:47.787187+0000","last_active":"2024-09-17T15:24:52.804409+0000","last_peered":"2024-09-17T15:24:52.804409+0000","last_clean":"2024-09-17T15:24:52.804409+0000","last_became_active":"2024-09-17T15:24:47.786871+0000","last_became_peered":"2024-09-17T15:24:47.786871+0000","last_unstale":"2024-09-17T15:24:52.804409+0000","last_undegraded":"2024-09-17T15:24:52.804409+0000","last_fullsized":"2024-09-17T15:24:52.804409+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:14:42.196510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T15:24:52.804424+0000","last_change":"2024-09-17T15:24:47.787090+0000","last_active":"2024-09-17T15:24:52.804424+0000","last_peered":"2024-09-17T15:24:52.804424+0000","last_clean":"2024-09-17T15:24:52.804424+0000","last_became_active":"2024-09-17T15:24:47.786830+0000","last_became_peered":"2024-09-17T15:24:47.786830+0000","last_unstale":"2024-09-17T15:24:52.804424+0000","last_undegraded":"2024-09-17T15:24:52.804424+0000","last_fullsized":"2024-09-17T15:24:52.804424+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:05:02.468465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":24,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T15:24:52.807724+0000","last_change":"2024-09-17T15:24:52.807635+0000","last_active":"2024-09-17T15:24:52.807724+0000","last_peered":"2024-09-17T15:24:52.807724+0000","last_clean":"2024-09-17T15:24:52.807724+0000","last_became_active":"2024-09-17T15:24:52.807322+0000","last_became_peered":"2024-09-17T15:24:52.807322+0000","last_unstale":"2024-09-17T15:24:52.807724+0000","last_undegraded":"2024-09-17T15:24:52.807724+0000","last_fullsized":"2024-09-17T15:24:52.807724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:24:33.490703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":32,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T15:24:52.804323+0000","last_change":"2024-09-17T15:24:41.635178+0000","last_active":"2024-09-17T15:24:52.804323+0000","last_peered":"2024-09-17T15:24:52.804323+0000","last_clean":"2024-09-17T15:24:52.804323+0000","last_became_active":"2024-09-17T15:24:39.625416+0000","last_became_peered":"2024-09-17T15:24:39.625416+0000","last_unstale":"2024-09-17T15:24:52.804323+0000","last_undegraded":"2024-09-17T15:24:52.804323+0000","last_fullsized":"2024-09-17T15:24:52.804323+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:58:18.531873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T15:24:52.812208+0000","last_change":"2024-09-17T15:24:52.812137+0000","last_active":"2024-09-17T15:24:52.812208+0000","last_peered":"2024-09-17T15:24:52.812208+0000","last_clean":"2024-09-17T15:24:52.812208+0000","last_became_active":"2024-09-17T15:24:52.811901+0000","last_became_peered":"2024-09-17T15:24:52.811901+0000","last_unstale":"2024-09-17T15:24:52.812208+0000","last_undegraded":"2024-09-17T15:24:52.812208+0000","last_fullsized":"2024-09-17T15:24:52.812208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:19:03.117972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T15:24:52.807466+0000","last_change":"2024-09-17T15:24:52.807364+0000","last_active":"2024-09-17T15:24:52.807466+0000","last_peered":"2024-09-17T15:24:52.807466+0000","last_clean":"2024-09-17T15:24:52.807466+0000","last_became_active":"2024-09-17T15:24:52.807065+0000","last_became_peered":"2024-09-17T15:24:52.807065+0000","last_unstale":"2024-09-17T15:24:52.807466+0000","last_undegraded":"2024-09-17T15:24:52.807466+0000","last_fullsized":"2024-09-17T15:24:52.807466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:52:00.850761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T15:24:52.802077+0000","last_change":"2024-09-17T15:24:47.786083+0000","last_active":"2024-09-17T15:24:52.802077+0000","last_peered":"2024-09-17T15:24:52.802077+0000","last_clean":"2024-09-17T15:24:52.802077+0000","last_became_active":"2024-09-17T15:24:47.785813+0000","last_became_peered":"2024-09-17T15:24:47.785813+0000","last_unstale":"2024-09-17T15:24:52.802077+0000","last_undegraded":"2024-09-17T15:24:52.802077+0000","last_fullsized":"2024-09-17T15:24:52.802077+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:07:13.003526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":34,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T15:24:52.802152+0000","last_change":"2024-09-17T15:24:41.632020+0000","last_active":"2024-09-17T15:24:52.802152+0000","last_peered":"2024-09-17T15:24:52.802152+0000","last_clean":"2024-09-17T15:24:52.802152+0000","last_became_active":"2024-09-17T15:24:39.625863+0000","last_became_peered":"2024-09-17T15:24:39.625863+0000","last_unstale":"2024-09-17T15:24:52.802152+0000","last_undegraded":"2024-09-17T15:24:52.802152+0000","last_fullsized":"2024-09-17T15:24:52.802152+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:59:34.094507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T15:24:52.809908+0000","last_change":"2024-09-17T15:24:52.809819+0000","last_active":"2024-09-17T15:24:52.809908+0000","last_peered":"2024-09-17T15:24:52.809908+0000","last_clean":"2024-09-17T15:24:52.809908+0000","last_became_active":"2024-09-17T15:24:52.809531+0000","last_became_peered":"2024-09-17T15:24:52.809531+0000","last_unstale":"2024-09-17T15:24:52.809908+0000","last_undegraded":"2024-09-17T15:24:52.809908+0000","last_fullsized":"2024-09-17T15:24:52.809908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:39:57.430321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":42,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T15:24:52.817495+0000","last_change":"2024-09-17T15:24:52.817403+0000","last_active":"2024-09-17T15:24:52.817495+0000","last_peered":"2024-09-17T15:24:52.817495+0000","last_clean":"2024-09-17T15:24:52.817495+0000","last_became_active":"2024-09-17T15:24:52.814394+0000","last_became_peered":"2024-09-17T15:24:52.814394+0000","last_unstale":"2024-09-17T15:24:52.817495+0000","last_undegraded":"2024-09-17T15:24:52.817495+0000","last_fullsized":"2024-09-17T15:24:52.817495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:09:18.730709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T15:24:52.809503+0000","last_change":"2024-09-17T15:24:52.809407+0000","last_active":"2024-09-17T15:24:52.809503+0000","last_peered":"2024-09-17T15:24:52.809503+0000","last_clean":"2024-09-17T15:24:52.809503+0000","last_became_active":"2024-09-17T15:24:52.808996+0000","last_became_peered":"2024-09-17T15:24:52.808996+0000","last_unstale":"2024-09-17T15:24:52.809503+0000","last_undegraded":"2024-09-17T15:24:52.809503+0000","last_fullsized":"2024-09-17T15:24:52.809503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:44:03.087308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T15:24:52.804684+0000","last_change":"2024-09-17T15:24:47.788403+0000","last_active":"2024-09-17T15:24:52.804684+0000","last_peered":"2024-09-17T15:24:52.804684+0000","last_clean":"2024-09-17T15:24:52.804684+0000","last_became_active":"2024-09-17T15:24:47.788135+0000","last_became_peered":"2024-09-17T15:24:47.788135+0000","last_unstale":"2024-09-17T15:24:52.804684+0000","last_undegraded":"2024-09-17T15:24:52.804684+0000","last_fullsized":"2024-09-17T15:24:52.804684+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:21:49.137928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":32,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T15:24:52.804500+0000","last_change":"2024-09-17T15:24:41.635548+0000","last_active":"2024-09-17T15:24:52.804500+0000","last_peered":"2024-09-17T15:24:52.804500+0000","last_clean":"2024-09-17T15:24:52.804500+0000","last_became_active":"2024-09-17T15:24:39.625795+0000","last_became_peered":"2024-09-17T15:24:39.625795+0000","last_unstale":"2024-09-17T15:24:52.804500+0000","last_undegraded":"2024-09-17T15:24:52.804500+0000","last_fullsized":"2024-09-17T15:24:52.804500+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:33:20.862791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T15:24:52.804599+0000","last_change":"2024-09-17T15:24:47.788932+0000","last_active":"2024-09-17T15:24:52.804599+0000","last_peered":"2024-09-17T15:24:52.804599+0000","last_clean":"2024-09-17T15:24:52.804599+0000","last_became_active":"2024-09-17T15:24:47.788675+0000","last_became_peered":"2024-09-17T15:24:47.788675+0000","last_unstale":"2024-09-17T15:24:52.804599+0000","last_undegraded":"2024-09-17T15:24:52.804599+0000","last_fullsized":"2024-09-17T15:24:52.804599+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:12:21.402742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T15:24:52.807423+0000","last_change":"2024-09-17T15:24:52.807322+0000","last_active":"2024-09-17T15:24:52.807423+0000","last_peered":"2024-09-17T15:24:52.807423+0000","last_clean":"2024-09-17T15:24:52.807423+0000","last_became_active":"2024-09-17T15:24:52.807036+0000","last_became_peered":"2024-09-17T15:24:52.807036+0000","last_unstale":"2024-09-17T15:24:52.807423+0000","last_undegraded":"2024-09-17T15:24:52.807423+0000","last_fullsized":"2024-09-17T15:24:52.807423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:43:14.530094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T15:24:52.804609+0000","last_change":"2024-09-17T15:24:47.789254+0000","last_active":"2024-09-17T15:24:52.804609+0000","last_peered":"2024-09-17T15:24:52.804609+0000","last_clean":"2024-09-17T15:24:52.804609+0000","last_became_active":"2024-09-17T15:24:47.788851+0000","last_became_peered":"2024-09-17T15:24:47.788851+0000","last_unstale":"2024-09-17T15:24:52.804609+0000","last_undegraded":"2024-09-17T15:24:52.804609+0000","last_fullsized":"2024-09-17T15:24:52.804609+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:44:17.715811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":32,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T15:24:52.804611+0000","last_change":"2024-09-17T15:24:41.635690+0000","last_active":"2024-09-17T15:24:52.804611+0000","last_peered":"2024-09-17T15:24:52.804611+0000","last_clean":"2024-09-17T15:24:52.804611+0000","last_became_active":"2024-09-17T15:24:39.625082+0000","last_became_peered":"2024-09-17T15:24:39.625082+0000","last_unstale":"2024-09-17T15:24:52.804611+0000","last_undegraded":"2024-09-17T15:24:52.804611+0000","last_fullsized":"2024-09-17T15:24:52.804611+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:51:35.823375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":37,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T15:24:52.809800+0000","last_change":"2024-09-17T15:24:52.809708+0000","last_active":"2024-09-17T15:24:52.809800+0000","last_peered":"2024-09-17T15:24:52.809800+0000","last_clean":"2024-09-17T15:24:52.809800+0000","last_became_active":"2024-09-17T15:24:52.809417+0000","last_became_peered":"2024-09-17T15:24:52.809417+0000","last_unstale":"2024-09-17T15:24:52.809800+0000","last_undegraded":"2024-09-17T15:24:52.809800+0000","last_fullsized":"2024-09-17T15:24:52.809800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:30:00.529543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"18'1","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T15:24:52.804473+0000","last_change":"2024-09-17T15:24:47.788133+0000","last_active":"2024-09-17T15:24:52.804473+0000","last_peered":"2024-09-17T15:24:52.804473+0000","last_clean":"2024-09-17T15:24:52.804473+0000","last_became_active":"2024-09-17T15:24:47.787836+0000","last_became_peered":"2024-09-17T15:24:47.787836+0000","last_unstale":"2024-09-17T15:24:52.804473+0000","last_undegraded":"2024-09-17T15:24:52.804473+0000","last_fullsized":"2024-09-17T15:24:52.804473+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:53:44.379421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738376,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":172032,"data_stored":44702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738376,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":172032,"data_stored":44702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:24:55.664 INFO:tasks.lost_unfound.ceph_manager:recovered! 2024-09-17T15:24:55.664 INFO:tasks.util.rados:rados -p unfound_pool put existing_1 /etc/fstab 2024-09-17T15:24:55.664 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_1 /etc/fstab 2024-09-17T15:24:55.715 INFO:tasks.util.rados:rados -p unfound_pool put existed_1 /etc/fstab 2024-09-17T15:24:55.715 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_1 /etc/fstab 2024-09-17T15:24:55.764 INFO:tasks.util.rados:rados -p unfound_pool rm existed_1 2024-09-17T15:24:55.764 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool rm existed_1 2024-09-17T15:24:55.816 INFO:tasks.util.rados:rados -p unfound_pool put existing_2 /etc/fstab 2024-09-17T15:24:55.816 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_2 /etc/fstab 2024-09-17T15:24:55.865 INFO:tasks.util.rados:rados -p unfound_pool put existed_2 /etc/fstab 2024-09-17T15:24:55.865 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_2 /etc/fstab 2024-09-17T15:24:55.915 INFO:tasks.util.rados:rados -p unfound_pool rm existed_2 2024-09-17T15:24:55.916 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool rm existed_2 2024-09-17T15:24:55.964 INFO:tasks.util.rados:rados -p unfound_pool put existing_3 /etc/fstab 2024-09-17T15:24:55.964 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_3 /etc/fstab 2024-09-17T15:24:56.014 INFO:tasks.util.rados:rados -p unfound_pool put existed_3 /etc/fstab 2024-09-17T15:24:56.014 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_3 /etc/fstab 2024-09-17T15:24:56.064 INFO:tasks.util.rados:rados -p unfound_pool rm existed_3 2024-09-17T15:24:56.064 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool rm existed_3 2024-09-17T15:24:56.113 INFO:tasks.util.rados:rados -p unfound_pool put existing_4 /etc/fstab 2024-09-17T15:24:56.113 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_4 /etc/fstab 2024-09-17T15:24:56.163 INFO:tasks.util.rados:rados -p unfound_pool put existed_4 /etc/fstab 2024-09-17T15:24:56.163 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_4 /etc/fstab 2024-09-17T15:24:56.213 INFO:tasks.util.rados:rados -p unfound_pool rm existed_4 2024-09-17T15:24:56.213 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool rm existed_4 2024-09-17T15:24:56.262 INFO:tasks.util.rados:rados -p unfound_pool put existing_5 /etc/fstab 2024-09-17T15:24:56.262 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_5 /etc/fstab 2024-09-17T15:24:56.312 INFO:tasks.util.rados:rados -p unfound_pool put existed_5 /etc/fstab 2024-09-17T15:24:56.313 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_5 /etc/fstab 2024-09-17T15:24:56.363 INFO:tasks.util.rados:rados -p unfound_pool rm existed_5 2024-09-17T15:24:56.363 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool rm existed_5 2024-09-17T15:24:56.412 INFO:tasks.util.rados:rados -p unfound_pool put existing_6 /etc/fstab 2024-09-17T15:24:56.412 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_6 /etc/fstab 2024-09-17T15:24:56.461 INFO:tasks.util.rados:rados -p unfound_pool put existed_6 /etc/fstab 2024-09-17T15:24:56.462 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_6 /etc/fstab 2024-09-17T15:24:56.512 INFO:tasks.util.rados:rados -p unfound_pool rm existed_6 2024-09-17T15:24:56.513 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool rm existed_6 2024-09-17T15:24:56.561 INFO:tasks.util.rados:rados -p unfound_pool put existing_7 /etc/fstab 2024-09-17T15:24:56.561 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_7 /etc/fstab 2024-09-17T15:24:56.611 INFO:tasks.util.rados:rados -p unfound_pool put existed_7 /etc/fstab 2024-09-17T15:24:56.611 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_7 /etc/fstab 2024-09-17T15:24:56.661 INFO:tasks.util.rados:rados -p unfound_pool rm existed_7 2024-09-17T15:24:56.662 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool rm existed_7 2024-09-17T15:24:56.711 INFO:tasks.util.rados:rados -p unfound_pool put existing_8 /etc/fstab 2024-09-17T15:24:56.711 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_8 /etc/fstab 2024-09-17T15:24:56.761 INFO:tasks.util.rados:rados -p unfound_pool put existed_8 /etc/fstab 2024-09-17T15:24:56.761 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_8 /etc/fstab 2024-09-17T15:24:56.811 INFO:tasks.util.rados:rados -p unfound_pool rm existed_8 2024-09-17T15:24:56.849 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool rm existed_8 2024-09-17T15:24:56.899 INFO:tasks.util.rados:rados -p unfound_pool put existing_9 /etc/fstab 2024-09-17T15:24:56.899 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_9 /etc/fstab 2024-09-17T15:24:56.949 INFO:tasks.util.rados:rados -p unfound_pool put existed_9 /etc/fstab 2024-09-17T15:24:56.949 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_9 /etc/fstab 2024-09-17T15:24:56.999 INFO:tasks.util.rados:rados -p unfound_pool rm existed_9 2024-09-17T15:24:57.000 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool rm existed_9 2024-09-17T15:24:57.048 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 injectargs '--osd-recovery-delay-start 1000 --osd-min-pg-log-entries 100000000' 2024-09-17T15:24:57.167 INFO:teuthology.orchestra.run.smithi083.stdout:{} 2024-09-17T15:24:57.167 INFO:teuthology.orchestra.run.smithi083.stderr:osd_min_pg_log_entries = '' (not observed, change may require restart) osd_recovery_delay_start = '' 2024-09-17T15:24:57.177 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-17T15:24:57.178 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T15:24:57.210 INFO:tasks.ceph.osd.0:Stopped 2024-09-17T15:24:57.210 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 0 2024-09-17T15:24:57.545 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-09-17T15:24:57.544+0000 7f514a174640 -1 mon.a@0(leader).osd e19 definitely_dead 0 2024-09-17T15:24:57.819 INFO:teuthology.orchestra.run.smithi083.stderr:marked down osd.0. 2024-09-17T15:24:57.832 INFO:tasks.util.rados:rados -p unfound_pool put new_1 /etc/fstab 2024-09-17T15:24:57.833 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put new_1 /etc/fstab 2024-09-17T15:24:58.833 INFO:tasks.util.rados:rados -p unfound_pool put existed_1 /etc/fstab 2024-09-17T15:24:58.833 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_1 /etc/fstab 2024-09-17T15:24:58.887 INFO:tasks.util.rados:rados -p unfound_pool put existing_1 /etc/fstab 2024-09-17T15:24:58.887 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_1 /etc/fstab 2024-09-17T15:24:58.938 INFO:tasks.util.rados:rados -p unfound_pool put new_2 /etc/fstab 2024-09-17T15:24:58.938 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put new_2 /etc/fstab 2024-09-17T15:24:58.987 INFO:tasks.util.rados:rados -p unfound_pool put existed_2 /etc/fstab 2024-09-17T15:24:58.988 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_2 /etc/fstab 2024-09-17T15:24:59.037 INFO:tasks.util.rados:rados -p unfound_pool put existing_2 /etc/fstab 2024-09-17T15:24:59.038 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_2 /etc/fstab 2024-09-17T15:24:59.087 INFO:tasks.util.rados:rados -p unfound_pool put new_3 /etc/fstab 2024-09-17T15:24:59.087 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put new_3 /etc/fstab 2024-09-17T15:24:59.137 INFO:tasks.util.rados:rados -p unfound_pool put existed_3 /etc/fstab 2024-09-17T15:24:59.137 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_3 /etc/fstab 2024-09-17T15:24:59.186 INFO:tasks.util.rados:rados -p unfound_pool put existing_3 /etc/fstab 2024-09-17T15:24:59.186 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_3 /etc/fstab 2024-09-17T15:24:59.236 INFO:tasks.util.rados:rados -p unfound_pool put new_4 /etc/fstab 2024-09-17T15:24:59.236 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put new_4 /etc/fstab 2024-09-17T15:24:59.286 INFO:tasks.util.rados:rados -p unfound_pool put existed_4 /etc/fstab 2024-09-17T15:24:59.286 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_4 /etc/fstab 2024-09-17T15:24:59.335 INFO:tasks.util.rados:rados -p unfound_pool put existing_4 /etc/fstab 2024-09-17T15:24:59.335 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_4 /etc/fstab 2024-09-17T15:24:59.385 INFO:tasks.util.rados:rados -p unfound_pool put new_5 /etc/fstab 2024-09-17T15:24:59.385 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put new_5 /etc/fstab 2024-09-17T15:24:59.435 INFO:tasks.util.rados:rados -p unfound_pool put existed_5 /etc/fstab 2024-09-17T15:24:59.435 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_5 /etc/fstab 2024-09-17T15:24:59.484 INFO:tasks.util.rados:rados -p unfound_pool put existing_5 /etc/fstab 2024-09-17T15:24:59.485 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_5 /etc/fstab 2024-09-17T15:24:59.534 INFO:tasks.util.rados:rados -p unfound_pool put new_6 /etc/fstab 2024-09-17T15:24:59.534 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put new_6 /etc/fstab 2024-09-17T15:24:59.584 INFO:tasks.util.rados:rados -p unfound_pool put existed_6 /etc/fstab 2024-09-17T15:24:59.584 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_6 /etc/fstab 2024-09-17T15:24:59.634 INFO:tasks.util.rados:rados -p unfound_pool put existing_6 /etc/fstab 2024-09-17T15:24:59.634 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_6 /etc/fstab 2024-09-17T15:24:59.684 INFO:tasks.util.rados:rados -p unfound_pool put new_7 /etc/fstab 2024-09-17T15:24:59.684 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put new_7 /etc/fstab 2024-09-17T15:24:59.734 INFO:tasks.util.rados:rados -p unfound_pool put existed_7 /etc/fstab 2024-09-17T15:24:59.735 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_7 /etc/fstab 2024-09-17T15:24:59.784 INFO:tasks.util.rados:rados -p unfound_pool put existing_7 /etc/fstab 2024-09-17T15:24:59.784 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_7 /etc/fstab 2024-09-17T15:24:59.836 INFO:tasks.util.rados:rados -p unfound_pool put new_8 /etc/fstab 2024-09-17T15:24:59.837 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put new_8 /etc/fstab 2024-09-17T15:24:59.886 INFO:tasks.util.rados:rados -p unfound_pool put existed_8 /etc/fstab 2024-09-17T15:24:59.887 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_8 /etc/fstab 2024-09-17T15:24:59.936 INFO:tasks.util.rados:rados -p unfound_pool put existing_8 /etc/fstab 2024-09-17T15:24:59.936 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_8 /etc/fstab 2024-09-17T15:24:59.986 INFO:tasks.util.rados:rados -p unfound_pool put new_9 /etc/fstab 2024-09-17T15:24:59.986 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put new_9 /etc/fstab 2024-09-17T15:25:00.036 INFO:tasks.util.rados:rados -p unfound_pool put existed_9 /etc/fstab 2024-09-17T15:25:00.036 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_9 /etc/fstab 2024-09-17T15:25:00.086 INFO:tasks.util.rados:rados -p unfound_pool put existing_9 /etc/fstab 2024-09-17T15:25:00.086 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_9 /etc/fstab 2024-09-17T15:25:00.135 INFO:tasks.lost_unfound:osd.0 command_args is foo 2024-09-17T15:25:00.136 INFO:tasks.lost_unfound:() 2024-09-17T15:25:00.136 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-17T15:25:00.136 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 --osd-recovery-delay-start 1000 2024-09-17T15:25:00.138 INFO:tasks.ceph.osd.0:Started 2024-09-17T15:25:00.139 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T15:25:00.236 INFO:teuthology.orchestra.run.smithi083.stderr:no valid command found; 10 closest matches: 2024-09-17T15:25:00.236 INFO:teuthology.orchestra.run.smithi083.stderr:0 2024-09-17T15:25:00.236 INFO:teuthology.orchestra.run.smithi083.stderr:1 2024-09-17T15:25:00.236 INFO:teuthology.orchestra.run.smithi083.stderr:2 2024-09-17T15:25:00.236 INFO:teuthology.orchestra.run.smithi083.stderr:abort 2024-09-17T15:25:00.237 INFO:teuthology.orchestra.run.smithi083.stderr:assert 2024-09-17T15:25:00.237 INFO:teuthology.orchestra.run.smithi083.stderr:config diff 2024-09-17T15:25:00.237 INFO:teuthology.orchestra.run.smithi083.stderr:config diff get 2024-09-17T15:25:00.237 INFO:teuthology.orchestra.run.smithi083.stderr:config get 2024-09-17T15:25:00.237 INFO:teuthology.orchestra.run.smithi083.stderr:config help [] 2024-09-17T15:25:00.237 INFO:teuthology.orchestra.run.smithi083.stderr:config set ... 2024-09-17T15:25:00.237 INFO:teuthology.orchestra.run.smithi083.stderr:admin_socket: invalid command 2024-09-17T15:25:00.239 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T15:25:00.239 INFO:tasks.lost_unfound.ceph_manager:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-17T15:25:01.053 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:01.052+0000 7fa938cef640 -1 Falling back to public interface 2024-09-17T15:25:04.573 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:04.572+0000 7fa938cef640 -1 osd.0 19 log_to_monitors true 2024-09-17T15:25:05.240 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T15:25:05.351 INFO:teuthology.orchestra.run.smithi083.stderr:no valid command found; 10 closest matches: 2024-09-17T15:25:05.351 INFO:teuthology.orchestra.run.smithi083.stderr:0 2024-09-17T15:25:05.351 INFO:teuthology.orchestra.run.smithi083.stderr:1 2024-09-17T15:25:05.351 INFO:teuthology.orchestra.run.smithi083.stderr:2 2024-09-17T15:25:05.352 INFO:teuthology.orchestra.run.smithi083.stderr:abort 2024-09-17T15:25:05.352 INFO:teuthology.orchestra.run.smithi083.stderr:assert 2024-09-17T15:25:05.352 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs debug_inject_read_zeros 2024-09-17T15:25:05.352 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs files list 2024-09-17T15:25:05.352 INFO:teuthology.orchestra.run.smithi083.stderr:bluefs stats 2024-09-17T15:25:05.352 INFO:teuthology.orchestra.run.smithi083.stderr:bluestore allocator dump block 2024-09-17T15:25:05.352 INFO:teuthology.orchestra.run.smithi083.stderr:bluestore allocator fragmentation block 2024-09-17T15:25:05.352 INFO:teuthology.orchestra.run.smithi083.stderr:admin_socket: invalid command 2024-09-17T15:25:05.354 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T15:25:05.354 INFO:tasks.lost_unfound.ceph_manager:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-17T15:25:05.421 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:05.420+0000 7fa92ca9d640 -1 osd.0 19 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T15:25:10.355 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T15:25:10.470 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T15:25:10.470 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T15:25:10.470 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T15:25:10.470 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T15:25:10.480 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 0 2024-09-17T15:25:10.774 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-09-17T15:25:10.774+0000 7f514a174640 -1 mon.a@0(leader).osd e24 definitely_dead 0 2024-09-17T15:25:10.775 INFO:teuthology.orchestra.run.smithi083.stderr:osd.0 is already in. 2024-09-17T15:25:10.787 INFO:tasks.lost_unfound.ceph_manager:waiting for osd.0 to be up 2024-09-17T15:25:10.787 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T15:25:11.069 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:25:11.069 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","created":"2024-09-17T15:24:23.007214+0000","modified":"2024-09-17T15:25:07.426886+0000","last_up_change":"2024-09-17T15:25:06.423560+0000","last_in_change":"2024-09-17T15:24:51.791044+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-17T15:24:38.423278+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":13,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8700000047683716,"score_stable":1.8700000047683716,"optimal_score":0.67000001668930054,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfound_pool","create_time":"2024-09-17T15:24:47.449679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8700000047683716,"score_stable":1.8700000047683716,"optimal_score":0.67000001668930054,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9defe403-224a-4a6c-9184-dcb72bef6bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":19,"up_from":23,"up_thru":23,"down_at":20,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":1704571166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6805","nonce":1704571166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6807","nonce":1704571166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":1704571166}]},"public_addr":"172.21.15.83:6804/1704571166","cluster_addr":"172.21.15.83:6805/1704571166","heartbeat_back_addr":"172.21.15.83:6807/1704571166","heartbeat_front_addr":"172.21.15.83:6806/1704571166","state":["exists","up"]},{"osd":1,"uuid":"d7d78e01-f761-444b-98c4-62587b991738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":1258824819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6801","nonce":1258824819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6803","nonce":1258824819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":1258824819}]},"public_addr":"172.21.15.83:6800/1258824819","cluster_addr":"172.21.15.83:6801/1258824819","heartbeat_back_addr":"172.21.15.83:6803/1258824819","heartbeat_front_addr":"172.21.15.83:6802/1258824819","state":["exists","up"]},{"osd":2,"uuid":"23c2275c-8731-45f3-97f9-5100b6676f99","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":14,"down_at":17,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":239948269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6805","nonce":239948269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6807","nonce":239948269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":239948269}]},"public_addr":"172.21.15.83:6804/239948269","cluster_addr":"172.21.15.83:6805/239948269","heartbeat_back_addr":"172.21.15.83:6807/239948269","heartbeat_front_addr":"172.21.15.83:6806/239948269","state":["exists"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T15:24:31.547535+0000","dead_epoch":20},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T15:24:30.867578+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":65536,"last_purged_snaps_scrub":"2024-09-17T15:24:30.866429+0000","dead_epoch":17}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T15:25:11.081 INFO:tasks.lost_unfound.ceph_manager:osd.0 is up 2024-09-17T15:25:11.081 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-17T15:25:11.082 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-17T15:25:11.216 INFO:teuthology.orchestra.run.smithi083.stdout:34359738380 2024-09-17T15:25:11.216 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T15:25:11.217 INFO:teuthology.orchestra.run.smithi083.stdout:98784247811 2024-09-17T15:25:11.217 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T15:25:11.548 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-09-17T15:25:11.549 INFO:teuthology.orchestra.run.smithi083.stdout:34359738379 2024-09-17T15:25:11.561 INFO:tasks.lost_unfound.ceph_manager:need seq 98784247811 got 1 for osd.0 2024-09-17T15:25:11.562 INFO:tasks.lost_unfound.ceph_manager:need seq 34359738380 got 34359738379 for osd.1 2024-09-17T15:25:12.563 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T15:25:12.563 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T15:25:12.863 INFO:teuthology.orchestra.run.smithi083.stdout:34359738380 2024-09-17T15:25:12.864 INFO:teuthology.orchestra.run.smithi083.stdout:98784247811 2024-09-17T15:25:12.881 INFO:tasks.lost_unfound.ceph_manager:need seq 34359738380 got 34359738380 for osd.1 2024-09-17T15:25:12.881 DEBUG:teuthology.parallel:result is None 2024-09-17T15:25:12.882 INFO:tasks.lost_unfound.ceph_manager:need seq 98784247811 got 98784247811 for osd.0 2024-09-17T15:25:12.882 DEBUG:teuthology.parallel:result is None 2024-09-17T15:25:12.882 INFO:tasks.lost_unfound.ceph_manager:waiting till active 2024-09-17T15:25:12.882 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:25:13.160 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:25:13.161 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:25:13.172 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2024-09-17T15:25:11.577001+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":13543,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":7,"num_objects_missing":7,"num_objects_degraded":27,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58,"ondisk_log_size":58,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55076,"kb_used_data":560,"kb_used_omap":5,"kb_used_meta":54458,"kb_avail":188688604,"statfs":{"total":193273528320,"available":193217130496,"internally_reserved":0,"allocated":573440,"data_stored":137730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6035,"internal_metadata":55765101},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":7,"num_objects_missing":7,"num_objects_degraded":-3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.749688"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-17T15:25:07.441898+0000","last_change":"2024-09-17T15:25:07.441777+0000","last_active":"2024-09-17T15:25:07.441898+0000","last_peered":"2024-09-17T15:25:07.441898+0000","last_clean":"2024-09-17T15:25:07.441898+0000","last_became_active":"2024-09-17T15:25:07.441462+0000","last_became_peered":"2024-09-17T15:25:07.441462+0000","last_unstale":"2024-09-17T15:25:07.441898+0000","last_undegraded":"2024-09-17T15:25:07.441898+0000","last_fullsized":"2024-09-17T15:25:07.441898+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:53:05.846968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"21'4","reported_seq":24,"reported_epoch":24,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:07.441407+0000","last_change":"2024-09-17T15:25:07.441588+0000","last_active":"2024-09-17T15:25:07.441407+0000","last_peered":"2024-09-17T15:25:07.441407+0000","last_clean":"2024-09-17T15:24:56.161111+0000","last_became_active":"2024-09-17T15:25:07.441314+0000","last_became_peered":"2024-09-17T15:25:07.441314+0000","last_unstale":"2024-09-17T15:25:07.441407+0000","last_undegraded":"2024-09-17T15:25:07.434880+0000","last_fullsized":"2024-09-17T15:25:07.441407+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'3","reported_seq":44,"reported_epoch":24,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:07.441534+0000","last_change":"2024-09-17T15:25:07.441705+0000","last_active":"2024-09-17T15:25:07.441534+0000","last_peered":"2024-09-17T15:25:07.441534+0000","last_clean":"2024-09-17T15:24:56.708693+0000","last_became_active":"2024-09-17T15:25:07.441417+0000","last_became_peered":"2024-09-17T15:25:07.441417+0000","last_unstale":"2024-09-17T15:25:07.441534+0000","last_undegraded":"2024-09-17T15:25:07.431265+0000","last_fullsized":"2024-09-17T15:25:07.441534+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-17T15:25:07.441459+0000","last_change":"2024-09-17T15:25:07.441363+0000","last_active":"2024-09-17T15:25:07.441459+0000","last_peered":"2024-09-17T15:25:07.441459+0000","last_clean":"2024-09-17T15:25:07.441459+0000","last_became_active":"2024-09-17T15:25:07.441011+0000","last_became_peered":"2024-09-17T15:25:07.441011+0000","last_unstale":"2024-09-17T15:25:07.441459+0000","last_undegraded":"2024-09-17T15:25:07.441459+0000","last_fullsized":"2024-09-17T15:25:07.441459+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:10:08.279875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"21'5","reported_seq":37,"reported_epoch":24,"state":"active+recovering+degraded","last_fresh":"2024-09-17T15:25:07.439201+0000","last_change":"2024-09-17T15:25:07.439793+0000","last_active":"2024-09-17T15:25:07.439201+0000","last_peered":"2024-09-17T15:25:07.439201+0000","last_clean":"2024-09-17T15:24:56.061756+0000","last_became_active":"2024-09-17T15:25:07.438828+0000","last_became_peered":"2024-09-17T15:25:07.438828+0000","last_unstale":"2024-09-17T15:25:07.439201+0000","last_undegraded":"2024-09-17T15:25:07.433692+0000","last_fullsized":"2024-09-17T15:25:07.439201+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"21'3","reported_seq":39,"reported_epoch":24,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:07.441304+0000","last_change":"2024-09-17T15:25:07.441532+0000","last_active":"2024-09-17T15:25:07.441304+0000","last_peered":"2024-09-17T15:25:07.441304+0000","last_clean":"2024-09-17T15:24:56.947098+0000","last_became_active":"2024-09-17T15:25:07.441201+0000","last_became_peered":"2024-09-17T15:25:07.441201+0000","last_unstale":"2024-09-17T15:25:07.441304+0000","last_undegraded":"2024-09-17T15:25:07.431230+0000","last_fullsized":"2024-09-17T15:25:07.441304+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"21'4","reported_seq":41,"reported_epoch":24,"state":"active+recovering+degraded","last_fresh":"2024-09-17T15:25:07.438919+0000","last_change":"2024-09-17T15:25:07.439785+0000","last_active":"2024-09-17T15:25:07.438919+0000","last_peered":"2024-09-17T15:25:07.438919+0000","last_clean":"2024-09-17T15:24:56.310518+0000","last_became_active":"2024-09-17T15:25:07.438632+0000","last_became_peered":"2024-09-17T15:25:07.438632+0000","last_unstale":"2024-09-17T15:25:07.438919+0000","last_undegraded":"2024-09-17T15:25:07.431418+0000","last_fullsized":"2024-09-17T15:25:07.438919+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"21'1","reported_seq":33,"reported_epoch":24,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:07.439078+0000","last_change":"2024-09-17T15:25:07.439256+0000","last_active":"2024-09-17T15:25:07.439078+0000","last_peered":"2024-09-17T15:25:07.439078+0000","last_clean":"2024-09-17T15:24:49.051582+0000","last_became_active":"2024-09-17T15:25:07.438978+0000","last_became_peered":"2024-09-17T15:25:07.438978+0000","last_unstale":"2024-09-17T15:25:07.439078+0000","last_undegraded":"2024-09-17T15:25:07.433841+0000","last_fullsized":"2024-09-17T15:25:07.439078+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":51,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-17T15:25:07.438449+0000","last_change":"2024-09-17T15:25:07.438351+0000","last_active":"2024-09-17T15:25:07.438449+0000","last_peered":"2024-09-17T15:25:07.438449+0000","last_clean":"2024-09-17T15:25:07.438449+0000","last_became_active":"2024-09-17T15:25:07.438078+0000","last_became_peered":"2024-09-17T15:25:07.438078+0000","last_unstale":"2024-09-17T15:25:07.438449+0000","last_undegraded":"2024-09-17T15:25:07.438449+0000","last_fullsized":"2024-09-17T15:25:07.438449+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:12:46.153849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"21'5","reported_seq":46,"reported_epoch":24,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:07.443760+0000","last_change":"2024-09-17T15:25:07.444097+0000","last_active":"2024-09-17T15:25:07.443760+0000","last_peered":"2024-09-17T15:25:07.443760+0000","last_clean":"2024-09-17T15:24:56.758735+0000","last_became_active":"2024-09-17T15:25:07.443569+0000","last_became_peered":"2024-09-17T15:25:07.443569+0000","last_unstale":"2024-09-17T15:25:07.443760+0000","last_undegraded":"2024-09-17T15:25:07.431720+0000","last_fullsized":"2024-09-17T15:25:07.443760+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-17T15:25:07.442831+0000","last_change":"2024-09-17T15:25:07.442375+0000","last_active":"2024-09-17T15:25:07.442831+0000","last_peered":"2024-09-17T15:25:07.442831+0000","last_clean":"2024-09-17T15:25:07.442831+0000","last_became_active":"2024-09-17T15:25:07.442046+0000","last_became_peered":"2024-09-17T15:25:07.442046+0000","last_unstale":"2024-09-17T15:25:07.442831+0000","last_undegraded":"2024-09-17T15:25:07.442831+0000","last_fullsized":"2024-09-17T15:25:07.442831+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:42:50.869443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'6","reported_seq":32,"reported_epoch":24,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:07.441532+0000","last_change":"2024-09-17T15:25:07.441877+0000","last_active":"2024-09-17T15:25:07.441532+0000","last_peered":"2024-09-17T15:25:07.441532+0000","last_clean":"2024-09-17T15:24:56.997318+0000","last_became_active":"2024-09-17T15:25:07.441443+0000","last_became_peered":"2024-09-17T15:25:07.441443+0000","last_unstale":"2024-09-17T15:25:07.441532+0000","last_undegraded":"2024-09-17T15:25:07.434031+0000","last_fullsized":"2024-09-17T15:25:07.441532+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":40,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-17T15:25:07.442185+0000","last_change":"2024-09-17T15:25:07.442076+0000","last_active":"2024-09-17T15:25:07.442185+0000","last_peered":"2024-09-17T15:25:07.442185+0000","last_clean":"2024-09-17T15:25:07.442185+0000","last_became_active":"2024-09-17T15:25:07.441685+0000","last_became_peered":"2024-09-17T15:25:07.441685+0000","last_unstale":"2024-09-17T15:25:07.442185+0000","last_undegraded":"2024-09-17T15:25:07.442185+0000","last_fullsized":"2024-09-17T15:25:07.442185+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:23:51.873444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"21'3","reported_seq":33,"reported_epoch":24,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:07.439010+0000","last_change":"2024-09-17T15:25:07.439256+0000","last_active":"2024-09-17T15:25:07.439010+0000","last_peered":"2024-09-17T15:25:07.439010+0000","last_clean":"2024-09-17T15:24:56.012036+0000","last_became_active":"2024-09-17T15:25:07.438770+0000","last_became_peered":"2024-09-17T15:25:07.438770+0000","last_unstale":"2024-09-17T15:25:07.439010+0000","last_undegraded":"2024-09-17T15:25:07.431359+0000","last_fullsized":"2024-09-17T15:25:07.439010+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":61,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-17T15:25:07.439234+0000","last_change":"2024-09-17T15:25:07.439129+0000","last_active":"2024-09-17T15:25:07.439234+0000","last_peered":"2024-09-17T15:25:07.439234+0000","last_clean":"2024-09-17T15:25:07.439234+0000","last_became_active":"2024-09-17T15:25:07.438669+0000","last_became_peered":"2024-09-17T15:25:07.438669+0000","last_unstale":"2024-09-17T15:25:07.439234+0000","last_undegraded":"2024-09-17T15:25:07.439234+0000","last_fullsized":"2024-09-17T15:25:07.439234+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:35:32.406532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"21'4","reported_seq":34,"reported_epoch":24,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:07.442656+0000","last_change":"2024-09-17T15:25:07.442832+0000","last_active":"2024-09-17T15:25:07.442656+0000","last_peered":"2024-09-17T15:25:07.442656+0000","last_clean":"2024-09-17T15:24:56.409666+0000","last_became_active":"2024-09-17T15:25:07.442558+0000","last_became_peered":"2024-09-17T15:25:07.442558+0000","last_unstale":"2024-09-17T15:25:07.442656+0000","last_undegraded":"2024-09-17T15:25:07.431756+0000","last_fullsized":"2024-09-17T15:25:07.442656+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"21'1","reported_seq":36,"reported_epoch":24,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:07.442556+0000","last_change":"2024-09-17T15:25:07.442754+0000","last_active":"2024-09-17T15:25:07.442556+0000","last_peered":"2024-09-17T15:25:07.442556+0000","last_clean":"2024-09-17T15:24:52.804722+0000","last_became_active":"2024-09-17T15:25:07.442427+0000","last_became_peered":"2024-09-17T15:25:07.442427+0000","last_unstale":"2024-09-17T15:25:07.442556+0000","last_undegraded":"2024-09-17T15:25:07.431777+0000","last_fullsized":"2024-09-17T15:25:07.442556+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":51,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-17T15:25:07.438457+0000","last_change":"2024-09-17T15:25:07.438389+0000","last_active":"2024-09-17T15:25:07.438457+0000","last_peered":"2024-09-17T15:25:07.438457+0000","last_clean":"2024-09-17T15:25:07.438457+0000","last_became_active":"2024-09-17T15:25:07.438131+0000","last_became_peered":"2024-09-17T15:25:07.438131+0000","last_unstale":"2024-09-17T15:25:07.438457+0000","last_undegraded":"2024-09-17T15:25:07.438457+0000","last_fullsized":"2024-09-17T15:25:07.438457+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:57:32.999616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'5","reported_seq":41,"reported_epoch":24,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:07.443257+0000","last_change":"2024-09-17T15:25:07.443584+0000","last_active":"2024-09-17T15:25:07.443257+0000","last_peered":"2024-09-17T15:25:07.443257+0000","last_clean":"2024-09-17T15:24:56.897031+0000","last_became_active":"2024-09-17T15:25:07.443171+0000","last_became_peered":"2024-09-17T15:25:07.443171+0000","last_unstale":"2024-09-17T15:25:07.443257+0000","last_undegraded":"2024-09-17T15:25:07.431277+0000","last_fullsized":"2024-09-17T15:25:07.443257+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":3}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-17T15:25:07.442565+0000","last_change":"2024-09-17T15:25:07.442301+0000","last_active":"2024-09-17T15:25:07.442565+0000","last_peered":"2024-09-17T15:25:07.442565+0000","last_clean":"2024-09-17T15:25:07.442565+0000","last_became_active":"2024-09-17T15:25:07.441967+0000","last_became_peered":"2024-09-17T15:25:07.441967+0000","last_unstale":"2024-09-17T15:25:07.442565+0000","last_undegraded":"2024-09-17T15:25:07.442565+0000","last_fullsized":"2024-09-17T15:25:07.442565+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:08:31.989482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"21'2","reported_seq":37,"reported_epoch":24,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:07.442552+0000","last_change":"2024-09-17T15:25:07.442750+0000","last_active":"2024-09-17T15:25:07.442552+0000","last_peered":"2024-09-17T15:25:07.442552+0000","last_clean":"2024-09-17T15:24:52.804663+0000","last_became_active":"2024-09-17T15:25:07.442449+0000","last_became_peered":"2024-09-17T15:25:07.442449+0000","last_unstale":"2024-09-17T15:25:07.442552+0000","last_undegraded":"2024-09-17T15:25:07.431835+0000","last_fullsized":"2024-09-17T15:25:07.442552+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":51,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-17T15:25:07.443856+0000","last_change":"2024-09-17T15:25:07.443740+0000","last_active":"2024-09-17T15:25:07.443856+0000","last_peered":"2024-09-17T15:25:07.443856+0000","last_clean":"2024-09-17T15:25:07.443856+0000","last_became_active":"2024-09-17T15:25:07.443213+0000","last_became_peered":"2024-09-17T15:25:07.443213+0000","last_unstale":"2024-09-17T15:25:07.443856+0000","last_undegraded":"2024-09-17T15:25:07.443856+0000","last_fullsized":"2024-09-17T15:25:07.443856+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:23:57.302175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":56,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-17T15:25:07.443519+0000","last_change":"2024-09-17T15:25:07.443422+0000","last_active":"2024-09-17T15:25:07.443519+0000","last_peered":"2024-09-17T15:25:07.443519+0000","last_clean":"2024-09-17T15:25:07.443519+0000","last_became_active":"2024-09-17T15:25:07.443001+0000","last_became_peered":"2024-09-17T15:25:07.443001+0000","last_unstale":"2024-09-17T15:25:07.443519+0000","last_undegraded":"2024-09-17T15:25:07.443519+0000","last_fullsized":"2024-09-17T15:25:07.443519+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:30:00.115375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"21'9","reported_seq":45,"reported_epoch":24,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:07.443087+0000","last_change":"2024-09-17T15:25:07.443319+0000","last_active":"2024-09-17T15:25:07.443087+0000","last_peered":"2024-09-17T15:25:07.443087+0000","last_clean":"2024-09-17T15:24:56.609264+0000","last_became_active":"2024-09-17T15:25:07.442967+0000","last_became_peered":"2024-09-17T15:25:07.442967+0000","last_unstale":"2024-09-17T15:25:07.443087+0000","last_undegraded":"2024-09-17T15:25:07.430994+0000","last_fullsized":"2024-09-17T15:25:07.443087+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1932,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"0,1","objects":1},{"shards":"1","objects":3}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":13524,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":7,"num_objects_missing":7,"num_objects_degraded":27,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":18354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738380,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":323584,"data_stored":73212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247811,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":244,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":249856,"data_stored":64518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":28110504},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:25:13.173 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:25:13.518 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:25:13.518 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 27/60 objects degraded (45.000%), 14 pgs degraded","count":14},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":50,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":24,"num_osds":3,"num_up_osds":2,"osd_up_since":1726586706,"num_in_osds":2,"osd_in_since":1726586691,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+recovery_wait+degraded","count":12},{"state_name":"active+clean","count":10},{"state_name":"active+recovering+degraded","count":2}],"num_pgs":24,"num_pools":2,"num_objects":30,"data_bytes":13543,"bytes_used":56397824,"bytes_avail":193217130496,"bytes_total":193273528320,"degraded_objects":27,"degraded_total":60,"degraded_ratio":0.45000000000000001},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-09-17T15:24:23.008081+0000","services":{}},"progress_events":{}} 2024-09-17T15:25:13.531 INFO:tasks.lost_unfound.ceph_manager:{'fsid': '5d1df884-ae2f-44a8-abc8-b077329edada', 'health': {'status': 'HEALTH_WARN', 'checks': {'PG_DEGRADED': {'severity': 'HEALTH_WARN', 'summary': {'message': 'Degraded data redundancy: 27/60 objects degraded (45.000%), 14 pgs degraded', 'count': 14}, 'muted': False}}, 'mutes': []}, 'election_epoch': 10, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 50, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 24, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1726586706, 'num_in_osds': 2, 'osd_in_since': 1726586691, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'active+recovery_wait+degraded', 'count': 12}, {'state_name': 'active+clean', 'count': 10}, {'state_name': 'active+recovering+degraded', 'count': 2}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 30, 'data_bytes': 13543, 'bytes_used': 56397824, 'bytes_avail': 193217130496, 'bytes_total': 193273528320, 'degraded_objects': 27, 'degraded_total': 60, 'degraded_ratio': 0.45}, 'fsmap': {'epoch': 1, 'btime': '2024-09-17T15:24:23:004974+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 1, 'modified': '2024-09-17T15:24:23.008081+0000', 'services': {}}, 'progress_events': {}} 2024-09-17T15:25:13.531 INFO:tasks.lost_unfound.ceph_manager:active! 2024-09-17T15:25:13.531 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-17T15:25:13.531 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T15:25:13.560 INFO:tasks.ceph.osd.1:Stopped 2024-09-17T15:25:13.560 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 1 2024-09-17T15:25:13.894 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-09-17T15:25:13.893+0000 7f514a174640 -1 mon.a@0(leader).osd e25 definitely_dead 0 2024-09-17T15:25:13.894 INFO:teuthology.orchestra.run.smithi083.stderr:osd.1 is already down. 2024-09-17T15:25:13.907 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 1 2024-09-17T15:25:14.197 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-09-17T15:25:14.196+0000 7f514a174640 -1 mon.a@0(leader).osd e25 definitely_dead 0 2024-09-17T15:25:14.643 INFO:teuthology.orchestra.run.smithi083.stderr:marked out osd.1. 2024-09-17T15:25:14.659 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd lost 1 --yes-i-really-mean-it 2024-09-17T15:25:15.652 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:15.650+0000 7fa91b150640 -1 log_channel(cluster) log [ERR] : 2.0 has 2 objects unfound and apparently lost 2024-09-17T15:25:15.652 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:15.651+0000 7fa917148640 -1 log_channel(cluster) log [ERR] : 2.8 has 1 objects unfound and apparently lost 2024-09-17T15:25:15.652 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:15.651+0000 7fa916947640 -1 log_channel(cluster) log [ERR] : 2.1 has 2 objects unfound and apparently lost 2024-09-17T15:25:15.652 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:15.651+0000 7fa91a94f640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-17T15:25:15.652 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:15.651+0000 7fa91914c640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-17T15:25:15.652 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:15.651+0000 7fa916146640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-17T15:25:15.653 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:15.651+0000 7fa91a14e640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-17T15:25:15.653 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:15.651+0000 7fa91994d640 -1 log_channel(cluster) log [ERR] : 2.b has 2 objects unfound and apparently lost 2024-09-17T15:25:15.654 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:15.651+0000 7fa915945640 -1 log_channel(cluster) log [ERR] : 2.3 has 2 objects unfound and apparently lost 2024-09-17T15:25:15.654 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:15.652+0000 7fa91894b640 -1 log_channel(cluster) log [ERR] : 2.d has 1 objects unfound and apparently lost 2024-09-17T15:25:15.654 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:15.652+0000 7fa914943640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-17T15:25:15.654 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:15.652+0000 7fa91814a640 -1 log_channel(cluster) log [ERR] : 2.e has 2 objects unfound and apparently lost 2024-09-17T15:25:15.654 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:15.652+0000 7fa914142640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-17T15:25:15.654 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:15.652+0000 7fa913941640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-17T15:25:15.654 INFO:teuthology.orchestra.run.smithi083.stderr:marked osd lost in epoch 25 2024-09-17T15:25:15.669 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-17T15:25:15.669 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-17T15:25:15.672 INFO:tasks.ceph.osd.2:Started 2024-09-17T15:25:15.672 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T15:25:15.775 INFO:teuthology.orchestra.run.smithi083.stderr:no valid command found; 10 closest matches: 2024-09-17T15:25:15.775 INFO:teuthology.orchestra.run.smithi083.stderr:0 2024-09-17T15:25:15.776 INFO:teuthology.orchestra.run.smithi083.stderr:1 2024-09-17T15:25:15.776 INFO:teuthology.orchestra.run.smithi083.stderr:2 2024-09-17T15:25:15.776 INFO:teuthology.orchestra.run.smithi083.stderr:abort 2024-09-17T15:25:15.776 INFO:teuthology.orchestra.run.smithi083.stderr:assert 2024-09-17T15:25:15.776 INFO:teuthology.orchestra.run.smithi083.stderr:config diff 2024-09-17T15:25:15.776 INFO:teuthology.orchestra.run.smithi083.stderr:config diff get 2024-09-17T15:25:15.776 INFO:teuthology.orchestra.run.smithi083.stderr:config get 2024-09-17T15:25:15.776 INFO:teuthology.orchestra.run.smithi083.stderr:config help [] 2024-09-17T15:25:15.776 INFO:teuthology.orchestra.run.smithi083.stderr:config set ... 2024-09-17T15:25:15.776 INFO:teuthology.orchestra.run.smithi083.stderr:admin_socket: invalid command 2024-09-17T15:25:15.778 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T15:25:15.778 INFO:tasks.lost_unfound.ceph_manager:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-17T15:25:17.325 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T15:25:17.324+0000 7fcc13558640 -1 Falling back to public interface 2024-09-17T15:25:20.779 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T15:25:20.877 INFO:teuthology.orchestra.run.smithi083.stderr:no valid command found; 10 closest matches: 2024-09-17T15:25:20.877 INFO:teuthology.orchestra.run.smithi083.stderr:0 2024-09-17T15:25:20.877 INFO:teuthology.orchestra.run.smithi083.stderr:1 2024-09-17T15:25:20.877 INFO:teuthology.orchestra.run.smithi083.stderr:2 2024-09-17T15:25:20.877 INFO:teuthology.orchestra.run.smithi083.stderr:abort 2024-09-17T15:25:20.877 INFO:teuthology.orchestra.run.smithi083.stderr:assert 2024-09-17T15:25:20.878 INFO:teuthology.orchestra.run.smithi083.stderr:config diff 2024-09-17T15:25:20.878 INFO:teuthology.orchestra.run.smithi083.stderr:config diff get 2024-09-17T15:25:20.878 INFO:teuthology.orchestra.run.smithi083.stderr:config get 2024-09-17T15:25:20.878 INFO:teuthology.orchestra.run.smithi083.stderr:config help [] 2024-09-17T15:25:20.878 INFO:teuthology.orchestra.run.smithi083.stderr:config set ... 2024-09-17T15:25:20.878 INFO:teuthology.orchestra.run.smithi083.stderr:admin_socket: invalid command 2024-09-17T15:25:20.879 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T15:25:20.880 INFO:tasks.lost_unfound.ceph_manager:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-17T15:25:21.584 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T15:25:21.584+0000 7fcc13558640 -1 osd.2 16 log_to_monitors true 2024-09-17T15:25:21.667 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:21.664+0000 7fa91b150640 -1 log_channel(cluster) log [ERR] : 2.0 has 2 objects unfound and apparently lost 2024-09-17T15:25:21.667 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:21.664+0000 7fa91a94f640 -1 log_channel(cluster) log [ERR] : 2.1 has 2 objects unfound and apparently lost 2024-09-17T15:25:21.667 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:21.664+0000 7fa917148640 -1 log_channel(cluster) log [ERR] : 2.8 has 1 objects unfound and apparently lost 2024-09-17T15:25:21.667 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:21.664+0000 7fa916947640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-17T15:25:21.667 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:21.664+0000 7fa91814a640 -1 log_channel(cluster) log [ERR] : 2.e has 2 objects unfound and apparently lost 2024-09-17T15:25:21.667 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:21.665+0000 7fa915945640 -1 log_channel(cluster) log [ERR] : 2.b has 2 objects unfound and apparently lost 2024-09-17T15:25:21.667 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:21.665+0000 7fa915144640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-17T15:25:21.667 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:21.665+0000 7fa914142640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-17T15:25:21.667 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:21.665+0000 7fa91994d640 -1 log_channel(cluster) log [ERR] : 2.3 has 2 objects unfound and apparently lost 2024-09-17T15:25:21.667 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:21.665+0000 7fa913941640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-17T15:25:21.667 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:21.665+0000 7fa914943640 -1 log_channel(cluster) log [ERR] : 2.d has 1 objects unfound and apparently lost 2024-09-17T15:25:21.668 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:21.665+0000 7fa91894b640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-17T15:25:21.668 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:21.665+0000 7fa916146640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-17T15:25:21.668 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:21.665+0000 7fa91a14e640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-17T15:25:21.670 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-09-17T15:25:21.670+0000 7fcc0b30e640 -1 osd.2 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T15:25:22.681 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:22.680+0000 7fa91b150640 -1 log_channel(cluster) log [ERR] : 2.8 has 1 objects unfound and apparently lost 2024-09-17T15:25:22.681 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:22.680+0000 7fa917148640 -1 log_channel(cluster) log [ERR] : 2.0 has 2 objects unfound and apparently lost 2024-09-17T15:25:22.681 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:22.680+0000 7fa916947640 -1 log_channel(cluster) log [ERR] : 2.1 has 2 objects unfound and apparently lost 2024-09-17T15:25:22.681 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:22.680+0000 7fa91994d640 -1 log_channel(cluster) log [ERR] : 2.3 has 2 objects unfound and apparently lost 2024-09-17T15:25:22.682 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:22.680+0000 7fa915945640 -1 log_channel(cluster) log [ERR] : 2.b has 2 objects unfound and apparently lost 2024-09-17T15:25:22.682 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:22.680+0000 7fa914142640 -1 log_channel(cluster) log [ERR] : 2.e has 2 objects unfound and apparently lost 2024-09-17T15:25:22.683 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:22.681+0000 7fa91a14e640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-17T15:25:22.683 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:22.681+0000 7fa91a94f640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-17T15:25:22.683 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:22.681+0000 7fa914142640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-17T15:25:22.683 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:22.681+0000 7fa917949640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-17T15:25:22.683 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:22.681+0000 7fa914943640 -1 log_channel(cluster) log [ERR] : 2.d has 1 objects unfound and apparently lost 2024-09-17T15:25:22.683 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:22.681+0000 7fa916146640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-17T15:25:22.683 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:22.681+0000 7fa914943640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-17T15:25:22.683 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:22.682+0000 7fa915144640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-17T15:25:22.782 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:22.779+0000 7fa915144640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-17T15:25:22.782 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:22.780+0000 7fa916146640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-17T15:25:22.782 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:22.780+0000 7fa91a14e640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-17T15:25:22.782 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:22.781+0000 7fa913941640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-17T15:25:22.782 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:22.781+0000 7fa916947640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-17T15:25:22.783 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:22.781+0000 7fa91814a640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-17T15:25:22.783 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:22.782+0000 7fa91894b640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-17T15:25:23.781 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:23.780+0000 7fa91a14e640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-17T15:25:23.781 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:23.780+0000 7fa91a14e640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-17T15:25:23.782 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:23.780+0000 7fa91a94f640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-17T15:25:23.782 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:23.780+0000 7fa91814a640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-17T15:25:23.782 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:23.781+0000 7fa915144640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-17T15:25:23.783 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:23.781+0000 7fa91894b640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-17T15:25:23.783 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:23.781+0000 7fa917949640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-17T15:25:25.880 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T15:25:25.996 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T15:25:25.996 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [], 2024-09-17T15:25:25.996 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 0 2024-09-17T15:25:25.996 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T15:25:26.005 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 2 2024-09-17T15:25:26.297 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-09-17T15:25:26.296+0000 7f514a174640 -1 mon.a@0(leader).osd e31 definitely_dead 0 2024-09-17T15:25:26.836 INFO:teuthology.orchestra.run.smithi083.stderr:marked in osd.2. 2024-09-17T15:25:26.856 INFO:tasks.lost_unfound.ceph_manager:waiting for osd.2 to be up 2024-09-17T15:25:26.856 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T15:25:27.140 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:25:27.140 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":32,"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","created":"2024-09-17T15:24:23.007214+0000","modified":"2024-09-17T15:25:26.831995+0000","last_up_change":"2024-09-17T15:25:22.657596+0000","last_in_change":"2024-09-17T15:25:26.831995+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-17T15:24:38.423278+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":13,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":1.4900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":1,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfound_pool","create_time":"2024-09-17T15:24:47.449679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":1.8700000047683716,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9defe403-224a-4a6c-9184-dcb72bef6bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":19,"up_from":23,"up_thru":30,"down_at":20,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":1704571166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6805","nonce":1704571166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6807","nonce":1704571166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":1704571166}]},"public_addr":"172.21.15.83:6804/1704571166","cluster_addr":"172.21.15.83:6805/1704571166","heartbeat_back_addr":"172.21.15.83:6807/1704571166","heartbeat_front_addr":"172.21.15.83:6806/1704571166","state":["exists","up"]},{"osd":1,"uuid":"d7d78e01-f761-444b-98c4-62587b991738","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":23,"down_at":25,"lost_at":25,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":1258824819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6801","nonce":1258824819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6803","nonce":1258824819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":1258824819}]},"public_addr":"172.21.15.83:6800/1258824819","cluster_addr":"172.21.15.83:6801/1258824819","heartbeat_back_addr":"172.21.15.83:6803/1258824819","heartbeat_front_addr":"172.21.15.83:6802/1258824819","state":["exists"]},{"osd":2,"uuid":"23c2275c-8731-45f3-97f9-5100b6676f99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":16,"up_from":29,"up_thru":14,"down_at":17,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3672266049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6801","nonce":3672266049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6803","nonce":3672266049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3672266049}]},"public_addr":"172.21.15.83:6800/3672266049","cluster_addr":"172.21.15.83:6801/3672266049","heartbeat_back_addr":"172.21.15.83:6803/3672266049","heartbeat_front_addr":"172.21.15.83:6802/3672266049","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T15:24:31.547535+0000","dead_epoch":20},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":65536,"last_purged_snaps_scrub":"2024-09-17T15:24:30.867578+0000","dead_epoch":25},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T15:24:30.866429+0000","dead_epoch":17}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[{"pgid":"1.1","osds":[0,2]},{"pgid":"1.3","osds":[0,2]},{"pgid":"1.5","osds":[0,2]},{"pgid":"1.7","osds":[0,2]},{"pgid":"2.0","osds":[0,2]},{"pgid":"2.1","osds":[0,2]},{"pgid":"2.3","osds":[0,2]},{"pgid":"2.c","osds":[0,2]},{"pgid":"2.d","osds":[0,2]},{"pgid":"2.e","osds":[0,2]}],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T15:25:27.153 INFO:tasks.lost_unfound.ceph_manager:osd.2 is up 2024-09-17T15:25:27.153 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-17T15:25:27.154 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-17T15:25:27.286 INFO:teuthology.orchestra.run.smithi083.stdout:124554051587 2024-09-17T15:25:27.287 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T15:25:27.287 INFO:teuthology.orchestra.run.smithi083.stdout:98784247815 2024-09-17T15:25:27.288 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T15:25:27.619 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-09-17T15:25:27.621 INFO:teuthology.orchestra.run.smithi083.stdout:98784247814 2024-09-17T15:25:27.633 INFO:tasks.lost_unfound.ceph_manager:need seq 124554051587 got 1 for osd.2 2024-09-17T15:25:27.635 INFO:tasks.lost_unfound.ceph_manager:need seq 98784247815 got 98784247814 for osd.0 2024-09-17T15:25:28.634 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T15:25:28.635 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T15:25:28.784 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:28.781+0000 7fa91a14e640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-17T15:25:28.784 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:28.782+0000 7fa917148640 -1 log_channel(cluster) log [ERR] : 2.8 has 1 objects unfound and apparently lost 2024-09-17T15:25:28.784 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:28.782+0000 7fa91a14e640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-17T15:25:28.784 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:28.782+0000 7fa915945640 -1 log_channel(cluster) log [ERR] : 2.b has 2 objects unfound and apparently lost 2024-09-17T15:25:28.784 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:28.782+0000 7fa917949640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-17T15:25:28.784 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:28.782+0000 7fa914943640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-17T15:25:28.784 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:28.782+0000 7fa914142640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-17T15:25:28.785 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:28.783+0000 7fa91914c640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-17T15:25:28.785 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-09-17T15:25:28.783+0000 7fa916947640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-17T15:25:28.943 INFO:teuthology.orchestra.run.smithi083.stdout:98784247815 2024-09-17T15:25:28.946 INFO:teuthology.orchestra.run.smithi083.stdout:124554051587 2024-09-17T15:25:28.956 INFO:tasks.lost_unfound.ceph_manager:need seq 98784247815 got 98784247815 for osd.0 2024-09-17T15:25:28.956 DEBUG:teuthology.parallel:result is None 2024-09-17T15:25:28.960 INFO:tasks.lost_unfound.ceph_manager:need seq 124554051587 got 124554051587 for osd.2 2024-09-17T15:25:28.961 DEBUG:teuthology.parallel:result is None 2024-09-17T15:25:28.961 INFO:tasks.lost_unfound.ceph_manager:waiting till active 2024-09-17T15:25:28.961 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:25:29.279 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:25:29.279 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:25:29.291 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":59,"stamp":"2024-09-17T15:25:28.778302+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":13543,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58,"ondisk_log_size":58,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55592,"kb_used_data":556,"kb_used_omap":8,"kb_used_meta":54967,"kb_avail":188688088,"statfs":{"total":193273528320,"available":193216602112,"internally_reserved":0,"allocated":569344,"data_stored":167129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8876,"internal_metadata":56286548},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-57,"num_objects_misplaced":0,"num_objects_unfound":-27,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.198561"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":40,"reported_epoch":31,"state":"peering","last_fresh":"2024-09-17T15:25:23.782142+0000","last_change":"2024-09-17T15:25:26.845582+0000","last_active":"2024-09-17T15:25:26.845582+0000","last_peered":"2024-09-17T15:25:23.782142+0000","last_clean":"2024-09-17T15:25:07.444279+0000","last_became_active":"2024-09-17T15:25:14.644891+0000","last_became_peered":"2024-09-17T15:25:14.644891+0000","last_unstale":"2024-09-17T15:25:23.782142+0000","last_undegraded":"2024-09-17T15:25:23.782142+0000","last_fullsized":"2024-09-17T15:25:14.642536+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"21'4","reported_seq":44,"reported_epoch":31,"state":"remapped+peering","last_fresh":"2024-09-17T15:25:23.787031+0000","last_change":"2024-09-17T15:25:26.845392+0000","last_active":"2024-09-17T15:25:26.845392+0000","last_peered":"2024-09-17T15:25:23.787031+0000","last_clean":"2024-09-17T15:24:56.161111+0000","last_became_active":"2024-09-17T15:25:23.786778+0000","last_became_peered":"2024-09-17T15:25:23.786778+0000","last_unstale":"2024-09-17T15:25:23.787031+0000","last_undegraded":"2024-09-17T15:25:23.781769+0000","last_fullsized":"2024-09-17T15:25:23.787031+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'3","reported_seq":60,"reported_epoch":31,"state":"remapped+peering","last_fresh":"2024-09-17T15:25:23.788002+0000","last_change":"2024-09-17T15:25:26.845656+0000","last_active":"2024-09-17T15:25:26.845656+0000","last_peered":"2024-09-17T15:25:23.788002+0000","last_clean":"2024-09-17T15:24:56.708693+0000","last_became_active":"2024-09-17T15:25:23.787735+0000","last_became_peered":"2024-09-17T15:25:23.787735+0000","last_unstale":"2024-09-17T15:25:23.788002+0000","last_undegraded":"2024-09-17T15:25:23.782606+0000","last_fullsized":"2024-09-17T15:25:23.788002+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":49,"reported_epoch":31,"state":"remapped+peering","last_fresh":"2024-09-17T15:25:23.789107+0000","last_change":"2024-09-17T15:25:26.842500+0000","last_active":"2024-09-17T15:25:26.842500+0000","last_peered":"2024-09-17T15:25:23.789107+0000","last_clean":"2024-09-17T15:25:23.789107+0000","last_became_active":"2024-09-17T15:25:23.785973+0000","last_became_peered":"2024-09-17T15:25:23.785973+0000","last_unstale":"2024-09-17T15:25:23.789107+0000","last_undegraded":"2024-09-17T15:25:23.789107+0000","last_fullsized":"2024-09-17T15:25:23.789107+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"21'5","reported_seq":58,"reported_epoch":32,"state":"peering","last_fresh":"2024-09-17T15:25:26.843385+0000","last_change":"2024-09-17T15:25:26.843385+0000","last_active":"2024-09-17T15:25:26.843385+0000","last_peered":"2024-09-17T15:25:23.788133+0000","last_clean":"2024-09-17T15:24:56.061756+0000","last_became_active":"2024-09-17T15:25:23.787629+0000","last_became_peered":"2024-09-17T15:25:23.787629+0000","last_unstale":"2024-09-17T15:25:26.843385+0000","last_undegraded":"2024-09-17T15:25:26.843385+0000","last_fullsized":"2024-09-17T15:25:26.843385+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"21'3","reported_seq":52,"reported_epoch":31,"state":"peering","last_fresh":"2024-09-17T15:25:23.781109+0000","last_change":"2024-09-17T15:25:26.842545+0000","last_active":"2024-09-17T15:25:26.842545+0000","last_peered":"2024-09-17T15:25:23.781109+0000","last_clean":"2024-09-17T15:24:56.947098+0000","last_became_active":"2024-09-17T15:25:14.647801+0000","last_became_peered":"2024-09-17T15:25:14.647801+0000","last_unstale":"2024-09-17T15:25:23.781109+0000","last_undegraded":"2024-09-17T15:25:14.643907+0000","last_fullsized":"2024-09-17T15:25:14.643744+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"21'4","reported_seq":54,"reported_epoch":31,"state":"peering","last_fresh":"2024-09-17T15:25:23.781870+0000","last_change":"2024-09-17T15:25:26.843966+0000","last_active":"2024-09-17T15:25:26.843966+0000","last_peered":"2024-09-17T15:25:23.781870+0000","last_clean":"2024-09-17T15:24:56.310518+0000","last_became_active":"2024-09-17T15:25:14.644231+0000","last_became_peered":"2024-09-17T15:25:14.644231+0000","last_unstale":"2024-09-17T15:25:23.781870+0000","last_undegraded":"2024-09-17T15:25:14.642309+0000","last_fullsized":"2024-09-17T15:25:14.642184+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"21'1","reported_seq":54,"reported_epoch":32,"state":"peering","last_fresh":"2024-09-17T15:25:26.842333+0000","last_change":"2024-09-17T15:25:26.842333+0000","last_active":"2024-09-17T15:25:26.842333+0000","last_peered":"2024-09-17T15:25:23.788331+0000","last_clean":"2024-09-17T15:24:49.051582+0000","last_became_active":"2024-09-17T15:25:23.787579+0000","last_became_peered":"2024-09-17T15:25:23.787579+0000","last_unstale":"2024-09-17T15:25:26.842333+0000","last_undegraded":"2024-09-17T15:25:26.842333+0000","last_fullsized":"2024-09-17T15:25:26.842333+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":31,"state":"peering","last_fresh":"2024-09-17T15:25:23.780907+0000","last_change":"2024-09-17T15:25:26.843242+0000","last_active":"2024-09-17T15:25:26.843242+0000","last_peered":"2024-09-17T15:25:23.780907+0000","last_clean":"2024-09-17T15:24:52.804370+0000","last_became_active":"2024-09-17T15:25:14.643129+0000","last_became_peered":"2024-09-17T15:25:14.643129+0000","last_unstale":"2024-09-17T15:25:23.780907+0000","last_undegraded":"2024-09-17T15:25:23.780907+0000","last_fullsized":"2024-09-17T15:25:14.641860+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"21'5","reported_seq":62,"reported_epoch":31,"state":"remapped+peering","last_fresh":"2024-09-17T15:25:23.788235+0000","last_change":"2024-09-17T15:25:26.843348+0000","last_active":"2024-09-17T15:25:26.843348+0000","last_peered":"2024-09-17T15:25:23.788235+0000","last_clean":"2024-09-17T15:24:56.758735+0000","last_became_active":"2024-09-17T15:25:23.787849+0000","last_became_peered":"2024-09-17T15:25:23.787849+0000","last_unstale":"2024-09-17T15:25:23.788235+0000","last_undegraded":"2024-09-17T15:25:23.782375+0000","last_fullsized":"2024-09-17T15:25:23.788235+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":49,"reported_epoch":31,"state":"remapped+peering","last_fresh":"2024-09-17T15:25:23.789031+0000","last_change":"2024-09-17T15:25:26.843227+0000","last_active":"2024-09-17T15:25:26.843227+0000","last_peered":"2024-09-17T15:25:23.789031+0000","last_clean":"2024-09-17T15:25:23.789031+0000","last_became_active":"2024-09-17T15:25:23.785709+0000","last_became_peered":"2024-09-17T15:25:23.785709+0000","last_unstale":"2024-09-17T15:25:23.789031+0000","last_undegraded":"2024-09-17T15:25:23.789031+0000","last_fullsized":"2024-09-17T15:25:23.789031+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'6","reported_seq":49,"reported_epoch":31,"state":"peering","last_fresh":"2024-09-17T15:25:23.781368+0000","last_change":"2024-09-17T15:25:26.844078+0000","last_active":"2024-09-17T15:25:26.844078+0000","last_peered":"2024-09-17T15:25:23.781368+0000","last_clean":"2024-09-17T15:24:56.997318+0000","last_became_active":"2024-09-17T15:25:14.646574+0000","last_became_peered":"2024-09-17T15:25:14.646574+0000","last_unstale":"2024-09-17T15:25:23.781368+0000","last_undegraded":"2024-09-17T15:25:14.643714+0000","last_fullsized":"2024-09-17T15:25:14.643599+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":58,"reported_epoch":31,"state":"peering","last_fresh":"2024-09-17T15:25:23.781475+0000","last_change":"2024-09-17T15:25:26.843793+0000","last_active":"2024-09-17T15:25:26.843793+0000","last_peered":"2024-09-17T15:25:23.781475+0000","last_clean":"2024-09-17T15:25:07.444351+0000","last_became_active":"2024-09-17T15:25:14.646077+0000","last_became_peered":"2024-09-17T15:25:14.646077+0000","last_unstale":"2024-09-17T15:25:23.781475+0000","last_undegraded":"2024-09-17T15:25:14.643241+0000","last_fullsized":"2024-09-17T15:25:14.643148+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"21'3","reported_seq":50,"reported_epoch":31,"state":"remapped+peering","last_fresh":"2024-09-17T15:25:23.786607+0000","last_change":"2024-09-17T15:25:26.842342+0000","last_active":"2024-09-17T15:25:26.842342+0000","last_peered":"2024-09-17T15:25:23.786607+0000","last_clean":"2024-09-17T15:24:56.012036+0000","last_became_active":"2024-09-17T15:25:23.786166+0000","last_became_peered":"2024-09-17T15:25:23.786166+0000","last_unstale":"2024-09-17T15:25:23.786607+0000","last_undegraded":"2024-09-17T15:25:23.781725+0000","last_fullsized":"2024-09-17T15:25:23.786607+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":55,"reported_epoch":31,"state":"remapped+peering","last_fresh":"2024-09-17T15:25:23.789720+0000","last_change":"2024-09-17T15:25:26.844347+0000","last_active":"2024-09-17T15:25:26.844347+0000","last_peered":"2024-09-17T15:25:23.789720+0000","last_clean":"2024-09-17T15:25:23.789720+0000","last_became_active":"2024-09-17T15:25:23.788061+0000","last_became_peered":"2024-09-17T15:25:23.788061+0000","last_unstale":"2024-09-17T15:25:23.789720+0000","last_undegraded":"2024-09-17T15:25:23.789720+0000","last_fullsized":"2024-09-17T15:25:23.789720+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"21'4","reported_seq":50,"reported_epoch":31,"state":"remapped+peering","last_fresh":"2024-09-17T15:25:23.786450+0000","last_change":"2024-09-17T15:25:26.842760+0000","last_active":"2024-09-17T15:25:26.842760+0000","last_peered":"2024-09-17T15:25:23.786450+0000","last_clean":"2024-09-17T15:24:56.409666+0000","last_became_active":"2024-09-17T15:25:23.786156+0000","last_became_peered":"2024-09-17T15:25:23.786156+0000","last_unstale":"2024-09-17T15:25:23.786450+0000","last_undegraded":"2024-09-17T15:25:23.781227+0000","last_fullsized":"2024-09-17T15:25:23.786450+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"21'1","reported_seq":49,"reported_epoch":31,"state":"peering","last_fresh":"2024-09-17T15:25:23.782633+0000","last_change":"2024-09-17T15:25:26.844901+0000","last_active":"2024-09-17T15:25:26.844901+0000","last_peered":"2024-09-17T15:25:23.782633+0000","last_clean":"2024-09-17T15:24:52.804722+0000","last_became_active":"2024-09-17T15:25:14.645464+0000","last_became_peered":"2024-09-17T15:25:14.645464+0000","last_unstale":"2024-09-17T15:25:23.782633+0000","last_undegraded":"2024-09-17T15:25:14.642584+0000","last_fullsized":"2024-09-17T15:25:14.642473+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":31,"state":"peering","last_fresh":"2024-09-17T15:25:23.782338+0000","last_change":"2024-09-17T15:25:26.844924+0000","last_active":"2024-09-17T15:25:26.844924+0000","last_peered":"2024-09-17T15:25:23.782338+0000","last_clean":"2024-09-17T15:24:52.804714+0000","last_became_active":"2024-09-17T15:25:14.646551+0000","last_became_peered":"2024-09-17T15:25:14.646551+0000","last_unstale":"2024-09-17T15:25:23.782338+0000","last_undegraded":"2024-09-17T15:25:23.782338+0000","last_fullsized":"2024-09-17T15:25:14.643177+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"21'5","reported_seq":54,"reported_epoch":31,"state":"peering","last_fresh":"2024-09-17T15:25:23.781996+0000","last_change":"2024-09-17T15:25:26.846147+0000","last_active":"2024-09-17T15:25:26.846147+0000","last_peered":"2024-09-17T15:25:23.781996+0000","last_clean":"2024-09-17T15:24:56.897031+0000","last_became_active":"2024-09-17T15:25:14.647365+0000","last_became_peered":"2024-09-17T15:25:14.647365+0000","last_unstale":"2024-09-17T15:25:23.781996+0000","last_undegraded":"2024-09-17T15:25:14.642743+0000","last_fullsized":"2024-09-17T15:25:14.642599+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":49,"reported_epoch":31,"state":"remapped+peering","last_fresh":"2024-09-17T15:25:23.789708+0000","last_change":"2024-09-17T15:25:26.846407+0000","last_active":"2024-09-17T15:25:26.846407+0000","last_peered":"2024-09-17T15:25:23.789708+0000","last_clean":"2024-09-17T15:25:23.789708+0000","last_became_active":"2024-09-17T15:25:23.787526+0000","last_became_peered":"2024-09-17T15:25:23.787526+0000","last_unstale":"2024-09-17T15:25:23.789708+0000","last_undegraded":"2024-09-17T15:25:23.789708+0000","last_fullsized":"2024-09-17T15:25:23.789708+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"21'2","reported_seq":50,"reported_epoch":31,"state":"peering","last_fresh":"2024-09-17T15:25:23.782533+0000","last_change":"2024-09-17T15:25:26.847325+0000","last_active":"2024-09-17T15:25:26.847325+0000","last_peered":"2024-09-17T15:25:23.782533+0000","last_clean":"2024-09-17T15:24:52.804663+0000","last_became_active":"2024-09-17T15:25:14.647451+0000","last_became_peered":"2024-09-17T15:25:14.647451+0000","last_unstale":"2024-09-17T15:25:23.782533+0000","last_undegraded":"2024-09-17T15:25:14.643189+0000","last_fullsized":"2024-09-17T15:25:14.643055+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":31,"state":"peering","last_fresh":"2024-09-17T15:25:23.781291+0000","last_change":"2024-09-17T15:25:26.842537+0000","last_active":"2024-09-17T15:25:26.842537+0000","last_peered":"2024-09-17T15:25:23.781291+0000","last_clean":"2024-09-17T15:24:52.804652+0000","last_became_active":"2024-09-17T15:25:14.644963+0000","last_became_peered":"2024-09-17T15:25:14.644963+0000","last_unstale":"2024-09-17T15:25:23.781291+0000","last_undegraded":"2024-09-17T15:25:23.781291+0000","last_fullsized":"2024-09-17T15:25:14.642294+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":31,"state":"remapped+peering","last_fresh":"2024-09-17T15:25:23.788908+0000","last_change":"2024-09-17T15:25:26.844192+0000","last_active":"2024-09-17T15:25:26.844192+0000","last_peered":"2024-09-17T15:25:23.788908+0000","last_clean":"2024-09-17T15:25:23.788908+0000","last_became_active":"2024-09-17T15:25:23.785955+0000","last_became_peered":"2024-09-17T15:25:23.785955+0000","last_unstale":"2024-09-17T15:25:23.788908+0000","last_undegraded":"2024-09-17T15:25:23.788908+0000","last_fullsized":"2024-09-17T15:25:23.788908+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"21'9","reported_seq":58,"reported_epoch":31,"state":"peering","last_fresh":"2024-09-17T15:25:23.782136+0000","last_change":"2024-09-17T15:25:26.843026+0000","last_active":"2024-09-17T15:25:26.843026+0000","last_peered":"2024-09-17T15:25:23.782136+0000","last_clean":"2024-09-17T15:24:56.609264+0000","last_became_active":"2024-09-17T15:25:14.646790+0000","last_became_peered":"2024-09-17T15:25:14.646790+0000","last_unstale":"2024-09-17T15:25:23.782136+0000","last_undegraded":"2024-09-17T15:25:14.643027+0000","last_fullsized":"2024-09-17T15:25:14.642589+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1932,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":13524,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":18354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051587,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":256,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":262144,"data_stored":81140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":28176044},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247815,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27784,"kb_used_data":300,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344056,"statfs":{"total":96636764160,"available":96608313344,"internally_reserved":0,"allocated":307200,"data_stored":85989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":28110504},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:25:29.292 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:25:29.639 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:25:29.640 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":66,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":34,"num_osds":3,"num_up_osds":2,"osd_up_since":1726586722,"num_in_osds":2,"osd_in_since":1726586726,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"peering","count":14},{"state_name":"remapped+peering","count":10}],"num_pgs":24,"num_pools":2,"num_objects":30,"data_bytes":13543,"bytes_used":56926208,"bytes_avail":193216602112,"bytes_total":193273528320,"inactive_pgs_ratio":1},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-09-17T15:24:23.008081+0000","services":{}},"progress_events":{}} 2024-09-17T15:25:29.652 INFO:tasks.lost_unfound.ceph_manager:{'fsid': '5d1df884-ae2f-44a8-abc8-b077329edada', 'health': {'status': 'HEALTH_OK', 'checks': {}, 'mutes': []}, 'election_epoch': 10, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 66, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 34, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1726586722, 'num_in_osds': 2, 'osd_in_since': 1726586726, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'peering', 'count': 14}, {'state_name': 'remapped+peering', 'count': 10}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 30, 'data_bytes': 13543, 'bytes_used': 56926208, 'bytes_avail': 193216602112, 'bytes_total': 193273528320, 'inactive_pgs_ratio': 1}, 'fsmap': {'epoch': 1, 'btime': '2024-09-17T15:24:23:004974+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 1, 'modified': '2024-09-17T15:24:23.008081+0000', 'services': {}}, 'progress_events': {}} 2024-09-17T15:25:32.653 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:25:32.946 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:25:32.946 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:25:32.957 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2024-09-17T15:25:31.581540+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":13543,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":18,"num_objects_dirty":30,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58,"ondisk_log_size":58,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55608,"kb_used_data":580,"kb_used_omap":8,"kb_used_meta":54967,"kb_avail":188688072,"statfs":{"total":193273528320,"available":193216585728,"internally_reserved":0,"allocated":593920,"data_stored":176091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8876,"internal_metadata":56286548},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-20,"num_objects_misplaced":-1,"num_objects_unfound":-9,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001138"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":51,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.783502+0000","last_change":"2024-09-17T15:25:27.786800+0000","last_active":"2024-09-17T15:25:28.783502+0000","last_peered":"2024-09-17T15:25:28.783502+0000","last_clean":"2024-09-17T15:25:28.783502+0000","last_became_active":"2024-09-17T15:25:27.786453+0000","last_became_peered":"2024-09-17T15:25:27.786453+0000","last_unstale":"2024-09-17T15:25:28.783502+0000","last_undegraded":"2024-09-17T15:25:28.783502+0000","last_fullsized":"2024-09-17T15:25:28.783502+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:46:04.427346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"21'4","reported_seq":44,"reported_epoch":31,"state":"remapped+peering","last_fresh":"2024-09-17T15:25:23.787031+0000","last_change":"2024-09-17T15:25:26.845392+0000","last_active":"2024-09-17T15:25:26.845392+0000","last_peered":"2024-09-17T15:25:23.787031+0000","last_clean":"2024-09-17T15:24:56.161111+0000","last_became_active":"2024-09-17T15:25:23.786778+0000","last_became_peered":"2024-09-17T15:25:23.786778+0000","last_unstale":"2024-09-17T15:25:23.787031+0000","last_undegraded":"2024-09-17T15:25:23.781769+0000","last_fullsized":"2024-09-17T15:25:23.787031+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'3","reported_seq":60,"reported_epoch":31,"state":"remapped+peering","last_fresh":"2024-09-17T15:25:23.788002+0000","last_change":"2024-09-17T15:25:26.845656+0000","last_active":"2024-09-17T15:25:26.845656+0000","last_peered":"2024-09-17T15:25:23.788002+0000","last_clean":"2024-09-17T15:24:56.708693+0000","last_became_active":"2024-09-17T15:25:23.787735+0000","last_became_peered":"2024-09-17T15:25:23.787735+0000","last_unstale":"2024-09-17T15:25:23.788002+0000","last_undegraded":"2024-09-17T15:25:23.782606+0000","last_fullsized":"2024-09-17T15:25:23.788002+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":49,"reported_epoch":31,"state":"remapped+peering","last_fresh":"2024-09-17T15:25:23.789107+0000","last_change":"2024-09-17T15:25:26.842500+0000","last_active":"2024-09-17T15:25:26.842500+0000","last_peered":"2024-09-17T15:25:23.789107+0000","last_clean":"2024-09-17T15:25:23.789107+0000","last_became_active":"2024-09-17T15:25:23.785973+0000","last_became_peered":"2024-09-17T15:25:23.785973+0000","last_unstale":"2024-09-17T15:25:23.789107+0000","last_undegraded":"2024-09-17T15:25:23.789107+0000","last_fullsized":"2024-09-17T15:25:23.789107+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"21'5","reported_seq":66,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:28.783166+0000","last_change":"2024-09-17T15:25:27.782128+0000","last_active":"2024-09-17T15:25:28.783166+0000","last_peered":"2024-09-17T15:25:28.783166+0000","last_clean":"2024-09-17T15:24:56.061756+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:25:28.783166+0000","last_undegraded":"2024-09-17T15:25:27.778309+0000","last_fullsized":"2024-09-17T15:25:28.783166+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"21'3","reported_seq":60,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:28.782832+0000","last_change":"2024-09-17T15:25:27.789375+0000","last_active":"2024-09-17T15:25:28.782832+0000","last_peered":"2024-09-17T15:25:28.782832+0000","last_clean":"2024-09-17T15:24:56.947098+0000","last_became_active":"2024-09-17T15:25:27.789142+0000","last_became_peered":"2024-09-17T15:25:27.789142+0000","last_unstale":"2024-09-17T15:25:28.782832+0000","last_undegraded":"2024-09-17T15:25:27.779842+0000","last_fullsized":"2024-09-17T15:25:28.782832+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"21'4","reported_seq":62,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:28.784241+0000","last_change":"2024-09-17T15:25:27.784357+0000","last_active":"2024-09-17T15:25:28.784241+0000","last_peered":"2024-09-17T15:25:28.784241+0000","last_clean":"2024-09-17T15:24:56.310518+0000","last_became_active":"2024-09-17T15:25:27.784135+0000","last_became_peered":"2024-09-17T15:25:27.784135+0000","last_unstale":"2024-09-17T15:25:28.784241+0000","last_undegraded":"2024-09-17T15:25:27.779462+0000","last_fullsized":"2024-09-17T15:25:28.784241+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"21'1","reported_seq":63,"reported_epoch":34,"state":"active+recovering","last_fresh":"2024-09-17T15:25:28.783075+0000","last_change":"2024-09-17T15:25:28.783075+0000","last_active":"2024-09-17T15:25:28.783075+0000","last_peered":"2024-09-17T15:25:28.783075+0000","last_clean":"2024-09-17T15:24:49.051582+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:25:28.783075+0000","last_undegraded":"2024-09-17T15:25:28.783075+0000","last_fullsized":"2024-09-17T15:25:28.783075+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.782961+0000","last_change":"2024-09-17T15:25:27.786793+0000","last_active":"2024-09-17T15:25:28.782961+0000","last_peered":"2024-09-17T15:25:28.782961+0000","last_clean":"2024-09-17T15:25:28.782961+0000","last_became_active":"2024-09-17T15:25:27.786505+0000","last_became_peered":"2024-09-17T15:25:27.786505+0000","last_unstale":"2024-09-17T15:25:28.782961+0000","last_undegraded":"2024-09-17T15:25:28.782961+0000","last_fullsized":"2024-09-17T15:25:28.782961+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:22:15.104318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"21'5","reported_seq":62,"reported_epoch":31,"state":"remapped+peering","last_fresh":"2024-09-17T15:25:23.788235+0000","last_change":"2024-09-17T15:25:26.843348+0000","last_active":"2024-09-17T15:25:26.843348+0000","last_peered":"2024-09-17T15:25:23.788235+0000","last_clean":"2024-09-17T15:24:56.758735+0000","last_became_active":"2024-09-17T15:25:23.787849+0000","last_became_peered":"2024-09-17T15:25:23.787849+0000","last_unstale":"2024-09-17T15:25:23.788235+0000","last_undegraded":"2024-09-17T15:25:23.782375+0000","last_fullsized":"2024-09-17T15:25:23.788235+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":49,"reported_epoch":31,"state":"remapped+peering","last_fresh":"2024-09-17T15:25:23.789031+0000","last_change":"2024-09-17T15:25:26.843227+0000","last_active":"2024-09-17T15:25:26.843227+0000","last_peered":"2024-09-17T15:25:23.789031+0000","last_clean":"2024-09-17T15:25:23.789031+0000","last_became_active":"2024-09-17T15:25:23.785709+0000","last_became_peered":"2024-09-17T15:25:23.785709+0000","last_unstale":"2024-09-17T15:25:23.789031+0000","last_undegraded":"2024-09-17T15:25:23.789031+0000","last_fullsized":"2024-09-17T15:25:23.789031+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'6","reported_seq":57,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:28.783160+0000","last_change":"2024-09-17T15:25:27.789502+0000","last_active":"2024-09-17T15:25:28.783160+0000","last_peered":"2024-09-17T15:25:28.783160+0000","last_clean":"2024-09-17T15:24:56.997318+0000","last_became_active":"2024-09-17T15:25:27.789253+0000","last_became_peered":"2024-09-17T15:25:27.789253+0000","last_unstale":"2024-09-17T15:25:28.783160+0000","last_undegraded":"2024-09-17T15:25:27.780228+0000","last_fullsized":"2024-09-17T15:25:28.783160+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":66,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:28.783096+0000","last_change":"2024-09-17T15:25:27.786740+0000","last_active":"2024-09-17T15:25:28.783096+0000","last_peered":"2024-09-17T15:25:28.783096+0000","last_clean":"2024-09-17T15:25:07.444351+0000","last_became_active":"2024-09-17T15:25:27.786489+0000","last_became_peered":"2024-09-17T15:25:27.786489+0000","last_unstale":"2024-09-17T15:25:28.783096+0000","last_undegraded":"2024-09-17T15:25:27.779442+0000","last_fullsized":"2024-09-17T15:25:28.783096+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"21'3","reported_seq":50,"reported_epoch":31,"state":"remapped+peering","last_fresh":"2024-09-17T15:25:23.786607+0000","last_change":"2024-09-17T15:25:26.842342+0000","last_active":"2024-09-17T15:25:26.842342+0000","last_peered":"2024-09-17T15:25:23.786607+0000","last_clean":"2024-09-17T15:24:56.012036+0000","last_became_active":"2024-09-17T15:25:23.786166+0000","last_became_peered":"2024-09-17T15:25:23.786166+0000","last_unstale":"2024-09-17T15:25:23.786607+0000","last_undegraded":"2024-09-17T15:25:23.781725+0000","last_fullsized":"2024-09-17T15:25:23.786607+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":55,"reported_epoch":31,"state":"remapped+peering","last_fresh":"2024-09-17T15:25:23.789720+0000","last_change":"2024-09-17T15:25:26.844347+0000","last_active":"2024-09-17T15:25:26.844347+0000","last_peered":"2024-09-17T15:25:23.789720+0000","last_clean":"2024-09-17T15:25:23.789720+0000","last_became_active":"2024-09-17T15:25:23.788061+0000","last_became_peered":"2024-09-17T15:25:23.788061+0000","last_unstale":"2024-09-17T15:25:23.789720+0000","last_undegraded":"2024-09-17T15:25:23.789720+0000","last_fullsized":"2024-09-17T15:25:23.789720+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"21'4","reported_seq":50,"reported_epoch":31,"state":"remapped+peering","last_fresh":"2024-09-17T15:25:23.786450+0000","last_change":"2024-09-17T15:25:26.842760+0000","last_active":"2024-09-17T15:25:26.842760+0000","last_peered":"2024-09-17T15:25:23.786450+0000","last_clean":"2024-09-17T15:24:56.409666+0000","last_became_active":"2024-09-17T15:25:23.786156+0000","last_became_peered":"2024-09-17T15:25:23.786156+0000","last_unstale":"2024-09-17T15:25:23.786450+0000","last_undegraded":"2024-09-17T15:25:23.781227+0000","last_fullsized":"2024-09-17T15:25:23.786450+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"21'1","reported_seq":57,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:28.783294+0000","last_change":"2024-09-17T15:25:27.789404+0000","last_active":"2024-09-17T15:25:28.783294+0000","last_peered":"2024-09-17T15:25:28.783294+0000","last_clean":"2024-09-17T15:24:52.804722+0000","last_became_active":"2024-09-17T15:25:27.789191+0000","last_became_peered":"2024-09-17T15:25:27.789191+0000","last_unstale":"2024-09-17T15:25:28.783294+0000","last_undegraded":"2024-09-17T15:25:27.780610+0000","last_fullsized":"2024-09-17T15:25:28.783294+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.784109+0000","last_change":"2024-09-17T15:25:27.789397+0000","last_active":"2024-09-17T15:25:28.784109+0000","last_peered":"2024-09-17T15:25:28.784109+0000","last_clean":"2024-09-17T15:25:28.784109+0000","last_became_active":"2024-09-17T15:25:27.789137+0000","last_became_peered":"2024-09-17T15:25:27.789137+0000","last_unstale":"2024-09-17T15:25:28.784109+0000","last_undegraded":"2024-09-17T15:25:28.784109+0000","last_fullsized":"2024-09-17T15:25:28.784109+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:26:53.538418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"21'5","reported_seq":62,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:28.783708+0000","last_change":"2024-09-17T15:25:27.789579+0000","last_active":"2024-09-17T15:25:28.783708+0000","last_peered":"2024-09-17T15:25:28.783708+0000","last_clean":"2024-09-17T15:24:56.897031+0000","last_became_active":"2024-09-17T15:25:27.789325+0000","last_became_peered":"2024-09-17T15:25:27.789325+0000","last_unstale":"2024-09-17T15:25:28.783708+0000","last_undegraded":"2024-09-17T15:25:27.781559+0000","last_fullsized":"2024-09-17T15:25:28.783708+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":3}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":49,"reported_epoch":31,"state":"remapped+peering","last_fresh":"2024-09-17T15:25:23.789708+0000","last_change":"2024-09-17T15:25:26.846407+0000","last_active":"2024-09-17T15:25:26.846407+0000","last_peered":"2024-09-17T15:25:23.789708+0000","last_clean":"2024-09-17T15:25:23.789708+0000","last_became_active":"2024-09-17T15:25:23.787526+0000","last_became_peered":"2024-09-17T15:25:23.787526+0000","last_unstale":"2024-09-17T15:25:23.789708+0000","last_undegraded":"2024-09-17T15:25:23.789708+0000","last_fullsized":"2024-09-17T15:25:23.789708+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"21'2","reported_seq":58,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:28.783656+0000","last_change":"2024-09-17T15:25:27.784424+0000","last_active":"2024-09-17T15:25:28.783656+0000","last_peered":"2024-09-17T15:25:28.783656+0000","last_clean":"2024-09-17T15:24:52.804663+0000","last_became_active":"2024-09-17T15:25:27.784195+0000","last_became_peered":"2024-09-17T15:25:27.784195+0000","last_unstale":"2024-09-17T15:25:28.783656+0000","last_undegraded":"2024-09-17T15:25:27.779406+0000","last_fullsized":"2024-09-17T15:25:28.783656+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.783601+0000","last_change":"2024-09-17T15:25:27.786796+0000","last_active":"2024-09-17T15:25:28.783601+0000","last_peered":"2024-09-17T15:25:28.783601+0000","last_clean":"2024-09-17T15:25:28.783601+0000","last_became_active":"2024-09-17T15:25:27.786506+0000","last_became_peered":"2024-09-17T15:25:27.786506+0000","last_unstale":"2024-09-17T15:25:28.783601+0000","last_undegraded":"2024-09-17T15:25:28.783601+0000","last_fullsized":"2024-09-17T15:25:28.783601+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:55:57.547976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":31,"state":"remapped+peering","last_fresh":"2024-09-17T15:25:23.788908+0000","last_change":"2024-09-17T15:25:26.844192+0000","last_active":"2024-09-17T15:25:26.844192+0000","last_peered":"2024-09-17T15:25:23.788908+0000","last_clean":"2024-09-17T15:25:23.788908+0000","last_became_active":"2024-09-17T15:25:23.785955+0000","last_became_peered":"2024-09-17T15:25:23.785955+0000","last_unstale":"2024-09-17T15:25:23.788908+0000","last_undegraded":"2024-09-17T15:25:23.788908+0000","last_fullsized":"2024-09-17T15:25:23.788908+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"21'9","reported_seq":66,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:28.784095+0000","last_change":"2024-09-17T15:25:27.789556+0000","last_active":"2024-09-17T15:25:28.784095+0000","last_peered":"2024-09-17T15:25:28.784095+0000","last_clean":"2024-09-17T15:24:56.609264+0000","last_became_active":"2024-09-17T15:25:27.789304+0000","last_became_peered":"2024-09-17T15:25:27.789304+0000","last_unstale":"2024-09-17T15:25:28.784095+0000","last_undegraded":"2024-09-17T15:25:27.782129+0000","last_fullsized":"2024-09-17T15:25:28.784095+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1932,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":3}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":13524,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":35,"num_objects_misplaced":0,"num_objects_unfound":18,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":18354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051587,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":256,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":262144,"data_stored":81140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":28176044},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247816,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":324,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":331776,"data_stored":94951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":28110504},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:25:32.959 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:25:33.305 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:25:33.306 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"OBJECT_UNFOUND":{"severity":"HEALTH_WARN","summary":{"message":"18/30 objects unfound (60.000%)","count":18},"muted":false},"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36/60 objects degraded (60.000%), 9 pgs degraded","count":9},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":70,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":34,"num_osds":3,"num_up_osds":2,"osd_up_since":1726586722,"num_in_osds":2,"osd_in_since":1726586726,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"remapped+peering","count":10},{"state_name":"active+recovery_wait+degraded","count":9},{"state_name":"active+clean","count":4},{"state_name":"active+recovering","count":1}],"num_pgs":24,"num_pools":2,"num_objects":30,"data_bytes":13543,"bytes_used":56942592,"bytes_avail":193216585728,"bytes_total":193273528320,"inactive_pgs_ratio":0.4166666567325592,"degraded_objects":36,"degraded_total":60,"degraded_ratio":0.59999999999999998,"unfound_objects":18,"unfound_total":30,"unfound_ratio":0.59999999999999998},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-17T15:25:29.581240+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-17T15:25:33.318 INFO:tasks.lost_unfound.ceph_manager:{'fsid': '5d1df884-ae2f-44a8-abc8-b077329edada', 'health': {'status': 'HEALTH_WARN', 'checks': {'OBJECT_UNFOUND': {'severity': 'HEALTH_WARN', 'summary': {'message': '18/30 objects unfound (60.000%)', 'count': 18}, 'muted': False}, 'PG_DEGRADED': {'severity': 'HEALTH_WARN', 'summary': {'message': 'Degraded data redundancy: 36/60 objects degraded (60.000%), 9 pgs degraded', 'count': 9}, 'muted': False}}, 'mutes': []}, 'election_epoch': 10, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 70, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 34, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1726586722, 'num_in_osds': 2, 'osd_in_since': 1726586726, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'remapped+peering', 'count': 10}, {'state_name': 'active+recovery_wait+degraded', 'count': 9}, {'state_name': 'active+clean', 'count': 4}, {'state_name': 'active+recovering', 'count': 1}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 30, 'data_bytes': 13543, 'bytes_used': 56942592, 'bytes_avail': 193216585728, 'bytes_total': 193273528320, 'inactive_pgs_ratio': 0.4166666567325592, 'degraded_objects': 36, 'degraded_total': 60, 'degraded_ratio': 0.6, 'unfound_objects': 18, 'unfound_total': 30, 'unfound_ratio': 0.6}, 'fsmap': {'epoch': 1, 'btime': '2024-09-17T15:24:23:004974+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 2, 'modified': '2024-09-17T15:25:29.581240+0000', 'services': {'osd': {'daemons': {'summary': '', '0': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, '1': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, '2': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}}}, 'progress_events': {}} 2024-09-17T15:25:33.904 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-09-17T15:25:33.904+0000 7f514c979640 -1 log_channel(cluster) log [ERR] : Health check failed: Possible data damage: 5 pgs recovery_unfound (PG_DAMAGED) 2024-09-17T15:25:36.319 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:25:36.611 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:25:36.611 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:25:36.623 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2024-09-17T15:25:35.582433+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":13543,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":54,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":30,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58,"ondisk_log_size":58,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55624,"kb_used_data":604,"kb_used_omap":8,"kb_used_meta":54967,"kb_avail":188688056,"statfs":{"total":193273528320,"available":193216569344,"internally_reserved":0,"allocated":618496,"data_stored":185053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8876,"internal_metadata":56286548},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":54,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001541"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":51,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.783502+0000","last_change":"2024-09-17T15:25:27.786800+0000","last_active":"2024-09-17T15:25:28.783502+0000","last_peered":"2024-09-17T15:25:28.783502+0000","last_clean":"2024-09-17T15:25:28.783502+0000","last_became_active":"2024-09-17T15:25:27.786453+0000","last_became_peered":"2024-09-17T15:25:27.786453+0000","last_unstale":"2024-09-17T15:25:28.783502+0000","last_undegraded":"2024-09-17T15:25:28.783502+0000","last_fullsized":"2024-09-17T15:25:28.783502+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:46:04.427346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"21'4","reported_seq":48,"reported_epoch":34,"state":"active+recovery_unfound","last_fresh":"2024-09-17T15:25:28.788616+0000","last_change":"2024-09-17T15:25:28.788883+0000","last_active":"2024-09-17T15:25:28.788616+0000","last_peered":"2024-09-17T15:25:28.788616+0000","last_clean":"2024-09-17T15:24:56.161111+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:25:28.788616+0000","last_undegraded":"2024-09-17T15:25:28.780074+0000","last_fullsized":"2024-09-17T15:25:28.788616+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"21'3","reported_seq":65,"reported_epoch":34,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-17T15:25:28.789975+0000","last_change":"2024-09-17T15:25:31.672367+0000","last_active":"2024-09-17T15:25:28.789975+0000","last_peered":"2024-09-17T15:25:28.789975+0000","last_clean":"2024-09-17T15:24:56.708693+0000","last_became_active":"2024-09-17T15:25:28.785518+0000","last_became_peered":"2024-09-17T15:25:28.785518+0000","last_unstale":"2024-09-17T15:25:28.789975+0000","last_undegraded":"2024-09-17T15:25:28.789975+0000","last_fullsized":"2024-09-17T15:25:28.789975+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":50,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.787961+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:25:28.787961+0000","last_peered":"2024-09-17T15:25:28.787961+0000","last_clean":"2024-09-17T15:25:28.787961+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:25:28.787961+0000","last_undegraded":"2024-09-17T15:25:28.787961+0000","last_fullsized":"2024-09-17T15:25:28.787961+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"21'5","reported_seq":66,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:28.783166+0000","last_change":"2024-09-17T15:25:27.782128+0000","last_active":"2024-09-17T15:25:28.783166+0000","last_peered":"2024-09-17T15:25:28.783166+0000","last_clean":"2024-09-17T15:24:56.061756+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:25:28.783166+0000","last_undegraded":"2024-09-17T15:25:27.778309+0000","last_fullsized":"2024-09-17T15:25:28.783166+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"21'3","reported_seq":60,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:28.782832+0000","last_change":"2024-09-17T15:25:27.789375+0000","last_active":"2024-09-17T15:25:28.782832+0000","last_peered":"2024-09-17T15:25:28.782832+0000","last_clean":"2024-09-17T15:24:56.947098+0000","last_became_active":"2024-09-17T15:25:27.789142+0000","last_became_peered":"2024-09-17T15:25:27.789142+0000","last_unstale":"2024-09-17T15:25:28.782832+0000","last_undegraded":"2024-09-17T15:25:27.779842+0000","last_fullsized":"2024-09-17T15:25:28.782832+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"21'4","reported_seq":62,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:28.784241+0000","last_change":"2024-09-17T15:25:27.784357+0000","last_active":"2024-09-17T15:25:28.784241+0000","last_peered":"2024-09-17T15:25:28.784241+0000","last_clean":"2024-09-17T15:24:56.310518+0000","last_became_active":"2024-09-17T15:25:27.784135+0000","last_became_peered":"2024-09-17T15:25:27.784135+0000","last_unstale":"2024-09-17T15:25:28.784241+0000","last_undegraded":"2024-09-17T15:25:27.779462+0000","last_fullsized":"2024-09-17T15:25:28.784241+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"21'1","reported_seq":63,"reported_epoch":34,"state":"active+recovering","last_fresh":"2024-09-17T15:25:28.783075+0000","last_change":"2024-09-17T15:25:28.783075+0000","last_active":"2024-09-17T15:25:28.783075+0000","last_peered":"2024-09-17T15:25:28.783075+0000","last_clean":"2024-09-17T15:24:49.051582+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:25:28.783075+0000","last_undegraded":"2024-09-17T15:25:28.783075+0000","last_fullsized":"2024-09-17T15:25:28.783075+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.782961+0000","last_change":"2024-09-17T15:25:27.786793+0000","last_active":"2024-09-17T15:25:28.782961+0000","last_peered":"2024-09-17T15:25:28.782961+0000","last_clean":"2024-09-17T15:25:28.782961+0000","last_became_active":"2024-09-17T15:25:27.786505+0000","last_became_peered":"2024-09-17T15:25:27.786505+0000","last_unstale":"2024-09-17T15:25:28.782961+0000","last_undegraded":"2024-09-17T15:25:28.782961+0000","last_fullsized":"2024-09-17T15:25:28.782961+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:22:15.104318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"21'5","reported_seq":67,"reported_epoch":34,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-17T15:25:28.789446+0000","last_change":"2024-09-17T15:25:31.672293+0000","last_active":"2024-09-17T15:25:28.789446+0000","last_peered":"2024-09-17T15:25:28.789446+0000","last_clean":"2024-09-17T15:24:56.758735+0000","last_became_active":"2024-09-17T15:25:28.785807+0000","last_became_peered":"2024-09-17T15:25:28.785807+0000","last_unstale":"2024-09-17T15:25:28.789446+0000","last_undegraded":"2024-09-17T15:25:28.789446+0000","last_fullsized":"2024-09-17T15:25:28.789446+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.787649+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:25:28.787649+0000","last_peered":"2024-09-17T15:25:28.787649+0000","last_clean":"2024-09-17T15:25:28.787649+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:25:28.787649+0000","last_undegraded":"2024-09-17T15:25:28.787649+0000","last_fullsized":"2024-09-17T15:25:28.787649+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"21'6","reported_seq":57,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:28.783160+0000","last_change":"2024-09-17T15:25:27.789502+0000","last_active":"2024-09-17T15:25:28.783160+0000","last_peered":"2024-09-17T15:25:28.783160+0000","last_clean":"2024-09-17T15:24:56.997318+0000","last_became_active":"2024-09-17T15:25:27.789253+0000","last_became_peered":"2024-09-17T15:25:27.789253+0000","last_unstale":"2024-09-17T15:25:28.783160+0000","last_undegraded":"2024-09-17T15:25:27.780228+0000","last_fullsized":"2024-09-17T15:25:28.783160+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":66,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:28.783096+0000","last_change":"2024-09-17T15:25:27.786740+0000","last_active":"2024-09-17T15:25:28.783096+0000","last_peered":"2024-09-17T15:25:28.783096+0000","last_clean":"2024-09-17T15:25:07.444351+0000","last_became_active":"2024-09-17T15:25:27.786489+0000","last_became_peered":"2024-09-17T15:25:27.786489+0000","last_unstale":"2024-09-17T15:25:28.783096+0000","last_undegraded":"2024-09-17T15:25:27.779442+0000","last_fullsized":"2024-09-17T15:25:28.783096+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"21'3","reported_seq":54,"reported_epoch":34,"state":"active+recovery_unfound","last_fresh":"2024-09-17T15:25:28.790281+0000","last_change":"2024-09-17T15:25:28.790471+0000","last_active":"2024-09-17T15:25:28.790281+0000","last_peered":"2024-09-17T15:25:28.790281+0000","last_clean":"2024-09-17T15:24:56.012036+0000","last_became_active":"2024-09-17T15:25:28.787663+0000","last_became_peered":"2024-09-17T15:25:28.787663+0000","last_unstale":"2024-09-17T15:25:28.790281+0000","last_undegraded":"2024-09-17T15:25:28.780859+0000","last_fullsized":"2024-09-17T15:25:28.790281+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.787312+0000","last_change":"2024-09-17T15:25:28.786053+0000","last_active":"2024-09-17T15:25:28.787312+0000","last_peered":"2024-09-17T15:25:28.787312+0000","last_clean":"2024-09-17T15:25:28.787312+0000","last_became_active":"2024-09-17T15:25:28.785773+0000","last_became_peered":"2024-09-17T15:25:28.785773+0000","last_unstale":"2024-09-17T15:25:28.787312+0000","last_undegraded":"2024-09-17T15:25:28.787312+0000","last_fullsized":"2024-09-17T15:25:28.787312+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:46:53.130296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"21'4","reported_seq":54,"reported_epoch":34,"state":"active+recovery_unfound","last_fresh":"2024-09-17T15:25:28.787441+0000","last_change":"2024-09-17T15:25:28.787777+0000","last_active":"2024-09-17T15:25:28.787441+0000","last_peered":"2024-09-17T15:25:28.787441+0000","last_clean":"2024-09-17T15:24:56.409666+0000","last_became_active":"2024-09-17T15:25:28.785157+0000","last_became_peered":"2024-09-17T15:25:28.785157+0000","last_unstale":"2024-09-17T15:25:28.787441+0000","last_undegraded":"2024-09-17T15:25:28.779736+0000","last_fullsized":"2024-09-17T15:25:28.787441+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"21'1","reported_seq":57,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:28.783294+0000","last_change":"2024-09-17T15:25:27.789404+0000","last_active":"2024-09-17T15:25:28.783294+0000","last_peered":"2024-09-17T15:25:28.783294+0000","last_clean":"2024-09-17T15:24:52.804722+0000","last_became_active":"2024-09-17T15:25:27.789191+0000","last_became_peered":"2024-09-17T15:25:27.789191+0000","last_unstale":"2024-09-17T15:25:28.783294+0000","last_undegraded":"2024-09-17T15:25:27.780610+0000","last_fullsized":"2024-09-17T15:25:28.783294+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.784109+0000","last_change":"2024-09-17T15:25:27.789397+0000","last_active":"2024-09-17T15:25:28.784109+0000","last_peered":"2024-09-17T15:25:28.784109+0000","last_clean":"2024-09-17T15:25:28.784109+0000","last_became_active":"2024-09-17T15:25:27.789137+0000","last_became_peered":"2024-09-17T15:25:27.789137+0000","last_unstale":"2024-09-17T15:25:28.784109+0000","last_undegraded":"2024-09-17T15:25:28.784109+0000","last_fullsized":"2024-09-17T15:25:28.784109+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:26:53.538418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"21'5","reported_seq":62,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:28.783708+0000","last_change":"2024-09-17T15:25:27.789579+0000","last_active":"2024-09-17T15:25:28.783708+0000","last_peered":"2024-09-17T15:25:28.783708+0000","last_clean":"2024-09-17T15:24:56.897031+0000","last_became_active":"2024-09-17T15:25:27.789325+0000","last_became_peered":"2024-09-17T15:25:27.789325+0000","last_unstale":"2024-09-17T15:25:28.783708+0000","last_undegraded":"2024-09-17T15:25:27.781559+0000","last_fullsized":"2024-09-17T15:25:28.783708+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":3}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.785793+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:25:28.785793+0000","last_peered":"2024-09-17T15:25:28.785793+0000","last_clean":"2024-09-17T15:25:28.785793+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:25:28.785793+0000","last_undegraded":"2024-09-17T15:25:28.785793+0000","last_fullsized":"2024-09-17T15:25:28.785793+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"21'2","reported_seq":58,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:28.783656+0000","last_change":"2024-09-17T15:25:27.784424+0000","last_active":"2024-09-17T15:25:28.783656+0000","last_peered":"2024-09-17T15:25:28.783656+0000","last_clean":"2024-09-17T15:24:52.804663+0000","last_became_active":"2024-09-17T15:25:27.784195+0000","last_became_peered":"2024-09-17T15:25:27.784195+0000","last_unstale":"2024-09-17T15:25:28.783656+0000","last_undegraded":"2024-09-17T15:25:27.779406+0000","last_fullsized":"2024-09-17T15:25:28.783656+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.783601+0000","last_change":"2024-09-17T15:25:27.786796+0000","last_active":"2024-09-17T15:25:28.783601+0000","last_peered":"2024-09-17T15:25:28.783601+0000","last_clean":"2024-09-17T15:25:28.783601+0000","last_became_active":"2024-09-17T15:25:27.786506+0000","last_became_peered":"2024-09-17T15:25:27.786506+0000","last_unstale":"2024-09-17T15:25:28.783601+0000","last_undegraded":"2024-09-17T15:25:28.783601+0000","last_fullsized":"2024-09-17T15:25:28.783601+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:55:57.547976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.786536+0000","last_change":"2024-09-17T15:25:28.785423+0000","last_active":"2024-09-17T15:25:28.786536+0000","last_peered":"2024-09-17T15:25:28.786536+0000","last_clean":"2024-09-17T15:25:28.786536+0000","last_became_active":"2024-09-17T15:25:28.785151+0000","last_became_peered":"2024-09-17T15:25:28.785151+0000","last_unstale":"2024-09-17T15:25:28.786536+0000","last_undegraded":"2024-09-17T15:25:28.786536+0000","last_fullsized":"2024-09-17T15:25:28.786536+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:47:12.307723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"21'9","reported_seq":66,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:28.784095+0000","last_change":"2024-09-17T15:25:27.789556+0000","last_active":"2024-09-17T15:25:28.784095+0000","last_peered":"2024-09-17T15:25:28.784095+0000","last_clean":"2024-09-17T15:24:56.609264+0000","last_became_active":"2024-09-17T15:25:27.789304+0000","last_became_peered":"2024-09-17T15:25:27.789304+0000","last_unstale":"2024-09-17T15:25:28.784095+0000","last_undegraded":"2024-09-17T15:25:27.782129+0000","last_fullsized":"2024-09-17T15:25:28.784095+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1932,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":3}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":13524,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":53,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":18354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051588,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":280,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":286720,"data_stored":90102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":28176044},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247817,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":324,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":331776,"data_stored":94951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":28110504},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:25:36.624 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:25:36.973 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:25:36.973 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_ERR","checks":{"OBJECT_UNFOUND":{"severity":"HEALTH_WARN","summary":{"message":"27/30 objects unfound (90.000%)","count":27},"muted":false},"PG_DAMAGED":{"severity":"HEALTH_ERR","summary":{"message":"Possible data damage: 5 pgs recovery_unfound","count":5},"muted":false},"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 54/60 objects degraded (90.000%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":73,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":34,"num_osds":3,"num_up_osds":2,"osd_up_since":1726586722,"num_in_osds":2,"osd_in_since":1726586726,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":9},{"state_name":"active+recovery_wait+degraded","count":9},{"state_name":"active+recovery_unfound","count":3},{"state_name":"active+recovery_unfound+degraded","count":2},{"state_name":"active+recovering","count":1}],"num_pgs":24,"num_pools":2,"num_objects":30,"data_bytes":13543,"bytes_used":56958976,"bytes_avail":193216569344,"bytes_total":193273528320,"degraded_objects":54,"degraded_total":60,"degraded_ratio":0.90000000000000002,"unfound_objects":27,"unfound_total":30,"unfound_ratio":0.90000000000000002},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-17T15:25:35.582525+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-17T15:25:36.986 INFO:tasks.lost_unfound.ceph_manager:{'fsid': '5d1df884-ae2f-44a8-abc8-b077329edada', 'health': {'status': 'HEALTH_ERR', 'checks': {'OBJECT_UNFOUND': {'severity': 'HEALTH_WARN', 'summary': {'message': '27/30 objects unfound (90.000%)', 'count': 27}, 'muted': False}, 'PG_DAMAGED': {'severity': 'HEALTH_ERR', 'summary': {'message': 'Possible data damage: 5 pgs recovery_unfound', 'count': 5}, 'muted': False}, 'PG_DEGRADED': {'severity': 'HEALTH_WARN', 'summary': {'message': 'Degraded data redundancy: 54/60 objects degraded (90.000%), 11 pgs degraded', 'count': 11}, 'muted': False}}, 'mutes': []}, 'election_epoch': 10, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 73, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 34, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1726586722, 'num_in_osds': 2, 'osd_in_since': 1726586726, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'active+clean', 'count': 9}, {'state_name': 'active+recovery_wait+degraded', 'count': 9}, {'state_name': 'active+recovery_unfound', 'count': 3}, {'state_name': 'active+recovery_unfound+degraded', 'count': 2}, {'state_name': 'active+recovering', 'count': 1}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 30, 'data_bytes': 13543, 'bytes_used': 56958976, 'bytes_avail': 193216569344, 'bytes_total': 193273528320, 'degraded_objects': 54, 'degraded_total': 60, 'degraded_ratio': 0.9, 'unfound_objects': 27, 'unfound_total': 30, 'unfound_ratio': 0.9}, 'fsmap': {'epoch': 1, 'btime': '2024-09-17T15:24:23:004974+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 3, 'modified': '2024-09-17T15:25:35.582525+0000', 'services': {'osd': {'daemons': {'summary': '', '0': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, '2': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}}}, 'progress_events': {}} 2024-09-17T15:25:36.987 INFO:tasks.lost_unfound.ceph_manager:active! 2024-09-17T15:25:36.987 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-17T15:25:36.987 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-17T15:25:37.121 INFO:teuthology.orchestra.run.smithi083.stdout:98784247818 2024-09-17T15:25:37.121 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T15:25:37.122 INFO:teuthology.orchestra.run.smithi083.stdout:124554051590 2024-09-17T15:25:37.122 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T15:25:37.450 INFO:teuthology.orchestra.run.smithi083.stdout:98784247817 2024-09-17T15:25:37.454 INFO:teuthology.orchestra.run.smithi083.stdout:124554051588 2024-09-17T15:25:37.464 INFO:tasks.lost_unfound.ceph_manager:need seq 98784247818 got 98784247817 for osd.0 2024-09-17T15:25:37.468 INFO:tasks.lost_unfound.ceph_manager:need seq 124554051590 got 124554051588 for osd.2 2024-09-17T15:25:38.465 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T15:25:38.469 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T15:25:38.779 INFO:teuthology.orchestra.run.smithi083.stdout:98784247818 2024-09-17T15:25:38.792 INFO:tasks.lost_unfound.ceph_manager:need seq 98784247818 got 98784247818 for osd.0 2024-09-17T15:25:38.792 DEBUG:teuthology.parallel:result is None 2024-09-17T15:25:38.815 INFO:teuthology.orchestra.run.smithi083.stdout:124554051590 2024-09-17T15:25:38.828 INFO:tasks.lost_unfound.ceph_manager:need seq 124554051590 got 124554051590 for osd.2 2024-09-17T15:25:38.828 DEBUG:teuthology.parallel:result is None 2024-09-17T15:25:38.829 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:25:39.218 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:25:39.218 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_ERR","checks":{"OBJECT_UNFOUND":{"severity":"HEALTH_WARN","summary":{"message":"27/30 objects unfound (90.000%)","count":27},"muted":false},"PG_DAMAGED":{"severity":"HEALTH_ERR","summary":{"message":"Possible data damage: 5 pgs recovery_unfound","count":5},"muted":false},"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 54/60 objects degraded (90.000%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":76,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":34,"num_osds":3,"num_up_osds":2,"osd_up_since":1726586722,"num_in_osds":2,"osd_in_since":1726586726,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":9},{"state_name":"active+recovery_wait+degraded","count":9},{"state_name":"active+recovery_unfound","count":3},{"state_name":"active+recovery_unfound+degraded","count":2},{"state_name":"active+recovering","count":1}],"num_pgs":24,"num_pools":2,"num_objects":30,"data_bytes":13543,"bytes_used":56958976,"bytes_avail":193216569344,"bytes_total":193273528320,"degraded_objects":54,"degraded_total":60,"degraded_ratio":0.90000000000000002,"unfound_objects":27,"unfound_total":30,"unfound_ratio":0.90000000000000002},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-17T15:25:35.582525+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-17T15:25:39.231 INFO:tasks.lost_unfound.ceph_manager:{'fsid': '5d1df884-ae2f-44a8-abc8-b077329edada', 'health': {'status': 'HEALTH_ERR', 'checks': {'OBJECT_UNFOUND': {'severity': 'HEALTH_WARN', 'summary': {'message': '27/30 objects unfound (90.000%)', 'count': 27}, 'muted': False}, 'PG_DAMAGED': {'severity': 'HEALTH_ERR', 'summary': {'message': 'Possible data damage: 5 pgs recovery_unfound', 'count': 5}, 'muted': False}, 'PG_DEGRADED': {'severity': 'HEALTH_WARN', 'summary': {'message': 'Degraded data redundancy: 54/60 objects degraded (90.000%), 11 pgs degraded', 'count': 11}, 'muted': False}}, 'mutes': []}, 'election_epoch': 10, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 76, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 34, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1726586722, 'num_in_osds': 2, 'osd_in_since': 1726586726, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'active+clean', 'count': 9}, {'state_name': 'active+recovery_wait+degraded', 'count': 9}, {'state_name': 'active+recovery_unfound', 'count': 3}, {'state_name': 'active+recovery_unfound+degraded', 'count': 2}, {'state_name': 'active+recovering', 'count': 1}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 30, 'data_bytes': 13543, 'bytes_used': 56958976, 'bytes_avail': 193216569344, 'bytes_total': 193273528320, 'degraded_objects': 54, 'degraded_total': 60, 'degraded_ratio': 0.9, 'unfound_objects': 27, 'unfound_total': 30, 'unfound_ratio': 0.9}, 'fsmap': {'epoch': 1, 'btime': '2024-09-17T15:24:23:004974+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 3, 'modified': '2024-09-17T15:25:35.582525+0000', 'services': {'osd': {'daemons': {'summary': '', '0': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, '2': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}}}, 'progress_events': {}} 2024-09-17T15:25:39.231 INFO:tasks.lost_unfound:there are 27 unfound objects 2024-09-17T15:25:39.232 DEBUG:teuthology.orchestra.run.smithi083:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.admin -b 4096 -p unfound_pool -t 20 bench 240 write' 2024-09-17T15:25:39.283 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout:hints = 1 2024-09-17T15:25:39.283 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout:Maintaining 20 concurrent writes of 4096 bytes to objects of size 4096 for up to 240 seconds or 0 objects 2024-09-17T15:25:39.283 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout:Object prefix: benchmark_data_smithi083_41818 2024-09-17T15:25:39.283 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-17T15:25:39.283 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 0 0 0 0 0 0 - 0 2024-09-17T15:25:40.283 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 1 20 3069 3049 11.9095 11.9102 0.00588086 0.006534 2024-09-17T15:25:41.284 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 2 20 6023 6003 11.7238 11.5391 0.00911397 0.00664444 2024-09-17T15:25:42.283 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 3 20 8736 8716 11.3481 10.5977 0.0124692 0.00686618 2024-09-17T15:25:43.284 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 4 20 11414 11394 11.1261 10.4609 0.00758974 0.00699589 2024-09-17T15:25:44.284 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 5 20 13526 13506 10.55 8.25 0.00813323 0.00739558 2024-09-17T15:25:45.284 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 6 20 15983 15963 10.3912 9.59766 0.010538 0.00750747 2024-09-17T15:25:46.284 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 7 20 18393 18373 10.2512 9.41406 0.00926775 0.00761337 2024-09-17T15:25:47.284 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 8 20 20674 20654 10.083 8.91016 0.00539539 0.00774066 2024-09-17T15:25:48.285 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 9 20 22505 22485 9.75736 7.15234 0.0152081 0.00799957 2024-09-17T15:25:49.234 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:25:49.285 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 10 20 24672 24652 9.62792 8.46484 0.00839658 0.00810667 2024-09-17T15:25:49.762 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:25:49.763 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:25:49.786 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":70,"stamp":"2024-09-17T15:25:49.585788+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":67937511,"num_objects":16613,"num_object_clones":0,"num_object_copies":33226,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":54,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":16613,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16640,"num_write_kb":66380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16641,"ondisk_log_size":16641,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":242808,"kb_used_data":133656,"kb_used_omap":8,"kb_used_meta":109111,"kb_avail":188500872,"statfs":{"total":193273528320,"available":193024892928,"internally_reserved":0,"allocated":136863744,"data_stored":136430301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8876,"internal_metadata":111730004},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,10,6,8,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":67923968,"num_objects":16583,"num_object_clones":0,"num_object_copies":33166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16583,"num_write_kb":66332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002815"},"pg_stats":[{"pgid":"2.f","version":"34'921","reported_seq":972,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:44.837700+0000","last_change":"2024-09-17T15:25:27.786800+0000","last_active":"2024-09-17T15:25:44.837700+0000","last_peered":"2024-09-17T15:25:44.837700+0000","last_clean":"2024-09-17T15:25:44.837700+0000","last_became_active":"2024-09-17T15:25:27.786453+0000","last_became_peered":"2024-09-17T15:25:27.786453+0000","last_unstale":"2024-09-17T15:25:44.837700+0000","last_undegraded":"2024-09-17T15:25:44.837700+0000","last_fullsized":"2024-09-17T15:25:44.837700+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":921,"log_dups_size":0,"ondisk_log_size":921,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:46:04.427346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3772416,"num_objects":921,"num_object_clones":0,"num_object_copies":1842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":921,"num_write_kb":3684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"34'1190","reported_seq":1234,"reported_epoch":34,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-17T15:25:46.667064+0000","last_change":"2024-09-17T15:25:39.297496+0000","last_active":"2024-09-17T15:25:46.667064+0000","last_peered":"2024-09-17T15:25:46.667064+0000","last_clean":"2024-09-17T15:24:56.161111+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:25:46.667064+0000","last_undegraded":"2024-09-17T15:25:28.788883+0000","last_fullsized":"2024-09-17T15:25:46.667064+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":1190,"log_dups_size":0,"ondisk_log_size":1190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4858822,"num_objects":1188,"num_object_clones":0,"num_object_copies":2376,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1190,"num_write_kb":4748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1188}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"34'1269","reported_seq":1331,"reported_epoch":34,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-17T15:25:46.672262+0000","last_change":"2024-09-17T15:25:31.672367+0000","last_active":"2024-09-17T15:25:46.672262+0000","last_peered":"2024-09-17T15:25:46.672262+0000","last_clean":"2024-09-17T15:24:56.708693+0000","last_became_active":"2024-09-17T15:25:28.785518+0000","last_became_peered":"2024-09-17T15:25:28.785518+0000","last_unstale":"2024-09-17T15:25:46.672262+0000","last_undegraded":"2024-09-17T15:25:28.789975+0000","last_fullsized":"2024-09-17T15:25:46.672262+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":1269,"log_dups_size":0,"ondisk_log_size":1269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5186019,"num_objects":1267,"num_object_clones":0,"num_object_copies":2534,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1269,"num_write_kb":5066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1267}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"34'1151","reported_seq":1201,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:46.658971+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:25:46.658971+0000","last_peered":"2024-09-17T15:25:46.658971+0000","last_clean":"2024-09-17T15:25:46.658971+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:25:46.658971+0000","last_undegraded":"2024-09-17T15:25:46.658971+0000","last_fullsized":"2024-09-17T15:25:46.658971+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":1151,"log_dups_size":0,"ondisk_log_size":1151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4714496,"num_objects":1151,"num_object_clones":0,"num_object_copies":2302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1151,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1151,"num_write_kb":4604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"34'922","reported_seq":982,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:44.846968+0000","last_change":"2024-09-17T15:25:27.782128+0000","last_active":"2024-09-17T15:25:44.846968+0000","last_peered":"2024-09-17T15:25:44.846968+0000","last_clean":"2024-09-17T15:24:56.061756+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:25:44.846968+0000","last_undegraded":"2024-09-17T15:25:27.778309+0000","last_fullsized":"2024-09-17T15:25:44.846968+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3756998,"num_objects":919,"num_object_clones":0,"num_object_copies":1838,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":922,"num_write_kb":3672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":919}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"34'999","reported_seq":1056,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:44.829938+0000","last_change":"2024-09-17T15:25:27.789375+0000","last_active":"2024-09-17T15:25:44.829938+0000","last_peered":"2024-09-17T15:25:44.829938+0000","last_clean":"2024-09-17T15:24:56.947098+0000","last_became_active":"2024-09-17T15:25:27.789142+0000","last_became_peered":"2024-09-17T15:25:27.789142+0000","last_unstale":"2024-09-17T15:25:44.829938+0000","last_undegraded":"2024-09-17T15:25:27.779842+0000","last_fullsized":"2024-09-17T15:25:44.829938+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":999,"log_dups_size":0,"ondisk_log_size":999,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4080582,"num_objects":998,"num_object_clones":0,"num_object_copies":1996,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":999,"num_write_kb":3987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":998}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"34'954","reported_seq":1011,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:44.845526+0000","last_change":"2024-09-17T15:25:27.784357+0000","last_active":"2024-09-17T15:25:44.845526+0000","last_peered":"2024-09-17T15:25:44.845526+0000","last_clean":"2024-09-17T15:24:56.310518+0000","last_became_active":"2024-09-17T15:25:27.784135+0000","last_became_peered":"2024-09-17T15:25:27.784135+0000","last_unstale":"2024-09-17T15:25:44.845526+0000","last_undegraded":"2024-09-17T15:25:27.779462+0000","last_fullsized":"2024-09-17T15:25:44.845526+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":954,"log_dups_size":0,"ondisk_log_size":954,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3892166,"num_objects":952,"num_object_clones":0,"num_object_copies":1904,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":954,"num_write_kb":3804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":952}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"34'953","reported_seq":1015,"reported_epoch":34,"state":"active+recovering","last_fresh":"2024-09-17T15:25:44.819184+0000","last_change":"2024-09-17T15:25:28.783075+0000","last_active":"2024-09-17T15:25:44.819184+0000","last_peered":"2024-09-17T15:25:44.819184+0000","last_clean":"2024-09-17T15:24:49.051582+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:25:44.819184+0000","last_undegraded":"2024-09-17T15:25:44.819184+0000","last_fullsized":"2024-09-17T15:25:44.819184+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":953,"log_dups_size":0,"ondisk_log_size":953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3899875,"num_objects":953,"num_object_clones":0,"num_object_copies":1906,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":953,"num_write_kb":3809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":953}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.782961+0000","last_change":"2024-09-17T15:25:27.786793+0000","last_active":"2024-09-17T15:25:28.782961+0000","last_peered":"2024-09-17T15:25:28.782961+0000","last_clean":"2024-09-17T15:25:28.782961+0000","last_became_active":"2024-09-17T15:25:27.786505+0000","last_became_peered":"2024-09-17T15:25:27.786505+0000","last_unstale":"2024-09-17T15:25:28.782961+0000","last_undegraded":"2024-09-17T15:25:28.782961+0000","last_fullsized":"2024-09-17T15:25:28.782961+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:22:15.104318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"34'1220","reported_seq":1280,"reported_epoch":34,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-17T15:25:46.675822+0000","last_change":"2024-09-17T15:25:31.672293+0000","last_active":"2024-09-17T15:25:46.675822+0000","last_peered":"2024-09-17T15:25:46.675822+0000","last_clean":"2024-09-17T15:24:56.758735+0000","last_became_active":"2024-09-17T15:25:28.785807+0000","last_became_peered":"2024-09-17T15:25:28.785807+0000","last_unstale":"2024-09-17T15:25:46.675822+0000","last_undegraded":"2024-09-17T15:25:28.789446+0000","last_fullsized":"2024-09-17T15:25:46.675822+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":1220,"log_dups_size":0,"ondisk_log_size":1220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4977606,"num_objects":1217,"num_object_clones":0,"num_object_copies":2434,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1217,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1220,"num_write_kb":4864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1217}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.787649+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:25:28.787649+0000","last_peered":"2024-09-17T15:25:28.787649+0000","last_clean":"2024-09-17T15:25:28.787649+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:25:28.787649+0000","last_undegraded":"2024-09-17T15:25:28.787649+0000","last_fullsized":"2024-09-17T15:25:28.787649+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"34'916","reported_seq":967,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:44.833723+0000","last_change":"2024-09-17T15:25:27.789502+0000","last_active":"2024-09-17T15:25:44.833723+0000","last_peered":"2024-09-17T15:25:44.833723+0000","last_clean":"2024-09-17T15:24:56.997318+0000","last_became_active":"2024-09-17T15:25:27.789253+0000","last_became_peered":"2024-09-17T15:25:27.789253+0000","last_unstale":"2024-09-17T15:25:44.833723+0000","last_undegraded":"2024-09-17T15:25:27.780228+0000","last_fullsized":"2024-09-17T15:25:44.833723+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":916,"log_dups_size":0,"ondisk_log_size":916,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3728326,"num_objects":912,"num_object_clones":0,"num_object_copies":1824,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":916,"num_write_kb":3644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":912}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":66,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:28.783096+0000","last_change":"2024-09-17T15:25:27.786740+0000","last_active":"2024-09-17T15:25:28.783096+0000","last_peered":"2024-09-17T15:25:28.783096+0000","last_clean":"2024-09-17T15:25:07.444351+0000","last_became_active":"2024-09-17T15:25:27.786489+0000","last_became_peered":"2024-09-17T15:25:27.786489+0000","last_unstale":"2024-09-17T15:25:28.783096+0000","last_undegraded":"2024-09-17T15:25:27.779442+0000","last_fullsized":"2024-09-17T15:25:28.783096+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"34'1200","reported_seq":1251,"reported_epoch":34,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-17T15:25:46.666962+0000","last_change":"2024-09-17T15:25:39.288227+0000","last_active":"2024-09-17T15:25:46.666962+0000","last_peered":"2024-09-17T15:25:46.666962+0000","last_clean":"2024-09-17T15:24:56.012036+0000","last_became_active":"2024-09-17T15:25:28.787663+0000","last_became_peered":"2024-09-17T15:25:28.787663+0000","last_unstale":"2024-09-17T15:25:46.666962+0000","last_undegraded":"2024-09-17T15:25:28.790471+0000","last_fullsized":"2024-09-17T15:25:46.666962+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":1200,"log_dups_size":0,"ondisk_log_size":1200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4903878,"num_objects":1199,"num_object_clones":0,"num_object_copies":2398,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1200,"num_write_kb":4791,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1199}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.787312+0000","last_change":"2024-09-17T15:25:28.786053+0000","last_active":"2024-09-17T15:25:28.787312+0000","last_peered":"2024-09-17T15:25:28.787312+0000","last_clean":"2024-09-17T15:25:28.787312+0000","last_became_active":"2024-09-17T15:25:28.785773+0000","last_became_peered":"2024-09-17T15:25:28.785773+0000","last_unstale":"2024-09-17T15:25:28.787312+0000","last_undegraded":"2024-09-17T15:25:28.787312+0000","last_fullsized":"2024-09-17T15:25:28.787312+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:46:53.130296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"34'1194","reported_seq":1244,"reported_epoch":34,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-17T15:25:46.673102+0000","last_change":"2024-09-17T15:25:39.288719+0000","last_active":"2024-09-17T15:25:46.673102+0000","last_peered":"2024-09-17T15:25:46.673102+0000","last_clean":"2024-09-17T15:24:56.409666+0000","last_became_active":"2024-09-17T15:25:28.785157+0000","last_became_peered":"2024-09-17T15:25:28.785157+0000","last_unstale":"2024-09-17T15:25:46.673102+0000","last_undegraded":"2024-09-17T15:25:28.787777+0000","last_fullsized":"2024-09-17T15:25:46.673102+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":1194,"log_dups_size":0,"ondisk_log_size":1194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4875206,"num_objects":1192,"num_object_clones":0,"num_object_copies":2384,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1194,"num_write_kb":4763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1192}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"34'950","reported_seq":1006,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:44.842926+0000","last_change":"2024-09-17T15:25:27.789404+0000","last_active":"2024-09-17T15:25:44.842926+0000","last_peered":"2024-09-17T15:25:44.842926+0000","last_clean":"2024-09-17T15:24:52.804722+0000","last_became_active":"2024-09-17T15:25:27.789191+0000","last_became_peered":"2024-09-17T15:25:27.789191+0000","last_unstale":"2024-09-17T15:25:44.842926+0000","last_undegraded":"2024-09-17T15:25:27.780610+0000","last_fullsized":"2024-09-17T15:25:44.842926+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":950,"log_dups_size":0,"ondisk_log_size":950,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3887587,"num_objects":950,"num_object_clones":0,"num_object_copies":1900,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":950,"num_write_kb":3797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":950}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.784109+0000","last_change":"2024-09-17T15:25:27.789397+0000","last_active":"2024-09-17T15:25:28.784109+0000","last_peered":"2024-09-17T15:25:28.784109+0000","last_clean":"2024-09-17T15:25:28.784109+0000","last_became_active":"2024-09-17T15:25:27.789137+0000","last_became_peered":"2024-09-17T15:25:27.789137+0000","last_unstale":"2024-09-17T15:25:28.784109+0000","last_undegraded":"2024-09-17T15:25:28.784109+0000","last_fullsized":"2024-09-17T15:25:28.784109+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:26:53.538418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"34'968","reported_seq":1025,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:44.837803+0000","last_change":"2024-09-17T15:25:27.789579+0000","last_active":"2024-09-17T15:25:44.837803+0000","last_peered":"2024-09-17T15:25:44.837803+0000","last_clean":"2024-09-17T15:24:56.897031+0000","last_became_active":"2024-09-17T15:25:27.789325+0000","last_became_peered":"2024-09-17T15:25:27.789325+0000","last_unstale":"2024-09-17T15:25:44.837803+0000","last_undegraded":"2024-09-17T15:25:27.781559+0000","last_fullsized":"2024-09-17T15:25:44.837803+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":968,"log_dups_size":0,"ondisk_log_size":968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3945897,"num_objects":966,"num_object_clones":0,"num_object_copies":1932,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":968,"num_write_kb":3856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":966}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.785793+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:25:28.785793+0000","last_peered":"2024-09-17T15:25:28.785793+0000","last_clean":"2024-09-17T15:25:28.785793+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:25:28.785793+0000","last_undegraded":"2024-09-17T15:25:28.785793+0000","last_fullsized":"2024-09-17T15:25:28.785793+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"34'918","reported_seq":974,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:44.833709+0000","last_change":"2024-09-17T15:25:27.784424+0000","last_active":"2024-09-17T15:25:44.833709+0000","last_peered":"2024-09-17T15:25:44.833709+0000","last_clean":"2024-09-17T15:24:52.804663+0000","last_became_active":"2024-09-17T15:25:27.784195+0000","last_became_peered":"2024-09-17T15:25:27.784195+0000","last_unstale":"2024-09-17T15:25:44.833709+0000","last_undegraded":"2024-09-17T15:25:27.779406+0000","last_fullsized":"2024-09-17T15:25:44.833709+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":918,"log_dups_size":0,"ondisk_log_size":918,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3752902,"num_objects":918,"num_object_clones":0,"num_object_copies":1836,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":918,"num_write_kb":3666,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":918}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.783601+0000","last_change":"2024-09-17T15:25:27.786796+0000","last_active":"2024-09-17T15:25:28.783601+0000","last_peered":"2024-09-17T15:25:28.783601+0000","last_clean":"2024-09-17T15:25:28.783601+0000","last_became_active":"2024-09-17T15:25:27.786506+0000","last_became_peered":"2024-09-17T15:25:27.786506+0000","last_unstale":"2024-09-17T15:25:28.783601+0000","last_undegraded":"2024-09-17T15:25:28.783601+0000","last_fullsized":"2024-09-17T15:25:28.783601+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:55:57.547976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.786536+0000","last_change":"2024-09-17T15:25:28.785423+0000","last_active":"2024-09-17T15:25:28.786536+0000","last_peered":"2024-09-17T15:25:28.786536+0000","last_clean":"2024-09-17T15:25:28.786536+0000","last_became_active":"2024-09-17T15:25:28.785151+0000","last_became_peered":"2024-09-17T15:25:28.785151+0000","last_unstale":"2024-09-17T15:25:28.786536+0000","last_undegraded":"2024-09-17T15:25:28.786536+0000","last_fullsized":"2024-09-17T15:25:28.786536+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:47:12.307723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"34'913","reported_seq":970,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:44.835810+0000","last_change":"2024-09-17T15:25:27.789556+0000","last_active":"2024-09-17T15:25:44.835810+0000","last_peered":"2024-09-17T15:25:44.835810+0000","last_clean":"2024-09-17T15:24:56.609264+0000","last_became_active":"2024-09-17T15:25:27.789304+0000","last_became_peered":"2024-09-17T15:25:27.789304+0000","last_unstale":"2024-09-17T15:25:44.835810+0000","last_undegraded":"2024-09-17T15:25:27.782129+0000","last_fullsized":"2024-09-17T15:25:44.835810+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":913,"log_dups_size":0,"ondisk_log_size":913,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3704716,"num_objects":908,"num_object_clones":0,"num_object_copies":1816,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":908,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":913,"num_write_kb":3623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":907}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":67937492,"num_objects":16611,"num_object_clones":0,"num_object_copies":33222,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":53,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":16611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16638,"num_write_kb":66378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":140619776,"data_stored":140482482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16638,"ondisk_log_size":16638,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051592,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":138060,"kb_used_data":74428,"kb_used_omap":4,"kb_used_meta":63611,"kb_avail":94233780,"statfs":{"total":96636764160,"available":96495390720,"internally_reserved":0,"allocated":76214272,"data_stored":76017654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":65138348},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,8,1,4,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247820,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104748,"kb_used_data":59228,"kb_used_omap":4,"kb_used_meta":45499,"kb_avail":94267092,"statfs":{"total":96636764160,"available":96529502208,"internally_reserved":0,"allocated":60649472,"data_stored":60412647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":46591656},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,2,5,4],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61321216,"data_stored":61285086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":79183872,"data_stored":79183872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:25:49.788 INFO:tasks.lost_unfound:no unfound in 2.f 2024-09-17T15:25:49.788 INFO:tasks.lost_unfound:listing missing/lost in 2.e state active+recovery_unfound+degraded 2024-09-17T15:25:49.788 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.e list_unfound '{}' 2024-09-17T15:25:50.004 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T15:25:50.005 INFO:teuthology.orchestra.run.smithi083.stdout: "offset": { 2024-09-17T15:25:50.005 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": "", 2024-09-17T15:25:50.005 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "", 2024-09-17T15:25:50.005 INFO:teuthology.orchestra.run.smithi083.stdout: "snapid": 0, 2024-09-17T15:25:50.005 INFO:teuthology.orchestra.run.smithi083.stdout: "hash": 0, 2024-09-17T15:25:50.005 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 0, 2024-09-17T15:25:50.005 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": -9223372036854775808, 2024-09-17T15:25:50.005 INFO:teuthology.orchestra.run.smithi083.stdout: "namespace": "" 2024-09-17T15:25:50.005 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:50.005 INFO:teuthology.orchestra.run.smithi083.stdout: "num_missing": 2, 2024-09-17T15:25:50.006 INFO:teuthology.orchestra.run.smithi083.stdout: "num_unfound": 2, 2024-09-17T15:25:50.006 INFO:teuthology.orchestra.run.smithi083.stdout: "objects": [ 2024-09-17T15:25:50.006 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:25:50.006 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": { 2024-09-17T15:25:50.006 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": "existing_4", 2024-09-17T15:25:50.006 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "", 2024-09-17T15:25:50.006 INFO:teuthology.orchestra.run.smithi083.stdout: "snapid": -2, 2024-09-17T15:25:50.006 INFO:teuthology.orchestra.run.smithi083.stdout: "hash": 1224592654, 2024-09-17T15:25:50.006 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 0, 2024-09-17T15:25:50.006 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 2, 2024-09-17T15:25:50.006 INFO:teuthology.orchestra.run.smithi083.stdout: "namespace": "" 2024-09-17T15:25:50.006 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:50.006 INFO:teuthology.orchestra.run.smithi083.stdout: "need": "21'3", 2024-09-17T15:25:50.007 INFO:teuthology.orchestra.run.smithi083.stdout: "have": "19'1", 2024-09-17T15:25:50.007 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": "none", 2024-09-17T15:25:50.007 INFO:teuthology.orchestra.run.smithi083.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-17T15:25:50.007 INFO:teuthology.orchestra.run.smithi083.stdout: "locations": [] 2024-09-17T15:25:50.007 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:50.007 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:25:50.007 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": { 2024-09-17T15:25:50.007 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": "existing_6", 2024-09-17T15:25:50.007 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "", 2024-09-17T15:25:50.007 INFO:teuthology.orchestra.run.smithi083.stdout: "snapid": -2, 2024-09-17T15:25:50.007 INFO:teuthology.orchestra.run.smithi083.stdout: "hash": 3101443614, 2024-09-17T15:25:50.007 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 0, 2024-09-17T15:25:50.007 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 2, 2024-09-17T15:25:50.007 INFO:teuthology.orchestra.run.smithi083.stdout: "namespace": "" 2024-09-17T15:25:50.008 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:50.008 INFO:teuthology.orchestra.run.smithi083.stdout: "need": "21'4", 2024-09-17T15:25:50.008 INFO:teuthology.orchestra.run.smithi083.stdout: "have": "19'2", 2024-09-17T15:25:50.008 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": "none", 2024-09-17T15:25:50.008 INFO:teuthology.orchestra.run.smithi083.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-17T15:25:50.008 INFO:teuthology.orchestra.run.smithi083.stdout: "locations": [] 2024-09-17T15:25:50.008 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T15:25:50.008 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T15:25:50.008 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "NotRecovering", 2024-09-17T15:25:50.008 INFO:teuthology.orchestra.run.smithi083.stdout: "available_might_have_unfound": true, 2024-09-17T15:25:50.008 INFO:teuthology.orchestra.run.smithi083.stdout: "might_have_unfound": [ 2024-09-17T15:25:50.008 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:25:50.008 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "1", 2024-09-17T15:25:50.008 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "osd is down" 2024-09-17T15:25:50.009 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T15:25:50.009 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T15:25:50.009 INFO:teuthology.orchestra.run.smithi083.stdout: "more": false 2024-09-17T15:25:50.009 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T15:25:50.024 INFO:tasks.lost_unfound:reverting unfound in 2.e on osd.2 2024-09-17T15:25:50.024 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.e mark_unfound_lost revert 2024-09-17T15:25:50.242 INFO:teuthology.orchestra.run.smithi083.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-17T15:25:50.262 INFO:tasks.lost_unfound:listing missing/lost in 2.d state active+recovery_unfound+degraded 2024-09-17T15:25:50.262 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.d list_unfound '{}' 2024-09-17T15:25:50.285 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 11 20 26522 26502 9.40959 7.22656 0.0104989 0.00829591 2024-09-17T15:25:50.462 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T15:25:50.462 INFO:teuthology.orchestra.run.smithi083.stdout: "offset": { 2024-09-17T15:25:50.462 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": "", 2024-09-17T15:25:50.462 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "", 2024-09-17T15:25:50.462 INFO:teuthology.orchestra.run.smithi083.stdout: "snapid": 0, 2024-09-17T15:25:50.463 INFO:teuthology.orchestra.run.smithi083.stdout: "hash": 0, 2024-09-17T15:25:50.463 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 0, 2024-09-17T15:25:50.463 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": -9223372036854775808, 2024-09-17T15:25:50.463 INFO:teuthology.orchestra.run.smithi083.stdout: "namespace": "" 2024-09-17T15:25:50.463 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:50.463 INFO:teuthology.orchestra.run.smithi083.stdout: "num_missing": 1, 2024-09-17T15:25:50.463 INFO:teuthology.orchestra.run.smithi083.stdout: "num_unfound": 1, 2024-09-17T15:25:50.463 INFO:teuthology.orchestra.run.smithi083.stdout: "objects": [ 2024-09-17T15:25:50.463 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:25:50.463 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": { 2024-09-17T15:25:50.463 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": "existed_7", 2024-09-17T15:25:50.463 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "", 2024-09-17T15:25:50.463 INFO:teuthology.orchestra.run.smithi083.stdout: "snapid": -2, 2024-09-17T15:25:50.463 INFO:teuthology.orchestra.run.smithi083.stdout: "hash": 2358579165, 2024-09-17T15:25:50.464 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 0, 2024-09-17T15:25:50.464 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 2, 2024-09-17T15:25:50.464 INFO:teuthology.orchestra.run.smithi083.stdout: "namespace": "" 2024-09-17T15:25:50.464 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:50.464 INFO:teuthology.orchestra.run.smithi083.stdout: "need": "21'3", 2024-09-17T15:25:50.464 INFO:teuthology.orchestra.run.smithi083.stdout: "have": "0'0", 2024-09-17T15:25:50.464 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": "none", 2024-09-17T15:25:50.464 INFO:teuthology.orchestra.run.smithi083.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-17T15:25:50.464 INFO:teuthology.orchestra.run.smithi083.stdout: "locations": [] 2024-09-17T15:25:50.464 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T15:25:50.464 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T15:25:50.464 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "NotRecovering", 2024-09-17T15:25:50.464 INFO:teuthology.orchestra.run.smithi083.stdout: "available_might_have_unfound": true, 2024-09-17T15:25:50.464 INFO:teuthology.orchestra.run.smithi083.stdout: "might_have_unfound": [ 2024-09-17T15:25:50.465 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:25:50.465 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "1", 2024-09-17T15:25:50.465 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "osd is down" 2024-09-17T15:25:50.465 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T15:25:50.465 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T15:25:50.465 INFO:teuthology.orchestra.run.smithi083.stdout: "more": false 2024-09-17T15:25:50.465 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T15:25:50.484 INFO:tasks.lost_unfound:reverting unfound in 2.d on osd.2 2024-09-17T15:25:50.484 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.d mark_unfound_lost revert 2024-09-17T15:25:50.753 INFO:teuthology.orchestra.run.smithi083.stderr:pg has 1 objects unfound and apparently lost marking 2024-09-17T15:25:50.773 INFO:tasks.lost_unfound:no unfound in 2.c 2024-09-17T15:25:50.773 INFO:tasks.lost_unfound:listing missing/lost in 2.b state active+recovery_wait+degraded 2024-09-17T15:25:50.773 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.b list_unfound '{}' 2024-09-17T15:25:50.971 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T15:25:50.971 INFO:teuthology.orchestra.run.smithi083.stdout: "offset": { 2024-09-17T15:25:50.971 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": "", 2024-09-17T15:25:50.971 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "", 2024-09-17T15:25:50.971 INFO:teuthology.orchestra.run.smithi083.stdout: "snapid": 0, 2024-09-17T15:25:50.971 INFO:teuthology.orchestra.run.smithi083.stdout: "hash": 0, 2024-09-17T15:25:50.971 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 0, 2024-09-17T15:25:50.971 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": -9223372036854775808, 2024-09-17T15:25:50.972 INFO:teuthology.orchestra.run.smithi083.stdout: "namespace": "" 2024-09-17T15:25:50.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:50.972 INFO:teuthology.orchestra.run.smithi083.stdout: "num_missing": 2, 2024-09-17T15:25:50.972 INFO:teuthology.orchestra.run.smithi083.stdout: "num_unfound": 2, 2024-09-17T15:25:50.972 INFO:teuthology.orchestra.run.smithi083.stdout: "objects": [ 2024-09-17T15:25:50.972 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:25:50.972 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": { 2024-09-17T15:25:50.972 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": "existed_3", 2024-09-17T15:25:50.972 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "", 2024-09-17T15:25:50.972 INFO:teuthology.orchestra.run.smithi083.stdout: "snapid": -2, 2024-09-17T15:25:50.972 INFO:teuthology.orchestra.run.smithi083.stdout: "hash": 943960171, 2024-09-17T15:25:50.972 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 0, 2024-09-17T15:25:50.972 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 2, 2024-09-17T15:25:50.972 INFO:teuthology.orchestra.run.smithi083.stdout: "namespace": "" 2024-09-17T15:25:50.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:50.973 INFO:teuthology.orchestra.run.smithi083.stdout: "need": "21'5", 2024-09-17T15:25:50.973 INFO:teuthology.orchestra.run.smithi083.stdout: "have": "0'0", 2024-09-17T15:25:50.973 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": "none", 2024-09-17T15:25:50.973 INFO:teuthology.orchestra.run.smithi083.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-17T15:25:50.973 INFO:teuthology.orchestra.run.smithi083.stdout: "locations": [] 2024-09-17T15:25:50.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:50.973 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:25:50.973 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": { 2024-09-17T15:25:50.973 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": "existing_1", 2024-09-17T15:25:50.973 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "", 2024-09-17T15:25:50.973 INFO:teuthology.orchestra.run.smithi083.stdout: "snapid": -2, 2024-09-17T15:25:50.973 INFO:teuthology.orchestra.run.smithi083.stdout: "hash": 2885884443, 2024-09-17T15:25:50.973 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 0, 2024-09-17T15:25:50.974 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 2, 2024-09-17T15:25:50.974 INFO:teuthology.orchestra.run.smithi083.stdout: "namespace": "" 2024-09-17T15:25:50.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:50.974 INFO:teuthology.orchestra.run.smithi083.stdout: "need": "21'4", 2024-09-17T15:25:50.974 INFO:teuthology.orchestra.run.smithi083.stdout: "have": "19'1", 2024-09-17T15:25:50.974 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": "none", 2024-09-17T15:25:50.974 INFO:teuthology.orchestra.run.smithi083.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-17T15:25:50.974 INFO:teuthology.orchestra.run.smithi083.stdout: "locations": [] 2024-09-17T15:25:50.974 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T15:25:50.974 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T15:25:50.974 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "Active", 2024-09-17T15:25:50.974 INFO:teuthology.orchestra.run.smithi083.stdout: "available_might_have_unfound": true, 2024-09-17T15:25:50.974 INFO:teuthology.orchestra.run.smithi083.stdout: "might_have_unfound": [ 2024-09-17T15:25:50.974 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:25:50.975 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "1", 2024-09-17T15:25:50.975 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "osd is down" 2024-09-17T15:25:50.975 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T15:25:50.975 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T15:25:50.975 INFO:teuthology.orchestra.run.smithi083.stdout: "more": false 2024-09-17T15:25:50.975 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T15:25:50.986 INFO:tasks.lost_unfound:reverting unfound in 2.b on osd.0 2024-09-17T15:25:50.986 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.b mark_unfound_lost revert 2024-09-17T15:25:51.210 INFO:teuthology.orchestra.run.smithi083.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-17T15:25:51.230 INFO:tasks.lost_unfound:listing missing/lost in 2.a state active+recovery_wait+degraded 2024-09-17T15:25:51.230 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.a list_unfound '{}' 2024-09-17T15:25:51.285 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 12 20 28204 28184 9.17296 6.57031 0.00667797 0.00851048 2024-09-17T15:25:51.421 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T15:25:51.421 INFO:teuthology.orchestra.run.smithi083.stdout: "offset": { 2024-09-17T15:25:51.421 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": "", 2024-09-17T15:25:51.421 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "", 2024-09-17T15:25:51.421 INFO:teuthology.orchestra.run.smithi083.stdout: "snapid": 0, 2024-09-17T15:25:51.422 INFO:teuthology.orchestra.run.smithi083.stdout: "hash": 0, 2024-09-17T15:25:51.422 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 0, 2024-09-17T15:25:51.422 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": -9223372036854775808, 2024-09-17T15:25:51.422 INFO:teuthology.orchestra.run.smithi083.stdout: "namespace": "" 2024-09-17T15:25:51.422 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:51.422 INFO:teuthology.orchestra.run.smithi083.stdout: "num_missing": 2, 2024-09-17T15:25:51.422 INFO:teuthology.orchestra.run.smithi083.stdout: "num_unfound": 2, 2024-09-17T15:25:51.422 INFO:teuthology.orchestra.run.smithi083.stdout: "objects": [ 2024-09-17T15:25:51.422 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:25:51.422 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": { 2024-09-17T15:25:51.422 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": "existing_9", 2024-09-17T15:25:51.422 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "", 2024-09-17T15:25:51.422 INFO:teuthology.orchestra.run.smithi083.stdout: "snapid": -2, 2024-09-17T15:25:51.422 INFO:teuthology.orchestra.run.smithi083.stdout: "hash": 1217469130, 2024-09-17T15:25:51.422 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 0, 2024-09-17T15:25:51.423 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 2, 2024-09-17T15:25:51.423 INFO:teuthology.orchestra.run.smithi083.stdout: "namespace": "" 2024-09-17T15:25:51.423 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:51.423 INFO:teuthology.orchestra.run.smithi083.stdout: "need": "21'3", 2024-09-17T15:25:51.423 INFO:teuthology.orchestra.run.smithi083.stdout: "have": "19'1", 2024-09-17T15:25:51.423 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": "none", 2024-09-17T15:25:51.423 INFO:teuthology.orchestra.run.smithi083.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-17T15:25:51.423 INFO:teuthology.orchestra.run.smithi083.stdout: "locations": [] 2024-09-17T15:25:51.423 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:51.423 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:25:51.423 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": { 2024-09-17T15:25:51.423 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": "new_9", 2024-09-17T15:25:51.423 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "", 2024-09-17T15:25:51.424 INFO:teuthology.orchestra.run.smithi083.stdout: "snapid": -2, 2024-09-17T15:25:51.424 INFO:teuthology.orchestra.run.smithi083.stdout: "hash": 170713514, 2024-09-17T15:25:51.424 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 0, 2024-09-17T15:25:51.424 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 2, 2024-09-17T15:25:51.424 INFO:teuthology.orchestra.run.smithi083.stdout: "namespace": "" 2024-09-17T15:25:51.424 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:51.424 INFO:teuthology.orchestra.run.smithi083.stdout: "need": "21'2", 2024-09-17T15:25:51.424 INFO:teuthology.orchestra.run.smithi083.stdout: "have": "0'0", 2024-09-17T15:25:51.424 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": "none", 2024-09-17T15:25:51.424 INFO:teuthology.orchestra.run.smithi083.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-17T15:25:51.424 INFO:teuthology.orchestra.run.smithi083.stdout: "locations": [] 2024-09-17T15:25:51.424 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T15:25:51.425 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T15:25:51.425 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "Active", 2024-09-17T15:25:51.425 INFO:teuthology.orchestra.run.smithi083.stdout: "available_might_have_unfound": true, 2024-09-17T15:25:51.425 INFO:teuthology.orchestra.run.smithi083.stdout: "might_have_unfound": [ 2024-09-17T15:25:51.425 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:25:51.425 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "1", 2024-09-17T15:25:51.425 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "osd is down" 2024-09-17T15:25:51.425 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T15:25:51.425 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T15:25:51.425 INFO:teuthology.orchestra.run.smithi083.stdout: "more": false 2024-09-17T15:25:51.425 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T15:25:51.440 INFO:tasks.lost_unfound:reverting unfound in 2.a on osd.0 2024-09-17T15:25:51.440 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.a mark_unfound_lost revert 2024-09-17T15:25:51.673 INFO:teuthology.orchestra.run.smithi083.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-17T15:25:51.695 INFO:tasks.lost_unfound:listing missing/lost in 2.9 state active+recovery_wait+degraded 2024-09-17T15:25:51.695 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.9 list_unfound '{}' 2024-09-17T15:25:51.899 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T15:25:51.899 INFO:teuthology.orchestra.run.smithi083.stdout: "offset": { 2024-09-17T15:25:51.899 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": "", 2024-09-17T15:25:51.899 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "", 2024-09-17T15:25:51.899 INFO:teuthology.orchestra.run.smithi083.stdout: "snapid": 0, 2024-09-17T15:25:51.899 INFO:teuthology.orchestra.run.smithi083.stdout: "hash": 0, 2024-09-17T15:25:51.899 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 0, 2024-09-17T15:25:51.899 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": -9223372036854775808, 2024-09-17T15:25:51.899 INFO:teuthology.orchestra.run.smithi083.stdout: "namespace": "" 2024-09-17T15:25:51.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:51.900 INFO:teuthology.orchestra.run.smithi083.stdout: "num_missing": 2, 2024-09-17T15:25:51.900 INFO:teuthology.orchestra.run.smithi083.stdout: "num_unfound": 2, 2024-09-17T15:25:51.900 INFO:teuthology.orchestra.run.smithi083.stdout: "objects": [ 2024-09-17T15:25:51.900 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:25:51.900 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": { 2024-09-17T15:25:51.900 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": "existing_5", 2024-09-17T15:25:51.900 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "", 2024-09-17T15:25:51.900 INFO:teuthology.orchestra.run.smithi083.stdout: "snapid": -2, 2024-09-17T15:25:51.900 INFO:teuthology.orchestra.run.smithi083.stdout: "hash": 2998549577, 2024-09-17T15:25:51.900 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 0, 2024-09-17T15:25:51.900 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 2, 2024-09-17T15:25:51.900 INFO:teuthology.orchestra.run.smithi083.stdout: "namespace": "" 2024-09-17T15:25:51.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:51.900 INFO:teuthology.orchestra.run.smithi083.stdout: "need": "21'4", 2024-09-17T15:25:51.901 INFO:teuthology.orchestra.run.smithi083.stdout: "have": "19'2", 2024-09-17T15:25:51.901 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": "none", 2024-09-17T15:25:51.901 INFO:teuthology.orchestra.run.smithi083.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-17T15:25:51.901 INFO:teuthology.orchestra.run.smithi083.stdout: "locations": [] 2024-09-17T15:25:51.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:51.901 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:25:51.901 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": { 2024-09-17T15:25:51.901 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": "existing_2", 2024-09-17T15:25:51.901 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "", 2024-09-17T15:25:51.901 INFO:teuthology.orchestra.run.smithi083.stdout: "snapid": -2, 2024-09-17T15:25:51.901 INFO:teuthology.orchestra.run.smithi083.stdout: "hash": 4032093161, 2024-09-17T15:25:51.901 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 0, 2024-09-17T15:25:51.901 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 2, 2024-09-17T15:25:51.901 INFO:teuthology.orchestra.run.smithi083.stdout: "namespace": "" 2024-09-17T15:25:51.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:51.902 INFO:teuthology.orchestra.run.smithi083.stdout: "need": "21'3", 2024-09-17T15:25:51.902 INFO:teuthology.orchestra.run.smithi083.stdout: "have": "19'1", 2024-09-17T15:25:51.902 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": "none", 2024-09-17T15:25:51.902 INFO:teuthology.orchestra.run.smithi083.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-17T15:25:51.902 INFO:teuthology.orchestra.run.smithi083.stdout: "locations": [] 2024-09-17T15:25:51.902 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T15:25:51.902 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T15:25:51.902 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "Active", 2024-09-17T15:25:51.902 INFO:teuthology.orchestra.run.smithi083.stdout: "available_might_have_unfound": true, 2024-09-17T15:25:51.902 INFO:teuthology.orchestra.run.smithi083.stdout: "might_have_unfound": [ 2024-09-17T15:25:51.902 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:25:51.902 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "1", 2024-09-17T15:25:51.902 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "osd is down" 2024-09-17T15:25:51.902 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T15:25:51.902 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T15:25:51.903 INFO:teuthology.orchestra.run.smithi083.stdout: "more": false 2024-09-17T15:25:51.903 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T15:25:51.923 INFO:tasks.lost_unfound:reverting unfound in 2.9 on osd.0 2024-09-17T15:25:51.923 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.9 mark_unfound_lost revert 2024-09-17T15:25:52.151 INFO:teuthology.orchestra.run.smithi083.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-17T15:25:52.174 INFO:tasks.lost_unfound:listing missing/lost in 2.8 state active+recovering 2024-09-17T15:25:52.175 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.8 list_unfound '{}' 2024-09-17T15:25:52.285 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 13 20 29858 29838 8.96431 6.46094 0.0100678 0.00870813 2024-09-17T15:25:52.378 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T15:25:52.378 INFO:teuthology.orchestra.run.smithi083.stdout: "offset": { 2024-09-17T15:25:52.378 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": "", 2024-09-17T15:25:52.378 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "", 2024-09-17T15:25:52.378 INFO:teuthology.orchestra.run.smithi083.stdout: "snapid": 0, 2024-09-17T15:25:52.378 INFO:teuthology.orchestra.run.smithi083.stdout: "hash": 0, 2024-09-17T15:25:52.378 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 0, 2024-09-17T15:25:52.378 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": -9223372036854775808, 2024-09-17T15:25:52.378 INFO:teuthology.orchestra.run.smithi083.stdout: "namespace": "" 2024-09-17T15:25:52.378 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:52.379 INFO:teuthology.orchestra.run.smithi083.stdout: "num_missing": 1, 2024-09-17T15:25:52.379 INFO:teuthology.orchestra.run.smithi083.stdout: "num_unfound": 1, 2024-09-17T15:25:52.379 INFO:teuthology.orchestra.run.smithi083.stdout: "objects": [ 2024-09-17T15:25:52.379 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:25:52.379 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": { 2024-09-17T15:25:52.379 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": "new_8", 2024-09-17T15:25:52.379 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "", 2024-09-17T15:25:52.379 INFO:teuthology.orchestra.run.smithi083.stdout: "snapid": -2, 2024-09-17T15:25:52.379 INFO:teuthology.orchestra.run.smithi083.stdout: "hash": 2730327960, 2024-09-17T15:25:52.379 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 0, 2024-09-17T15:25:52.379 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 2, 2024-09-17T15:25:52.379 INFO:teuthology.orchestra.run.smithi083.stdout: "namespace": "" 2024-09-17T15:25:52.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:52.379 INFO:teuthology.orchestra.run.smithi083.stdout: "need": "21'1", 2024-09-17T15:25:52.379 INFO:teuthology.orchestra.run.smithi083.stdout: "have": "0'0", 2024-09-17T15:25:52.380 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": "none", 2024-09-17T15:25:52.380 INFO:teuthology.orchestra.run.smithi083.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-17T15:25:52.380 INFO:teuthology.orchestra.run.smithi083.stdout: "locations": [] 2024-09-17T15:25:52.380 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T15:25:52.380 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T15:25:52.380 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "Active", 2024-09-17T15:25:52.380 INFO:teuthology.orchestra.run.smithi083.stdout: "available_might_have_unfound": true, 2024-09-17T15:25:52.380 INFO:teuthology.orchestra.run.smithi083.stdout: "might_have_unfound": [ 2024-09-17T15:25:52.380 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:25:52.380 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "1", 2024-09-17T15:25:52.380 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "osd is down" 2024-09-17T15:25:52.380 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T15:25:52.380 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T15:25:52.380 INFO:teuthology.orchestra.run.smithi083.stdout: "more": false 2024-09-17T15:25:52.381 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T15:25:52.400 INFO:tasks.lost_unfound:reverting unfound in 2.8 on osd.0 2024-09-17T15:25:52.400 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.8 mark_unfound_lost revert 2024-09-17T15:25:52.644 INFO:teuthology.orchestra.run.smithi083.stderr:pg has 1 objects unfound and apparently lost marking 2024-09-17T15:25:52.660 INFO:tasks.lost_unfound:no unfound in 1.0 2024-09-17T15:25:52.660 INFO:tasks.lost_unfound:listing missing/lost in 2.3 state active+recovery_unfound+degraded 2024-09-17T15:25:52.660 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.3 list_unfound '{}' 2024-09-17T15:25:52.867 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T15:25:52.867 INFO:teuthology.orchestra.run.smithi083.stdout: "offset": { 2024-09-17T15:25:52.867 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": "", 2024-09-17T15:25:52.867 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "", 2024-09-17T15:25:52.867 INFO:teuthology.orchestra.run.smithi083.stdout: "snapid": 0, 2024-09-17T15:25:52.867 INFO:teuthology.orchestra.run.smithi083.stdout: "hash": 0, 2024-09-17T15:25:52.867 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 0, 2024-09-17T15:25:52.867 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": -9223372036854775808, 2024-09-17T15:25:52.867 INFO:teuthology.orchestra.run.smithi083.stdout: "namespace": "" 2024-09-17T15:25:52.867 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:52.867 INFO:teuthology.orchestra.run.smithi083.stdout: "num_missing": 2, 2024-09-17T15:25:52.867 INFO:teuthology.orchestra.run.smithi083.stdout: "num_unfound": 2, 2024-09-17T15:25:52.868 INFO:teuthology.orchestra.run.smithi083.stdout: "objects": [ 2024-09-17T15:25:52.868 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:25:52.868 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": { 2024-09-17T15:25:52.868 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": "existing_8", 2024-09-17T15:25:52.868 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "", 2024-09-17T15:25:52.868 INFO:teuthology.orchestra.run.smithi083.stdout: "snapid": -2, 2024-09-17T15:25:52.868 INFO:teuthology.orchestra.run.smithi083.stdout: "hash": 1901787523, 2024-09-17T15:25:52.868 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 0, 2024-09-17T15:25:52.868 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 2, 2024-09-17T15:25:52.868 INFO:teuthology.orchestra.run.smithi083.stdout: "namespace": "" 2024-09-17T15:25:52.868 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:52.868 INFO:teuthology.orchestra.run.smithi083.stdout: "need": "21'5", 2024-09-17T15:25:52.868 INFO:teuthology.orchestra.run.smithi083.stdout: "have": "19'3", 2024-09-17T15:25:52.868 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": "none", 2024-09-17T15:25:52.869 INFO:teuthology.orchestra.run.smithi083.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-17T15:25:52.869 INFO:teuthology.orchestra.run.smithi083.stdout: "locations": [] 2024-09-17T15:25:52.869 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:52.869 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:25:52.869 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": { 2024-09-17T15:25:52.869 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": "existed_1", 2024-09-17T15:25:52.869 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "", 2024-09-17T15:25:52.869 INFO:teuthology.orchestra.run.smithi083.stdout: "snapid": -2, 2024-09-17T15:25:52.869 INFO:teuthology.orchestra.run.smithi083.stdout: "hash": 3122379363, 2024-09-17T15:25:52.869 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 0, 2024-09-17T15:25:52.869 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 2, 2024-09-17T15:25:52.869 INFO:teuthology.orchestra.run.smithi083.stdout: "namespace": "" 2024-09-17T15:25:52.869 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:52.869 INFO:teuthology.orchestra.run.smithi083.stdout: "need": "21'4", 2024-09-17T15:25:52.870 INFO:teuthology.orchestra.run.smithi083.stdout: "have": "0'0", 2024-09-17T15:25:52.870 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": "none", 2024-09-17T15:25:52.870 INFO:teuthology.orchestra.run.smithi083.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-17T15:25:52.870 INFO:teuthology.orchestra.run.smithi083.stdout: "locations": [] 2024-09-17T15:25:52.870 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T15:25:52.870 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T15:25:52.870 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "NotRecovering", 2024-09-17T15:25:52.870 INFO:teuthology.orchestra.run.smithi083.stdout: "available_might_have_unfound": true, 2024-09-17T15:25:52.870 INFO:teuthology.orchestra.run.smithi083.stdout: "might_have_unfound": [ 2024-09-17T15:25:52.870 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:25:52.870 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "1", 2024-09-17T15:25:52.870 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "osd is down" 2024-09-17T15:25:52.870 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T15:25:52.870 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T15:25:52.871 INFO:teuthology.orchestra.run.smithi083.stdout: "more": false 2024-09-17T15:25:52.871 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T15:25:52.887 INFO:tasks.lost_unfound:reverting unfound in 2.3 on osd.2 2024-09-17T15:25:52.888 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.3 mark_unfound_lost revert 2024-09-17T15:25:53.108 INFO:teuthology.orchestra.run.smithi083.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-17T15:25:53.127 INFO:tasks.lost_unfound:no unfound in 1.1 2024-09-17T15:25:53.127 INFO:tasks.lost_unfound:listing missing/lost in 2.2 state active+recovery_wait+degraded 2024-09-17T15:25:53.127 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.2 list_unfound '{}' 2024-09-17T15:25:53.285 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 14 20 31572 31552 8.80222 6.69531 0.0111811 0.00886684 2024-09-17T15:25:53.330 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T15:25:53.331 INFO:teuthology.orchestra.run.smithi083.stdout: "offset": { 2024-09-17T15:25:53.331 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": "", 2024-09-17T15:25:53.331 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "", 2024-09-17T15:25:53.331 INFO:teuthology.orchestra.run.smithi083.stdout: "snapid": 0, 2024-09-17T15:25:53.331 INFO:teuthology.orchestra.run.smithi083.stdout: "hash": 0, 2024-09-17T15:25:53.331 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 0, 2024-09-17T15:25:53.331 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": -9223372036854775808, 2024-09-17T15:25:53.331 INFO:teuthology.orchestra.run.smithi083.stdout: "namespace": "" 2024-09-17T15:25:53.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:53.331 INFO:teuthology.orchestra.run.smithi083.stdout: "num_missing": 2, 2024-09-17T15:25:53.331 INFO:teuthology.orchestra.run.smithi083.stdout: "num_unfound": 2, 2024-09-17T15:25:53.332 INFO:teuthology.orchestra.run.smithi083.stdout: "objects": [ 2024-09-17T15:25:53.332 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:25:53.332 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": { 2024-09-17T15:25:53.332 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": "existed_6", 2024-09-17T15:25:53.332 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "", 2024-09-17T15:25:53.332 INFO:teuthology.orchestra.run.smithi083.stdout: "snapid": -2, 2024-09-17T15:25:53.332 INFO:teuthology.orchestra.run.smithi083.stdout: "hash": 1708673474, 2024-09-17T15:25:53.332 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 0, 2024-09-17T15:25:53.332 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 2, 2024-09-17T15:25:53.332 INFO:teuthology.orchestra.run.smithi083.stdout: "namespace": "" 2024-09-17T15:25:53.332 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:53.332 INFO:teuthology.orchestra.run.smithi083.stdout: "need": "21'5", 2024-09-17T15:25:53.332 INFO:teuthology.orchestra.run.smithi083.stdout: "have": "0'0", 2024-09-17T15:25:53.333 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": "none", 2024-09-17T15:25:53.333 INFO:teuthology.orchestra.run.smithi083.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-17T15:25:53.333 INFO:teuthology.orchestra.run.smithi083.stdout: "locations": [] 2024-09-17T15:25:53.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:53.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:25:53.333 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": { 2024-09-17T15:25:53.333 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": "existed_9", 2024-09-17T15:25:53.333 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "", 2024-09-17T15:25:53.333 INFO:teuthology.orchestra.run.smithi083.stdout: "snapid": -2, 2024-09-17T15:25:53.333 INFO:teuthology.orchestra.run.smithi083.stdout: "hash": 831402034, 2024-09-17T15:25:53.333 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 0, 2024-09-17T15:25:53.333 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 2, 2024-09-17T15:25:53.334 INFO:teuthology.orchestra.run.smithi083.stdout: "namespace": "" 2024-09-17T15:25:53.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:53.334 INFO:teuthology.orchestra.run.smithi083.stdout: "need": "21'6", 2024-09-17T15:25:53.334 INFO:teuthology.orchestra.run.smithi083.stdout: "have": "0'0", 2024-09-17T15:25:53.334 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": "none", 2024-09-17T15:25:53.334 INFO:teuthology.orchestra.run.smithi083.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-17T15:25:53.334 INFO:teuthology.orchestra.run.smithi083.stdout: "locations": [] 2024-09-17T15:25:53.334 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T15:25:53.334 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T15:25:53.334 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "Active", 2024-09-17T15:25:53.334 INFO:teuthology.orchestra.run.smithi083.stdout: "available_might_have_unfound": true, 2024-09-17T15:25:53.334 INFO:teuthology.orchestra.run.smithi083.stdout: "might_have_unfound": [ 2024-09-17T15:25:53.334 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:25:53.335 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "1", 2024-09-17T15:25:53.335 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "osd is down" 2024-09-17T15:25:53.335 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T15:25:53.335 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T15:25:53.335 INFO:teuthology.orchestra.run.smithi083.stdout: "more": false 2024-09-17T15:25:53.335 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T15:25:53.353 INFO:tasks.lost_unfound:reverting unfound in 2.2 on osd.0 2024-09-17T15:25:53.353 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.2 mark_unfound_lost revert 2024-09-17T15:25:53.575 INFO:teuthology.orchestra.run.smithi083.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-17T15:25:53.593 INFO:tasks.lost_unfound:no unfound in 1.2 2024-09-17T15:25:53.593 INFO:tasks.lost_unfound:listing missing/lost in 2.1 state active+recovery_unfound+degraded 2024-09-17T15:25:53.593 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.1 list_unfound '{}' 2024-09-17T15:25:53.812 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T15:25:53.812 INFO:teuthology.orchestra.run.smithi083.stdout: "offset": { 2024-09-17T15:25:53.812 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": "", 2024-09-17T15:25:53.813 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "", 2024-09-17T15:25:53.813 INFO:teuthology.orchestra.run.smithi083.stdout: "snapid": 0, 2024-09-17T15:25:53.813 INFO:teuthology.orchestra.run.smithi083.stdout: "hash": 0, 2024-09-17T15:25:53.813 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 0, 2024-09-17T15:25:53.813 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": -9223372036854775808, 2024-09-17T15:25:53.813 INFO:teuthology.orchestra.run.smithi083.stdout: "namespace": "" 2024-09-17T15:25:53.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:53.813 INFO:teuthology.orchestra.run.smithi083.stdout: "num_missing": 2, 2024-09-17T15:25:53.813 INFO:teuthology.orchestra.run.smithi083.stdout: "num_unfound": 2, 2024-09-17T15:25:53.813 INFO:teuthology.orchestra.run.smithi083.stdout: "objects": [ 2024-09-17T15:25:53.813 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:25:53.813 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": { 2024-09-17T15:25:53.813 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": "existing_3", 2024-09-17T15:25:53.814 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "", 2024-09-17T15:25:53.814 INFO:teuthology.orchestra.run.smithi083.stdout: "snapid": -2, 2024-09-17T15:25:53.814 INFO:teuthology.orchestra.run.smithi083.stdout: "hash": 459768081, 2024-09-17T15:25:53.814 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 0, 2024-09-17T15:25:53.814 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 2, 2024-09-17T15:25:53.814 INFO:teuthology.orchestra.run.smithi083.stdout: "namespace": "" 2024-09-17T15:25:53.814 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:53.814 INFO:teuthology.orchestra.run.smithi083.stdout: "need": "21'3", 2024-09-17T15:25:53.814 INFO:teuthology.orchestra.run.smithi083.stdout: "have": "19'1", 2024-09-17T15:25:53.814 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": "none", 2024-09-17T15:25:53.814 INFO:teuthology.orchestra.run.smithi083.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-17T15:25:53.814 INFO:teuthology.orchestra.run.smithi083.stdout: "locations": [] 2024-09-17T15:25:53.815 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:53.815 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:25:53.815 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": { 2024-09-17T15:25:53.815 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": "new_3", 2024-09-17T15:25:53.815 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "", 2024-09-17T15:25:53.815 INFO:teuthology.orchestra.run.smithi083.stdout: "snapid": -2, 2024-09-17T15:25:53.815 INFO:teuthology.orchestra.run.smithi083.stdout: "hash": 480221105, 2024-09-17T15:25:53.815 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 0, 2024-09-17T15:25:53.815 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 2, 2024-09-17T15:25:53.815 INFO:teuthology.orchestra.run.smithi083.stdout: "namespace": "" 2024-09-17T15:25:53.815 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:53.815 INFO:teuthology.orchestra.run.smithi083.stdout: "need": "21'2", 2024-09-17T15:25:53.816 INFO:teuthology.orchestra.run.smithi083.stdout: "have": "0'0", 2024-09-17T15:25:53.816 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": "none", 2024-09-17T15:25:53.816 INFO:teuthology.orchestra.run.smithi083.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-17T15:25:53.816 INFO:teuthology.orchestra.run.smithi083.stdout: "locations": [] 2024-09-17T15:25:53.816 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T15:25:53.816 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T15:25:53.816 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "NotRecovering", 2024-09-17T15:25:53.816 INFO:teuthology.orchestra.run.smithi083.stdout: "available_might_have_unfound": true, 2024-09-17T15:25:53.816 INFO:teuthology.orchestra.run.smithi083.stdout: "might_have_unfound": [ 2024-09-17T15:25:53.816 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:25:53.816 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "1", 2024-09-17T15:25:53.816 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "osd is down" 2024-09-17T15:25:53.817 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T15:25:53.817 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T15:25:53.817 INFO:teuthology.orchestra.run.smithi083.stdout: "more": false 2024-09-17T15:25:53.817 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T15:25:53.830 INFO:tasks.lost_unfound:reverting unfound in 2.1 on osd.2 2024-09-17T15:25:53.830 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.1 mark_unfound_lost revert 2024-09-17T15:25:53.977 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-09-17T15:25:53.976+0000 7f514c979640 -1 log_channel(cluster) log [ERR] : Health check update: Possible data damage: 3 pgs recovery_unfound (PG_DAMAGED) 2024-09-17T15:25:54.054 INFO:teuthology.orchestra.run.smithi083.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-17T15:25:54.073 INFO:tasks.lost_unfound:no unfound in 1.3 2024-09-17T15:25:54.073 INFO:tasks.lost_unfound:listing missing/lost in 2.0 state active+recovery_unfound+degraded 2024-09-17T15:25:54.074 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.0 list_unfound '{}' 2024-09-17T15:25:54.285 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 15 20 33043 33023 8.59846 5.74609 0.0135195 0.00904459 2024-09-17T15:25:54.352 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T15:25:54.352 INFO:teuthology.orchestra.run.smithi083.stdout: "offset": { 2024-09-17T15:25:54.352 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": "", 2024-09-17T15:25:54.352 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "", 2024-09-17T15:25:54.352 INFO:teuthology.orchestra.run.smithi083.stdout: "snapid": 0, 2024-09-17T15:25:54.353 INFO:teuthology.orchestra.run.smithi083.stdout: "hash": 0, 2024-09-17T15:25:54.353 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 0, 2024-09-17T15:25:54.353 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": -9223372036854775808, 2024-09-17T15:25:54.353 INFO:teuthology.orchestra.run.smithi083.stdout: "namespace": "" 2024-09-17T15:25:54.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:54.353 INFO:teuthology.orchestra.run.smithi083.stdout: "num_missing": 2, 2024-09-17T15:25:54.353 INFO:teuthology.orchestra.run.smithi083.stdout: "num_unfound": 2, 2024-09-17T15:25:54.353 INFO:teuthology.orchestra.run.smithi083.stdout: "objects": [ 2024-09-17T15:25:54.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:25:54.353 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": { 2024-09-17T15:25:54.353 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": "new_5", 2024-09-17T15:25:54.353 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "", 2024-09-17T15:25:54.353 INFO:teuthology.orchestra.run.smithi083.stdout: "snapid": -2, 2024-09-17T15:25:54.353 INFO:teuthology.orchestra.run.smithi083.stdout: "hash": 4248945696, 2024-09-17T15:25:54.354 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 0, 2024-09-17T15:25:54.354 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 2, 2024-09-17T15:25:54.354 INFO:teuthology.orchestra.run.smithi083.stdout: "namespace": "" 2024-09-17T15:25:54.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:54.354 INFO:teuthology.orchestra.run.smithi083.stdout: "need": "21'3", 2024-09-17T15:25:54.354 INFO:teuthology.orchestra.run.smithi083.stdout: "have": "0'0", 2024-09-17T15:25:54.354 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": "none", 2024-09-17T15:25:54.354 INFO:teuthology.orchestra.run.smithi083.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-17T15:25:54.354 INFO:teuthology.orchestra.run.smithi083.stdout: "locations": [] 2024-09-17T15:25:54.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:54.354 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:25:54.354 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": { 2024-09-17T15:25:54.354 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": "existed_5", 2024-09-17T15:25:54.354 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "", 2024-09-17T15:25:54.355 INFO:teuthology.orchestra.run.smithi083.stdout: "snapid": -2, 2024-09-17T15:25:54.355 INFO:teuthology.orchestra.run.smithi083.stdout: "hash": 4212789968, 2024-09-17T15:25:54.355 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 0, 2024-09-17T15:25:54.355 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 2, 2024-09-17T15:25:54.355 INFO:teuthology.orchestra.run.smithi083.stdout: "namespace": "" 2024-09-17T15:25:54.355 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:54.355 INFO:teuthology.orchestra.run.smithi083.stdout: "need": "21'4", 2024-09-17T15:25:54.355 INFO:teuthology.orchestra.run.smithi083.stdout: "have": "0'0", 2024-09-17T15:25:54.355 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": "none", 2024-09-17T15:25:54.355 INFO:teuthology.orchestra.run.smithi083.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-17T15:25:54.355 INFO:teuthology.orchestra.run.smithi083.stdout: "locations": [] 2024-09-17T15:25:54.355 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T15:25:54.355 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T15:25:54.355 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "NotRecovering", 2024-09-17T15:25:54.356 INFO:teuthology.orchestra.run.smithi083.stdout: "available_might_have_unfound": true, 2024-09-17T15:25:54.356 INFO:teuthology.orchestra.run.smithi083.stdout: "might_have_unfound": [ 2024-09-17T15:25:54.356 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:25:54.356 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "1", 2024-09-17T15:25:54.356 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "osd is down" 2024-09-17T15:25:54.356 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T15:25:54.356 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T15:25:54.356 INFO:teuthology.orchestra.run.smithi083.stdout: "more": false 2024-09-17T15:25:54.356 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T15:25:54.370 INFO:tasks.lost_unfound:reverting unfound in 2.0 on osd.2 2024-09-17T15:25:54.371 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.0 mark_unfound_lost revert 2024-09-17T15:25:54.592 INFO:teuthology.orchestra.run.smithi083.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-17T15:25:54.613 INFO:tasks.lost_unfound:listing missing/lost in 2.7 state active+recovery_wait+degraded 2024-09-17T15:25:54.613 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.7 list_unfound '{}' 2024-09-17T15:25:54.829 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T15:25:54.829 INFO:teuthology.orchestra.run.smithi083.stdout: "offset": { 2024-09-17T15:25:54.829 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": "", 2024-09-17T15:25:54.829 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "", 2024-09-17T15:25:54.829 INFO:teuthology.orchestra.run.smithi083.stdout: "snapid": 0, 2024-09-17T15:25:54.829 INFO:teuthology.orchestra.run.smithi083.stdout: "hash": 0, 2024-09-17T15:25:54.829 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 0, 2024-09-17T15:25:54.829 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": -9223372036854775808, 2024-09-17T15:25:54.829 INFO:teuthology.orchestra.run.smithi083.stdout: "namespace": "" 2024-09-17T15:25:54.829 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:54.830 INFO:teuthology.orchestra.run.smithi083.stdout: "num_missing": 1, 2024-09-17T15:25:54.830 INFO:teuthology.orchestra.run.smithi083.stdout: "num_unfound": 1, 2024-09-17T15:25:54.830 INFO:teuthology.orchestra.run.smithi083.stdout: "objects": [ 2024-09-17T15:25:54.830 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:25:54.830 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": { 2024-09-17T15:25:54.830 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": "new_7", 2024-09-17T15:25:54.830 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "", 2024-09-17T15:25:54.830 INFO:teuthology.orchestra.run.smithi083.stdout: "snapid": -2, 2024-09-17T15:25:54.830 INFO:teuthology.orchestra.run.smithi083.stdout: "hash": 3761017527, 2024-09-17T15:25:54.830 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 0, 2024-09-17T15:25:54.830 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 2, 2024-09-17T15:25:54.831 INFO:teuthology.orchestra.run.smithi083.stdout: "namespace": "" 2024-09-17T15:25:54.831 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:54.831 INFO:teuthology.orchestra.run.smithi083.stdout: "need": "21'1", 2024-09-17T15:25:54.831 INFO:teuthology.orchestra.run.smithi083.stdout: "have": "0'0", 2024-09-17T15:25:54.831 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": "none", 2024-09-17T15:25:54.831 INFO:teuthology.orchestra.run.smithi083.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-17T15:25:54.831 INFO:teuthology.orchestra.run.smithi083.stdout: "locations": [] 2024-09-17T15:25:54.831 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T15:25:54.831 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T15:25:54.831 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "Active", 2024-09-17T15:25:54.831 INFO:teuthology.orchestra.run.smithi083.stdout: "available_might_have_unfound": true, 2024-09-17T15:25:54.831 INFO:teuthology.orchestra.run.smithi083.stdout: "might_have_unfound": [ 2024-09-17T15:25:54.831 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:25:54.832 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "1", 2024-09-17T15:25:54.832 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "osd is down" 2024-09-17T15:25:54.832 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T15:25:54.832 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T15:25:54.832 INFO:teuthology.orchestra.run.smithi083.stdout: "more": false 2024-09-17T15:25:54.832 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T15:25:54.850 INFO:tasks.lost_unfound:reverting unfound in 2.7 on osd.0 2024-09-17T15:25:54.850 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.7 mark_unfound_lost revert 2024-09-17T15:25:55.084 INFO:teuthology.orchestra.run.smithi083.stderr:pg has 1 objects unfound and apparently lost marking 2024-09-17T15:25:55.106 INFO:tasks.lost_unfound:no unfound in 1.4 2024-09-17T15:25:55.106 INFO:tasks.lost_unfound:listing missing/lost in 2.6 state active+recovery_wait+degraded 2024-09-17T15:25:55.106 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.6 list_unfound '{}' 2024-09-17T15:25:55.285 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 16 20 34534 34514 8.42504 5.82422 0.0100456 0.00926588 2024-09-17T15:25:55.329 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T15:25:55.329 INFO:teuthology.orchestra.run.smithi083.stdout: "offset": { 2024-09-17T15:25:55.330 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": "", 2024-09-17T15:25:55.330 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "", 2024-09-17T15:25:55.330 INFO:teuthology.orchestra.run.smithi083.stdout: "snapid": 0, 2024-09-17T15:25:55.330 INFO:teuthology.orchestra.run.smithi083.stdout: "hash": 0, 2024-09-17T15:25:55.330 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 0, 2024-09-17T15:25:55.330 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": -9223372036854775808, 2024-09-17T15:25:55.330 INFO:teuthology.orchestra.run.smithi083.stdout: "namespace": "" 2024-09-17T15:25:55.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:55.330 INFO:teuthology.orchestra.run.smithi083.stdout: "num_missing": 3, 2024-09-17T15:25:55.330 INFO:teuthology.orchestra.run.smithi083.stdout: "num_unfound": 3, 2024-09-17T15:25:55.330 INFO:teuthology.orchestra.run.smithi083.stdout: "objects": [ 2024-09-17T15:25:55.330 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:25:55.331 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": { 2024-09-17T15:25:55.331 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": "existed_8", 2024-09-17T15:25:55.331 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "", 2024-09-17T15:25:55.331 INFO:teuthology.orchestra.run.smithi083.stdout: "snapid": -2, 2024-09-17T15:25:55.331 INFO:teuthology.orchestra.run.smithi083.stdout: "hash": 1861424966, 2024-09-17T15:25:55.331 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 0, 2024-09-17T15:25:55.331 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 2, 2024-09-17T15:25:55.331 INFO:teuthology.orchestra.run.smithi083.stdout: "namespace": "" 2024-09-17T15:25:55.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:55.331 INFO:teuthology.orchestra.run.smithi083.stdout: "need": "21'5", 2024-09-17T15:25:55.331 INFO:teuthology.orchestra.run.smithi083.stdout: "have": "0'0", 2024-09-17T15:25:55.331 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": "none", 2024-09-17T15:25:55.331 INFO:teuthology.orchestra.run.smithi083.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-17T15:25:55.331 INFO:teuthology.orchestra.run.smithi083.stdout: "locations": [] 2024-09-17T15:25:55.332 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:55.332 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:25:55.332 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": { 2024-09-17T15:25:55.332 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": "new_6", 2024-09-17T15:25:55.332 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "", 2024-09-17T15:25:55.332 INFO:teuthology.orchestra.run.smithi083.stdout: "snapid": -2, 2024-09-17T15:25:55.332 INFO:teuthology.orchestra.run.smithi083.stdout: "hash": 3910131350, 2024-09-17T15:25:55.332 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 0, 2024-09-17T15:25:55.332 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 2, 2024-09-17T15:25:55.332 INFO:teuthology.orchestra.run.smithi083.stdout: "namespace": "" 2024-09-17T15:25:55.332 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:55.332 INFO:teuthology.orchestra.run.smithi083.stdout: "need": "21'4", 2024-09-17T15:25:55.332 INFO:teuthology.orchestra.run.smithi083.stdout: "have": "0'0", 2024-09-17T15:25:55.333 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": "none", 2024-09-17T15:25:55.333 INFO:teuthology.orchestra.run.smithi083.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-17T15:25:55.333 INFO:teuthology.orchestra.run.smithi083.stdout: "locations": [] 2024-09-17T15:25:55.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:55.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:25:55.333 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": { 2024-09-17T15:25:55.333 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": "new_4", 2024-09-17T15:25:55.333 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "", 2024-09-17T15:25:55.333 INFO:teuthology.orchestra.run.smithi083.stdout: "snapid": -2, 2024-09-17T15:25:55.333 INFO:teuthology.orchestra.run.smithi083.stdout: "hash": 2787839990, 2024-09-17T15:25:55.333 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 0, 2024-09-17T15:25:55.333 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 2, 2024-09-17T15:25:55.333 INFO:teuthology.orchestra.run.smithi083.stdout: "namespace": "" 2024-09-17T15:25:55.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:55.334 INFO:teuthology.orchestra.run.smithi083.stdout: "need": "21'3", 2024-09-17T15:25:55.334 INFO:teuthology.orchestra.run.smithi083.stdout: "have": "0'0", 2024-09-17T15:25:55.334 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": "none", 2024-09-17T15:25:55.334 INFO:teuthology.orchestra.run.smithi083.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-17T15:25:55.334 INFO:teuthology.orchestra.run.smithi083.stdout: "locations": [] 2024-09-17T15:25:55.334 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T15:25:55.334 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T15:25:55.334 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "Active", 2024-09-17T15:25:55.334 INFO:teuthology.orchestra.run.smithi083.stdout: "available_might_have_unfound": true, 2024-09-17T15:25:55.334 INFO:teuthology.orchestra.run.smithi083.stdout: "might_have_unfound": [ 2024-09-17T15:25:55.334 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:25:55.334 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "1", 2024-09-17T15:25:55.334 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "osd is down" 2024-09-17T15:25:55.335 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T15:25:55.335 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T15:25:55.335 INFO:teuthology.orchestra.run.smithi083.stdout: "more": false 2024-09-17T15:25:55.335 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T15:25:55.353 INFO:tasks.lost_unfound:reverting unfound in 2.6 on osd.0 2024-09-17T15:25:55.354 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.6 mark_unfound_lost revert 2024-09-17T15:25:55.572 INFO:teuthology.orchestra.run.smithi083.stderr:pg has 3 objects unfound and apparently lost marking 2024-09-17T15:25:55.592 INFO:tasks.lost_unfound:no unfound in 1.5 2024-09-17T15:25:55.592 INFO:tasks.lost_unfound:listing missing/lost in 2.5 state active+recovery_wait+degraded 2024-09-17T15:25:55.592 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.5 list_unfound '{}' 2024-09-17T15:25:55.805 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T15:25:55.805 INFO:teuthology.orchestra.run.smithi083.stdout: "offset": { 2024-09-17T15:25:55.805 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": "", 2024-09-17T15:25:55.805 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "", 2024-09-17T15:25:55.805 INFO:teuthology.orchestra.run.smithi083.stdout: "snapid": 0, 2024-09-17T15:25:55.805 INFO:teuthology.orchestra.run.smithi083.stdout: "hash": 0, 2024-09-17T15:25:55.805 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 0, 2024-09-17T15:25:55.805 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": -9223372036854775808, 2024-09-17T15:25:55.805 INFO:teuthology.orchestra.run.smithi083.stdout: "namespace": "" 2024-09-17T15:25:55.805 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:55.805 INFO:teuthology.orchestra.run.smithi083.stdout: "num_missing": 2, 2024-09-17T15:25:55.805 INFO:teuthology.orchestra.run.smithi083.stdout: "num_unfound": 2, 2024-09-17T15:25:55.806 INFO:teuthology.orchestra.run.smithi083.stdout: "objects": [ 2024-09-17T15:25:55.806 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:25:55.806 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": { 2024-09-17T15:25:55.806 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": "new_1", 2024-09-17T15:25:55.806 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "", 2024-09-17T15:25:55.806 INFO:teuthology.orchestra.run.smithi083.stdout: "snapid": -2, 2024-09-17T15:25:55.806 INFO:teuthology.orchestra.run.smithi083.stdout: "hash": 26281365, 2024-09-17T15:25:55.806 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 0, 2024-09-17T15:25:55.806 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 2, 2024-09-17T15:25:55.806 INFO:teuthology.orchestra.run.smithi083.stdout: "namespace": "" 2024-09-17T15:25:55.806 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:55.806 INFO:teuthology.orchestra.run.smithi083.stdout: "need": "21'1", 2024-09-17T15:25:55.806 INFO:teuthology.orchestra.run.smithi083.stdout: "have": "0'0", 2024-09-17T15:25:55.806 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": "none", 2024-09-17T15:25:55.807 INFO:teuthology.orchestra.run.smithi083.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-17T15:25:55.807 INFO:teuthology.orchestra.run.smithi083.stdout: "locations": [] 2024-09-17T15:25:55.807 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:55.807 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:25:55.807 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": { 2024-09-17T15:25:55.807 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": "new_2", 2024-09-17T15:25:55.807 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "", 2024-09-17T15:25:55.807 INFO:teuthology.orchestra.run.smithi083.stdout: "snapid": -2, 2024-09-17T15:25:55.807 INFO:teuthology.orchestra.run.smithi083.stdout: "hash": 3196340981, 2024-09-17T15:25:55.807 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 0, 2024-09-17T15:25:55.807 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 2, 2024-09-17T15:25:55.807 INFO:teuthology.orchestra.run.smithi083.stdout: "namespace": "" 2024-09-17T15:25:55.807 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:55.808 INFO:teuthology.orchestra.run.smithi083.stdout: "need": "21'2", 2024-09-17T15:25:55.808 INFO:teuthology.orchestra.run.smithi083.stdout: "have": "0'0", 2024-09-17T15:25:55.808 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": "none", 2024-09-17T15:25:55.808 INFO:teuthology.orchestra.run.smithi083.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-17T15:25:55.808 INFO:teuthology.orchestra.run.smithi083.stdout: "locations": [] 2024-09-17T15:25:55.808 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T15:25:55.808 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T15:25:55.808 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "Active", 2024-09-17T15:25:55.808 INFO:teuthology.orchestra.run.smithi083.stdout: "available_might_have_unfound": true, 2024-09-17T15:25:55.808 INFO:teuthology.orchestra.run.smithi083.stdout: "might_have_unfound": [ 2024-09-17T15:25:55.808 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:25:55.808 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "1", 2024-09-17T15:25:55.808 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "osd is down" 2024-09-17T15:25:55.809 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T15:25:55.809 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T15:25:55.809 INFO:teuthology.orchestra.run.smithi083.stdout: "more": false 2024-09-17T15:25:55.809 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T15:25:55.821 INFO:tasks.lost_unfound:reverting unfound in 2.5 on osd.0 2024-09-17T15:25:55.821 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.5 mark_unfound_lost revert 2024-09-17T15:25:56.046 INFO:teuthology.orchestra.run.smithi083.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-17T15:25:56.067 INFO:tasks.lost_unfound:no unfound in 1.6 2024-09-17T15:25:56.067 INFO:tasks.lost_unfound:no unfound in 1.7 2024-09-17T15:25:56.067 INFO:tasks.lost_unfound:listing missing/lost in 2.4 state active+recovery_wait+degraded 2024-09-17T15:25:56.067 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.4 list_unfound '{}' 2024-09-17T15:25:56.274 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T15:25:56.275 INFO:teuthology.orchestra.run.smithi083.stdout: "offset": { 2024-09-17T15:25:56.275 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": "", 2024-09-17T15:25:56.275 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "", 2024-09-17T15:25:56.275 INFO:teuthology.orchestra.run.smithi083.stdout: "snapid": 0, 2024-09-17T15:25:56.275 INFO:teuthology.orchestra.run.smithi083.stdout: "hash": 0, 2024-09-17T15:25:56.275 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 0, 2024-09-17T15:25:56.275 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": -9223372036854775808, 2024-09-17T15:25:56.275 INFO:teuthology.orchestra.run.smithi083.stdout: "namespace": "" 2024-09-17T15:25:56.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:56.275 INFO:teuthology.orchestra.run.smithi083.stdout: "num_missing": 4, 2024-09-17T15:25:56.275 INFO:teuthology.orchestra.run.smithi083.stdout: "num_unfound": 3, 2024-09-17T15:25:56.276 INFO:teuthology.orchestra.run.smithi083.stdout: "objects": [ 2024-09-17T15:25:56.276 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:25:56.276 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": { 2024-09-17T15:25:56.276 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": "existed_4", 2024-09-17T15:25:56.276 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "", 2024-09-17T15:25:56.276 INFO:teuthology.orchestra.run.smithi083.stdout: "snapid": -2, 2024-09-17T15:25:56.276 INFO:teuthology.orchestra.run.smithi083.stdout: "hash": 873891716, 2024-09-17T15:25:56.276 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 0, 2024-09-17T15:25:56.276 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 2, 2024-09-17T15:25:56.276 INFO:teuthology.orchestra.run.smithi083.stdout: "namespace": "" 2024-09-17T15:25:56.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:56.276 INFO:teuthology.orchestra.run.smithi083.stdout: "need": "21'8", 2024-09-17T15:25:56.277 INFO:teuthology.orchestra.run.smithi083.stdout: "have": "0'0", 2024-09-17T15:25:56.277 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": "none", 2024-09-17T15:25:56.277 INFO:teuthology.orchestra.run.smithi083.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-17T15:25:56.277 INFO:teuthology.orchestra.run.smithi083.stdout: "locations": [] 2024-09-17T15:25:56.277 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:56.277 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:25:56.277 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": { 2024-09-17T15:25:56.277 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": "existing_7", 2024-09-17T15:25:56.277 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "", 2024-09-17T15:25:56.277 INFO:teuthology.orchestra.run.smithi083.stdout: "snapid": -2, 2024-09-17T15:25:56.277 INFO:teuthology.orchestra.run.smithi083.stdout: "hash": 3996155460, 2024-09-17T15:25:56.277 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 0, 2024-09-17T15:25:56.277 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 2, 2024-09-17T15:25:56.278 INFO:teuthology.orchestra.run.smithi083.stdout: "namespace": "" 2024-09-17T15:25:56.278 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:56.278 INFO:teuthology.orchestra.run.smithi083.stdout: "need": "21'9", 2024-09-17T15:25:56.278 INFO:teuthology.orchestra.run.smithi083.stdout: "have": "19'6", 2024-09-17T15:25:56.278 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": "none", 2024-09-17T15:25:56.278 INFO:teuthology.orchestra.run.smithi083.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-17T15:25:56.278 INFO:teuthology.orchestra.run.smithi083.stdout: "locations": [] 2024-09-17T15:25:56.278 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:56.278 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:25:56.278 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": { 2024-09-17T15:25:56.278 INFO:teuthology.orchestra.run.smithi083.stdout: "oid": "existed_2", 2024-09-17T15:25:56.278 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "", 2024-09-17T15:25:56.278 INFO:teuthology.orchestra.run.smithi083.stdout: "snapid": -2, 2024-09-17T15:25:56.278 INFO:teuthology.orchestra.run.smithi083.stdout: "hash": 1767667508, 2024-09-17T15:25:56.279 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 0, 2024-09-17T15:25:56.279 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 2, 2024-09-17T15:25:56.279 INFO:teuthology.orchestra.run.smithi083.stdout: "namespace": "" 2024-09-17T15:25:56.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:25:56.279 INFO:teuthology.orchestra.run.smithi083.stdout: "need": "21'7", 2024-09-17T15:25:56.279 INFO:teuthology.orchestra.run.smithi083.stdout: "have": "0'0", 2024-09-17T15:25:56.279 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": "none", 2024-09-17T15:25:56.279 INFO:teuthology.orchestra.run.smithi083.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-17T15:25:56.279 INFO:teuthology.orchestra.run.smithi083.stdout: "locations": [] 2024-09-17T15:25:56.279 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T15:25:56.279 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T15:25:56.279 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "Active", 2024-09-17T15:25:56.280 INFO:teuthology.orchestra.run.smithi083.stdout: "available_might_have_unfound": true, 2024-09-17T15:25:56.280 INFO:teuthology.orchestra.run.smithi083.stdout: "might_have_unfound": [ 2024-09-17T15:25:56.280 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:25:56.280 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "1", 2024-09-17T15:25:56.280 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "osd is down" 2024-09-17T15:25:56.280 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T15:25:56.280 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T15:25:56.280 INFO:teuthology.orchestra.run.smithi083.stdout: "more": false 2024-09-17T15:25:56.280 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T15:25:56.285 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 17 20 36014 35994 8.26951 5.78125 0.00835617 0.00944099 2024-09-17T15:25:56.294 INFO:tasks.lost_unfound:reverting unfound in 2.4 on osd.0 2024-09-17T15:25:56.294 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.4 mark_unfound_lost revert 2024-09-17T15:25:56.527 INFO:teuthology.orchestra.run.smithi083.stderr:pg has 3 objects unfound and apparently lost marking 2024-09-17T15:25:56.545 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 debug kick_recovery_wq 5 2024-09-17T15:25:56.766 INFO:teuthology.orchestra.run.smithi083.stderr:kicking recovery queue. set osd_recovery_delay_start to 5 2024-09-17T15:25:56.783 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 debug kick_recovery_wq 5 2024-09-17T15:25:56.990 INFO:teuthology.orchestra.run.smithi083.stderr:kicking recovery queue. set osd_recovery_delay_start to 5 2024-09-17T15:25:57.010 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-17T15:25:57.011 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-17T15:25:57.246 INFO:teuthology.orchestra.run.smithi083.stdout:124554051595 2024-09-17T15:25:57.246 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T15:25:57.259 INFO:teuthology.orchestra.run.smithi083.stdout:98784247823 2024-09-17T15:25:57.259 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T15:25:57.285 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 18 20 37531 37511 8.1393 5.92578 0.0143636 0.00959022 2024-09-17T15:25:57.792 INFO:teuthology.orchestra.run.smithi083.stdout:124554051593 2024-09-17T15:25:57.815 INFO:tasks.lost_unfound.ceph_manager:need seq 124554051595 got 124554051593 for osd.2 2024-09-17T15:25:57.845 INFO:teuthology.orchestra.run.smithi083.stdout:98784247822 2024-09-17T15:25:57.870 INFO:tasks.lost_unfound.ceph_manager:need seq 98784247823 got 98784247822 for osd.0 2024-09-17T15:25:58.287 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 19 20 39028 39008 8.01867 5.84766 0.00968078 0.00973842 2024-09-17T15:25:58.816 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T15:25:58.871 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T15:25:59.286 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout:2024-09-17T15:25:59.286239+0000 min lat: 0.00187689 max lat: 0.213069 avg lat: 0.00985807 2024-09-17T15:25:59.286 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-17T15:25:59.286 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 20 20 40572 40552 7.91927 6.03125 0.0204755 0.00985807 2024-09-17T15:25:59.346 INFO:teuthology.orchestra.run.smithi083.stdout:124554051595 2024-09-17T15:25:59.366 INFO:tasks.lost_unfound.ceph_manager:need seq 124554051595 got 124554051595 for osd.2 2024-09-17T15:25:59.367 DEBUG:teuthology.parallel:result is None 2024-09-17T15:25:59.399 INFO:teuthology.orchestra.run.smithi083.stdout:98784247823 2024-09-17T15:25:59.421 INFO:tasks.lost_unfound.ceph_manager:need seq 98784247823 got 98784247823 for osd.0 2024-09-17T15:25:59.421 DEBUG:teuthology.parallel:result is None 2024-09-17T15:25:59.421 INFO:tasks.lost_unfound.ceph_manager:waiting for recovery to complete 2024-09-17T15:25:59.421 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:25:59.982 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:25:59.983 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:26:00.005 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2024-09-17T15:25:59.588243+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":153324408,"num_objects":37455,"num_object_clones":0,"num_object_copies":74910,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":5,"num_objects_dirty":37455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27667,"num_write_kb":110548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37515,"ondisk_log_size":37515,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":503724,"kb_used_data":291848,"kb_used_omap":8,"kb_used_meta":211831,"kb_avail":188239956,"statfs":{"total":193273528320,"available":192757714944,"internally_reserved":0,"allocated":298852352,"data_stored":298404457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8876,"internal_metadata":216915284},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1,16,4,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":85386897,"num_objects":20842,"num_object_clones":0,"num_object_copies":41684,"num_objects_missing_on_primary":-9,"num_objects_missing":-9,"num_objects_degraded":-18,"num_objects_misplaced":0,"num_objects_unfound":-22,"num_objects_dirty":20842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11027,"num_write_kb":44168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002964"},"pg_stats":[{"pgid":"2.f","version":"34'2296","reported_seq":2347,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:57.196769+0000","last_change":"2024-09-17T15:25:27.786800+0000","last_active":"2024-09-17T15:25:57.196769+0000","last_peered":"2024-09-17T15:25:57.196769+0000","last_clean":"2024-09-17T15:25:57.196769+0000","last_became_active":"2024-09-17T15:25:27.786453+0000","last_became_peered":"2024-09-17T15:25:27.786453+0000","last_unstale":"2024-09-17T15:25:57.196769+0000","last_undegraded":"2024-09-17T15:25:57.196769+0000","last_fullsized":"2024-09-17T15:25:57.196769+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2296,"log_dups_size":0,"ondisk_log_size":2296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:46:04.427346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9404416,"num_objects":2296,"num_object_clones":0,"num_object_copies":4592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2296,"num_write_kb":9184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"34'2268","reported_seq":2349,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:57.193198+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:25:57.193198+0000","last_peered":"2024-09-17T15:25:57.193198+0000","last_clean":"2024-09-17T15:25:57.193198+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:25:57.193198+0000","last_undegraded":"2024-09-17T15:25:57.193198+0000","last_fullsized":"2024-09-17T15:25:57.193198+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":2268,"log_dups_size":0,"ondisk_log_size":2268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9266118,"num_objects":2264,"num_object_clones":0,"num_object_copies":4528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":625,"num_write_kb":2500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"34'2395","reported_seq":2493,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:57.192852+0000","last_change":"2024-09-17T15:25:51.885496+0000","last_active":"2024-09-17T15:25:57.192852+0000","last_peered":"2024-09-17T15:25:57.192852+0000","last_clean":"2024-09-17T15:25:57.192852+0000","last_became_active":"2024-09-17T15:25:28.785518+0000","last_became_peered":"2024-09-17T15:25:28.785518+0000","last_unstale":"2024-09-17T15:25:57.192852+0000","last_undegraded":"2024-09-17T15:25:57.192852+0000","last_fullsized":"2024-09-17T15:25:57.192852+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":2395,"log_dups_size":0,"ondisk_log_size":2395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:59:02.229332+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9793536,"num_objects":2391,"num_object_clones":0,"num_object_copies":4782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":552,"num_write_kb":2208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"34'2303","reported_seq":2353,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:57.214332+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:25:57.214332+0000","last_peered":"2024-09-17T15:25:57.214332+0000","last_clean":"2024-09-17T15:25:57.214332+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:25:57.214332+0000","last_undegraded":"2024-09-17T15:25:57.214332+0000","last_fullsized":"2024-09-17T15:25:57.214332+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2303,"log_dups_size":0,"ondisk_log_size":2303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9433088,"num_objects":2303,"num_object_clones":0,"num_object_copies":4606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2303,"num_write_kb":9212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"34'2300","reported_seq":2360,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:57.186712+0000","last_change":"2024-09-17T15:25:27.782128+0000","last_active":"2024-09-17T15:25:57.186712+0000","last_peered":"2024-09-17T15:25:57.186712+0000","last_clean":"2024-09-17T15:24:56.061756+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:25:57.186712+0000","last_undegraded":"2024-09-17T15:25:27.778309+0000","last_fullsized":"2024-09-17T15:25:57.186712+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2300,"log_dups_size":0,"ondisk_log_size":2300,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9393094,"num_objects":2295,"num_object_clones":0,"num_object_copies":4590,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2298,"num_write_kb":9176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0,2","objects":2294}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"34'2420","reported_seq":2476,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:57.220828+0000","last_change":"2024-09-17T15:25:27.789375+0000","last_active":"2024-09-17T15:25:57.220828+0000","last_peered":"2024-09-17T15:25:57.220828+0000","last_clean":"2024-09-17T15:24:56.947098+0000","last_became_active":"2024-09-17T15:25:27.789142+0000","last_became_peered":"2024-09-17T15:25:27.789142+0000","last_unstale":"2024-09-17T15:25:57.220828+0000","last_undegraded":"2024-09-17T15:25:27.779842+0000","last_fullsized":"2024-09-17T15:25:57.220828+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2420,"log_dups_size":0,"ondisk_log_size":2420,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9892806,"num_objects":2417,"num_object_clones":0,"num_object_copies":4834,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2417,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2418,"num_write_kb":9663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0,2","objects":2416}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"34'2390","reported_seq":2447,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:57.217756+0000","last_change":"2024-09-17T15:25:27.784357+0000","last_active":"2024-09-17T15:25:57.217756+0000","last_peered":"2024-09-17T15:25:57.217756+0000","last_clean":"2024-09-17T15:24:56.310518+0000","last_became_active":"2024-09-17T15:25:27.784135+0000","last_became_peered":"2024-09-17T15:25:27.784135+0000","last_unstale":"2024-09-17T15:25:57.217756+0000","last_undegraded":"2024-09-17T15:25:27.779462+0000","last_fullsized":"2024-09-17T15:25:57.217756+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2390,"log_dups_size":0,"ondisk_log_size":2390,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9765830,"num_objects":2386,"num_object_clones":0,"num_object_copies":4772,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2388,"num_write_kb":9540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":2},{"shards":"0,2","objects":2384}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"34'2363","reported_seq":2425,"reported_epoch":34,"state":"active+recovering","last_fresh":"2024-09-17T15:25:57.217475+0000","last_change":"2024-09-17T15:25:28.783075+0000","last_active":"2024-09-17T15:25:57.217475+0000","last_peered":"2024-09-17T15:25:57.217475+0000","last_clean":"2024-09-17T15:24:49.051582+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:25:57.217475+0000","last_undegraded":"2024-09-17T15:25:57.217475+0000","last_fullsized":"2024-09-17T15:25:57.217475+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2363,"log_dups_size":0,"ondisk_log_size":2363,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9671139,"num_objects":2362,"num_object_clones":0,"num_object_copies":4724,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2362,"num_write_kb":9445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2362}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.782961+0000","last_change":"2024-09-17T15:25:27.786793+0000","last_active":"2024-09-17T15:25:28.782961+0000","last_peered":"2024-09-17T15:25:28.782961+0000","last_clean":"2024-09-17T15:25:28.782961+0000","last_became_active":"2024-09-17T15:25:27.786505+0000","last_became_peered":"2024-09-17T15:25:27.786505+0000","last_unstale":"2024-09-17T15:25:28.782961+0000","last_undegraded":"2024-09-17T15:25:28.782961+0000","last_fullsized":"2024-09-17T15:25:28.782961+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:22:15.104318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"34'2312","reported_seq":2412,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:57.176217+0000","last_change":"2024-09-17T15:25:53.959204+0000","last_active":"2024-09-17T15:25:57.176217+0000","last_peered":"2024-09-17T15:25:57.176217+0000","last_clean":"2024-09-17T15:25:57.176217+0000","last_became_active":"2024-09-17T15:25:28.785807+0000","last_became_peered":"2024-09-17T15:25:28.785807+0000","last_unstale":"2024-09-17T15:25:57.176217+0000","last_undegraded":"2024-09-17T15:25:57.176217+0000","last_fullsized":"2024-09-17T15:25:57.176217+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":2312,"log_dups_size":0,"ondisk_log_size":2312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:14:31.995973+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9441763,"num_objects":2306,"num_object_clones":0,"num_object_copies":4612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":312,"num_write_kb":1248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.787649+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:25:28.787649+0000","last_peered":"2024-09-17T15:25:28.787649+0000","last_clean":"2024-09-17T15:25:28.787649+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:25:28.787649+0000","last_undegraded":"2024-09-17T15:25:28.787649+0000","last_fullsized":"2024-09-17T15:25:28.787649+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"34'2304","reported_seq":2353,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:57.228962+0000","last_change":"2024-09-17T15:25:27.789502+0000","last_active":"2024-09-17T15:25:57.228962+0000","last_peered":"2024-09-17T15:25:57.228962+0000","last_clean":"2024-09-17T15:24:56.997318+0000","last_became_active":"2024-09-17T15:25:27.789253+0000","last_became_peered":"2024-09-17T15:25:27.789253+0000","last_unstale":"2024-09-17T15:25:57.228962+0000","last_undegraded":"2024-09-17T15:25:27.780228+0000","last_fullsized":"2024-09-17T15:25:57.228962+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2304,"log_dups_size":0,"ondisk_log_size":2304,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9405382,"num_objects":2298,"num_object_clones":0,"num_object_copies":4596,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2302,"num_write_kb":9188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2298}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":66,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:28.783096+0000","last_change":"2024-09-17T15:25:27.786740+0000","last_active":"2024-09-17T15:25:28.783096+0000","last_peered":"2024-09-17T15:25:28.783096+0000","last_clean":"2024-09-17T15:25:07.444351+0000","last_became_active":"2024-09-17T15:25:27.786489+0000","last_became_peered":"2024-09-17T15:25:27.786489+0000","last_unstale":"2024-09-17T15:25:28.783096+0000","last_undegraded":"2024-09-17T15:25:27.779442+0000","last_fullsized":"2024-09-17T15:25:28.783096+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"34'2332","reported_seq":2424,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:57.210434+0000","last_change":"2024-09-17T15:25:55.123333+0000","last_active":"2024-09-17T15:25:57.210434+0000","last_peered":"2024-09-17T15:25:57.210434+0000","last_clean":"2024-09-17T15:25:57.210434+0000","last_became_active":"2024-09-17T15:25:28.787663+0000","last_became_peered":"2024-09-17T15:25:28.787663+0000","last_unstale":"2024-09-17T15:25:57.210434+0000","last_undegraded":"2024-09-17T15:25:57.210434+0000","last_fullsized":"2024-09-17T15:25:57.210434+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":2332,"log_dups_size":0,"ondisk_log_size":2332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:10:55.294151+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9531875,"num_objects":2328,"num_object_clones":0,"num_object_copies":4656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":227,"num_write_kb":908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.787312+0000","last_change":"2024-09-17T15:25:28.786053+0000","last_active":"2024-09-17T15:25:28.787312+0000","last_peered":"2024-09-17T15:25:28.787312+0000","last_clean":"2024-09-17T15:25:28.787312+0000","last_became_active":"2024-09-17T15:25:28.785773+0000","last_became_peered":"2024-09-17T15:25:28.785773+0000","last_unstale":"2024-09-17T15:25:28.787312+0000","last_undegraded":"2024-09-17T15:25:28.787312+0000","last_fullsized":"2024-09-17T15:25:28.787312+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:46:53.130296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"34'2373","reported_seq":2465,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:57.200570+0000","last_change":"2024-09-17T15:25:56.130418+0000","last_active":"2024-09-17T15:25:57.200570+0000","last_peered":"2024-09-17T15:25:57.200570+0000","last_clean":"2024-09-17T15:25:57.200570+0000","last_became_active":"2024-09-17T15:25:28.785157+0000","last_became_peered":"2024-09-17T15:25:28.785157+0000","last_unstale":"2024-09-17T15:25:57.200570+0000","last_undegraded":"2024-09-17T15:25:57.200570+0000","last_fullsized":"2024-09-17T15:25:57.200570+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":2373,"log_dups_size":0,"ondisk_log_size":2373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:03:03.960144+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9695232,"num_objects":2367,"num_object_clones":0,"num_object_copies":4734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"34'2391","reported_seq":2447,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:57.213271+0000","last_change":"2024-09-17T15:25:27.789404+0000","last_active":"2024-09-17T15:25:57.213271+0000","last_peered":"2024-09-17T15:25:57.213271+0000","last_clean":"2024-09-17T15:24:52.804722+0000","last_became_active":"2024-09-17T15:25:27.789191+0000","last_became_peered":"2024-09-17T15:25:27.789191+0000","last_unstale":"2024-09-17T15:25:57.213271+0000","last_undegraded":"2024-09-17T15:25:27.780610+0000","last_fullsized":"2024-09-17T15:25:57.213271+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2391,"log_dups_size":0,"ondisk_log_size":2391,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9785827,"num_objects":2390,"num_object_clones":0,"num_object_copies":4780,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2390,"num_write_kb":9557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2390}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.784109+0000","last_change":"2024-09-17T15:25:27.789397+0000","last_active":"2024-09-17T15:25:28.784109+0000","last_peered":"2024-09-17T15:25:28.784109+0000","last_clean":"2024-09-17T15:25:28.784109+0000","last_became_active":"2024-09-17T15:25:27.789137+0000","last_became_peered":"2024-09-17T15:25:27.789137+0000","last_unstale":"2024-09-17T15:25:28.784109+0000","last_undegraded":"2024-09-17T15:25:28.784109+0000","last_fullsized":"2024-09-17T15:25:28.784109+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:26:53.538418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"34'2373","reported_seq":2428,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:57.217287+0000","last_change":"2024-09-17T15:25:27.789579+0000","last_active":"2024-09-17T15:25:57.217287+0000","last_peered":"2024-09-17T15:25:57.217287+0000","last_clean":"2024-09-17T15:24:56.897031+0000","last_became_active":"2024-09-17T15:25:27.789325+0000","last_became_peered":"2024-09-17T15:25:27.789325+0000","last_unstale":"2024-09-17T15:25:57.217287+0000","last_undegraded":"2024-09-17T15:25:27.781559+0000","last_fullsized":"2024-09-17T15:25:57.217287+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2373,"log_dups_size":0,"ondisk_log_size":2373,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9688489,"num_objects":2368,"num_object_clones":0,"num_object_copies":4736,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2368,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2370,"num_write_kb":9464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2368}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.785793+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:25:28.785793+0000","last_peered":"2024-09-17T15:25:28.785793+0000","last_clean":"2024-09-17T15:25:28.785793+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:25:28.785793+0000","last_undegraded":"2024-09-17T15:25:28.785793+0000","last_fullsized":"2024-09-17T15:25:28.785793+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"34'2347","reported_seq":2402,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:57.216769+0000","last_change":"2024-09-17T15:25:27.784424+0000","last_active":"2024-09-17T15:25:57.216769+0000","last_peered":"2024-09-17T15:25:57.216769+0000","last_clean":"2024-09-17T15:24:52.804663+0000","last_became_active":"2024-09-17T15:25:27.784195+0000","last_became_peered":"2024-09-17T15:25:27.784195+0000","last_unstale":"2024-09-17T15:25:57.216769+0000","last_undegraded":"2024-09-17T15:25:27.779406+0000","last_fullsized":"2024-09-17T15:25:57.216769+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2347,"log_dups_size":0,"ondisk_log_size":2347,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9597894,"num_objects":2345,"num_object_clones":0,"num_object_copies":4690,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2345,"num_write_kb":9374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2345}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.783601+0000","last_change":"2024-09-17T15:25:27.786796+0000","last_active":"2024-09-17T15:25:28.783601+0000","last_peered":"2024-09-17T15:25:28.783601+0000","last_clean":"2024-09-17T15:25:28.783601+0000","last_became_active":"2024-09-17T15:25:27.786506+0000","last_became_peered":"2024-09-17T15:25:27.786506+0000","last_unstale":"2024-09-17T15:25:28.783601+0000","last_undegraded":"2024-09-17T15:25:28.783601+0000","last_fullsized":"2024-09-17T15:25:28.783601+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:55:57.547976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.786536+0000","last_change":"2024-09-17T15:25:28.785423+0000","last_active":"2024-09-17T15:25:28.786536+0000","last_peered":"2024-09-17T15:25:28.786536+0000","last_clean":"2024-09-17T15:25:28.786536+0000","last_became_active":"2024-09-17T15:25:28.785151+0000","last_became_peered":"2024-09-17T15:25:28.785151+0000","last_unstale":"2024-09-17T15:25:28.786536+0000","last_undegraded":"2024-09-17T15:25:28.786536+0000","last_fullsized":"2024-09-17T15:25:28.786536+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:47:12.307723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"34'2345","reported_seq":2398,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:57.207922+0000","last_change":"2024-09-17T15:25:27.789556+0000","last_active":"2024-09-17T15:25:57.207922+0000","last_peered":"2024-09-17T15:25:57.207922+0000","last_clean":"2024-09-17T15:24:56.609264+0000","last_became_active":"2024-09-17T15:25:27.789304+0000","last_became_peered":"2024-09-17T15:25:27.789304+0000","last_unstale":"2024-09-17T15:25:57.207922+0000","last_undegraded":"2024-09-17T15:25:27.782129+0000","last_fullsized":"2024-09-17T15:25:57.207922+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2345,"log_dups_size":0,"ondisk_log_size":2345,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9557900,"num_objects":2337,"num_object_clones":0,"num_object_copies":4674,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2342,"num_write_kb":9339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0","objects":1},{"shards":"0,2","objects":2335}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":153324389,"num_objects":37453,"num_object_clones":0,"num_object_copies":74906,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":35,"num_objects_misplaced":0,"num_objects_unfound":5,"num_objects_dirty":37453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27665,"num_write_kb":110546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":306876416,"data_stored":306724670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37512,"ondisk_log_size":37512,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051595,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":248676,"kb_used_data":144784,"kb_used_omap":4,"kb_used_meta":103867,"kb_avail":94123164,"statfs":{"total":96636764160,"available":96382119936,"internally_reserved":0,"allocated":148258816,"data_stored":148047746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":106360492},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1,3,3,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247823,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":255048,"kb_used_data":147064,"kb_used_omap":4,"kb_used_meta":107963,"kb_avail":94116792,"statfs":{"total":96636764160,"available":96375595008,"internally_reserved":0,"allocated":150593536,"data_stored":150356711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":110554792},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,13,1,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":153436160,"data_stored":153400030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":153325568,"data_stored":153311116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:26:00.007 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:26:00.286 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 21 20 42047 42027 7.81651 5.76172 0.0165193 0.0099888 2024-09-17T15:26:00.544 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:26:00.545 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:26:00.565 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2024-09-17T15:25:59.588243+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":153324408,"num_objects":37455,"num_object_clones":0,"num_object_copies":74910,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":5,"num_objects_dirty":37455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27667,"num_write_kb":110548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37515,"ondisk_log_size":37515,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":503724,"kb_used_data":291848,"kb_used_omap":8,"kb_used_meta":211831,"kb_avail":188239956,"statfs":{"total":193273528320,"available":192757714944,"internally_reserved":0,"allocated":298852352,"data_stored":298404457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8876,"internal_metadata":216915284},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1,16,4,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":85386897,"num_objects":20842,"num_object_clones":0,"num_object_copies":41684,"num_objects_missing_on_primary":-9,"num_objects_missing":-9,"num_objects_degraded":-18,"num_objects_misplaced":0,"num_objects_unfound":-22,"num_objects_dirty":20842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11027,"num_write_kb":44168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002964"},"pg_stats":[{"pgid":"2.f","version":"34'2296","reported_seq":2347,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:57.196769+0000","last_change":"2024-09-17T15:25:27.786800+0000","last_active":"2024-09-17T15:25:57.196769+0000","last_peered":"2024-09-17T15:25:57.196769+0000","last_clean":"2024-09-17T15:25:57.196769+0000","last_became_active":"2024-09-17T15:25:27.786453+0000","last_became_peered":"2024-09-17T15:25:27.786453+0000","last_unstale":"2024-09-17T15:25:57.196769+0000","last_undegraded":"2024-09-17T15:25:57.196769+0000","last_fullsized":"2024-09-17T15:25:57.196769+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2296,"log_dups_size":0,"ondisk_log_size":2296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:46:04.427346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9404416,"num_objects":2296,"num_object_clones":0,"num_object_copies":4592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2296,"num_write_kb":9184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"34'2268","reported_seq":2349,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:57.193198+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:25:57.193198+0000","last_peered":"2024-09-17T15:25:57.193198+0000","last_clean":"2024-09-17T15:25:57.193198+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:25:57.193198+0000","last_undegraded":"2024-09-17T15:25:57.193198+0000","last_fullsized":"2024-09-17T15:25:57.193198+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":2268,"log_dups_size":0,"ondisk_log_size":2268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9266118,"num_objects":2264,"num_object_clones":0,"num_object_copies":4528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":625,"num_write_kb":2500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"34'2395","reported_seq":2493,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:57.192852+0000","last_change":"2024-09-17T15:25:51.885496+0000","last_active":"2024-09-17T15:25:57.192852+0000","last_peered":"2024-09-17T15:25:57.192852+0000","last_clean":"2024-09-17T15:25:57.192852+0000","last_became_active":"2024-09-17T15:25:28.785518+0000","last_became_peered":"2024-09-17T15:25:28.785518+0000","last_unstale":"2024-09-17T15:25:57.192852+0000","last_undegraded":"2024-09-17T15:25:57.192852+0000","last_fullsized":"2024-09-17T15:25:57.192852+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":2395,"log_dups_size":0,"ondisk_log_size":2395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:59:02.229332+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9793536,"num_objects":2391,"num_object_clones":0,"num_object_copies":4782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":552,"num_write_kb":2208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"34'2303","reported_seq":2353,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:57.214332+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:25:57.214332+0000","last_peered":"2024-09-17T15:25:57.214332+0000","last_clean":"2024-09-17T15:25:57.214332+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:25:57.214332+0000","last_undegraded":"2024-09-17T15:25:57.214332+0000","last_fullsized":"2024-09-17T15:25:57.214332+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2303,"log_dups_size":0,"ondisk_log_size":2303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9433088,"num_objects":2303,"num_object_clones":0,"num_object_copies":4606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2303,"num_write_kb":9212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"34'2300","reported_seq":2360,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:57.186712+0000","last_change":"2024-09-17T15:25:27.782128+0000","last_active":"2024-09-17T15:25:57.186712+0000","last_peered":"2024-09-17T15:25:57.186712+0000","last_clean":"2024-09-17T15:24:56.061756+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:25:57.186712+0000","last_undegraded":"2024-09-17T15:25:27.778309+0000","last_fullsized":"2024-09-17T15:25:57.186712+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2300,"log_dups_size":0,"ondisk_log_size":2300,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9393094,"num_objects":2295,"num_object_clones":0,"num_object_copies":4590,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2298,"num_write_kb":9176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0,2","objects":2294}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"34'2420","reported_seq":2476,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:57.220828+0000","last_change":"2024-09-17T15:25:27.789375+0000","last_active":"2024-09-17T15:25:57.220828+0000","last_peered":"2024-09-17T15:25:57.220828+0000","last_clean":"2024-09-17T15:24:56.947098+0000","last_became_active":"2024-09-17T15:25:27.789142+0000","last_became_peered":"2024-09-17T15:25:27.789142+0000","last_unstale":"2024-09-17T15:25:57.220828+0000","last_undegraded":"2024-09-17T15:25:27.779842+0000","last_fullsized":"2024-09-17T15:25:57.220828+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2420,"log_dups_size":0,"ondisk_log_size":2420,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9892806,"num_objects":2417,"num_object_clones":0,"num_object_copies":4834,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2417,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2418,"num_write_kb":9663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0,2","objects":2416}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"34'2390","reported_seq":2447,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:57.217756+0000","last_change":"2024-09-17T15:25:27.784357+0000","last_active":"2024-09-17T15:25:57.217756+0000","last_peered":"2024-09-17T15:25:57.217756+0000","last_clean":"2024-09-17T15:24:56.310518+0000","last_became_active":"2024-09-17T15:25:27.784135+0000","last_became_peered":"2024-09-17T15:25:27.784135+0000","last_unstale":"2024-09-17T15:25:57.217756+0000","last_undegraded":"2024-09-17T15:25:27.779462+0000","last_fullsized":"2024-09-17T15:25:57.217756+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2390,"log_dups_size":0,"ondisk_log_size":2390,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9765830,"num_objects":2386,"num_object_clones":0,"num_object_copies":4772,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2388,"num_write_kb":9540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":2},{"shards":"0,2","objects":2384}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"34'2363","reported_seq":2425,"reported_epoch":34,"state":"active+recovering","last_fresh":"2024-09-17T15:25:57.217475+0000","last_change":"2024-09-17T15:25:28.783075+0000","last_active":"2024-09-17T15:25:57.217475+0000","last_peered":"2024-09-17T15:25:57.217475+0000","last_clean":"2024-09-17T15:24:49.051582+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:25:57.217475+0000","last_undegraded":"2024-09-17T15:25:57.217475+0000","last_fullsized":"2024-09-17T15:25:57.217475+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2363,"log_dups_size":0,"ondisk_log_size":2363,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9671139,"num_objects":2362,"num_object_clones":0,"num_object_copies":4724,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2362,"num_write_kb":9445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2362}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.782961+0000","last_change":"2024-09-17T15:25:27.786793+0000","last_active":"2024-09-17T15:25:28.782961+0000","last_peered":"2024-09-17T15:25:28.782961+0000","last_clean":"2024-09-17T15:25:28.782961+0000","last_became_active":"2024-09-17T15:25:27.786505+0000","last_became_peered":"2024-09-17T15:25:27.786505+0000","last_unstale":"2024-09-17T15:25:28.782961+0000","last_undegraded":"2024-09-17T15:25:28.782961+0000","last_fullsized":"2024-09-17T15:25:28.782961+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:22:15.104318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"34'2312","reported_seq":2412,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:57.176217+0000","last_change":"2024-09-17T15:25:53.959204+0000","last_active":"2024-09-17T15:25:57.176217+0000","last_peered":"2024-09-17T15:25:57.176217+0000","last_clean":"2024-09-17T15:25:57.176217+0000","last_became_active":"2024-09-17T15:25:28.785807+0000","last_became_peered":"2024-09-17T15:25:28.785807+0000","last_unstale":"2024-09-17T15:25:57.176217+0000","last_undegraded":"2024-09-17T15:25:57.176217+0000","last_fullsized":"2024-09-17T15:25:57.176217+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":2312,"log_dups_size":0,"ondisk_log_size":2312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:14:31.995973+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9441763,"num_objects":2306,"num_object_clones":0,"num_object_copies":4612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":312,"num_write_kb":1248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.787649+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:25:28.787649+0000","last_peered":"2024-09-17T15:25:28.787649+0000","last_clean":"2024-09-17T15:25:28.787649+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:25:28.787649+0000","last_undegraded":"2024-09-17T15:25:28.787649+0000","last_fullsized":"2024-09-17T15:25:28.787649+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"34'2304","reported_seq":2353,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:57.228962+0000","last_change":"2024-09-17T15:25:27.789502+0000","last_active":"2024-09-17T15:25:57.228962+0000","last_peered":"2024-09-17T15:25:57.228962+0000","last_clean":"2024-09-17T15:24:56.997318+0000","last_became_active":"2024-09-17T15:25:27.789253+0000","last_became_peered":"2024-09-17T15:25:27.789253+0000","last_unstale":"2024-09-17T15:25:57.228962+0000","last_undegraded":"2024-09-17T15:25:27.780228+0000","last_fullsized":"2024-09-17T15:25:57.228962+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2304,"log_dups_size":0,"ondisk_log_size":2304,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9405382,"num_objects":2298,"num_object_clones":0,"num_object_copies":4596,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2302,"num_write_kb":9188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2298}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":66,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:28.783096+0000","last_change":"2024-09-17T15:25:27.786740+0000","last_active":"2024-09-17T15:25:28.783096+0000","last_peered":"2024-09-17T15:25:28.783096+0000","last_clean":"2024-09-17T15:25:07.444351+0000","last_became_active":"2024-09-17T15:25:27.786489+0000","last_became_peered":"2024-09-17T15:25:27.786489+0000","last_unstale":"2024-09-17T15:25:28.783096+0000","last_undegraded":"2024-09-17T15:25:27.779442+0000","last_fullsized":"2024-09-17T15:25:28.783096+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"34'2332","reported_seq":2424,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:57.210434+0000","last_change":"2024-09-17T15:25:55.123333+0000","last_active":"2024-09-17T15:25:57.210434+0000","last_peered":"2024-09-17T15:25:57.210434+0000","last_clean":"2024-09-17T15:25:57.210434+0000","last_became_active":"2024-09-17T15:25:28.787663+0000","last_became_peered":"2024-09-17T15:25:28.787663+0000","last_unstale":"2024-09-17T15:25:57.210434+0000","last_undegraded":"2024-09-17T15:25:57.210434+0000","last_fullsized":"2024-09-17T15:25:57.210434+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":2332,"log_dups_size":0,"ondisk_log_size":2332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:10:55.294151+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9531875,"num_objects":2328,"num_object_clones":0,"num_object_copies":4656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":227,"num_write_kb":908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.787312+0000","last_change":"2024-09-17T15:25:28.786053+0000","last_active":"2024-09-17T15:25:28.787312+0000","last_peered":"2024-09-17T15:25:28.787312+0000","last_clean":"2024-09-17T15:25:28.787312+0000","last_became_active":"2024-09-17T15:25:28.785773+0000","last_became_peered":"2024-09-17T15:25:28.785773+0000","last_unstale":"2024-09-17T15:25:28.787312+0000","last_undegraded":"2024-09-17T15:25:28.787312+0000","last_fullsized":"2024-09-17T15:25:28.787312+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:46:53.130296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"34'2373","reported_seq":2465,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:57.200570+0000","last_change":"2024-09-17T15:25:56.130418+0000","last_active":"2024-09-17T15:25:57.200570+0000","last_peered":"2024-09-17T15:25:57.200570+0000","last_clean":"2024-09-17T15:25:57.200570+0000","last_became_active":"2024-09-17T15:25:28.785157+0000","last_became_peered":"2024-09-17T15:25:28.785157+0000","last_unstale":"2024-09-17T15:25:57.200570+0000","last_undegraded":"2024-09-17T15:25:57.200570+0000","last_fullsized":"2024-09-17T15:25:57.200570+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":2373,"log_dups_size":0,"ondisk_log_size":2373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:03:03.960144+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9695232,"num_objects":2367,"num_object_clones":0,"num_object_copies":4734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"34'2391","reported_seq":2447,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:57.213271+0000","last_change":"2024-09-17T15:25:27.789404+0000","last_active":"2024-09-17T15:25:57.213271+0000","last_peered":"2024-09-17T15:25:57.213271+0000","last_clean":"2024-09-17T15:24:52.804722+0000","last_became_active":"2024-09-17T15:25:27.789191+0000","last_became_peered":"2024-09-17T15:25:27.789191+0000","last_unstale":"2024-09-17T15:25:57.213271+0000","last_undegraded":"2024-09-17T15:25:27.780610+0000","last_fullsized":"2024-09-17T15:25:57.213271+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2391,"log_dups_size":0,"ondisk_log_size":2391,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9785827,"num_objects":2390,"num_object_clones":0,"num_object_copies":4780,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2390,"num_write_kb":9557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2390}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.784109+0000","last_change":"2024-09-17T15:25:27.789397+0000","last_active":"2024-09-17T15:25:28.784109+0000","last_peered":"2024-09-17T15:25:28.784109+0000","last_clean":"2024-09-17T15:25:28.784109+0000","last_became_active":"2024-09-17T15:25:27.789137+0000","last_became_peered":"2024-09-17T15:25:27.789137+0000","last_unstale":"2024-09-17T15:25:28.784109+0000","last_undegraded":"2024-09-17T15:25:28.784109+0000","last_fullsized":"2024-09-17T15:25:28.784109+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:26:53.538418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"34'2373","reported_seq":2428,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:57.217287+0000","last_change":"2024-09-17T15:25:27.789579+0000","last_active":"2024-09-17T15:25:57.217287+0000","last_peered":"2024-09-17T15:25:57.217287+0000","last_clean":"2024-09-17T15:24:56.897031+0000","last_became_active":"2024-09-17T15:25:27.789325+0000","last_became_peered":"2024-09-17T15:25:27.789325+0000","last_unstale":"2024-09-17T15:25:57.217287+0000","last_undegraded":"2024-09-17T15:25:27.781559+0000","last_fullsized":"2024-09-17T15:25:57.217287+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2373,"log_dups_size":0,"ondisk_log_size":2373,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9688489,"num_objects":2368,"num_object_clones":0,"num_object_copies":4736,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2368,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2370,"num_write_kb":9464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2368}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.785793+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:25:28.785793+0000","last_peered":"2024-09-17T15:25:28.785793+0000","last_clean":"2024-09-17T15:25:28.785793+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:25:28.785793+0000","last_undegraded":"2024-09-17T15:25:28.785793+0000","last_fullsized":"2024-09-17T15:25:28.785793+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"34'2347","reported_seq":2402,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:57.216769+0000","last_change":"2024-09-17T15:25:27.784424+0000","last_active":"2024-09-17T15:25:57.216769+0000","last_peered":"2024-09-17T15:25:57.216769+0000","last_clean":"2024-09-17T15:24:52.804663+0000","last_became_active":"2024-09-17T15:25:27.784195+0000","last_became_peered":"2024-09-17T15:25:27.784195+0000","last_unstale":"2024-09-17T15:25:57.216769+0000","last_undegraded":"2024-09-17T15:25:27.779406+0000","last_fullsized":"2024-09-17T15:25:57.216769+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2347,"log_dups_size":0,"ondisk_log_size":2347,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9597894,"num_objects":2345,"num_object_clones":0,"num_object_copies":4690,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2345,"num_write_kb":9374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2345}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.783601+0000","last_change":"2024-09-17T15:25:27.786796+0000","last_active":"2024-09-17T15:25:28.783601+0000","last_peered":"2024-09-17T15:25:28.783601+0000","last_clean":"2024-09-17T15:25:28.783601+0000","last_became_active":"2024-09-17T15:25:27.786506+0000","last_became_peered":"2024-09-17T15:25:27.786506+0000","last_unstale":"2024-09-17T15:25:28.783601+0000","last_undegraded":"2024-09-17T15:25:28.783601+0000","last_fullsized":"2024-09-17T15:25:28.783601+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:55:57.547976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.786536+0000","last_change":"2024-09-17T15:25:28.785423+0000","last_active":"2024-09-17T15:25:28.786536+0000","last_peered":"2024-09-17T15:25:28.786536+0000","last_clean":"2024-09-17T15:25:28.786536+0000","last_became_active":"2024-09-17T15:25:28.785151+0000","last_became_peered":"2024-09-17T15:25:28.785151+0000","last_unstale":"2024-09-17T15:25:28.786536+0000","last_undegraded":"2024-09-17T15:25:28.786536+0000","last_fullsized":"2024-09-17T15:25:28.786536+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:47:12.307723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"34'2345","reported_seq":2398,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:57.207922+0000","last_change":"2024-09-17T15:25:27.789556+0000","last_active":"2024-09-17T15:25:57.207922+0000","last_peered":"2024-09-17T15:25:57.207922+0000","last_clean":"2024-09-17T15:24:56.609264+0000","last_became_active":"2024-09-17T15:25:27.789304+0000","last_became_peered":"2024-09-17T15:25:27.789304+0000","last_unstale":"2024-09-17T15:25:57.207922+0000","last_undegraded":"2024-09-17T15:25:27.782129+0000","last_fullsized":"2024-09-17T15:25:57.207922+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2345,"log_dups_size":0,"ondisk_log_size":2345,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9557900,"num_objects":2337,"num_object_clones":0,"num_object_copies":4674,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2342,"num_write_kb":9339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0","objects":1},{"shards":"0,2","objects":2335}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":153324389,"num_objects":37453,"num_object_clones":0,"num_object_copies":74906,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":35,"num_objects_misplaced":0,"num_objects_unfound":5,"num_objects_dirty":37453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27665,"num_write_kb":110546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":306876416,"data_stored":306724670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37512,"ondisk_log_size":37512,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051595,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":248676,"kb_used_data":144784,"kb_used_omap":4,"kb_used_meta":103867,"kb_avail":94123164,"statfs":{"total":96636764160,"available":96382119936,"internally_reserved":0,"allocated":148258816,"data_stored":148047746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":106360492},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1,3,3,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247823,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":255048,"kb_used_data":147064,"kb_used_omap":4,"kb_used_meta":107963,"kb_avail":94116792,"statfs":{"total":96636764160,"available":96375595008,"internally_reserved":0,"allocated":150593536,"data_stored":150356711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":110554792},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,13,1,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":153436160,"data_stored":153400030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":153325568,"data_stored":153311116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:26:00.566 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:26:01.080 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:26:01.080 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:26:01.104 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2024-09-17T15:25:59.588243+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":153324408,"num_objects":37455,"num_object_clones":0,"num_object_copies":74910,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":5,"num_objects_dirty":37455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27667,"num_write_kb":110548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37515,"ondisk_log_size":37515,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":503724,"kb_used_data":291848,"kb_used_omap":8,"kb_used_meta":211831,"kb_avail":188239956,"statfs":{"total":193273528320,"available":192757714944,"internally_reserved":0,"allocated":298852352,"data_stored":298404457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8876,"internal_metadata":216915284},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1,16,4,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":85386897,"num_objects":20842,"num_object_clones":0,"num_object_copies":41684,"num_objects_missing_on_primary":-9,"num_objects_missing":-9,"num_objects_degraded":-18,"num_objects_misplaced":0,"num_objects_unfound":-22,"num_objects_dirty":20842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11027,"num_write_kb":44168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002964"},"pg_stats":[{"pgid":"2.f","version":"34'2296","reported_seq":2347,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:57.196769+0000","last_change":"2024-09-17T15:25:27.786800+0000","last_active":"2024-09-17T15:25:57.196769+0000","last_peered":"2024-09-17T15:25:57.196769+0000","last_clean":"2024-09-17T15:25:57.196769+0000","last_became_active":"2024-09-17T15:25:27.786453+0000","last_became_peered":"2024-09-17T15:25:27.786453+0000","last_unstale":"2024-09-17T15:25:57.196769+0000","last_undegraded":"2024-09-17T15:25:57.196769+0000","last_fullsized":"2024-09-17T15:25:57.196769+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2296,"log_dups_size":0,"ondisk_log_size":2296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:46:04.427346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9404416,"num_objects":2296,"num_object_clones":0,"num_object_copies":4592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2296,"num_write_kb":9184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"34'2268","reported_seq":2349,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:57.193198+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:25:57.193198+0000","last_peered":"2024-09-17T15:25:57.193198+0000","last_clean":"2024-09-17T15:25:57.193198+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:25:57.193198+0000","last_undegraded":"2024-09-17T15:25:57.193198+0000","last_fullsized":"2024-09-17T15:25:57.193198+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":2268,"log_dups_size":0,"ondisk_log_size":2268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9266118,"num_objects":2264,"num_object_clones":0,"num_object_copies":4528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":625,"num_write_kb":2500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"34'2395","reported_seq":2493,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:57.192852+0000","last_change":"2024-09-17T15:25:51.885496+0000","last_active":"2024-09-17T15:25:57.192852+0000","last_peered":"2024-09-17T15:25:57.192852+0000","last_clean":"2024-09-17T15:25:57.192852+0000","last_became_active":"2024-09-17T15:25:28.785518+0000","last_became_peered":"2024-09-17T15:25:28.785518+0000","last_unstale":"2024-09-17T15:25:57.192852+0000","last_undegraded":"2024-09-17T15:25:57.192852+0000","last_fullsized":"2024-09-17T15:25:57.192852+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":2395,"log_dups_size":0,"ondisk_log_size":2395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:59:02.229332+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9793536,"num_objects":2391,"num_object_clones":0,"num_object_copies":4782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":552,"num_write_kb":2208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"34'2303","reported_seq":2353,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:57.214332+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:25:57.214332+0000","last_peered":"2024-09-17T15:25:57.214332+0000","last_clean":"2024-09-17T15:25:57.214332+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:25:57.214332+0000","last_undegraded":"2024-09-17T15:25:57.214332+0000","last_fullsized":"2024-09-17T15:25:57.214332+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2303,"log_dups_size":0,"ondisk_log_size":2303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9433088,"num_objects":2303,"num_object_clones":0,"num_object_copies":4606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2303,"num_write_kb":9212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"34'2300","reported_seq":2360,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:57.186712+0000","last_change":"2024-09-17T15:25:27.782128+0000","last_active":"2024-09-17T15:25:57.186712+0000","last_peered":"2024-09-17T15:25:57.186712+0000","last_clean":"2024-09-17T15:24:56.061756+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:25:57.186712+0000","last_undegraded":"2024-09-17T15:25:27.778309+0000","last_fullsized":"2024-09-17T15:25:57.186712+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2300,"log_dups_size":0,"ondisk_log_size":2300,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9393094,"num_objects":2295,"num_object_clones":0,"num_object_copies":4590,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2298,"num_write_kb":9176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0,2","objects":2294}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"34'2420","reported_seq":2476,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:57.220828+0000","last_change":"2024-09-17T15:25:27.789375+0000","last_active":"2024-09-17T15:25:57.220828+0000","last_peered":"2024-09-17T15:25:57.220828+0000","last_clean":"2024-09-17T15:24:56.947098+0000","last_became_active":"2024-09-17T15:25:27.789142+0000","last_became_peered":"2024-09-17T15:25:27.789142+0000","last_unstale":"2024-09-17T15:25:57.220828+0000","last_undegraded":"2024-09-17T15:25:27.779842+0000","last_fullsized":"2024-09-17T15:25:57.220828+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2420,"log_dups_size":0,"ondisk_log_size":2420,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9892806,"num_objects":2417,"num_object_clones":0,"num_object_copies":4834,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2417,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2418,"num_write_kb":9663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0,2","objects":2416}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"34'2390","reported_seq":2447,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:57.217756+0000","last_change":"2024-09-17T15:25:27.784357+0000","last_active":"2024-09-17T15:25:57.217756+0000","last_peered":"2024-09-17T15:25:57.217756+0000","last_clean":"2024-09-17T15:24:56.310518+0000","last_became_active":"2024-09-17T15:25:27.784135+0000","last_became_peered":"2024-09-17T15:25:27.784135+0000","last_unstale":"2024-09-17T15:25:57.217756+0000","last_undegraded":"2024-09-17T15:25:27.779462+0000","last_fullsized":"2024-09-17T15:25:57.217756+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2390,"log_dups_size":0,"ondisk_log_size":2390,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9765830,"num_objects":2386,"num_object_clones":0,"num_object_copies":4772,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2388,"num_write_kb":9540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":2},{"shards":"0,2","objects":2384}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"34'2363","reported_seq":2425,"reported_epoch":34,"state":"active+recovering","last_fresh":"2024-09-17T15:25:57.217475+0000","last_change":"2024-09-17T15:25:28.783075+0000","last_active":"2024-09-17T15:25:57.217475+0000","last_peered":"2024-09-17T15:25:57.217475+0000","last_clean":"2024-09-17T15:24:49.051582+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:25:57.217475+0000","last_undegraded":"2024-09-17T15:25:57.217475+0000","last_fullsized":"2024-09-17T15:25:57.217475+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2363,"log_dups_size":0,"ondisk_log_size":2363,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9671139,"num_objects":2362,"num_object_clones":0,"num_object_copies":4724,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2362,"num_write_kb":9445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2362}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.782961+0000","last_change":"2024-09-17T15:25:27.786793+0000","last_active":"2024-09-17T15:25:28.782961+0000","last_peered":"2024-09-17T15:25:28.782961+0000","last_clean":"2024-09-17T15:25:28.782961+0000","last_became_active":"2024-09-17T15:25:27.786505+0000","last_became_peered":"2024-09-17T15:25:27.786505+0000","last_unstale":"2024-09-17T15:25:28.782961+0000","last_undegraded":"2024-09-17T15:25:28.782961+0000","last_fullsized":"2024-09-17T15:25:28.782961+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:22:15.104318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"34'2312","reported_seq":2412,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:57.176217+0000","last_change":"2024-09-17T15:25:53.959204+0000","last_active":"2024-09-17T15:25:57.176217+0000","last_peered":"2024-09-17T15:25:57.176217+0000","last_clean":"2024-09-17T15:25:57.176217+0000","last_became_active":"2024-09-17T15:25:28.785807+0000","last_became_peered":"2024-09-17T15:25:28.785807+0000","last_unstale":"2024-09-17T15:25:57.176217+0000","last_undegraded":"2024-09-17T15:25:57.176217+0000","last_fullsized":"2024-09-17T15:25:57.176217+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":2312,"log_dups_size":0,"ondisk_log_size":2312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:14:31.995973+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9441763,"num_objects":2306,"num_object_clones":0,"num_object_copies":4612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":312,"num_write_kb":1248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.787649+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:25:28.787649+0000","last_peered":"2024-09-17T15:25:28.787649+0000","last_clean":"2024-09-17T15:25:28.787649+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:25:28.787649+0000","last_undegraded":"2024-09-17T15:25:28.787649+0000","last_fullsized":"2024-09-17T15:25:28.787649+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"34'2304","reported_seq":2353,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:57.228962+0000","last_change":"2024-09-17T15:25:27.789502+0000","last_active":"2024-09-17T15:25:57.228962+0000","last_peered":"2024-09-17T15:25:57.228962+0000","last_clean":"2024-09-17T15:24:56.997318+0000","last_became_active":"2024-09-17T15:25:27.789253+0000","last_became_peered":"2024-09-17T15:25:27.789253+0000","last_unstale":"2024-09-17T15:25:57.228962+0000","last_undegraded":"2024-09-17T15:25:27.780228+0000","last_fullsized":"2024-09-17T15:25:57.228962+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2304,"log_dups_size":0,"ondisk_log_size":2304,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9405382,"num_objects":2298,"num_object_clones":0,"num_object_copies":4596,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2302,"num_write_kb":9188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2298}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":66,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:28.783096+0000","last_change":"2024-09-17T15:25:27.786740+0000","last_active":"2024-09-17T15:25:28.783096+0000","last_peered":"2024-09-17T15:25:28.783096+0000","last_clean":"2024-09-17T15:25:07.444351+0000","last_became_active":"2024-09-17T15:25:27.786489+0000","last_became_peered":"2024-09-17T15:25:27.786489+0000","last_unstale":"2024-09-17T15:25:28.783096+0000","last_undegraded":"2024-09-17T15:25:27.779442+0000","last_fullsized":"2024-09-17T15:25:28.783096+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"34'2332","reported_seq":2424,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:57.210434+0000","last_change":"2024-09-17T15:25:55.123333+0000","last_active":"2024-09-17T15:25:57.210434+0000","last_peered":"2024-09-17T15:25:57.210434+0000","last_clean":"2024-09-17T15:25:57.210434+0000","last_became_active":"2024-09-17T15:25:28.787663+0000","last_became_peered":"2024-09-17T15:25:28.787663+0000","last_unstale":"2024-09-17T15:25:57.210434+0000","last_undegraded":"2024-09-17T15:25:57.210434+0000","last_fullsized":"2024-09-17T15:25:57.210434+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":2332,"log_dups_size":0,"ondisk_log_size":2332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:10:55.294151+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9531875,"num_objects":2328,"num_object_clones":0,"num_object_copies":4656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":227,"num_write_kb":908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.787312+0000","last_change":"2024-09-17T15:25:28.786053+0000","last_active":"2024-09-17T15:25:28.787312+0000","last_peered":"2024-09-17T15:25:28.787312+0000","last_clean":"2024-09-17T15:25:28.787312+0000","last_became_active":"2024-09-17T15:25:28.785773+0000","last_became_peered":"2024-09-17T15:25:28.785773+0000","last_unstale":"2024-09-17T15:25:28.787312+0000","last_undegraded":"2024-09-17T15:25:28.787312+0000","last_fullsized":"2024-09-17T15:25:28.787312+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:46:53.130296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"34'2373","reported_seq":2465,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:57.200570+0000","last_change":"2024-09-17T15:25:56.130418+0000","last_active":"2024-09-17T15:25:57.200570+0000","last_peered":"2024-09-17T15:25:57.200570+0000","last_clean":"2024-09-17T15:25:57.200570+0000","last_became_active":"2024-09-17T15:25:28.785157+0000","last_became_peered":"2024-09-17T15:25:28.785157+0000","last_unstale":"2024-09-17T15:25:57.200570+0000","last_undegraded":"2024-09-17T15:25:57.200570+0000","last_fullsized":"2024-09-17T15:25:57.200570+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":2373,"log_dups_size":0,"ondisk_log_size":2373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:03:03.960144+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9695232,"num_objects":2367,"num_object_clones":0,"num_object_copies":4734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"34'2391","reported_seq":2447,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:57.213271+0000","last_change":"2024-09-17T15:25:27.789404+0000","last_active":"2024-09-17T15:25:57.213271+0000","last_peered":"2024-09-17T15:25:57.213271+0000","last_clean":"2024-09-17T15:24:52.804722+0000","last_became_active":"2024-09-17T15:25:27.789191+0000","last_became_peered":"2024-09-17T15:25:27.789191+0000","last_unstale":"2024-09-17T15:25:57.213271+0000","last_undegraded":"2024-09-17T15:25:27.780610+0000","last_fullsized":"2024-09-17T15:25:57.213271+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2391,"log_dups_size":0,"ondisk_log_size":2391,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9785827,"num_objects":2390,"num_object_clones":0,"num_object_copies":4780,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2390,"num_write_kb":9557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2390}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.784109+0000","last_change":"2024-09-17T15:25:27.789397+0000","last_active":"2024-09-17T15:25:28.784109+0000","last_peered":"2024-09-17T15:25:28.784109+0000","last_clean":"2024-09-17T15:25:28.784109+0000","last_became_active":"2024-09-17T15:25:27.789137+0000","last_became_peered":"2024-09-17T15:25:27.789137+0000","last_unstale":"2024-09-17T15:25:28.784109+0000","last_undegraded":"2024-09-17T15:25:28.784109+0000","last_fullsized":"2024-09-17T15:25:28.784109+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:26:53.538418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"34'2373","reported_seq":2428,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:57.217287+0000","last_change":"2024-09-17T15:25:27.789579+0000","last_active":"2024-09-17T15:25:57.217287+0000","last_peered":"2024-09-17T15:25:57.217287+0000","last_clean":"2024-09-17T15:24:56.897031+0000","last_became_active":"2024-09-17T15:25:27.789325+0000","last_became_peered":"2024-09-17T15:25:27.789325+0000","last_unstale":"2024-09-17T15:25:57.217287+0000","last_undegraded":"2024-09-17T15:25:27.781559+0000","last_fullsized":"2024-09-17T15:25:57.217287+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2373,"log_dups_size":0,"ondisk_log_size":2373,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9688489,"num_objects":2368,"num_object_clones":0,"num_object_copies":4736,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2368,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2370,"num_write_kb":9464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2368}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.785793+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:25:28.785793+0000","last_peered":"2024-09-17T15:25:28.785793+0000","last_clean":"2024-09-17T15:25:28.785793+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:25:28.785793+0000","last_undegraded":"2024-09-17T15:25:28.785793+0000","last_fullsized":"2024-09-17T15:25:28.785793+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"34'2347","reported_seq":2402,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:57.216769+0000","last_change":"2024-09-17T15:25:27.784424+0000","last_active":"2024-09-17T15:25:57.216769+0000","last_peered":"2024-09-17T15:25:57.216769+0000","last_clean":"2024-09-17T15:24:52.804663+0000","last_became_active":"2024-09-17T15:25:27.784195+0000","last_became_peered":"2024-09-17T15:25:27.784195+0000","last_unstale":"2024-09-17T15:25:57.216769+0000","last_undegraded":"2024-09-17T15:25:27.779406+0000","last_fullsized":"2024-09-17T15:25:57.216769+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2347,"log_dups_size":0,"ondisk_log_size":2347,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9597894,"num_objects":2345,"num_object_clones":0,"num_object_copies":4690,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2345,"num_write_kb":9374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2345}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.783601+0000","last_change":"2024-09-17T15:25:27.786796+0000","last_active":"2024-09-17T15:25:28.783601+0000","last_peered":"2024-09-17T15:25:28.783601+0000","last_clean":"2024-09-17T15:25:28.783601+0000","last_became_active":"2024-09-17T15:25:27.786506+0000","last_became_peered":"2024-09-17T15:25:27.786506+0000","last_unstale":"2024-09-17T15:25:28.783601+0000","last_undegraded":"2024-09-17T15:25:28.783601+0000","last_fullsized":"2024-09-17T15:25:28.783601+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:55:57.547976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.786536+0000","last_change":"2024-09-17T15:25:28.785423+0000","last_active":"2024-09-17T15:25:28.786536+0000","last_peered":"2024-09-17T15:25:28.786536+0000","last_clean":"2024-09-17T15:25:28.786536+0000","last_became_active":"2024-09-17T15:25:28.785151+0000","last_became_peered":"2024-09-17T15:25:28.785151+0000","last_unstale":"2024-09-17T15:25:28.786536+0000","last_undegraded":"2024-09-17T15:25:28.786536+0000","last_fullsized":"2024-09-17T15:25:28.786536+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:47:12.307723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"34'2345","reported_seq":2398,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:57.207922+0000","last_change":"2024-09-17T15:25:27.789556+0000","last_active":"2024-09-17T15:25:57.207922+0000","last_peered":"2024-09-17T15:25:57.207922+0000","last_clean":"2024-09-17T15:24:56.609264+0000","last_became_active":"2024-09-17T15:25:27.789304+0000","last_became_peered":"2024-09-17T15:25:27.789304+0000","last_unstale":"2024-09-17T15:25:57.207922+0000","last_undegraded":"2024-09-17T15:25:27.782129+0000","last_fullsized":"2024-09-17T15:25:57.207922+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2345,"log_dups_size":0,"ondisk_log_size":2345,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9557900,"num_objects":2337,"num_object_clones":0,"num_object_copies":4674,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2342,"num_write_kb":9339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0","objects":1},{"shards":"0,2","objects":2335}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":153324389,"num_objects":37453,"num_object_clones":0,"num_object_copies":74906,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":35,"num_objects_misplaced":0,"num_objects_unfound":5,"num_objects_dirty":37453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27665,"num_write_kb":110546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":306876416,"data_stored":306724670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37512,"ondisk_log_size":37512,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051595,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":248676,"kb_used_data":144784,"kb_used_omap":4,"kb_used_meta":103867,"kb_avail":94123164,"statfs":{"total":96636764160,"available":96382119936,"internally_reserved":0,"allocated":148258816,"data_stored":148047746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":106360492},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1,3,3,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247823,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":255048,"kb_used_data":147064,"kb_used_omap":4,"kb_used_meta":107963,"kb_avail":94116792,"statfs":{"total":96636764160,"available":96375595008,"internally_reserved":0,"allocated":150593536,"data_stored":150356711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":110554792},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,13,1,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":153436160,"data_stored":153400030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":153325568,"data_stored":153311116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:26:01.286 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 22 20 43557 43537 7.72931 5.89844 0.00715011 0.0101016 2024-09-17T15:26:02.286 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 23 20 45211 45191 7.67414 6.46094 0.0142532 0.0101745 2024-09-17T15:26:03.286 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 24 20 46829 46809 7.61771 6.32031 0.013041 0.0102496 2024-09-17T15:26:04.106 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:26:04.286 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 25 20 48330 48310 7.54752 5.86328 0.0142144 0.0103447 2024-09-17T15:26:04.625 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:26:04.626 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:26:04.651 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2024-09-17T15:26:03.589074+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":173943672,"num_objects":42489,"num_object_clones":0,"num_object_copies":84978,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":5,"num_objects_dirty":42489,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32701,"num_write_kb":130684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42549,"ondisk_log_size":42549,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":563296,"kb_used_data":338944,"kb_used_omap":8,"kb_used_meta":224311,"kb_avail":188180384,"statfs":{"total":193273528320,"available":192696713216,"internally_reserved":0,"allocated":347078656,"data_stored":346630761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8876,"internal_metadata":229694804},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1,16,4,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":78300817,"num_objects":19112,"num_object_clones":0,"num_object_copies":38224,"num_objects_missing_on_primary":-9,"num_objects_missing":-9,"num_objects_degraded":-18,"num_objects_misplaced":0,"num_objects_unfound":-22,"num_objects_dirty":19112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9297,"num_write_kb":37248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002783"},"pg_stats":[{"pgid":"2.f","version":"34'2534","reported_seq":2585,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:59.807256+0000","last_change":"2024-09-17T15:25:27.786800+0000","last_active":"2024-09-17T15:25:59.807256+0000","last_peered":"2024-09-17T15:25:59.807256+0000","last_clean":"2024-09-17T15:25:59.807256+0000","last_became_active":"2024-09-17T15:25:27.786453+0000","last_became_peered":"2024-09-17T15:25:27.786453+0000","last_unstale":"2024-09-17T15:25:59.807256+0000","last_undegraded":"2024-09-17T15:25:59.807256+0000","last_fullsized":"2024-09-17T15:25:59.807256+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2534,"log_dups_size":0,"ondisk_log_size":2534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:46:04.427346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10379264,"num_objects":2534,"num_object_clones":0,"num_object_copies":5068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2534,"num_write_kb":10136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"34'2696","reported_seq":2777,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:26:01.685666+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:26:01.685666+0000","last_peered":"2024-09-17T15:26:01.685666+0000","last_clean":"2024-09-17T15:26:01.685666+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:26:01.685666+0000","last_undegraded":"2024-09-17T15:26:01.685666+0000","last_fullsized":"2024-09-17T15:26:01.685666+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":2696,"log_dups_size":0,"ondisk_log_size":2696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11019206,"num_objects":2692,"num_object_clones":0,"num_object_copies":5384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1053,"num_write_kb":4212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"34'2835","reported_seq":2933,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:26:01.643609+0000","last_change":"2024-09-17T15:25:51.885496+0000","last_active":"2024-09-17T15:26:01.643609+0000","last_peered":"2024-09-17T15:26:01.643609+0000","last_clean":"2024-09-17T15:26:01.643609+0000","last_became_active":"2024-09-17T15:25:28.785518+0000","last_became_peered":"2024-09-17T15:25:28.785518+0000","last_unstale":"2024-09-17T15:26:01.643609+0000","last_undegraded":"2024-09-17T15:26:01.643609+0000","last_fullsized":"2024-09-17T15:26:01.643609+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":2835,"log_dups_size":0,"ondisk_log_size":2835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:59:02.229332+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11595776,"num_objects":2831,"num_object_clones":0,"num_object_copies":5662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2831,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":992,"num_write_kb":3968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"34'2707","reported_seq":2757,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:26:01.670273+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:26:01.670273+0000","last_peered":"2024-09-17T15:26:01.670273+0000","last_clean":"2024-09-17T15:26:01.670273+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:26:01.670273+0000","last_undegraded":"2024-09-17T15:26:01.670273+0000","last_fullsized":"2024-09-17T15:26:01.670273+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2707,"log_dups_size":0,"ondisk_log_size":2707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11087872,"num_objects":2707,"num_object_clones":0,"num_object_copies":5414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2707,"num_write_kb":10828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"34'2529","reported_seq":2589,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:59.872755+0000","last_change":"2024-09-17T15:25:27.782128+0000","last_active":"2024-09-17T15:25:59.872755+0000","last_peered":"2024-09-17T15:25:59.872755+0000","last_clean":"2024-09-17T15:24:56.061756+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:25:59.872755+0000","last_undegraded":"2024-09-17T15:25:27.778309+0000","last_fullsized":"2024-09-17T15:25:59.872755+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2529,"log_dups_size":0,"ondisk_log_size":2529,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10331078,"num_objects":2524,"num_object_clones":0,"num_object_copies":5048,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2527,"num_write_kb":10092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0,2","objects":2523}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"34'2695","reported_seq":2751,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:59.857116+0000","last_change":"2024-09-17T15:25:27.789375+0000","last_active":"2024-09-17T15:25:59.857116+0000","last_peered":"2024-09-17T15:25:59.857116+0000","last_clean":"2024-09-17T15:24:56.947098+0000","last_became_active":"2024-09-17T15:25:27.789142+0000","last_became_peered":"2024-09-17T15:25:27.789142+0000","last_unstale":"2024-09-17T15:25:59.857116+0000","last_undegraded":"2024-09-17T15:25:27.779842+0000","last_fullsized":"2024-09-17T15:25:59.857116+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2695,"log_dups_size":0,"ondisk_log_size":2695,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11019206,"num_objects":2692,"num_object_clones":0,"num_object_copies":5384,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2693,"num_write_kb":10763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0,2","objects":2691}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"34'2656","reported_seq":2711,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:59.856039+0000","last_change":"2024-09-17T15:25:27.784357+0000","last_active":"2024-09-17T15:25:59.856039+0000","last_peered":"2024-09-17T15:25:59.856039+0000","last_clean":"2024-09-17T15:24:56.310518+0000","last_became_active":"2024-09-17T15:25:27.784135+0000","last_became_peered":"2024-09-17T15:25:27.784135+0000","last_unstale":"2024-09-17T15:25:59.856039+0000","last_undegraded":"2024-09-17T15:25:27.779462+0000","last_fullsized":"2024-09-17T15:25:59.856039+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2656,"log_dups_size":0,"ondisk_log_size":2656,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10855366,"num_objects":2652,"num_object_clones":0,"num_object_copies":5304,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2654,"num_write_kb":10604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":2},{"shards":"0,2","objects":2650}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"34'2626","reported_seq":2688,"reported_epoch":34,"state":"active+recovering","last_fresh":"2024-09-17T15:25:59.854210+0000","last_change":"2024-09-17T15:25:28.783075+0000","last_active":"2024-09-17T15:25:59.854210+0000","last_peered":"2024-09-17T15:25:59.854210+0000","last_clean":"2024-09-17T15:24:49.051582+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:25:59.854210+0000","last_undegraded":"2024-09-17T15:25:59.854210+0000","last_fullsized":"2024-09-17T15:25:59.854210+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2626,"log_dups_size":0,"ondisk_log_size":2626,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10748387,"num_objects":2625,"num_object_clones":0,"num_object_copies":5250,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2625,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2625,"num_write_kb":10497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2625}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.782961+0000","last_change":"2024-09-17T15:25:27.786793+0000","last_active":"2024-09-17T15:25:28.782961+0000","last_peered":"2024-09-17T15:25:28.782961+0000","last_clean":"2024-09-17T15:25:28.782961+0000","last_became_active":"2024-09-17T15:25:27.786505+0000","last_became_peered":"2024-09-17T15:25:27.786505+0000","last_unstale":"2024-09-17T15:25:28.782961+0000","last_undegraded":"2024-09-17T15:25:28.782961+0000","last_fullsized":"2024-09-17T15:25:28.782961+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:22:15.104318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"34'2722","reported_seq":2821,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:26:01.676083+0000","last_change":"2024-09-17T15:25:53.959204+0000","last_active":"2024-09-17T15:26:01.676083+0000","last_peered":"2024-09-17T15:26:01.676083+0000","last_clean":"2024-09-17T15:26:01.676083+0000","last_became_active":"2024-09-17T15:25:28.785807+0000","last_became_peered":"2024-09-17T15:25:28.785807+0000","last_unstale":"2024-09-17T15:26:01.676083+0000","last_undegraded":"2024-09-17T15:26:01.676083+0000","last_fullsized":"2024-09-17T15:26:01.676083+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":2722,"log_dups_size":0,"ondisk_log_size":2722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:14:31.995973+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11121123,"num_objects":2716,"num_object_clones":0,"num_object_copies":5432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":722,"num_write_kb":2888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.787649+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:25:28.787649+0000","last_peered":"2024-09-17T15:25:28.787649+0000","last_clean":"2024-09-17T15:25:28.787649+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:25:28.787649+0000","last_undegraded":"2024-09-17T15:25:28.787649+0000","last_fullsized":"2024-09-17T15:25:28.787649+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"34'2562","reported_seq":2612,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:59.862064+0000","last_change":"2024-09-17T15:25:27.789502+0000","last_active":"2024-09-17T15:25:59.862064+0000","last_peered":"2024-09-17T15:25:59.862064+0000","last_clean":"2024-09-17T15:24:56.997318+0000","last_became_active":"2024-09-17T15:25:27.789253+0000","last_became_peered":"2024-09-17T15:25:27.789253+0000","last_unstale":"2024-09-17T15:25:59.862064+0000","last_undegraded":"2024-09-17T15:25:27.780228+0000","last_fullsized":"2024-09-17T15:25:59.862064+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2562,"log_dups_size":0,"ondisk_log_size":2562,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10462150,"num_objects":2556,"num_object_clones":0,"num_object_copies":5112,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2560,"num_write_kb":10220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2556}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":66,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:28.783096+0000","last_change":"2024-09-17T15:25:27.786740+0000","last_active":"2024-09-17T15:25:28.783096+0000","last_peered":"2024-09-17T15:25:28.783096+0000","last_clean":"2024-09-17T15:25:07.444351+0000","last_became_active":"2024-09-17T15:25:27.786489+0000","last_became_peered":"2024-09-17T15:25:27.786489+0000","last_unstale":"2024-09-17T15:25:28.783096+0000","last_undegraded":"2024-09-17T15:25:27.779442+0000","last_fullsized":"2024-09-17T15:25:28.783096+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"34'2724","reported_seq":2816,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:26:01.686207+0000","last_change":"2024-09-17T15:25:55.123333+0000","last_active":"2024-09-17T15:26:01.686207+0000","last_peered":"2024-09-17T15:26:01.686207+0000","last_clean":"2024-09-17T15:26:01.686207+0000","last_became_active":"2024-09-17T15:25:28.787663+0000","last_became_peered":"2024-09-17T15:25:28.787663+0000","last_unstale":"2024-09-17T15:26:01.686207+0000","last_undegraded":"2024-09-17T15:26:01.686207+0000","last_fullsized":"2024-09-17T15:26:01.686207+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":2724,"log_dups_size":0,"ondisk_log_size":2724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:10:55.294151+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11137507,"num_objects":2720,"num_object_clones":0,"num_object_copies":5440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":619,"num_write_kb":2476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.787312+0000","last_change":"2024-09-17T15:25:28.786053+0000","last_active":"2024-09-17T15:25:28.787312+0000","last_peered":"2024-09-17T15:25:28.787312+0000","last_clean":"2024-09-17T15:25:28.787312+0000","last_became_active":"2024-09-17T15:25:28.785773+0000","last_became_peered":"2024-09-17T15:25:28.785773+0000","last_unstale":"2024-09-17T15:25:28.787312+0000","last_undegraded":"2024-09-17T15:25:28.787312+0000","last_fullsized":"2024-09-17T15:25:28.787312+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:46:53.130296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"34'2780","reported_seq":2872,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:26:01.670756+0000","last_change":"2024-09-17T15:25:56.130418+0000","last_active":"2024-09-17T15:26:01.670756+0000","last_peered":"2024-09-17T15:26:01.670756+0000","last_clean":"2024-09-17T15:26:01.670756+0000","last_became_active":"2024-09-17T15:25:28.785157+0000","last_became_peered":"2024-09-17T15:25:28.785157+0000","last_unstale":"2024-09-17T15:26:01.670756+0000","last_undegraded":"2024-09-17T15:26:01.670756+0000","last_fullsized":"2024-09-17T15:26:01.670756+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":2780,"log_dups_size":0,"ondisk_log_size":2780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:03:03.960144+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11362304,"num_objects":2774,"num_object_clones":0,"num_object_copies":5548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":542,"num_write_kb":2168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"34'2643","reported_seq":2699,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:59.871137+0000","last_change":"2024-09-17T15:25:27.789404+0000","last_active":"2024-09-17T15:25:59.871137+0000","last_peered":"2024-09-17T15:25:59.871137+0000","last_clean":"2024-09-17T15:24:52.804722+0000","last_became_active":"2024-09-17T15:25:27.789191+0000","last_became_peered":"2024-09-17T15:25:27.789191+0000","last_unstale":"2024-09-17T15:25:59.871137+0000","last_undegraded":"2024-09-17T15:25:27.780610+0000","last_fullsized":"2024-09-17T15:25:59.871137+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2643,"log_dups_size":0,"ondisk_log_size":2643,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10818019,"num_objects":2642,"num_object_clones":0,"num_object_copies":5284,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2642,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2642,"num_write_kb":10565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2642}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.784109+0000","last_change":"2024-09-17T15:25:27.789397+0000","last_active":"2024-09-17T15:25:28.784109+0000","last_peered":"2024-09-17T15:25:28.784109+0000","last_clean":"2024-09-17T15:25:28.784109+0000","last_became_active":"2024-09-17T15:25:27.789137+0000","last_became_peered":"2024-09-17T15:25:27.789137+0000","last_unstale":"2024-09-17T15:25:28.784109+0000","last_undegraded":"2024-09-17T15:25:28.784109+0000","last_fullsized":"2024-09-17T15:25:28.784109+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:26:53.538418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"34'2649","reported_seq":2704,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:59.858974+0000","last_change":"2024-09-17T15:25:27.789579+0000","last_active":"2024-09-17T15:25:59.858974+0000","last_peered":"2024-09-17T15:25:59.858974+0000","last_clean":"2024-09-17T15:24:56.897031+0000","last_became_active":"2024-09-17T15:25:27.789325+0000","last_became_peered":"2024-09-17T15:25:27.789325+0000","last_unstale":"2024-09-17T15:25:59.858974+0000","last_undegraded":"2024-09-17T15:25:27.781559+0000","last_fullsized":"2024-09-17T15:25:59.858974+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2649,"log_dups_size":0,"ondisk_log_size":2649,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10818985,"num_objects":2644,"num_object_clones":0,"num_object_copies":5288,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2646,"num_write_kb":10568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2644}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.785793+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:25:28.785793+0000","last_peered":"2024-09-17T15:25:28.785793+0000","last_clean":"2024-09-17T15:25:28.785793+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:25:28.785793+0000","last_undegraded":"2024-09-17T15:25:28.785793+0000","last_fullsized":"2024-09-17T15:25:28.785793+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"34'2580","reported_seq":2635,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:59.866862+0000","last_change":"2024-09-17T15:25:27.784424+0000","last_active":"2024-09-17T15:25:59.866862+0000","last_peered":"2024-09-17T15:25:59.866862+0000","last_clean":"2024-09-17T15:24:52.804663+0000","last_became_active":"2024-09-17T15:25:27.784195+0000","last_became_peered":"2024-09-17T15:25:27.784195+0000","last_unstale":"2024-09-17T15:25:59.866862+0000","last_undegraded":"2024-09-17T15:25:27.779406+0000","last_fullsized":"2024-09-17T15:25:59.866862+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2580,"log_dups_size":0,"ondisk_log_size":2580,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10552262,"num_objects":2578,"num_object_clones":0,"num_object_copies":5156,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2578,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2578,"num_write_kb":10306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2578}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.783601+0000","last_change":"2024-09-17T15:25:27.786796+0000","last_active":"2024-09-17T15:25:28.783601+0000","last_peered":"2024-09-17T15:25:28.783601+0000","last_clean":"2024-09-17T15:25:28.783601+0000","last_became_active":"2024-09-17T15:25:27.786506+0000","last_became_peered":"2024-09-17T15:25:27.786506+0000","last_unstale":"2024-09-17T15:25:28.783601+0000","last_undegraded":"2024-09-17T15:25:28.783601+0000","last_fullsized":"2024-09-17T15:25:28.783601+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:55:57.547976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.786536+0000","last_change":"2024-09-17T15:25:28.785423+0000","last_active":"2024-09-17T15:25:28.786536+0000","last_peered":"2024-09-17T15:25:28.786536+0000","last_clean":"2024-09-17T15:25:28.786536+0000","last_became_active":"2024-09-17T15:25:28.785151+0000","last_became_peered":"2024-09-17T15:25:28.785151+0000","last_unstale":"2024-09-17T15:25:28.786536+0000","last_undegraded":"2024-09-17T15:25:28.786536+0000","last_fullsized":"2024-09-17T15:25:28.786536+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:47:12.307723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"34'2608","reported_seq":2660,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:59.847727+0000","last_change":"2024-09-17T15:25:27.789556+0000","last_active":"2024-09-17T15:25:59.847727+0000","last_peered":"2024-09-17T15:25:59.847727+0000","last_clean":"2024-09-17T15:24:56.609264+0000","last_became_active":"2024-09-17T15:25:27.789304+0000","last_became_peered":"2024-09-17T15:25:27.789304+0000","last_unstale":"2024-09-17T15:25:59.847727+0000","last_undegraded":"2024-09-17T15:25:27.782129+0000","last_fullsized":"2024-09-17T15:25:59.847727+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2608,"log_dups_size":0,"ondisk_log_size":2608,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10635148,"num_objects":2600,"num_object_clones":0,"num_object_copies":5200,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2600,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2605,"num_write_kb":10391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0","objects":1},{"shards":"0,2","objects":2598}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":173943653,"num_objects":42487,"num_object_clones":0,"num_object_copies":84974,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":35,"num_objects_misplaced":0,"num_objects_unfound":5,"num_objects_dirty":42487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32699,"num_write_kb":130682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":351145984,"data_stored":350994238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42546,"ondisk_log_size":42546,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051596,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":286572,"kb_used_data":174364,"kb_used_omap":4,"kb_used_meta":112187,"kb_avail":94085268,"statfs":{"total":96636764160,"available":96343314432,"internally_reserved":0,"allocated":178548736,"data_stored":178337666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":114880172},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1,3,3,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247824,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":276724,"kb_used_data":164580,"kb_used_omap":4,"kb_used_meta":112123,"kb_avail":94095116,"statfs":{"total":96636764160,"available":96353398784,"internally_reserved":0,"allocated":168529920,"data_stored":168293095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":114814632},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,13,1,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":169820160,"data_stored":169784030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":181211136,"data_stored":181196684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:26:04.653 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:26:05.181 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:26:05.181 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:26:05.201 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2024-09-17T15:26:03.589074+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":173943672,"num_objects":42489,"num_object_clones":0,"num_object_copies":84978,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":5,"num_objects_dirty":42489,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32701,"num_write_kb":130684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42549,"ondisk_log_size":42549,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":563296,"kb_used_data":338944,"kb_used_omap":8,"kb_used_meta":224311,"kb_avail":188180384,"statfs":{"total":193273528320,"available":192696713216,"internally_reserved":0,"allocated":347078656,"data_stored":346630761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8876,"internal_metadata":229694804},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1,16,4,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":78300817,"num_objects":19112,"num_object_clones":0,"num_object_copies":38224,"num_objects_missing_on_primary":-9,"num_objects_missing":-9,"num_objects_degraded":-18,"num_objects_misplaced":0,"num_objects_unfound":-22,"num_objects_dirty":19112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9297,"num_write_kb":37248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002783"},"pg_stats":[{"pgid":"2.f","version":"34'2534","reported_seq":2585,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:59.807256+0000","last_change":"2024-09-17T15:25:27.786800+0000","last_active":"2024-09-17T15:25:59.807256+0000","last_peered":"2024-09-17T15:25:59.807256+0000","last_clean":"2024-09-17T15:25:59.807256+0000","last_became_active":"2024-09-17T15:25:27.786453+0000","last_became_peered":"2024-09-17T15:25:27.786453+0000","last_unstale":"2024-09-17T15:25:59.807256+0000","last_undegraded":"2024-09-17T15:25:59.807256+0000","last_fullsized":"2024-09-17T15:25:59.807256+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2534,"log_dups_size":0,"ondisk_log_size":2534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:46:04.427346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10379264,"num_objects":2534,"num_object_clones":0,"num_object_copies":5068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2534,"num_write_kb":10136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"34'2696","reported_seq":2777,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:26:01.685666+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:26:01.685666+0000","last_peered":"2024-09-17T15:26:01.685666+0000","last_clean":"2024-09-17T15:26:01.685666+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:26:01.685666+0000","last_undegraded":"2024-09-17T15:26:01.685666+0000","last_fullsized":"2024-09-17T15:26:01.685666+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":2696,"log_dups_size":0,"ondisk_log_size":2696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11019206,"num_objects":2692,"num_object_clones":0,"num_object_copies":5384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1053,"num_write_kb":4212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"34'2835","reported_seq":2933,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:26:01.643609+0000","last_change":"2024-09-17T15:25:51.885496+0000","last_active":"2024-09-17T15:26:01.643609+0000","last_peered":"2024-09-17T15:26:01.643609+0000","last_clean":"2024-09-17T15:26:01.643609+0000","last_became_active":"2024-09-17T15:25:28.785518+0000","last_became_peered":"2024-09-17T15:25:28.785518+0000","last_unstale":"2024-09-17T15:26:01.643609+0000","last_undegraded":"2024-09-17T15:26:01.643609+0000","last_fullsized":"2024-09-17T15:26:01.643609+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":2835,"log_dups_size":0,"ondisk_log_size":2835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:59:02.229332+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11595776,"num_objects":2831,"num_object_clones":0,"num_object_copies":5662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2831,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":992,"num_write_kb":3968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"34'2707","reported_seq":2757,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:26:01.670273+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:26:01.670273+0000","last_peered":"2024-09-17T15:26:01.670273+0000","last_clean":"2024-09-17T15:26:01.670273+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:26:01.670273+0000","last_undegraded":"2024-09-17T15:26:01.670273+0000","last_fullsized":"2024-09-17T15:26:01.670273+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2707,"log_dups_size":0,"ondisk_log_size":2707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11087872,"num_objects":2707,"num_object_clones":0,"num_object_copies":5414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2707,"num_write_kb":10828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"34'2529","reported_seq":2589,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:59.872755+0000","last_change":"2024-09-17T15:25:27.782128+0000","last_active":"2024-09-17T15:25:59.872755+0000","last_peered":"2024-09-17T15:25:59.872755+0000","last_clean":"2024-09-17T15:24:56.061756+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:25:59.872755+0000","last_undegraded":"2024-09-17T15:25:27.778309+0000","last_fullsized":"2024-09-17T15:25:59.872755+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2529,"log_dups_size":0,"ondisk_log_size":2529,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10331078,"num_objects":2524,"num_object_clones":0,"num_object_copies":5048,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2527,"num_write_kb":10092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0,2","objects":2523}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"34'2695","reported_seq":2751,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:59.857116+0000","last_change":"2024-09-17T15:25:27.789375+0000","last_active":"2024-09-17T15:25:59.857116+0000","last_peered":"2024-09-17T15:25:59.857116+0000","last_clean":"2024-09-17T15:24:56.947098+0000","last_became_active":"2024-09-17T15:25:27.789142+0000","last_became_peered":"2024-09-17T15:25:27.789142+0000","last_unstale":"2024-09-17T15:25:59.857116+0000","last_undegraded":"2024-09-17T15:25:27.779842+0000","last_fullsized":"2024-09-17T15:25:59.857116+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2695,"log_dups_size":0,"ondisk_log_size":2695,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11019206,"num_objects":2692,"num_object_clones":0,"num_object_copies":5384,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2693,"num_write_kb":10763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0,2","objects":2691}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"34'2656","reported_seq":2711,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:59.856039+0000","last_change":"2024-09-17T15:25:27.784357+0000","last_active":"2024-09-17T15:25:59.856039+0000","last_peered":"2024-09-17T15:25:59.856039+0000","last_clean":"2024-09-17T15:24:56.310518+0000","last_became_active":"2024-09-17T15:25:27.784135+0000","last_became_peered":"2024-09-17T15:25:27.784135+0000","last_unstale":"2024-09-17T15:25:59.856039+0000","last_undegraded":"2024-09-17T15:25:27.779462+0000","last_fullsized":"2024-09-17T15:25:59.856039+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2656,"log_dups_size":0,"ondisk_log_size":2656,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10855366,"num_objects":2652,"num_object_clones":0,"num_object_copies":5304,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2654,"num_write_kb":10604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":2},{"shards":"0,2","objects":2650}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"34'2626","reported_seq":2688,"reported_epoch":34,"state":"active+recovering","last_fresh":"2024-09-17T15:25:59.854210+0000","last_change":"2024-09-17T15:25:28.783075+0000","last_active":"2024-09-17T15:25:59.854210+0000","last_peered":"2024-09-17T15:25:59.854210+0000","last_clean":"2024-09-17T15:24:49.051582+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:25:59.854210+0000","last_undegraded":"2024-09-17T15:25:59.854210+0000","last_fullsized":"2024-09-17T15:25:59.854210+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2626,"log_dups_size":0,"ondisk_log_size":2626,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10748387,"num_objects":2625,"num_object_clones":0,"num_object_copies":5250,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2625,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2625,"num_write_kb":10497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2625}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.782961+0000","last_change":"2024-09-17T15:25:27.786793+0000","last_active":"2024-09-17T15:25:28.782961+0000","last_peered":"2024-09-17T15:25:28.782961+0000","last_clean":"2024-09-17T15:25:28.782961+0000","last_became_active":"2024-09-17T15:25:27.786505+0000","last_became_peered":"2024-09-17T15:25:27.786505+0000","last_unstale":"2024-09-17T15:25:28.782961+0000","last_undegraded":"2024-09-17T15:25:28.782961+0000","last_fullsized":"2024-09-17T15:25:28.782961+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:22:15.104318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"34'2722","reported_seq":2821,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:26:01.676083+0000","last_change":"2024-09-17T15:25:53.959204+0000","last_active":"2024-09-17T15:26:01.676083+0000","last_peered":"2024-09-17T15:26:01.676083+0000","last_clean":"2024-09-17T15:26:01.676083+0000","last_became_active":"2024-09-17T15:25:28.785807+0000","last_became_peered":"2024-09-17T15:25:28.785807+0000","last_unstale":"2024-09-17T15:26:01.676083+0000","last_undegraded":"2024-09-17T15:26:01.676083+0000","last_fullsized":"2024-09-17T15:26:01.676083+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":2722,"log_dups_size":0,"ondisk_log_size":2722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:14:31.995973+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11121123,"num_objects":2716,"num_object_clones":0,"num_object_copies":5432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":722,"num_write_kb":2888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.787649+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:25:28.787649+0000","last_peered":"2024-09-17T15:25:28.787649+0000","last_clean":"2024-09-17T15:25:28.787649+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:25:28.787649+0000","last_undegraded":"2024-09-17T15:25:28.787649+0000","last_fullsized":"2024-09-17T15:25:28.787649+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"34'2562","reported_seq":2612,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:59.862064+0000","last_change":"2024-09-17T15:25:27.789502+0000","last_active":"2024-09-17T15:25:59.862064+0000","last_peered":"2024-09-17T15:25:59.862064+0000","last_clean":"2024-09-17T15:24:56.997318+0000","last_became_active":"2024-09-17T15:25:27.789253+0000","last_became_peered":"2024-09-17T15:25:27.789253+0000","last_unstale":"2024-09-17T15:25:59.862064+0000","last_undegraded":"2024-09-17T15:25:27.780228+0000","last_fullsized":"2024-09-17T15:25:59.862064+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2562,"log_dups_size":0,"ondisk_log_size":2562,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10462150,"num_objects":2556,"num_object_clones":0,"num_object_copies":5112,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2560,"num_write_kb":10220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2556}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":66,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:28.783096+0000","last_change":"2024-09-17T15:25:27.786740+0000","last_active":"2024-09-17T15:25:28.783096+0000","last_peered":"2024-09-17T15:25:28.783096+0000","last_clean":"2024-09-17T15:25:07.444351+0000","last_became_active":"2024-09-17T15:25:27.786489+0000","last_became_peered":"2024-09-17T15:25:27.786489+0000","last_unstale":"2024-09-17T15:25:28.783096+0000","last_undegraded":"2024-09-17T15:25:27.779442+0000","last_fullsized":"2024-09-17T15:25:28.783096+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"34'2724","reported_seq":2816,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:26:01.686207+0000","last_change":"2024-09-17T15:25:55.123333+0000","last_active":"2024-09-17T15:26:01.686207+0000","last_peered":"2024-09-17T15:26:01.686207+0000","last_clean":"2024-09-17T15:26:01.686207+0000","last_became_active":"2024-09-17T15:25:28.787663+0000","last_became_peered":"2024-09-17T15:25:28.787663+0000","last_unstale":"2024-09-17T15:26:01.686207+0000","last_undegraded":"2024-09-17T15:26:01.686207+0000","last_fullsized":"2024-09-17T15:26:01.686207+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":2724,"log_dups_size":0,"ondisk_log_size":2724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:10:55.294151+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11137507,"num_objects":2720,"num_object_clones":0,"num_object_copies":5440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":619,"num_write_kb":2476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.787312+0000","last_change":"2024-09-17T15:25:28.786053+0000","last_active":"2024-09-17T15:25:28.787312+0000","last_peered":"2024-09-17T15:25:28.787312+0000","last_clean":"2024-09-17T15:25:28.787312+0000","last_became_active":"2024-09-17T15:25:28.785773+0000","last_became_peered":"2024-09-17T15:25:28.785773+0000","last_unstale":"2024-09-17T15:25:28.787312+0000","last_undegraded":"2024-09-17T15:25:28.787312+0000","last_fullsized":"2024-09-17T15:25:28.787312+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:46:53.130296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"34'2780","reported_seq":2872,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:26:01.670756+0000","last_change":"2024-09-17T15:25:56.130418+0000","last_active":"2024-09-17T15:26:01.670756+0000","last_peered":"2024-09-17T15:26:01.670756+0000","last_clean":"2024-09-17T15:26:01.670756+0000","last_became_active":"2024-09-17T15:25:28.785157+0000","last_became_peered":"2024-09-17T15:25:28.785157+0000","last_unstale":"2024-09-17T15:26:01.670756+0000","last_undegraded":"2024-09-17T15:26:01.670756+0000","last_fullsized":"2024-09-17T15:26:01.670756+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":2780,"log_dups_size":0,"ondisk_log_size":2780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:03:03.960144+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11362304,"num_objects":2774,"num_object_clones":0,"num_object_copies":5548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":542,"num_write_kb":2168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"34'2643","reported_seq":2699,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:59.871137+0000","last_change":"2024-09-17T15:25:27.789404+0000","last_active":"2024-09-17T15:25:59.871137+0000","last_peered":"2024-09-17T15:25:59.871137+0000","last_clean":"2024-09-17T15:24:52.804722+0000","last_became_active":"2024-09-17T15:25:27.789191+0000","last_became_peered":"2024-09-17T15:25:27.789191+0000","last_unstale":"2024-09-17T15:25:59.871137+0000","last_undegraded":"2024-09-17T15:25:27.780610+0000","last_fullsized":"2024-09-17T15:25:59.871137+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2643,"log_dups_size":0,"ondisk_log_size":2643,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10818019,"num_objects":2642,"num_object_clones":0,"num_object_copies":5284,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2642,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2642,"num_write_kb":10565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2642}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.784109+0000","last_change":"2024-09-17T15:25:27.789397+0000","last_active":"2024-09-17T15:25:28.784109+0000","last_peered":"2024-09-17T15:25:28.784109+0000","last_clean":"2024-09-17T15:25:28.784109+0000","last_became_active":"2024-09-17T15:25:27.789137+0000","last_became_peered":"2024-09-17T15:25:27.789137+0000","last_unstale":"2024-09-17T15:25:28.784109+0000","last_undegraded":"2024-09-17T15:25:28.784109+0000","last_fullsized":"2024-09-17T15:25:28.784109+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:26:53.538418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"34'2649","reported_seq":2704,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:59.858974+0000","last_change":"2024-09-17T15:25:27.789579+0000","last_active":"2024-09-17T15:25:59.858974+0000","last_peered":"2024-09-17T15:25:59.858974+0000","last_clean":"2024-09-17T15:24:56.897031+0000","last_became_active":"2024-09-17T15:25:27.789325+0000","last_became_peered":"2024-09-17T15:25:27.789325+0000","last_unstale":"2024-09-17T15:25:59.858974+0000","last_undegraded":"2024-09-17T15:25:27.781559+0000","last_fullsized":"2024-09-17T15:25:59.858974+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2649,"log_dups_size":0,"ondisk_log_size":2649,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10818985,"num_objects":2644,"num_object_clones":0,"num_object_copies":5288,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2646,"num_write_kb":10568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2644}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.785793+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:25:28.785793+0000","last_peered":"2024-09-17T15:25:28.785793+0000","last_clean":"2024-09-17T15:25:28.785793+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:25:28.785793+0000","last_undegraded":"2024-09-17T15:25:28.785793+0000","last_fullsized":"2024-09-17T15:25:28.785793+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"34'2580","reported_seq":2635,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:59.866862+0000","last_change":"2024-09-17T15:25:27.784424+0000","last_active":"2024-09-17T15:25:59.866862+0000","last_peered":"2024-09-17T15:25:59.866862+0000","last_clean":"2024-09-17T15:24:52.804663+0000","last_became_active":"2024-09-17T15:25:27.784195+0000","last_became_peered":"2024-09-17T15:25:27.784195+0000","last_unstale":"2024-09-17T15:25:59.866862+0000","last_undegraded":"2024-09-17T15:25:27.779406+0000","last_fullsized":"2024-09-17T15:25:59.866862+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2580,"log_dups_size":0,"ondisk_log_size":2580,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10552262,"num_objects":2578,"num_object_clones":0,"num_object_copies":5156,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2578,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2578,"num_write_kb":10306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2578}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.783601+0000","last_change":"2024-09-17T15:25:27.786796+0000","last_active":"2024-09-17T15:25:28.783601+0000","last_peered":"2024-09-17T15:25:28.783601+0000","last_clean":"2024-09-17T15:25:28.783601+0000","last_became_active":"2024-09-17T15:25:27.786506+0000","last_became_peered":"2024-09-17T15:25:27.786506+0000","last_unstale":"2024-09-17T15:25:28.783601+0000","last_undegraded":"2024-09-17T15:25:28.783601+0000","last_fullsized":"2024-09-17T15:25:28.783601+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:55:57.547976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.786536+0000","last_change":"2024-09-17T15:25:28.785423+0000","last_active":"2024-09-17T15:25:28.786536+0000","last_peered":"2024-09-17T15:25:28.786536+0000","last_clean":"2024-09-17T15:25:28.786536+0000","last_became_active":"2024-09-17T15:25:28.785151+0000","last_became_peered":"2024-09-17T15:25:28.785151+0000","last_unstale":"2024-09-17T15:25:28.786536+0000","last_undegraded":"2024-09-17T15:25:28.786536+0000","last_fullsized":"2024-09-17T15:25:28.786536+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:47:12.307723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"34'2608","reported_seq":2660,"reported_epoch":34,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-17T15:25:59.847727+0000","last_change":"2024-09-17T15:25:27.789556+0000","last_active":"2024-09-17T15:25:59.847727+0000","last_peered":"2024-09-17T15:25:59.847727+0000","last_clean":"2024-09-17T15:24:56.609264+0000","last_became_active":"2024-09-17T15:25:27.789304+0000","last_became_peered":"2024-09-17T15:25:27.789304+0000","last_unstale":"2024-09-17T15:25:59.847727+0000","last_undegraded":"2024-09-17T15:25:27.782129+0000","last_fullsized":"2024-09-17T15:25:59.847727+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2608,"log_dups_size":0,"ondisk_log_size":2608,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10635148,"num_objects":2600,"num_object_clones":0,"num_object_copies":5200,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2600,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2605,"num_write_kb":10391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0","objects":1},{"shards":"0,2","objects":2598}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":173943653,"num_objects":42487,"num_object_clones":0,"num_object_copies":84974,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":35,"num_objects_misplaced":0,"num_objects_unfound":5,"num_objects_dirty":42487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32699,"num_write_kb":130682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":351145984,"data_stored":350994238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42546,"ondisk_log_size":42546,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051596,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":286572,"kb_used_data":174364,"kb_used_omap":4,"kb_used_meta":112187,"kb_avail":94085268,"statfs":{"total":96636764160,"available":96343314432,"internally_reserved":0,"allocated":178548736,"data_stored":178337666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":114880172},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1,3,3,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247824,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":276724,"kb_used_data":164580,"kb_used_omap":4,"kb_used_meta":112123,"kb_avail":94095116,"statfs":{"total":96636764160,"available":96353398784,"internally_reserved":0,"allocated":168529920,"data_stored":168293095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":114814632},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,13,1,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":169820160,"data_stored":169784030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":181211136,"data_stored":181196684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:26:05.286 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 26 20 49600 49580 7.44802 4.96094 0.0106795 0.0104827 2024-09-17T15:26:06.286 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 27 20 50997 50977 7.37427 5.45703 0.0147274 0.0105871 2024-09-17T15:26:07.286 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 28 20 52125 52105 7.26826 4.40625 0.0109454 0.0107441 2024-09-17T15:26:08.203 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:26:08.286 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 29 20 53540 53520 7.20822 5.52734 0.011476 0.010831 2024-09-17T15:26:08.732 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:26:08.732 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:26:08.754 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":79,"stamp":"2024-09-17T15:26:07.589899+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":204827029,"num_objects":50028,"num_object_clones":0,"num_object_copies":100056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50028,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37196,"num_write_kb":148667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":2917,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":50089,"ondisk_log_size":50089,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":629156,"kb_used_data":399236,"kb_used_omap":8,"kb_used_meta":229879,"kb_avail":188114524,"statfs":{"total":193273528320,"available":192629272576,"internally_reserved":0,"allocated":408817664,"data_stored":408344014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8876,"internal_metadata":235396436},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,6,9,8],"upper_bound":32},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":73880750,"num_objects":18032,"num_object_clones":0,"num_object_copies":36064,"num_objects_missing_on_primary":-24,"num_objects_missing":-24,"num_objects_degraded":-48,"num_objects_misplaced":0,"num_objects_unfound":-19,"num_objects_dirty":18032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6814,"num_write_kb":27307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":2917,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002620"},"pg_stats":[{"pgid":"2.f","version":"34'2981","reported_seq":3031,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:26:04.874549+0000","last_change":"2024-09-17T15:25:27.786800+0000","last_active":"2024-09-17T15:26:04.874549+0000","last_peered":"2024-09-17T15:26:04.874549+0000","last_clean":"2024-09-17T15:26:04.874549+0000","last_became_active":"2024-09-17T15:25:27.786453+0000","last_became_peered":"2024-09-17T15:25:27.786453+0000","last_unstale":"2024-09-17T15:26:04.874549+0000","last_undegraded":"2024-09-17T15:26:04.874549+0000","last_fullsized":"2024-09-17T15:26:04.874549+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":2981,"log_dups_size":0,"ondisk_log_size":2981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:46:04.427346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12210176,"num_objects":2981,"num_object_clones":0,"num_object_copies":5962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2981,"num_write_kb":11924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"34'3183","reported_seq":3264,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:26:06.647075+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:26:06.647075+0000","last_peered":"2024-09-17T15:26:06.647075+0000","last_clean":"2024-09-17T15:26:06.647075+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:26:06.647075+0000","last_undegraded":"2024-09-17T15:26:06.647075+0000","last_fullsized":"2024-09-17T15:26:06.647075+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":3183,"log_dups_size":0,"ondisk_log_size":3183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13013958,"num_objects":3179,"num_object_clones":0,"num_object_copies":6358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3179,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1540,"num_write_kb":6160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"34'3264","reported_seq":3362,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:26:06.625549+0000","last_change":"2024-09-17T15:25:51.885496+0000","last_active":"2024-09-17T15:26:06.625549+0000","last_peered":"2024-09-17T15:26:06.625549+0000","last_clean":"2024-09-17T15:26:06.625549+0000","last_became_active":"2024-09-17T15:25:28.785518+0000","last_became_peered":"2024-09-17T15:25:28.785518+0000","last_unstale":"2024-09-17T15:26:06.625549+0000","last_undegraded":"2024-09-17T15:26:06.625549+0000","last_fullsized":"2024-09-17T15:26:06.625549+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":3264,"log_dups_size":0,"ondisk_log_size":3264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:59:02.229332+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13352960,"num_objects":3260,"num_object_clones":0,"num_object_copies":6520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1421,"num_write_kb":5684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"34'3160","reported_seq":3210,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:26:06.692376+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:26:06.692376+0000","last_peered":"2024-09-17T15:26:06.692376+0000","last_clean":"2024-09-17T15:26:06.692376+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:26:06.692376+0000","last_undegraded":"2024-09-17T15:26:06.692376+0000","last_fullsized":"2024-09-17T15:26:06.692376+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":3160,"log_dups_size":0,"ondisk_log_size":3160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12943360,"num_objects":3160,"num_object_clones":0,"num_object_copies":6320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3160,"num_write_kb":12640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"34'2998","reported_seq":3079,"reported_epoch":34,"state":"active+clean+scrubbing","last_fresh":"2024-09-17T15:26:04.892463+0000","last_change":"2024-09-17T15:26:04.647957+0000","last_active":"2024-09-17T15:26:04.892463+0000","last_peered":"2024-09-17T15:26:04.892463+0000","last_clean":"2024-09-17T15:26:04.892463+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:26:04.892463+0000","last_undegraded":"2024-09-17T15:26:04.892463+0000","last_fullsized":"2024-09-17T15:26:04.892463+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":724,"log_size":2998,"log_dups_size":0,"ondisk_log_size":2998,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12252102,"num_objects":2993,"num_object_clones":0,"num_object_copies":5986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2993,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2996,"num_write_kb":11968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"34'3218","reported_seq":3284,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:26:04.862361+0000","last_change":"2024-09-17T15:26:02.942159+0000","last_active":"2024-09-17T15:26:04.862361+0000","last_peered":"2024-09-17T15:26:04.862361+0000","last_clean":"2024-09-17T15:26:04.862361+0000","last_became_active":"2024-09-17T15:25:27.789142+0000","last_became_peered":"2024-09-17T15:25:27.789142+0000","last_unstale":"2024-09-17T15:26:04.862361+0000","last_undegraded":"2024-09-17T15:26:04.862361+0000","last_fullsized":"2024-09-17T15:26:04.862361+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":3218,"log_dups_size":0,"ondisk_log_size":3218,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13161414,"num_objects":3215,"num_object_clones":0,"num_object_copies":6430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3215,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3216,"num_write_kb":12855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"34'3133","reported_seq":3198,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:26:04.868411+0000","last_change":"2024-09-17T15:26:02.827274+0000","last_active":"2024-09-17T15:26:04.868411+0000","last_peered":"2024-09-17T15:26:04.868411+0000","last_clean":"2024-09-17T15:26:04.868411+0000","last_became_active":"2024-09-17T15:25:27.784135+0000","last_became_peered":"2024-09-17T15:25:27.784135+0000","last_unstale":"2024-09-17T15:26:04.868411+0000","last_undegraded":"2024-09-17T15:26:04.868411+0000","last_fullsized":"2024-09-17T15:26:04.868411+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":3133,"log_dups_size":0,"ondisk_log_size":3133,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12809158,"num_objects":3129,"num_object_clones":0,"num_object_copies":6258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3131,"num_write_kb":12512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"34'3113","reported_seq":3221,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:26:04.872571+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:26:04.872571+0000","last_peered":"2024-09-17T15:26:04.872571+0000","last_clean":"2024-09-17T15:26:04.872571+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:26:04.872571+0000","last_undegraded":"2024-09-17T15:26:04.872571+0000","last_fullsized":"2024-09-17T15:26:04.872571+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":3113,"log_dups_size":0,"ondisk_log_size":3113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12742656,"num_objects":3111,"num_object_clones":0,"num_object_copies":6222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":67,"num_write_kb":268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.782961+0000","last_change":"2024-09-17T15:25:27.786793+0000","last_active":"2024-09-17T15:25:28.782961+0000","last_peered":"2024-09-17T15:25:28.782961+0000","last_clean":"2024-09-17T15:25:28.782961+0000","last_became_active":"2024-09-17T15:25:27.786505+0000","last_became_peered":"2024-09-17T15:25:27.786505+0000","last_unstale":"2024-09-17T15:25:28.782961+0000","last_undegraded":"2024-09-17T15:25:28.782961+0000","last_fullsized":"2024-09-17T15:25:28.782961+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:22:15.104318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"34'3193","reported_seq":3293,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:26:06.678469+0000","last_change":"2024-09-17T15:25:53.959204+0000","last_active":"2024-09-17T15:26:06.678469+0000","last_peered":"2024-09-17T15:26:06.678469+0000","last_clean":"2024-09-17T15:26:06.678469+0000","last_became_active":"2024-09-17T15:25:28.785807+0000","last_became_peered":"2024-09-17T15:25:28.785807+0000","last_unstale":"2024-09-17T15:26:06.678469+0000","last_undegraded":"2024-09-17T15:26:06.678469+0000","last_fullsized":"2024-09-17T15:26:06.678469+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":3193,"log_dups_size":0,"ondisk_log_size":3193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:14:31.995973+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13050339,"num_objects":3187,"num_object_clones":0,"num_object_copies":6374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3187,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1193,"num_write_kb":4772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.787649+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:25:28.787649+0000","last_peered":"2024-09-17T15:25:28.787649+0000","last_clean":"2024-09-17T15:25:28.787649+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:25:28.787649+0000","last_undegraded":"2024-09-17T15:25:28.787649+0000","last_fullsized":"2024-09-17T15:25:28.787649+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"34'3020","reported_seq":3080,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:26:04.886881+0000","last_change":"2024-09-17T15:26:03.035368+0000","last_active":"2024-09-17T15:26:04.886881+0000","last_peered":"2024-09-17T15:26:04.886881+0000","last_clean":"2024-09-17T15:26:04.886881+0000","last_became_active":"2024-09-17T15:25:27.789253+0000","last_became_peered":"2024-09-17T15:25:27.789253+0000","last_unstale":"2024-09-17T15:26:04.886881+0000","last_undegraded":"2024-09-17T15:26:04.886881+0000","last_fullsized":"2024-09-17T15:26:04.886881+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":3020,"log_dups_size":0,"ondisk_log_size":3020,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12338118,"num_objects":3014,"num_object_clones":0,"num_object_copies":6028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3018,"num_write_kb":12052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":73,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:26:03.220095+0000","last_change":"2024-09-17T15:26:03.219996+0000","last_active":"2024-09-17T15:26:03.220095+0000","last_peered":"2024-09-17T15:26:03.220095+0000","last_clean":"2024-09-17T15:26:03.220095+0000","last_became_active":"2024-09-17T15:25:27.786489+0000","last_became_peered":"2024-09-17T15:25:27.786489+0000","last_unstale":"2024-09-17T15:26:03.220095+0000","last_undegraded":"2024-09-17T15:26:03.220095+0000","last_fullsized":"2024-09-17T15:26:03.220095+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:45:30.573425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"34'3202","reported_seq":3294,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:26:06.686876+0000","last_change":"2024-09-17T15:25:55.123333+0000","last_active":"2024-09-17T15:26:06.686876+0000","last_peered":"2024-09-17T15:26:06.686876+0000","last_clean":"2024-09-17T15:26:06.686876+0000","last_became_active":"2024-09-17T15:25:28.787663+0000","last_became_peered":"2024-09-17T15:25:28.787663+0000","last_unstale":"2024-09-17T15:26:06.686876+0000","last_undegraded":"2024-09-17T15:26:06.686876+0000","last_fullsized":"2024-09-17T15:26:06.686876+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":3202,"log_dups_size":0,"ondisk_log_size":3202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:10:55.294151+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13095395,"num_objects":3198,"num_object_clones":0,"num_object_copies":6396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3198,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1097,"num_write_kb":4388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.787312+0000","last_change":"2024-09-17T15:25:28.786053+0000","last_active":"2024-09-17T15:25:28.787312+0000","last_peered":"2024-09-17T15:25:28.787312+0000","last_clean":"2024-09-17T15:25:28.787312+0000","last_became_active":"2024-09-17T15:25:28.785773+0000","last_became_peered":"2024-09-17T15:25:28.785773+0000","last_unstale":"2024-09-17T15:25:28.787312+0000","last_undegraded":"2024-09-17T15:25:28.787312+0000","last_fullsized":"2024-09-17T15:25:28.787312+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:46:53.130296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"34'3236","reported_seq":3328,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:26:06.684932+0000","last_change":"2024-09-17T15:25:56.130418+0000","last_active":"2024-09-17T15:26:06.684932+0000","last_peered":"2024-09-17T15:26:06.684932+0000","last_clean":"2024-09-17T15:26:06.684932+0000","last_became_active":"2024-09-17T15:25:28.785157+0000","last_became_peered":"2024-09-17T15:25:28.785157+0000","last_unstale":"2024-09-17T15:26:06.684932+0000","last_undegraded":"2024-09-17T15:26:06.684932+0000","last_fullsized":"2024-09-17T15:26:06.684932+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":3236,"log_dups_size":0,"ondisk_log_size":3236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:03:03.960144+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13230080,"num_objects":3230,"num_object_clones":0,"num_object_copies":6460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":998,"num_write_kb":3992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"34'3121","reported_seq":3183,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:26:04.875328+0000","last_change":"2024-09-17T15:26:02.973019+0000","last_active":"2024-09-17T15:26:04.875328+0000","last_peered":"2024-09-17T15:26:04.875328+0000","last_clean":"2024-09-17T15:26:04.875328+0000","last_became_active":"2024-09-17T15:25:27.789191+0000","last_became_peered":"2024-09-17T15:25:27.789191+0000","last_unstale":"2024-09-17T15:26:04.875328+0000","last_undegraded":"2024-09-17T15:26:04.875328+0000","last_fullsized":"2024-09-17T15:26:04.875328+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":3121,"log_dups_size":0,"ondisk_log_size":3121,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12775907,"num_objects":3120,"num_object_clones":0,"num_object_copies":6240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3120,"num_write_kb":12477,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.784109+0000","last_change":"2024-09-17T15:25:27.789397+0000","last_active":"2024-09-17T15:25:28.784109+0000","last_peered":"2024-09-17T15:25:28.784109+0000","last_clean":"2024-09-17T15:25:28.784109+0000","last_became_active":"2024-09-17T15:25:27.789137+0000","last_became_peered":"2024-09-17T15:25:27.789137+0000","last_unstale":"2024-09-17T15:25:28.784109+0000","last_undegraded":"2024-09-17T15:25:28.784109+0000","last_fullsized":"2024-09-17T15:25:28.784109+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:26:53.538418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"34'3136","reported_seq":3205,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:26:04.862212+0000","last_change":"2024-09-17T15:26:03.201802+0000","last_active":"2024-09-17T15:26:04.862212+0000","last_peered":"2024-09-17T15:26:04.862212+0000","last_clean":"2024-09-17T15:26:04.862212+0000","last_became_active":"2024-09-17T15:25:27.789325+0000","last_became_peered":"2024-09-17T15:25:27.789325+0000","last_unstale":"2024-09-17T15:26:04.862212+0000","last_undegraded":"2024-09-17T15:26:04.862212+0000","last_fullsized":"2024-09-17T15:26:04.862212+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":3136,"log_dups_size":0,"ondisk_log_size":3136,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12813737,"num_objects":3131,"num_object_clones":0,"num_object_copies":6262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3133,"num_write_kb":12516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.785793+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:25:28.785793+0000","last_peered":"2024-09-17T15:25:28.785793+0000","last_clean":"2024-09-17T15:25:28.785793+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:25:28.785793+0000","last_undegraded":"2024-09-17T15:25:28.785793+0000","last_fullsized":"2024-09-17T15:25:28.785793+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"34'3056","reported_seq":3120,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:26:04.881404+0000","last_change":"2024-09-17T15:26:02.874976+0000","last_active":"2024-09-17T15:26:04.881404+0000","last_peered":"2024-09-17T15:26:04.881404+0000","last_clean":"2024-09-17T15:26:04.881404+0000","last_became_active":"2024-09-17T15:25:27.784195+0000","last_became_peered":"2024-09-17T15:25:27.784195+0000","last_unstale":"2024-09-17T15:26:04.881404+0000","last_undegraded":"2024-09-17T15:26:04.881404+0000","last_fullsized":"2024-09-17T15:26:04.881404+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":3056,"log_dups_size":0,"ondisk_log_size":3056,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12501958,"num_objects":3054,"num_object_clones":0,"num_object_copies":6108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3054,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3054,"num_write_kb":12210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.783601+0000","last_change":"2024-09-17T15:25:27.786796+0000","last_active":"2024-09-17T15:25:28.783601+0000","last_peered":"2024-09-17T15:25:28.783601+0000","last_clean":"2024-09-17T15:25:28.783601+0000","last_became_active":"2024-09-17T15:25:27.786506+0000","last_became_peered":"2024-09-17T15:25:27.786506+0000","last_unstale":"2024-09-17T15:25:28.783601+0000","last_undegraded":"2024-09-17T15:25:28.783601+0000","last_fullsized":"2024-09-17T15:25:28.783601+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:55:57.547976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:25:28.786536+0000","last_change":"2024-09-17T15:25:28.785423+0000","last_active":"2024-09-17T15:25:28.786536+0000","last_peered":"2024-09-17T15:25:28.786536+0000","last_clean":"2024-09-17T15:25:28.786536+0000","last_became_active":"2024-09-17T15:25:28.785151+0000","last_became_peered":"2024-09-17T15:25:28.785151+0000","last_unstale":"2024-09-17T15:25:28.786536+0000","last_undegraded":"2024-09-17T15:25:28.786536+0000","last_fullsized":"2024-09-17T15:25:28.786536+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:47:12.307723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"34'3072","reported_seq":3142,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T15:26:04.840474+0000","last_change":"2024-09-17T15:26:03.112208+0000","last_active":"2024-09-17T15:26:04.840474+0000","last_peered":"2024-09-17T15:26:04.840474+0000","last_clean":"2024-09-17T15:26:04.840474+0000","last_became_active":"2024-09-17T15:25:27.789304+0000","last_became_peered":"2024-09-17T15:25:27.789304+0000","last_unstale":"2024-09-17T15:26:04.840474+0000","last_undegraded":"2024-09-17T15:26:04.840474+0000","last_fullsized":"2024-09-17T15:26:04.840474+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":3072,"log_dups_size":0,"ondisk_log_size":3072,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12535692,"num_objects":3064,"num_object_clones":0,"num_object_copies":6128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3069,"num_write_kb":12247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":204827010,"num_objects":50026,"num_object_clones":0,"num_object_copies":100052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37194,"num_write_kb":148665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":2898,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":412528640,"data_stored":412355216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":50086,"ondisk_log_size":50086,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051597,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":325716,"kb_used_data":203780,"kb_used_omap":4,"kb_used_meta":121915,"kb_avail":94046124,"statfs":{"total":96636764160,"available":96303230976,"internally_reserved":0,"allocated":208670720,"data_stored":208433895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":124841644},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,7,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247825,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303440,"kb_used_data":195456,"kb_used_omap":4,"kb_used_meta":107963,"kb_avail":94068400,"statfs":{"total":96636764160,"available":96326041600,"internally_reserved":0,"allocated":200146944,"data_stored":199910119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":110554792},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,4,2,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":201084928,"data_stored":201048798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":211329024,"data_stored":211292894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:26:08.755 INFO:tasks.lost_unfound.ceph_manager:recovered! 2024-09-17T15:26:08.755 INFO:tasks.util.rados:rados -p unfound_pool get new_1 - 2024-09-17T15:26:08.755 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get new_1 - 2024-09-17T15:26:08.845 INFO:teuthology.orchestra.run.smithi083.stderr:error getting unfound_pool/new_1: (2) No such file or directory 2024-09-17T15:26:08.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T15:26:08.856 INFO:tasks.util.rados:rados -p unfound_pool get existed_1 - 2024-09-17T15:26:08.857 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existed_1 - 2024-09-17T15:26:08.948 INFO:teuthology.orchestra.run.smithi083.stderr:error getting unfound_pool/existed_1: (2) No such file or directory 2024-09-17T15:26:08.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T15:26:08.956 INFO:tasks.util.rados:rados -p unfound_pool get existing_1 - 2024-09-17T15:26:08.957 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existing_1 - 2024-09-17T15:26:09.064 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:26:09.064 INFO:teuthology.orchestra.run.smithi083.stdout:# 2024-09-17T15:26:09.064 INFO:teuthology.orchestra.run.smithi083.stdout:# /etc/fstab 2024-09-17T15:26:09.064 INFO:teuthology.orchestra.run.smithi083.stdout:# Created by anaconda on Wed Aug 28 20:55:26 2024 2024-09-17T15:26:09.064 INFO:teuthology.orchestra.run.smithi083.stdout:# 2024-09-17T15:26:09.064 INFO:teuthology.orchestra.run.smithi083.stdout:# Accessible filesystems, by reference, are maintained under '/dev/disk/'. 2024-09-17T15:26:09.064 INFO:teuthology.orchestra.run.smithi083.stdout:# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info. 2024-09-17T15:26:09.064 INFO:teuthology.orchestra.run.smithi083.stdout:# 2024-09-17T15:26:09.065 INFO:teuthology.orchestra.run.smithi083.stdout:# After editing this file, run 'systemctl daemon-reload' to update systemd 2024-09-17T15:26:09.065 INFO:teuthology.orchestra.run.smithi083.stdout:# units generated from this file. 2024-09-17T15:26:09.065 INFO:teuthology.orchestra.run.smithi083.stdout:# 2024-09-17T15:26:09.065 INFO:teuthology.orchestra.run.smithi083.stdout:UUID=fa8d2923-554d-41e2-ac4a-c752ff079ceb / ext4 defaults 1 1 2024-09-17T15:26:09.065 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/vg_nvme/lv_5 /var/lib/ceph xfs defaults,nofail 0 0 2024-09-17T15:26:09.078 INFO:tasks.util.rados:rados -p unfound_pool get new_2 - 2024-09-17T15:26:09.078 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get new_2 - 2024-09-17T15:26:09.257 INFO:teuthology.orchestra.run.smithi083.stderr:error getting unfound_pool/new_2: (2) No such file or directory 2024-09-17T15:26:09.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T15:26:09.265 INFO:tasks.util.rados:rados -p unfound_pool get existed_2 - 2024-09-17T15:26:09.265 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existed_2 - 2024-09-17T15:26:09.286 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 30 20 54599 54579 7.10583 4.13672 0.0130026 0.0109883 2024-09-17T15:26:09.364 INFO:teuthology.orchestra.run.smithi083.stderr:error getting unfound_pool/existed_2: (2) No such file or directory 2024-09-17T15:26:09.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T15:26:09.370 INFO:tasks.util.rados:rados -p unfound_pool get existing_2 - 2024-09-17T15:26:09.371 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existing_2 - 2024-09-17T15:26:09.521 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:26:09.521 INFO:teuthology.orchestra.run.smithi083.stdout:# 2024-09-17T15:26:09.521 INFO:teuthology.orchestra.run.smithi083.stdout:# /etc/fstab 2024-09-17T15:26:09.521 INFO:teuthology.orchestra.run.smithi083.stdout:# Created by anaconda on Wed Aug 28 20:55:26 2024 2024-09-17T15:26:09.521 INFO:teuthology.orchestra.run.smithi083.stdout:# 2024-09-17T15:26:09.521 INFO:teuthology.orchestra.run.smithi083.stdout:# Accessible filesystems, by reference, are maintained under '/dev/disk/'. 2024-09-17T15:26:09.521 INFO:teuthology.orchestra.run.smithi083.stdout:# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info. 2024-09-17T15:26:09.521 INFO:teuthology.orchestra.run.smithi083.stdout:# 2024-09-17T15:26:09.521 INFO:teuthology.orchestra.run.smithi083.stdout:# After editing this file, run 'systemctl daemon-reload' to update systemd 2024-09-17T15:26:09.521 INFO:teuthology.orchestra.run.smithi083.stdout:# units generated from this file. 2024-09-17T15:26:09.522 INFO:teuthology.orchestra.run.smithi083.stdout:# 2024-09-17T15:26:09.522 INFO:teuthology.orchestra.run.smithi083.stdout:UUID=fa8d2923-554d-41e2-ac4a-c752ff079ceb / ext4 defaults 1 1 2024-09-17T15:26:09.522 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/vg_nvme/lv_5 /var/lib/ceph xfs defaults,nofail 0 0 2024-09-17T15:26:09.528 INFO:tasks.util.rados:rados -p unfound_pool get new_3 - 2024-09-17T15:26:09.529 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get new_3 - 2024-09-17T15:26:09.624 INFO:teuthology.orchestra.run.smithi083.stderr:error getting unfound_pool/new_3: (2) No such file or directory 2024-09-17T15:26:09.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T15:26:09.639 INFO:tasks.util.rados:rados -p unfound_pool get existed_3 - 2024-09-17T15:26:09.639 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existed_3 - 2024-09-17T15:26:09.747 INFO:teuthology.orchestra.run.smithi083.stderr:error getting unfound_pool/existed_3: (2) No such file or directory 2024-09-17T15:26:09.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T15:26:09.753 INFO:tasks.util.rados:rados -p unfound_pool get existing_3 - 2024-09-17T15:26:09.753 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existing_3 - 2024-09-17T15:26:09.839 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:26:09.839 INFO:teuthology.orchestra.run.smithi083.stdout:# 2024-09-17T15:26:09.839 INFO:teuthology.orchestra.run.smithi083.stdout:# /etc/fstab 2024-09-17T15:26:09.840 INFO:teuthology.orchestra.run.smithi083.stdout:# Created by anaconda on Wed Aug 28 20:55:26 2024 2024-09-17T15:26:09.840 INFO:teuthology.orchestra.run.smithi083.stdout:# 2024-09-17T15:26:09.840 INFO:teuthology.orchestra.run.smithi083.stdout:# Accessible filesystems, by reference, are maintained under '/dev/disk/'. 2024-09-17T15:26:09.840 INFO:teuthology.orchestra.run.smithi083.stdout:# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info. 2024-09-17T15:26:09.840 INFO:teuthology.orchestra.run.smithi083.stdout:# 2024-09-17T15:26:09.840 INFO:teuthology.orchestra.run.smithi083.stdout:# After editing this file, run 'systemctl daemon-reload' to update systemd 2024-09-17T15:26:09.840 INFO:teuthology.orchestra.run.smithi083.stdout:# units generated from this file. 2024-09-17T15:26:09.840 INFO:teuthology.orchestra.run.smithi083.stdout:# 2024-09-17T15:26:09.840 INFO:teuthology.orchestra.run.smithi083.stdout:UUID=fa8d2923-554d-41e2-ac4a-c752ff079ceb / ext4 defaults 1 1 2024-09-17T15:26:09.840 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/vg_nvme/lv_5 /var/lib/ceph xfs defaults,nofail 0 0 2024-09-17T15:26:09.852 INFO:tasks.util.rados:rados -p unfound_pool get new_4 - 2024-09-17T15:26:09.852 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get new_4 - 2024-09-17T15:26:09.941 INFO:teuthology.orchestra.run.smithi083.stderr:error getting unfound_pool/new_4: (2) No such file or directory 2024-09-17T15:26:09.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T15:26:09.950 INFO:tasks.util.rados:rados -p unfound_pool get existed_4 - 2024-09-17T15:26:09.951 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existed_4 - 2024-09-17T15:26:10.040 INFO:teuthology.orchestra.run.smithi083.stderr:error getting unfound_pool/existed_4: (2) No such file or directory 2024-09-17T15:26:10.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T15:26:10.048 INFO:tasks.util.rados:rados -p unfound_pool get existing_4 - 2024-09-17T15:26:10.048 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existing_4 - 2024-09-17T15:26:10.145 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:26:10.145 INFO:teuthology.orchestra.run.smithi083.stdout:# 2024-09-17T15:26:10.146 INFO:teuthology.orchestra.run.smithi083.stdout:# /etc/fstab 2024-09-17T15:26:10.146 INFO:teuthology.orchestra.run.smithi083.stdout:# Created by anaconda on Wed Aug 28 20:55:26 2024 2024-09-17T15:26:10.146 INFO:teuthology.orchestra.run.smithi083.stdout:# 2024-09-17T15:26:10.146 INFO:teuthology.orchestra.run.smithi083.stdout:# Accessible filesystems, by reference, are maintained under '/dev/disk/'. 2024-09-17T15:26:10.146 INFO:teuthology.orchestra.run.smithi083.stdout:# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info. 2024-09-17T15:26:10.146 INFO:teuthology.orchestra.run.smithi083.stdout:# 2024-09-17T15:26:10.146 INFO:teuthology.orchestra.run.smithi083.stdout:# After editing this file, run 'systemctl daemon-reload' to update systemd 2024-09-17T15:26:10.146 INFO:teuthology.orchestra.run.smithi083.stdout:# units generated from this file. 2024-09-17T15:26:10.146 INFO:teuthology.orchestra.run.smithi083.stdout:# 2024-09-17T15:26:10.146 INFO:teuthology.orchestra.run.smithi083.stdout:UUID=fa8d2923-554d-41e2-ac4a-c752ff079ceb / ext4 defaults 1 1 2024-09-17T15:26:10.146 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/vg_nvme/lv_5 /var/lib/ceph xfs defaults,nofail 0 0 2024-09-17T15:26:10.154 INFO:tasks.util.rados:rados -p unfound_pool get new_5 - 2024-09-17T15:26:10.155 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get new_5 - 2024-09-17T15:26:10.244 INFO:teuthology.orchestra.run.smithi083.stderr:error getting unfound_pool/new_5: (2) No such file or directory 2024-09-17T15:26:10.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T15:26:10.256 INFO:tasks.util.rados:rados -p unfound_pool get existed_5 - 2024-09-17T15:26:10.256 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existed_5 - 2024-09-17T15:26:10.287 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 31 20 55762 55742 7.02296 4.54297 0.0117579 0.011117 2024-09-17T15:26:10.345 INFO:teuthology.orchestra.run.smithi083.stderr:error getting unfound_pool/existed_5: (2) No such file or directory 2024-09-17T15:26:10.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T15:26:10.359 INFO:tasks.util.rados:rados -p unfound_pool get existing_5 - 2024-09-17T15:26:10.359 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existing_5 - 2024-09-17T15:26:10.495 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:26:10.495 INFO:teuthology.orchestra.run.smithi083.stdout:# 2024-09-17T15:26:10.496 INFO:teuthology.orchestra.run.smithi083.stdout:# /etc/fstab 2024-09-17T15:26:10.496 INFO:teuthology.orchestra.run.smithi083.stdout:# Created by anaconda on Wed Aug 28 20:55:26 2024 2024-09-17T15:26:10.496 INFO:teuthology.orchestra.run.smithi083.stdout:# 2024-09-17T15:26:10.496 INFO:teuthology.orchestra.run.smithi083.stdout:# Accessible filesystems, by reference, are maintained under '/dev/disk/'. 2024-09-17T15:26:10.496 INFO:teuthology.orchestra.run.smithi083.stdout:# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info. 2024-09-17T15:26:10.496 INFO:teuthology.orchestra.run.smithi083.stdout:# 2024-09-17T15:26:10.496 INFO:teuthology.orchestra.run.smithi083.stdout:# After editing this file, run 'systemctl daemon-reload' to update systemd 2024-09-17T15:26:10.496 INFO:teuthology.orchestra.run.smithi083.stdout:# units generated from this file. 2024-09-17T15:26:10.496 INFO:teuthology.orchestra.run.smithi083.stdout:# 2024-09-17T15:26:10.496 INFO:teuthology.orchestra.run.smithi083.stdout:UUID=fa8d2923-554d-41e2-ac4a-c752ff079ceb / ext4 defaults 1 1 2024-09-17T15:26:10.496 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/vg_nvme/lv_5 /var/lib/ceph xfs defaults,nofail 0 0 2024-09-17T15:26:10.506 INFO:tasks.util.rados:rados -p unfound_pool get new_6 - 2024-09-17T15:26:10.506 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get new_6 - 2024-09-17T15:26:10.604 INFO:teuthology.orchestra.run.smithi083.stderr:error getting unfound_pool/new_6: (2) No such file or directory 2024-09-17T15:26:10.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T15:26:10.610 INFO:tasks.util.rados:rados -p unfound_pool get existed_6 - 2024-09-17T15:26:10.610 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existed_6 - 2024-09-17T15:26:10.718 INFO:teuthology.orchestra.run.smithi083.stderr:error getting unfound_pool/existed_6: (2) No such file or directory 2024-09-17T15:26:10.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T15:26:10.732 INFO:tasks.util.rados:rados -p unfound_pool get existing_6 - 2024-09-17T15:26:10.732 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existing_6 - 2024-09-17T15:26:10.821 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:26:10.821 INFO:teuthology.orchestra.run.smithi083.stdout:# 2024-09-17T15:26:10.821 INFO:teuthology.orchestra.run.smithi083.stdout:# /etc/fstab 2024-09-17T15:26:10.821 INFO:teuthology.orchestra.run.smithi083.stdout:# Created by anaconda on Wed Aug 28 20:55:26 2024 2024-09-17T15:26:10.821 INFO:teuthology.orchestra.run.smithi083.stdout:# 2024-09-17T15:26:10.821 INFO:teuthology.orchestra.run.smithi083.stdout:# Accessible filesystems, by reference, are maintained under '/dev/disk/'. 2024-09-17T15:26:10.822 INFO:teuthology.orchestra.run.smithi083.stdout:# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info. 2024-09-17T15:26:10.822 INFO:teuthology.orchestra.run.smithi083.stdout:# 2024-09-17T15:26:10.822 INFO:teuthology.orchestra.run.smithi083.stdout:# After editing this file, run 'systemctl daemon-reload' to update systemd 2024-09-17T15:26:10.822 INFO:teuthology.orchestra.run.smithi083.stdout:# units generated from this file. 2024-09-17T15:26:10.822 INFO:teuthology.orchestra.run.smithi083.stdout:# 2024-09-17T15:26:10.822 INFO:teuthology.orchestra.run.smithi083.stdout:UUID=fa8d2923-554d-41e2-ac4a-c752ff079ceb / ext4 defaults 1 1 2024-09-17T15:26:10.822 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/vg_nvme/lv_5 /var/lib/ceph xfs defaults,nofail 0 0 2024-09-17T15:26:10.833 INFO:tasks.util.rados:rados -p unfound_pool get new_7 - 2024-09-17T15:26:10.833 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get new_7 - 2024-09-17T15:26:10.931 INFO:teuthology.orchestra.run.smithi083.stderr:error getting unfound_pool/new_7: (2) No such file or directory 2024-09-17T15:26:10.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T15:26:10.939 INFO:tasks.util.rados:rados -p unfound_pool get existed_7 - 2024-09-17T15:26:10.939 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existed_7 - 2024-09-17T15:26:11.036 INFO:teuthology.orchestra.run.smithi083.stderr:error getting unfound_pool/existed_7: (2) No such file or directory 2024-09-17T15:26:11.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T15:26:11.042 INFO:tasks.util.rados:rados -p unfound_pool get existing_7 - 2024-09-17T15:26:11.042 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existing_7 - 2024-09-17T15:26:11.137 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:26:11.137 INFO:teuthology.orchestra.run.smithi083.stdout:# 2024-09-17T15:26:11.137 INFO:teuthology.orchestra.run.smithi083.stdout:# /etc/fstab 2024-09-17T15:26:11.137 INFO:teuthology.orchestra.run.smithi083.stdout:# Created by anaconda on Wed Aug 28 20:55:26 2024 2024-09-17T15:26:11.137 INFO:teuthology.orchestra.run.smithi083.stdout:# 2024-09-17T15:26:11.137 INFO:teuthology.orchestra.run.smithi083.stdout:# Accessible filesystems, by reference, are maintained under '/dev/disk/'. 2024-09-17T15:26:11.137 INFO:teuthology.orchestra.run.smithi083.stdout:# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info. 2024-09-17T15:26:11.137 INFO:teuthology.orchestra.run.smithi083.stdout:# 2024-09-17T15:26:11.137 INFO:teuthology.orchestra.run.smithi083.stdout:# After editing this file, run 'systemctl daemon-reload' to update systemd 2024-09-17T15:26:11.138 INFO:teuthology.orchestra.run.smithi083.stdout:# units generated from this file. 2024-09-17T15:26:11.138 INFO:teuthology.orchestra.run.smithi083.stdout:# 2024-09-17T15:26:11.138 INFO:teuthology.orchestra.run.smithi083.stdout:UUID=fa8d2923-554d-41e2-ac4a-c752ff079ceb / ext4 defaults 1 1 2024-09-17T15:26:11.138 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/vg_nvme/lv_5 /var/lib/ceph xfs defaults,nofail 0 0 2024-09-17T15:26:11.142 INFO:tasks.util.rados:rados -p unfound_pool get new_8 - 2024-09-17T15:26:11.142 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get new_8 - 2024-09-17T15:26:11.246 INFO:teuthology.orchestra.run.smithi083.stderr:error getting unfound_pool/new_8: (2) No such file or directory 2024-09-17T15:26:11.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T15:26:11.255 INFO:tasks.util.rados:rados -p unfound_pool get existed_8 - 2024-09-17T15:26:11.255 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existed_8 - 2024-09-17T15:26:11.287 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 32 20 56900 56880 6.9424 4.44531 0.0119354 0.0112462 2024-09-17T15:26:11.361 INFO:teuthology.orchestra.run.smithi083.stderr:error getting unfound_pool/existed_8: (2) No such file or directory 2024-09-17T15:26:11.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T15:26:11.373 INFO:tasks.util.rados:rados -p unfound_pool get existing_8 - 2024-09-17T15:26:11.374 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existing_8 - 2024-09-17T15:26:11.518 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:26:11.518 INFO:teuthology.orchestra.run.smithi083.stdout:# 2024-09-17T15:26:11.518 INFO:teuthology.orchestra.run.smithi083.stdout:# /etc/fstab 2024-09-17T15:26:11.518 INFO:teuthology.orchestra.run.smithi083.stdout:# Created by anaconda on Wed Aug 28 20:55:26 2024 2024-09-17T15:26:11.518 INFO:teuthology.orchestra.run.smithi083.stdout:# 2024-09-17T15:26:11.519 INFO:teuthology.orchestra.run.smithi083.stdout:# Accessible filesystems, by reference, are maintained under '/dev/disk/'. 2024-09-17T15:26:11.519 INFO:teuthology.orchestra.run.smithi083.stdout:# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info. 2024-09-17T15:26:11.519 INFO:teuthology.orchestra.run.smithi083.stdout:# 2024-09-17T15:26:11.519 INFO:teuthology.orchestra.run.smithi083.stdout:# After editing this file, run 'systemctl daemon-reload' to update systemd 2024-09-17T15:26:11.519 INFO:teuthology.orchestra.run.smithi083.stdout:# units generated from this file. 2024-09-17T15:26:11.519 INFO:teuthology.orchestra.run.smithi083.stdout:# 2024-09-17T15:26:11.519 INFO:teuthology.orchestra.run.smithi083.stdout:UUID=fa8d2923-554d-41e2-ac4a-c752ff079ceb / ext4 defaults 1 1 2024-09-17T15:26:11.519 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/vg_nvme/lv_5 /var/lib/ceph xfs defaults,nofail 0 0 2024-09-17T15:26:11.533 INFO:tasks.util.rados:rados -p unfound_pool get new_9 - 2024-09-17T15:26:11.533 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get new_9 - 2024-09-17T15:26:11.649 INFO:teuthology.orchestra.run.smithi083.stderr:error getting unfound_pool/new_9: (2) No such file or directory 2024-09-17T15:26:11.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T15:26:11.656 INFO:tasks.util.rados:rados -p unfound_pool get existed_9 - 2024-09-17T15:26:11.656 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existed_9 - 2024-09-17T15:26:11.744 INFO:teuthology.orchestra.run.smithi083.stderr:error getting unfound_pool/existed_9: (2) No such file or directory 2024-09-17T15:26:11.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T15:26:11.753 INFO:tasks.util.rados:rados -p unfound_pool get existing_9 - 2024-09-17T15:26:11.753 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existing_9 - 2024-09-17T15:26:11.844 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:26:11.845 INFO:teuthology.orchestra.run.smithi083.stdout:# 2024-09-17T15:26:11.845 INFO:teuthology.orchestra.run.smithi083.stdout:# /etc/fstab 2024-09-17T15:26:11.845 INFO:teuthology.orchestra.run.smithi083.stdout:# Created by anaconda on Wed Aug 28 20:55:26 2024 2024-09-17T15:26:11.845 INFO:teuthology.orchestra.run.smithi083.stdout:# 2024-09-17T15:26:11.845 INFO:teuthology.orchestra.run.smithi083.stdout:# Accessible filesystems, by reference, are maintained under '/dev/disk/'. 2024-09-17T15:26:11.845 INFO:teuthology.orchestra.run.smithi083.stdout:# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info. 2024-09-17T15:26:11.846 INFO:teuthology.orchestra.run.smithi083.stdout:# 2024-09-17T15:26:11.846 INFO:teuthology.orchestra.run.smithi083.stdout:# After editing this file, run 'systemctl daemon-reload' to update systemd 2024-09-17T15:26:11.846 INFO:teuthology.orchestra.run.smithi083.stdout:# units generated from this file. 2024-09-17T15:26:11.846 INFO:teuthology.orchestra.run.smithi083.stdout:# 2024-09-17T15:26:11.846 INFO:teuthology.orchestra.run.smithi083.stdout:UUID=fa8d2923-554d-41e2-ac4a-c752ff079ceb / ext4 defaults 1 1 2024-09-17T15:26:11.846 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/vg_nvme/lv_5 /var/lib/ceph xfs defaults,nofail 0 0 2024-09-17T15:26:11.851 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 1 2024-09-17T15:26:12.289 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 33 20 57999 57979 6.86176 4.29297 0.0137273 0.0113788 2024-09-17T15:26:12.388 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-09-17T15:26:12.387+0000 7f514a174640 -1 mon.a@0(leader).osd e34 definitely_dead 0 2024-09-17T15:26:13.111 INFO:teuthology.orchestra.run.smithi083.stderr:marked in osd.1. 2024-09-17T15:26:13.139 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-17T15:26:13.139 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-17T15:26:13.142 INFO:tasks.ceph.osd.1:Started 2024-09-17T15:26:13.142 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T15:26:13.240 INFO:teuthology.orchestra.run.smithi083.stderr:no valid command found; 10 closest matches: 2024-09-17T15:26:13.240 INFO:teuthology.orchestra.run.smithi083.stderr:0 2024-09-17T15:26:13.240 INFO:teuthology.orchestra.run.smithi083.stderr:1 2024-09-17T15:26:13.240 INFO:teuthology.orchestra.run.smithi083.stderr:2 2024-09-17T15:26:13.240 INFO:teuthology.orchestra.run.smithi083.stderr:abort 2024-09-17T15:26:13.240 INFO:teuthology.orchestra.run.smithi083.stderr:assert 2024-09-17T15:26:13.240 INFO:teuthology.orchestra.run.smithi083.stderr:config diff 2024-09-17T15:26:13.240 INFO:teuthology.orchestra.run.smithi083.stderr:config diff get 2024-09-17T15:26:13.240 INFO:teuthology.orchestra.run.smithi083.stderr:config get 2024-09-17T15:26:13.241 INFO:teuthology.orchestra.run.smithi083.stderr:config help [] 2024-09-17T15:26:13.241 INFO:teuthology.orchestra.run.smithi083.stderr:config set ... 2024-09-17T15:26:13.241 INFO:teuthology.orchestra.run.smithi083.stderr:admin_socket: invalid command 2024-09-17T15:26:13.242 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T15:26:13.242 INFO:tasks.lost_unfound.ceph_manager:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-17T15:26:13.289 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 34 20 59101 59081 6.78654 4.30469 0.00281833 0.0114504 2024-09-17T15:26:14.290 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 35 20 59101 59081 6.59265 0 - 0.0114504 2024-09-17T15:26:14.320 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T15:26:14.320+0000 7f5aa3eb4640 -1 Falling back to public interface 2024-09-17T15:26:15.290 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 36 20 59287 59267 6.42971 0.363281 0.0155627 0.0121449 2024-09-17T15:26:16.289 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 37 20 60660 60640 6.40088 5.36328 0.0145904 0.0121996 2024-09-17T15:26:17.290 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 38 20 62000 61980 6.37018 5.23438 0.0138375 0.0122575 2024-09-17T15:26:18.243 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T15:26:18.289 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 39 20 63279 63259 6.33494 4.99609 0.0163474 0.0123249 2024-09-17T15:26:18.408 INFO:teuthology.orchestra.run.smithi083.stderr:no valid command found; 10 closest matches: 2024-09-17T15:26:18.408 INFO:teuthology.orchestra.run.smithi083.stderr:0 2024-09-17T15:26:18.408 INFO:teuthology.orchestra.run.smithi083.stderr:1 2024-09-17T15:26:18.408 INFO:teuthology.orchestra.run.smithi083.stderr:2 2024-09-17T15:26:18.408 INFO:teuthology.orchestra.run.smithi083.stderr:abort 2024-09-17T15:26:18.408 INFO:teuthology.orchestra.run.smithi083.stderr:assert 2024-09-17T15:26:18.408 INFO:teuthology.orchestra.run.smithi083.stderr:bluestore allocator dump block 2024-09-17T15:26:18.409 INFO:teuthology.orchestra.run.smithi083.stderr:bluestore allocator fragmentation block 2024-09-17T15:26:18.409 INFO:teuthology.orchestra.run.smithi083.stderr:bluestore allocator fragmentation histogram block [] [] 2024-09-17T15:26:18.409 INFO:teuthology.orchestra.run.smithi083.stderr:bluestore allocator score block 2024-09-17T15:26:18.409 INFO:teuthology.orchestra.run.smithi083.stderr:config diff 2024-09-17T15:26:18.409 INFO:teuthology.orchestra.run.smithi083.stderr:admin_socket: invalid command 2024-09-17T15:26:18.410 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T15:26:18.411 INFO:tasks.lost_unfound.ceph_manager:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-17T15:26:18.697 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T15:26:18.696+0000 7f5aa3eb4640 -1 osd.1 24 log_to_monitors true 2024-09-17T15:26:19.169 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-09-17T15:26:19.168+0000 7f5a9bc6a640 -1 osd.1 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T15:26:19.290 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout:2024-09-17T15:26:19.290459+0000 min lat: 0.00187689 max lat: 2.05778 avg lat: 0.0123942 2024-09-17T15:26:19.290 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-17T15:26:19.290 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 40 20 64547 64527 6.30039 4.95312 0.0196209 0.0123942 2024-09-17T15:26:20.290 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 41 20 65713 65693 6.25779 4.55469 0.0208221 0.0124446 2024-09-17T15:26:21.290 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 42 20 65713 65693 6.10881 0 - 0.0124446 2024-09-17T15:26:22.290 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 43 20 65713 65693 5.96675 0 - 0.0124446 2024-09-17T15:26:23.290 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 44 20 66115 66095 5.86684 0.523438 0.0407852 0.0133094 2024-09-17T15:26:23.411 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T15:26:23.601 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T15:26:23.602 INFO:teuthology.orchestra.run.smithi083.stdout: "ops": [ 2024-09-17T15:26:23.602 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.602 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5322.0:66308 2.0 e41/40 2:05bf2303:::benchmark_data_smithi083_41818_object66307:head v 41'4191, mlcod=21'4)", 2024-09-17T15:26:23.602 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T15:26:23.594732+0000", 2024-09-17T15:26:23.602 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0061099509999999998, 2024-09-17T15:26:23.602 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0061374020000000001, 2024-09-17T15:26:23.602 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T15:26:23.602 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T15:26:23.602 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T15:26:23.602 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T15:26:23.602 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.603 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T15:26:23.603 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.594732+0000", 2024-09-17T15:26:23.603 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T15:26:23.603 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.603 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.603 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T15:26:23.603 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.594732+0000", 2024-09-17T15:26:23.603 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T15:26:23.603 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.603 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.603 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T15:26:23.603 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.594733+0000", 2024-09-17T15:26:23.603 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.7100000000000011e-07 2024-09-17T15:26:23.604 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.604 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.604 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T15:26:23.604 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.594743+0000", 2024-09-17T15:26:23.604 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0383999999999999e-05 2024-09-17T15:26:23.604 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.604 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.604 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T15:26:23.604 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.594760+0000", 2024-09-17T15:26:23.604 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.658e-05 2024-09-17T15:26:23.604 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.605 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.605 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T15:26:23.605 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.594772+0000", 2024-09-17T15:26:23.605 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.252e-05 2024-09-17T15:26:23.605 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.605 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.605 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T15:26:23.605 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.595145+0000", 2024-09-17T15:26:23.605 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00037268099999999998 2024-09-17T15:26:23.605 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.605 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.605 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T15:26:23.605 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.595167+0000", 2024-09-17T15:26:23.606 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2133e-05 2024-09-17T15:26:23.606 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T15:26:23.606 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T15:26:23.606 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T15:26:23.606 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.606 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.606 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5322.0:66309 2.0 e41/40)", 2024-09-17T15:26:23.606 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T15:26:23.594797+0000", 2024-09-17T15:26:23.606 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0060443579999999997, 2024-09-17T15:26:23.606 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0061221460000000002, 2024-09-17T15:26:23.606 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T15:26:23.606 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T15:26:23.606 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "queued for pg", 2024-09-17T15:26:23.606 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T15:26:23.607 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.607 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T15:26:23.607 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.594797+0000", 2024-09-17T15:26:23.607 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T15:26:23.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.607 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.607 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T15:26:23.607 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.594797+0000", 2024-09-17T15:26:23.607 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T15:26:23.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.607 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.607 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T15:26:23.607 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.594798+0000", 2024-09-17T15:26:23.607 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0999999999999999e-07 2024-09-17T15:26:23.608 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.608 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.608 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T15:26:23.608 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.594810+0000", 2024-09-17T15:26:23.608 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.2051e-05 2024-09-17T15:26:23.608 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.608 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.608 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T15:26:23.608 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.594816+0000", 2024-09-17T15:26:23.608 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.8359999999999997e-06 2024-09-17T15:26:23.608 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.608 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.609 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T15:26:23.609 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.594820+0000", 2024-09-17T15:26:23.609 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5159999999999996e-06 2024-09-17T15:26:23.609 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T15:26:23.609 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T15:26:23.609 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T15:26:23.609 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.609 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.609 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5322.0:66306 2.2 e41/40 2:428d9b3a:::benchmark_data_smithi083_41818_object66305:head v 41'4055, mlcod=21'6)", 2024-09-17T15:26:23.609 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T15:26:23.595238+0000", 2024-09-17T15:26:23.609 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0056040939999999996, 2024-09-17T15:26:23.609 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0057203269999999999, 2024-09-17T15:26:23.609 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T15:26:23.610 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T15:26:23.610 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T15:26:23.610 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T15:26:23.610 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.610 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T15:26:23.610 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.595238+0000", 2024-09-17T15:26:23.610 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T15:26:23.610 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.610 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.610 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T15:26:23.610 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.595238+0000", 2024-09-17T15:26:23.610 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T15:26:23.610 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.611 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.611 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T15:26:23.611 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.595238+0000", 2024-09-17T15:26:23.611 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.8599999999999998e-07 2024-09-17T15:26:23.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.611 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.611 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T15:26:23.611 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.595245+0000", 2024-09-17T15:26:23.611 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2339999999999997e-06 2024-09-17T15:26:23.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.611 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.611 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T15:26:23.611 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.595253+0000", 2024-09-17T15:26:23.612 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.2429999999999999e-06 2024-09-17T15:26:23.612 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.612 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.612 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T15:26:23.612 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.595258+0000", 2024-09-17T15:26:23.612 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.1789999999999999e-06 2024-09-17T15:26:23.612 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.612 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.612 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T15:26:23.612 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.599104+0000", 2024-09-17T15:26:23.612 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0038459789999999998 2024-09-17T15:26:23.612 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.612 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.613 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T15:26:23.613 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.599130+0000", 2024-09-17T15:26:23.613 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6383000000000001e-05 2024-09-17T15:26:23.613 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T15:26:23.613 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T15:26:23.613 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T15:26:23.613 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.613 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.613 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5322.0:66307 2.5 e41/40 2:a65fa392:::benchmark_data_smithi083_41818_object66306:head v 41'4113, mlcod=21'2)", 2024-09-17T15:26:23.613 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T15:26:23.595275+0000", 2024-09-17T15:26:23.613 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0055673129999999999, 2024-09-17T15:26:23.614 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0057278870000000001, 2024-09-17T15:26:23.614 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T15:26:23.614 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T15:26:23.614 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T15:26:23.615 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T15:26:23.615 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.615 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T15:26:23.615 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.595275+0000", 2024-09-17T15:26:23.615 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T15:26:23.615 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.615 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.615 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T15:26:23.615 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.595275+0000", 2024-09-17T15:26:23.615 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T15:26:23.615 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.615 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.615 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T15:26:23.615 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.595275+0000", 2024-09-17T15:26:23.616 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.1399999999999998e-07 2024-09-17T15:26:23.616 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.616 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.616 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T15:26:23.616 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.595283+0000", 2024-09-17T15:26:23.616 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 8.4449999999999996e-06 2024-09-17T15:26:23.616 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.616 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.616 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T15:26:23.616 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.595290+0000", 2024-09-17T15:26:23.616 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.2820000000000002e-06 2024-09-17T15:26:23.616 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.616 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.617 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T15:26:23.617 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.595294+0000", 2024-09-17T15:26:23.617 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7040000000000002e-06 2024-09-17T15:26:23.617 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.617 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.617 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T15:26:23.617 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.596712+0000", 2024-09-17T15:26:23.617 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0014178719999999999 2024-09-17T15:26:23.617 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.617 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.617 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T15:26:23.617 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.596733+0000", 2024-09-17T15:26:23.618 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0886e-05 2024-09-17T15:26:23.618 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T15:26:23.618 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T15:26:23.618 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T15:26:23.618 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.618 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.618 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5322.0:66304 2.a e41/40)", 2024-09-17T15:26:23.618 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T15:26:23.595306+0000", 2024-09-17T15:26:23.618 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0055353640000000001, 2024-09-17T15:26:23.618 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0057389920000000001, 2024-09-17T15:26:23.618 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T15:26:23.618 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T15:26:23.618 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "queued for pg", 2024-09-17T15:26:23.619 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T15:26:23.619 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.619 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T15:26:23.619 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.595306+0000", 2024-09-17T15:26:23.619 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T15:26:23.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.619 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.619 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T15:26:23.619 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.595306+0000", 2024-09-17T15:26:23.619 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T15:26:23.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.619 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.619 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T15:26:23.619 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.595307+0000", 2024-09-17T15:26:23.620 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.67e-07 2024-09-17T15:26:23.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.620 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.620 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T15:26:23.620 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.595314+0000", 2024-09-17T15:26:23.620 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 7.7319999999999998e-06 2024-09-17T15:26:23.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.620 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.620 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T15:26:23.620 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.595319+0000", 2024-09-17T15:26:23.620 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.9699999999999998e-06 2024-09-17T15:26:23.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.621 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.621 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T15:26:23.621 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.595324+0000", 2024-09-17T15:26:23.621 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.262e-06 2024-09-17T15:26:23.621 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T15:26:23.621 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T15:26:23.621 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T15:26:23.621 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.621 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.621 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5322.0:66305 2.4 e41/40 2:2ed5b67e:::benchmark_data_smithi083_41818_object66304:head v 41'4155, mlcod=21'9)", 2024-09-17T15:26:23.621 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T15:26:23.595330+0000", 2024-09-17T15:26:23.621 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0055121859999999996, 2024-09-17T15:26:23.621 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.00575948, 2024-09-17T15:26:23.622 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T15:26:23.622 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T15:26:23.622 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T15:26:23.622 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T15:26:23.622 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.622 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T15:26:23.622 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.595330+0000", 2024-09-17T15:26:23.622 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T15:26:23.622 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.622 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.622 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T15:26:23.622 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.595330+0000", 2024-09-17T15:26:23.622 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T15:26:23.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.623 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.623 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T15:26:23.623 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.595330+0000", 2024-09-17T15:26:23.623 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.8900000000000001e-07 2024-09-17T15:26:23.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.623 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.623 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T15:26:23.623 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.595340+0000", 2024-09-17T15:26:23.623 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.0550999999999999e-05 2024-09-17T15:26:23.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.623 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.623 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T15:26:23.623 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.595345+0000", 2024-09-17T15:26:23.623 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.7990000000000001e-06 2024-09-17T15:26:23.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.624 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.624 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T15:26:23.624 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.595349+0000", 2024-09-17T15:26:23.624 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.0430000000000001e-06 2024-09-17T15:26:23.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.624 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.624 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T15:26:23.624 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.595395+0000", 2024-09-17T15:26:23.624 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.5800000000000002e-05 2024-09-17T15:26:23.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.624 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.624 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T15:26:23.625 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.595420+0000", 2024-09-17T15:26:23.625 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4689e-05 2024-09-17T15:26:23.625 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T15:26:23.625 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T15:26:23.625 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T15:26:23.625 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.625 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.625 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5322.0:66312 2.9 e41/40 2:93fb71ef:::benchmark_data_smithi083_41818_object66311:head v 41'4218, mlcod=21'4)", 2024-09-17T15:26:23.625 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T15:26:23.595360+0000", 2024-09-17T15:26:23.625 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.0054813639999999999, 2024-09-17T15:26:23.625 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0057727050000000004, 2024-09-17T15:26:23.625 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T15:26:23.625 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T15:26:23.626 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T15:26:23.626 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T15:26:23.626 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.626 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T15:26:23.626 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.595360+0000", 2024-09-17T15:26:23.626 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T15:26:23.626 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.626 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.626 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T15:26:23.626 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.595360+0000", 2024-09-17T15:26:23.626 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T15:26:23.626 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.626 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.626 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T15:26:23.627 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.595361+0000", 2024-09-17T15:26:23.627 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1299999999999999e-07 2024-09-17T15:26:23.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.627 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.627 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T15:26:23.627 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.595367+0000", 2024-09-17T15:26:23.627 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.737e-06 2024-09-17T15:26:23.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.627 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.627 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T15:26:23.627 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.595373+0000", 2024-09-17T15:26:23.627 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 5.2650000000000004e-06 2024-09-17T15:26:23.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.628 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.628 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T15:26:23.628 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.595377+0000", 2024-09-17T15:26:23.628 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2849999999999996e-06 2024-09-17T15:26:23.628 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.628 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.628 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T15:26:23.628 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.597442+0000", 2024-09-17T15:26:23.628 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.0020651269999999999 2024-09-17T15:26:23.628 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.628 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.628 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T15:26:23.628 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.597464+0000", 2024-09-17T15:26:23.629 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.1546e-05 2024-09-17T15:26:23.629 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T15:26:23.629 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T15:26:23.629 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T15:26:23.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.629 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.629 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5322.0:66294 2.f e41/40 2:ffe664f9:::benchmark_data_smithi083_41818_object66293:head v 41'4100, mlcod=0'0)", 2024-09-17T15:26:23.629 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T15:26:23.569265+0000", 2024-09-17T15:26:23.629 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.031576901999999997, 2024-09-17T15:26:23.629 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.031914900000000003, 2024-09-17T15:26:23.629 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T15:26:23.629 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T15:26:23.629 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T15:26:23.629 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T15:26:23.630 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.630 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T15:26:23.630 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.569265+0000", 2024-09-17T15:26:23.630 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T15:26:23.630 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.630 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.630 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T15:26:23.630 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.569265+0000", 2024-09-17T15:26:23.630 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T15:26:23.630 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.630 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.630 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T15:26:23.630 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.569265+0000", 2024-09-17T15:26:23.631 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.4400000000000001e-07 2024-09-17T15:26:23.631 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.631 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.631 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T15:26:23.631 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.569272+0000", 2024-09-17T15:26:23.631 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 6.3489999999999997e-06 2024-09-17T15:26:23.631 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.631 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.631 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T15:26:23.631 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.569276+0000", 2024-09-17T15:26:23.631 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 4.2810000000000002e-06 2024-09-17T15:26:23.631 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.631 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.631 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T15:26:23.632 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.569279+0000", 2024-09-17T15:26:23.632 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 3.4149999999999999e-06 2024-09-17T15:26:23.632 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.632 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.632 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T15:26:23.632 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.583512+0000", 2024-09-17T15:26:23.632 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.014232725 2024-09-17T15:26:23.632 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.632 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.632 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T15:26:23.632 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.583535+0000", 2024-09-17T15:26:23.632 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.2600999999999999e-05 2024-09-17T15:26:23.632 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T15:26:23.632 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T15:26:23.633 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T15:26:23.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.633 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.633 INFO:teuthology.orchestra.run.smithi083.stdout: "description": "osd_repop(client.5322.0:66300 2.f e41/40 2:f2d6b243:::benchmark_data_smithi083_41818_object66299:head v 41'4101, mlcod=0'0)", 2024-09-17T15:26:23.633 INFO:teuthology.orchestra.run.smithi083.stdout: "initiated_at": "2024-09-17T15:26:23.573312+0000", 2024-09-17T15:26:23.633 INFO:teuthology.orchestra.run.smithi083.stdout: "age": 0.027530246000000001, 2024-09-17T15:26:23.633 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.027914837000000001, 2024-09-17T15:26:23.633 INFO:teuthology.orchestra.run.smithi083.stdout: "continuous": false, 2024-09-17T15:26:23.633 INFO:teuthology.orchestra.run.smithi083.stdout: "type_data": { 2024-09-17T15:26:23.633 INFO:teuthology.orchestra.run.smithi083.stdout: "flag_point": "started", 2024-09-17T15:26:23.633 INFO:teuthology.orchestra.run.smithi083.stdout: "events": [ 2024-09-17T15:26:23.633 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.633 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "initiated", 2024-09-17T15:26:23.634 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.573312+0000", 2024-09-17T15:26:23.634 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T15:26:23.634 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.634 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.634 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "header_read", 2024-09-17T15:26:23.634 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.573312+0000", 2024-09-17T15:26:23.634 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0 2024-09-17T15:26:23.634 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.634 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.634 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "throttled", 2024-09-17T15:26:23.634 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.573314+0000", 2024-09-17T15:26:23.634 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.0959999999999999e-06 2024-09-17T15:26:23.634 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.635 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.635 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "all_read", 2024-09-17T15:26:23.635 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.573327+0000", 2024-09-17T15:26:23.635 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.3692e-05 2024-09-17T15:26:23.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.635 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.635 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "dispatched", 2024-09-17T15:26:23.635 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.573339+0000", 2024-09-17T15:26:23.635 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 1.1389000000000001e-05 2024-09-17T15:26:23.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.635 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.635 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "queued_for_pg", 2024-09-17T15:26:23.635 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.573348+0000", 2024-09-17T15:26:23.635 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 9.2220000000000008e-06 2024-09-17T15:26:23.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.636 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.636 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "reached_pg", 2024-09-17T15:26:23.636 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.591209+0000", 2024-09-17T15:26:23.636 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 0.017861273 2024-09-17T15:26:23.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-09-17T15:26:23.636 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-09-17T15:26:23.636 INFO:teuthology.orchestra.run.smithi083.stdout: "event": "started", 2024-09-17T15:26:23.636 INFO:teuthology.orchestra.run.smithi083.stdout: "time": "2024-09-17T15:26:23.591235+0000", 2024-09-17T15:26:23.636 INFO:teuthology.orchestra.run.smithi083.stdout: "duration": 2.6092999999999999e-05 2024-09-17T15:26:23.636 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T15:26:23.636 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-09-17T15:26:23.636 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T15:26:23.636 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-09-17T15:26:23.637 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-09-17T15:26:23.637 INFO:teuthology.orchestra.run.smithi083.stdout: "num_ops": 9 2024-09-17T15:26:23.637 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T15:26:23.638 INFO:tasks.lost_unfound.ceph_manager:waiting for osd.1 to be up 2024-09-17T15:26:23.638 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T15:26:24.200 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:26:24.200 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":41,"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","created":"2024-09-17T15:24:23.007214+0000","modified":"2024-09-17T15:26:22.177259+0000","last_up_change":"2024-09-17T15:26:20.158671+0000","last_in_change":"2024-09-17T15:26:13.095382+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-17T15:24:38.423278+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":13,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfound_pool","create_time":"2024-09-17T15:24:47.449679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9defe403-224a-4a6c-9184-dcb72bef6bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":19,"up_from":23,"up_thru":40,"down_at":20,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":1704571166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6805","nonce":1704571166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6807","nonce":1704571166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":1704571166}]},"public_addr":"172.21.15.83:6804/1704571166","cluster_addr":"172.21.15.83:6805/1704571166","heartbeat_back_addr":"172.21.15.83:6807/1704571166","heartbeat_front_addr":"172.21.15.83:6806/1704571166","state":["exists","up"]},{"osd":1,"uuid":"d7d78e01-f761-444b-98c4-62587b991738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":24,"up_from":39,"up_thru":40,"down_at":25,"lost_at":25,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2000475044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":2000475044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6811","nonce":2000475044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2000475044}]},"public_addr":"172.21.15.83:6808/2000475044","cluster_addr":"172.21.15.83:6809/2000475044","heartbeat_back_addr":"172.21.15.83:6811/2000475044","heartbeat_front_addr":"172.21.15.83:6810/2000475044","state":["exists","up"]},{"osd":2,"uuid":"23c2275c-8731-45f3-97f9-5100b6676f99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":16,"up_from":29,"up_thru":40,"down_at":17,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3672266049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6801","nonce":3672266049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6803","nonce":3672266049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3672266049}]},"public_addr":"172.21.15.83:6800/3672266049","cluster_addr":"172.21.15.83:6801/3672266049","heartbeat_back_addr":"172.21.15.83:6803/3672266049","heartbeat_front_addr":"172.21.15.83:6802/3672266049","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T15:24:31.547535+0000","dead_epoch":20},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T15:24:30.867578+0000","dead_epoch":25},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T15:24:30.866429+0000","dead_epoch":17}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[{"pgid":"2.0","osds":[2]},{"pgid":"2.1","osds":[2]},{"pgid":"2.2","osds":[0]},{"pgid":"2.3","osds":[2]},{"pgid":"2.4","osds":[0]},{"pgid":"2.5","osds":[0]},{"pgid":"2.6","osds":[0]},{"pgid":"2.7","osds":[0]},{"pgid":"2.9","osds":[0]},{"pgid":"2.a","osds":[0]},{"pgid":"2.d","osds":[2]},{"pgid":"2.f","osds":[0]}],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T15:26:24.228 INFO:tasks.lost_unfound.ceph_manager:osd.1 is up 2024-09-17T15:26:24.228 INFO:tasks.lost_unfound.ceph_manager:waiting for clean 2024-09-17T15:26:24.229 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:26:24.292 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 45 20 66731 66711 5.78977 2.40625 0.0458938 0.0134859 2024-09-17T15:26:24.769 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:26:24.770 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:26:24.806 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":94,"stamp":"2024-09-17T15:26:23.592782+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":268321521,"num_objects":65519,"num_object_clones":0,"num_object_copies":131038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":32833,"num_objects_unfound":0,"num_objects_dirty":65519,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":25626,"num_write_kb":102498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65592,"ondisk_log_size":65592,"up":34,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":72,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":839992,"kb_used_data":524216,"kb_used_omap":13,"kb_used_meta":315698,"kb_avail":282275528,"statfs":{"total":289910292480,"available":289050140672,"internally_reserved":0,"allocated":536797184,"data_stored":536026138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":323275775},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,1,3,3,4,4,0,0,0,9],"upper_bound":1024},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22999040,"num_objects":5615,"num_object_clones":0,"num_object_copies":11230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17469,"num_objects_unfound":0,"num_objects_dirty":5615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5615,"num_write_kb":22460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.426933"},"pg_stats":[{"pgid":"2.f","version":"38'4046","reported_seq":4110,"reported_epoch":38,"state":"active+clean+remapped","last_fresh":"2024-09-17T15:26:19.914562+0000","last_change":"2024-09-17T15:26:15.147288+0000","last_active":"2024-09-17T15:26:19.914562+0000","last_peered":"2024-09-17T15:26:19.914562+0000","last_clean":"2024-09-17T15:26:19.914562+0000","last_became_active":"2024-09-17T15:26:15.135641+0000","last_became_peered":"2024-09-17T15:26:15.135641+0000","last_unstale":"2024-09-17T15:26:19.914562+0000","last_undegraded":"2024-09-17T15:26:19.914562+0000","last_fullsized":"2024-09-17T15:26:19.914562+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":4046,"log_dups_size":0,"ondisk_log_size":4046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:20:02.579123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16572416,"num_objects":4046,"num_object_clones":0,"num_object_copies":8092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4046,"num_objects_unfound":0,"num_objects_dirty":4046,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4046,"num_write_kb":16184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":4046}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'4077","reported_seq":4172,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T15:26:21.183706+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:26:21.183706+0000","last_peered":"2024-09-17T15:26:21.183706+0000","last_clean":"2024-09-17T15:26:21.183706+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:26:21.183706+0000","last_undegraded":"2024-09-17T15:26:21.183706+0000","last_fullsized":"2024-09-17T15:26:21.183706+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":4077,"log_dups_size":0,"ondisk_log_size":4077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16675782,"num_objects":4073,"num_object_clones":0,"num_object_copies":8146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4073,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2434,"num_write_kb":9736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"38'4172","reported_seq":4286,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-17T15:26:21.184781+0000","last_change":"2024-09-17T15:26:20.172232+0000","last_active":"2024-09-17T15:26:20.172232+0000","last_peered":"2024-09-17T15:26:20.158176+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:15.138987+0000","last_became_peered":"2024-09-17T15:26:15.138987+0000","last_unstale":"2024-09-17T15:26:21.184781+0000","last_undegraded":"2024-09-17T15:26:21.184781+0000","last_fullsized":"2024-09-17T15:26:21.184781+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":4172,"log_dups_size":0,"ondisk_log_size":4172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17072128,"num_objects":4168,"num_object_clones":0,"num_object_copies":8336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2329,"num_write_kb":9316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'4064","reported_seq":4126,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T15:26:21.184921+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:26:21.184921+0000","last_peered":"2024-09-17T15:26:21.184921+0000","last_clean":"2024-09-17T15:26:21.184921+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:26:21.184921+0000","last_undegraded":"2024-09-17T15:26:21.184921+0000","last_fullsized":"2024-09-17T15:26:21.184921+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":4064,"log_dups_size":0,"ondisk_log_size":4064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16646144,"num_objects":4064,"num_object_clones":0,"num_object_copies":8128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4064,"num_write_kb":16256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"38'3989","reported_seq":4107,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T15:26:19.915922+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:26:19.915922+0000","last_peered":"2024-09-17T15:26:19.915922+0000","last_clean":"2024-09-17T15:26:19.915922+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:26:19.915922+0000","last_undegraded":"2024-09-17T15:26:19.915922+0000","last_fullsized":"2024-09-17T15:26:19.915922+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":3989,"log_dups_size":0,"ondisk_log_size":3989,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16310755,"num_objects":3983,"num_object_clones":0,"num_object_copies":7966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3983,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":971,"num_write_kb":3884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"38'4254","reported_seq":4376,"reported_epoch":38,"state":"active+clean+remapped","last_fresh":"2024-09-17T15:26:19.918041+0000","last_change":"2024-09-17T15:26:15.144492+0000","last_active":"2024-09-17T15:26:19.918041+0000","last_peered":"2024-09-17T15:26:19.918041+0000","last_clean":"2024-09-17T15:26:19.918041+0000","last_became_active":"2024-09-17T15:26:15.135835+0000","last_became_peered":"2024-09-17T15:26:15.135835+0000","last_unstale":"2024-09-17T15:26:19.918041+0000","last_undegraded":"2024-09-17T15:26:19.918041+0000","last_fullsized":"2024-09-17T15:26:19.918041+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":4254,"log_dups_size":0,"ondisk_log_size":4254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:58:31.430197+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17404387,"num_objects":4250,"num_object_clones":0,"num_object_copies":8500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4250,"num_objects_unfound":0,"num_objects_dirty":4250,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":753,"num_write_kb":3012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":4250}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"38'4158","reported_seq":4281,"reported_epoch":38,"state":"active+clean+remapped","last_fresh":"2024-09-17T15:26:19.934666+0000","last_change":"2024-09-17T15:26:15.144564+0000","last_active":"2024-09-17T15:26:19.934666+0000","last_peered":"2024-09-17T15:26:19.934666+0000","last_clean":"2024-09-17T15:26:19.934666+0000","last_became_active":"2024-09-17T15:26:15.135490+0000","last_became_peered":"2024-09-17T15:26:15.135490+0000","last_unstale":"2024-09-17T15:26:19.934666+0000","last_undegraded":"2024-09-17T15:26:19.934666+0000","last_fullsized":"2024-09-17T15:26:19.934666+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":4158,"log_dups_size":0,"ondisk_log_size":4158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:54:41.720662+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17007558,"num_objects":4154,"num_object_clones":0,"num_object_copies":8308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4154,"num_objects_unfound":0,"num_objects_dirty":4154,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":927,"num_write_kb":3708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"38'4115","reported_seq":4231,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T15:26:19.900752+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:26:19.900752+0000","last_peered":"2024-09-17T15:26:19.900752+0000","last_clean":"2024-09-17T15:26:19.900752+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:26:19.900752+0000","last_undegraded":"2024-09-17T15:26:19.900752+0000","last_fullsized":"2024-09-17T15:26:19.900752+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":4115,"log_dups_size":0,"ondisk_log_size":4115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16846848,"num_objects":4113,"num_object_clones":0,"num_object_copies":8226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1069,"num_write_kb":4276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":86,"reported_epoch":38,"state":"active+clean+remapped","last_fresh":"2024-09-17T15:26:19.170646+0000","last_change":"2024-09-17T15:26:15.137622+0000","last_active":"2024-09-17T15:26:19.170646+0000","last_peered":"2024-09-17T15:26:19.170646+0000","last_clean":"2024-09-17T15:26:19.170646+0000","last_became_active":"2024-09-17T15:26:15.135633+0000","last_became_peered":"2024-09-17T15:26:15.135633+0000","last_unstale":"2024-09-17T15:26:19.170646+0000","last_undegraded":"2024-09-17T15:26:19.170646+0000","last_fullsized":"2024-09-17T15:26:19.170646+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:03:41.999315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"38'4077","reported_seq":4193,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-17T15:26:21.183272+0000","last_change":"2024-09-17T15:26:20.172430+0000","last_active":"2024-09-17T15:26:20.172430+0000","last_peered":"2024-09-17T15:26:20.155646+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:15.132800+0000","last_became_peered":"2024-09-17T15:26:15.132800+0000","last_unstale":"2024-09-17T15:26:21.183272+0000","last_undegraded":"2024-09-17T15:26:21.183272+0000","last_fullsized":"2024-09-17T15:26:21.183272+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":4077,"log_dups_size":0,"ondisk_log_size":4077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16671203,"num_objects":4071,"num_object_clones":0,"num_object_copies":8142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4071,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2077,"num_write_kb":8308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":62,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T15:26:21.185047+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:21.185047+0000","last_peered":"2024-09-17T15:26:21.185047+0000","last_clean":"2024-09-17T15:26:21.185047+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:21.185047+0000","last_undegraded":"2024-09-17T15:26:21.185047+0000","last_fullsized":"2024-09-17T15:26:21.185047+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"38'3993","reported_seq":4109,"reported_epoch":38,"state":"active+clean+remapped","last_fresh":"2024-09-17T15:26:19.928508+0000","last_change":"2024-09-17T15:26:15.147907+0000","last_active":"2024-09-17T15:26:19.928508+0000","last_peered":"2024-09-17T15:26:19.928508+0000","last_clean":"2024-09-17T15:26:19.928508+0000","last_became_active":"2024-09-17T15:26:15.143449+0000","last_became_peered":"2024-09-17T15:26:15.143449+0000","last_unstale":"2024-09-17T15:26:19.928508+0000","last_undegraded":"2024-09-17T15:26:19.928508+0000","last_fullsized":"2024-09-17T15:26:19.928508+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":3993,"log_dups_size":0,"ondisk_log_size":3993,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:21:57.740831+0000","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16322560,"num_objects":3985,"num_object_clones":0,"num_object_copies":7970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3985,"num_objects_unfound":0,"num_objects_dirty":3985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":589,"num_write_kb":2356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":3985}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":86,"reported_epoch":38,"state":"active+clean+remapped","last_fresh":"2024-09-17T15:26:19.170730+0000","last_change":"2024-09-17T15:26:15.147768+0000","last_active":"2024-09-17T15:26:19.170730+0000","last_peered":"2024-09-17T15:26:19.170730+0000","last_clean":"2024-09-17T15:26:19.170730+0000","last_became_active":"2024-09-17T15:26:15.144910+0000","last_became_peered":"2024-09-17T15:26:15.144910+0000","last_unstale":"2024-09-17T15:26:19.170730+0000","last_undegraded":"2024-09-17T15:26:19.170730+0000","last_fullsized":"2024-09-17T15:26:19.170730+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:08:49.194044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"38'4081","reported_seq":4190,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-17T15:26:21.184771+0000","last_change":"2024-09-17T15:26:20.169734+0000","last_active":"2024-09-17T15:26:20.169734+0000","last_peered":"2024-09-17T15:26:20.152702+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:15.137344+0000","last_became_peered":"2024-09-17T15:26:15.137344+0000","last_unstale":"2024-09-17T15:26:21.184771+0000","last_undegraded":"2024-09-17T15:26:21.184771+0000","last_fullsized":"2024-09-17T15:26:21.184771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":4081,"log_dups_size":0,"ondisk_log_size":4081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16695779,"num_objects":4077,"num_object_clones":0,"num_object_copies":8154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4077,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1976,"num_write_kb":7904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":30,"reported_epoch":37,"state":"active+clean+remapped","last_fresh":"2024-09-17T15:26:15.133451+0000","last_change":"2024-09-17T15:26:15.133308+0000","last_active":"2024-09-17T15:26:15.133451+0000","last_peered":"2024-09-17T15:26:15.133451+0000","last_clean":"2024-09-17T15:26:15.133451+0000","last_became_active":"2024-09-17T15:26:15.132864+0000","last_became_peered":"2024-09-17T15:26:15.132864+0000","last_unstale":"2024-09-17T15:26:15.133451+0000","last_undegraded":"2024-09-17T15:26:15.133451+0000","last_fullsized":"2024-09-17T15:26:15.133451+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:23:20.565701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"38'4143","reported_seq":4250,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-17T15:26:21.183420+0000","last_change":"2024-09-17T15:26:20.170902+0000","last_active":"2024-09-17T15:26:20.170902+0000","last_peered":"2024-09-17T15:26:20.164921+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:15.139446+0000","last_became_peered":"2024-09-17T15:26:15.139446+0000","last_unstale":"2024-09-17T15:26:21.183420+0000","last_undegraded":"2024-09-17T15:26:21.183420+0000","last_fullsized":"2024-09-17T15:26:21.183420+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":4143,"log_dups_size":0,"ondisk_log_size":4143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16945152,"num_objects":4137,"num_object_clones":0,"num_object_copies":8274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1905,"num_write_kb":7620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"38'4127","reported_seq":4247,"reported_epoch":38,"state":"active+clean+remapped","last_fresh":"2024-09-17T15:26:19.928097+0000","last_change":"2024-09-17T15:26:15.154611+0000","last_active":"2024-09-17T15:26:19.928097+0000","last_peered":"2024-09-17T15:26:19.928097+0000","last_clean":"2024-09-17T15:26:19.928097+0000","last_became_active":"2024-09-17T15:26:15.142706+0000","last_became_peered":"2024-09-17T15:26:15.142706+0000","last_unstale":"2024-09-17T15:26:19.928097+0000","last_undegraded":"2024-09-17T15:26:19.928097+0000","last_fullsized":"2024-09-17T15:26:19.928097+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":4127,"log_dups_size":0,"ondisk_log_size":4127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:54:33.867193+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16896000,"num_objects":4125,"num_object_clones":0,"num_object_copies":8250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4125,"num_objects_unfound":0,"num_objects_dirty":4125,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":675,"num_write_kb":2700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":4125}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":86,"reported_epoch":38,"state":"active+clean+remapped","last_fresh":"2024-09-17T15:26:19.171015+0000","last_change":"2024-09-17T15:26:15.137379+0000","last_active":"2024-09-17T15:26:19.171015+0000","last_peered":"2024-09-17T15:26:19.171015+0000","last_clean":"2024-09-17T15:26:19.171015+0000","last_became_active":"2024-09-17T15:26:15.135848+0000","last_became_peered":"2024-09-17T15:26:15.135848+0000","last_unstale":"2024-09-17T15:26:19.171015+0000","last_undegraded":"2024-09-17T15:26:19.171015+0000","last_fullsized":"2024-09-17T15:26:19.171015+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:39:08.366449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"38'4150","reported_seq":4277,"reported_epoch":38,"state":"active+clean+remapped","last_fresh":"2024-09-17T15:26:19.896686+0000","last_change":"2024-09-17T15:26:15.145078+0000","last_active":"2024-09-17T15:26:19.896686+0000","last_peered":"2024-09-17T15:26:19.896686+0000","last_clean":"2024-09-17T15:26:19.896686+0000","last_became_active":"2024-09-17T15:26:15.135701+0000","last_became_peered":"2024-09-17T15:26:15.135701+0000","last_unstale":"2024-09-17T15:26:19.896686+0000","last_undegraded":"2024-09-17T15:26:19.896686+0000","last_fullsized":"2024-09-17T15:26:19.896686+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":4150,"log_dups_size":0,"ondisk_log_size":4150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:58:40.063370+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16965632,"num_objects":4142,"num_object_clones":0,"num_object_copies":8284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4142,"num_objects_unfound":0,"num_objects_dirty":4142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":437,"num_write_kb":1748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":4142}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":62,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T15:26:21.184340+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:21.184340+0000","last_peered":"2024-09-17T15:26:21.184340+0000","last_clean":"2024-09-17T15:26:21.184340+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:21.184340+0000","last_undegraded":"2024-09-17T15:26:21.184340+0000","last_fullsized":"2024-09-17T15:26:21.184340+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"38'4047","reported_seq":4166,"reported_epoch":38,"state":"active+clean+remapped","last_fresh":"2024-09-17T15:26:19.914342+0000","last_change":"2024-09-17T15:26:15.148287+0000","last_active":"2024-09-17T15:26:19.914342+0000","last_peered":"2024-09-17T15:26:19.914342+0000","last_clean":"2024-09-17T15:26:19.914342+0000","last_became_active":"2024-09-17T15:26:15.135645+0000","last_became_peered":"2024-09-17T15:26:15.135645+0000","last_unstale":"2024-09-17T15:26:19.914342+0000","last_undegraded":"2024-09-17T15:26:19.914342+0000","last_fullsized":"2024-09-17T15:26:19.914342+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":4047,"log_dups_size":0,"ondisk_log_size":4047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:11:51.064560+0000","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16560128,"num_objects":4043,"num_object_clones":0,"num_object_copies":8086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4043,"num_objects_unfound":0,"num_objects_dirty":4043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":834,"num_write_kb":3336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":4043}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":86,"reported_epoch":38,"state":"active+clean+remapped","last_fresh":"2024-09-17T15:26:19.170957+0000","last_change":"2024-09-17T15:26:15.136697+0000","last_active":"2024-09-17T15:26:19.170957+0000","last_peered":"2024-09-17T15:26:19.170957+0000","last_clean":"2024-09-17T15:26:19.170957+0000","last_became_active":"2024-09-17T15:26:15.135718+0000","last_became_peered":"2024-09-17T15:26:15.135718+0000","last_unstale":"2024-09-17T15:26:19.170957+0000","last_undegraded":"2024-09-17T15:26:19.170957+0000","last_fullsized":"2024-09-17T15:26:19.170957+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:33:45.454249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":86,"reported_epoch":37,"state":"active+clean+remapped","last_fresh":"2024-09-17T15:26:15.151327+0000","last_change":"2024-09-17T15:26:15.151202+0000","last_active":"2024-09-17T15:26:15.151327+0000","last_peered":"2024-09-17T15:26:15.151327+0000","last_clean":"2024-09-17T15:26:15.151327+0000","last_became_active":"2024-09-17T15:26:15.150841+0000","last_became_peered":"2024-09-17T15:26:15.150841+0000","last_unstale":"2024-09-17T15:26:15.151327+0000","last_undegraded":"2024-09-17T15:26:15.151327+0000","last_fullsized":"2024-09-17T15:26:15.151327+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:13:40.584905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"38'4096","reported_seq":4224,"reported_epoch":38,"state":"active+clean+remapped","last_fresh":"2024-09-17T15:26:19.905281+0000","last_change":"2024-09-17T15:26:15.154318+0000","last_active":"2024-09-17T15:26:19.905281+0000","last_peered":"2024-09-17T15:26:19.905281+0000","last_clean":"2024-09-17T15:26:19.905281+0000","last_became_active":"2024-09-17T15:26:15.136292+0000","last_became_peered":"2024-09-17T15:26:15.136292+0000","last_unstale":"2024-09-17T15:26:19.905281+0000","last_undegraded":"2024-09-17T15:26:19.905281+0000","last_fullsized":"2024-09-17T15:26:19.905281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":4096,"log_dups_size":0,"ondisk_log_size":4096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:11:42.118386+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16729030,"num_objects":4086,"num_object_clones":0,"num_object_copies":8172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4086,"num_objects_unfound":0,"num_objects_dirty":4086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":538,"num_write_kb":2152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":4086}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":268321502,"num_objects":65517,"num_object_clones":0,"num_object_copies":131034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":32831,"num_objects_unfound":0,"num_objects_dirty":65517,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":25624,"num_write_kb":102496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":537157632,"data_stored":536984208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65589,"ondisk_log_size":65589,"up":24,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":10,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051600,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":411476,"kb_used_data":263160,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":93960364,"statfs":{"total":96636764160,"available":96215412736,"internally_reserved":0,"allocated":269475840,"data_stored":269210760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":151842476},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,9],"upper_bound":1024},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":0,"seq":1,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":316,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":323584,"data_stored":73212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":28176043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247828,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":400664,"kb_used_data":260740,"kb_used_omap":4,"kb_used_meta":139899,"kb_avail":93971176,"statfs":{"total":96636764160,"available":96226484224,"internally_reserved":0,"allocated":266997760,"data_stored":266742166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":143257256},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,1,3,3,4,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":267907072,"data_stored":267870942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":269135872,"data_stored":269099742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:26:24.807 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:26:25.294 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 46 20 67264 67244 5.70917 2.08203 0.0222456 0.0136747 2024-09-17T15:26:25.430 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:26:25.432 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:26:25.460 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":94,"stamp":"2024-09-17T15:26:23.592782+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":268321521,"num_objects":65519,"num_object_clones":0,"num_object_copies":131038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":32833,"num_objects_unfound":0,"num_objects_dirty":65519,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":25626,"num_write_kb":102498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65592,"ondisk_log_size":65592,"up":34,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":72,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":839992,"kb_used_data":524216,"kb_used_omap":13,"kb_used_meta":315698,"kb_avail":282275528,"statfs":{"total":289910292480,"available":289050140672,"internally_reserved":0,"allocated":536797184,"data_stored":536026138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":323275775},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,1,3,3,4,4,0,0,0,9],"upper_bound":1024},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22999040,"num_objects":5615,"num_object_clones":0,"num_object_copies":11230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17469,"num_objects_unfound":0,"num_objects_dirty":5615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5615,"num_write_kb":22460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.426933"},"pg_stats":[{"pgid":"2.f","version":"38'4046","reported_seq":4110,"reported_epoch":38,"state":"active+clean+remapped","last_fresh":"2024-09-17T15:26:19.914562+0000","last_change":"2024-09-17T15:26:15.147288+0000","last_active":"2024-09-17T15:26:19.914562+0000","last_peered":"2024-09-17T15:26:19.914562+0000","last_clean":"2024-09-17T15:26:19.914562+0000","last_became_active":"2024-09-17T15:26:15.135641+0000","last_became_peered":"2024-09-17T15:26:15.135641+0000","last_unstale":"2024-09-17T15:26:19.914562+0000","last_undegraded":"2024-09-17T15:26:19.914562+0000","last_fullsized":"2024-09-17T15:26:19.914562+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":4046,"log_dups_size":0,"ondisk_log_size":4046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:20:02.579123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16572416,"num_objects":4046,"num_object_clones":0,"num_object_copies":8092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4046,"num_objects_unfound":0,"num_objects_dirty":4046,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4046,"num_write_kb":16184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":4046}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'4077","reported_seq":4172,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T15:26:21.183706+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:26:21.183706+0000","last_peered":"2024-09-17T15:26:21.183706+0000","last_clean":"2024-09-17T15:26:21.183706+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:26:21.183706+0000","last_undegraded":"2024-09-17T15:26:21.183706+0000","last_fullsized":"2024-09-17T15:26:21.183706+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":4077,"log_dups_size":0,"ondisk_log_size":4077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16675782,"num_objects":4073,"num_object_clones":0,"num_object_copies":8146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4073,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2434,"num_write_kb":9736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"38'4172","reported_seq":4286,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-17T15:26:21.184781+0000","last_change":"2024-09-17T15:26:20.172232+0000","last_active":"2024-09-17T15:26:20.172232+0000","last_peered":"2024-09-17T15:26:20.158176+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:15.138987+0000","last_became_peered":"2024-09-17T15:26:15.138987+0000","last_unstale":"2024-09-17T15:26:21.184781+0000","last_undegraded":"2024-09-17T15:26:21.184781+0000","last_fullsized":"2024-09-17T15:26:21.184781+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":4172,"log_dups_size":0,"ondisk_log_size":4172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17072128,"num_objects":4168,"num_object_clones":0,"num_object_copies":8336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2329,"num_write_kb":9316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'4064","reported_seq":4126,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T15:26:21.184921+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:26:21.184921+0000","last_peered":"2024-09-17T15:26:21.184921+0000","last_clean":"2024-09-17T15:26:21.184921+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:26:21.184921+0000","last_undegraded":"2024-09-17T15:26:21.184921+0000","last_fullsized":"2024-09-17T15:26:21.184921+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":4064,"log_dups_size":0,"ondisk_log_size":4064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16646144,"num_objects":4064,"num_object_clones":0,"num_object_copies":8128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4064,"num_write_kb":16256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"38'3989","reported_seq":4107,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T15:26:19.915922+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:26:19.915922+0000","last_peered":"2024-09-17T15:26:19.915922+0000","last_clean":"2024-09-17T15:26:19.915922+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:26:19.915922+0000","last_undegraded":"2024-09-17T15:26:19.915922+0000","last_fullsized":"2024-09-17T15:26:19.915922+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":3989,"log_dups_size":0,"ondisk_log_size":3989,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16310755,"num_objects":3983,"num_object_clones":0,"num_object_copies":7966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3983,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":971,"num_write_kb":3884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"38'4254","reported_seq":4376,"reported_epoch":38,"state":"active+clean+remapped","last_fresh":"2024-09-17T15:26:19.918041+0000","last_change":"2024-09-17T15:26:15.144492+0000","last_active":"2024-09-17T15:26:19.918041+0000","last_peered":"2024-09-17T15:26:19.918041+0000","last_clean":"2024-09-17T15:26:19.918041+0000","last_became_active":"2024-09-17T15:26:15.135835+0000","last_became_peered":"2024-09-17T15:26:15.135835+0000","last_unstale":"2024-09-17T15:26:19.918041+0000","last_undegraded":"2024-09-17T15:26:19.918041+0000","last_fullsized":"2024-09-17T15:26:19.918041+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":4254,"log_dups_size":0,"ondisk_log_size":4254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:58:31.430197+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17404387,"num_objects":4250,"num_object_clones":0,"num_object_copies":8500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4250,"num_objects_unfound":0,"num_objects_dirty":4250,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":753,"num_write_kb":3012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":4250}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"38'4158","reported_seq":4281,"reported_epoch":38,"state":"active+clean+remapped","last_fresh":"2024-09-17T15:26:19.934666+0000","last_change":"2024-09-17T15:26:15.144564+0000","last_active":"2024-09-17T15:26:19.934666+0000","last_peered":"2024-09-17T15:26:19.934666+0000","last_clean":"2024-09-17T15:26:19.934666+0000","last_became_active":"2024-09-17T15:26:15.135490+0000","last_became_peered":"2024-09-17T15:26:15.135490+0000","last_unstale":"2024-09-17T15:26:19.934666+0000","last_undegraded":"2024-09-17T15:26:19.934666+0000","last_fullsized":"2024-09-17T15:26:19.934666+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":4158,"log_dups_size":0,"ondisk_log_size":4158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:54:41.720662+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17007558,"num_objects":4154,"num_object_clones":0,"num_object_copies":8308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4154,"num_objects_unfound":0,"num_objects_dirty":4154,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":927,"num_write_kb":3708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"38'4115","reported_seq":4231,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T15:26:19.900752+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:26:19.900752+0000","last_peered":"2024-09-17T15:26:19.900752+0000","last_clean":"2024-09-17T15:26:19.900752+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:26:19.900752+0000","last_undegraded":"2024-09-17T15:26:19.900752+0000","last_fullsized":"2024-09-17T15:26:19.900752+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":4115,"log_dups_size":0,"ondisk_log_size":4115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16846848,"num_objects":4113,"num_object_clones":0,"num_object_copies":8226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1069,"num_write_kb":4276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":86,"reported_epoch":38,"state":"active+clean+remapped","last_fresh":"2024-09-17T15:26:19.170646+0000","last_change":"2024-09-17T15:26:15.137622+0000","last_active":"2024-09-17T15:26:19.170646+0000","last_peered":"2024-09-17T15:26:19.170646+0000","last_clean":"2024-09-17T15:26:19.170646+0000","last_became_active":"2024-09-17T15:26:15.135633+0000","last_became_peered":"2024-09-17T15:26:15.135633+0000","last_unstale":"2024-09-17T15:26:19.170646+0000","last_undegraded":"2024-09-17T15:26:19.170646+0000","last_fullsized":"2024-09-17T15:26:19.170646+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:03:41.999315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"38'4077","reported_seq":4193,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-17T15:26:21.183272+0000","last_change":"2024-09-17T15:26:20.172430+0000","last_active":"2024-09-17T15:26:20.172430+0000","last_peered":"2024-09-17T15:26:20.155646+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:15.132800+0000","last_became_peered":"2024-09-17T15:26:15.132800+0000","last_unstale":"2024-09-17T15:26:21.183272+0000","last_undegraded":"2024-09-17T15:26:21.183272+0000","last_fullsized":"2024-09-17T15:26:21.183272+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":4077,"log_dups_size":0,"ondisk_log_size":4077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16671203,"num_objects":4071,"num_object_clones":0,"num_object_copies":8142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4071,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2077,"num_write_kb":8308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":62,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T15:26:21.185047+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:21.185047+0000","last_peered":"2024-09-17T15:26:21.185047+0000","last_clean":"2024-09-17T15:26:21.185047+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:21.185047+0000","last_undegraded":"2024-09-17T15:26:21.185047+0000","last_fullsized":"2024-09-17T15:26:21.185047+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"38'3993","reported_seq":4109,"reported_epoch":38,"state":"active+clean+remapped","last_fresh":"2024-09-17T15:26:19.928508+0000","last_change":"2024-09-17T15:26:15.147907+0000","last_active":"2024-09-17T15:26:19.928508+0000","last_peered":"2024-09-17T15:26:19.928508+0000","last_clean":"2024-09-17T15:26:19.928508+0000","last_became_active":"2024-09-17T15:26:15.143449+0000","last_became_peered":"2024-09-17T15:26:15.143449+0000","last_unstale":"2024-09-17T15:26:19.928508+0000","last_undegraded":"2024-09-17T15:26:19.928508+0000","last_fullsized":"2024-09-17T15:26:19.928508+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":3993,"log_dups_size":0,"ondisk_log_size":3993,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:21:57.740831+0000","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16322560,"num_objects":3985,"num_object_clones":0,"num_object_copies":7970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3985,"num_objects_unfound":0,"num_objects_dirty":3985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":589,"num_write_kb":2356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":3985}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":86,"reported_epoch":38,"state":"active+clean+remapped","last_fresh":"2024-09-17T15:26:19.170730+0000","last_change":"2024-09-17T15:26:15.147768+0000","last_active":"2024-09-17T15:26:19.170730+0000","last_peered":"2024-09-17T15:26:19.170730+0000","last_clean":"2024-09-17T15:26:19.170730+0000","last_became_active":"2024-09-17T15:26:15.144910+0000","last_became_peered":"2024-09-17T15:26:15.144910+0000","last_unstale":"2024-09-17T15:26:19.170730+0000","last_undegraded":"2024-09-17T15:26:19.170730+0000","last_fullsized":"2024-09-17T15:26:19.170730+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:08:49.194044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"38'4081","reported_seq":4190,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-17T15:26:21.184771+0000","last_change":"2024-09-17T15:26:20.169734+0000","last_active":"2024-09-17T15:26:20.169734+0000","last_peered":"2024-09-17T15:26:20.152702+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:15.137344+0000","last_became_peered":"2024-09-17T15:26:15.137344+0000","last_unstale":"2024-09-17T15:26:21.184771+0000","last_undegraded":"2024-09-17T15:26:21.184771+0000","last_fullsized":"2024-09-17T15:26:21.184771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":4081,"log_dups_size":0,"ondisk_log_size":4081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16695779,"num_objects":4077,"num_object_clones":0,"num_object_copies":8154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4077,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1976,"num_write_kb":7904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":30,"reported_epoch":37,"state":"active+clean+remapped","last_fresh":"2024-09-17T15:26:15.133451+0000","last_change":"2024-09-17T15:26:15.133308+0000","last_active":"2024-09-17T15:26:15.133451+0000","last_peered":"2024-09-17T15:26:15.133451+0000","last_clean":"2024-09-17T15:26:15.133451+0000","last_became_active":"2024-09-17T15:26:15.132864+0000","last_became_peered":"2024-09-17T15:26:15.132864+0000","last_unstale":"2024-09-17T15:26:15.133451+0000","last_undegraded":"2024-09-17T15:26:15.133451+0000","last_fullsized":"2024-09-17T15:26:15.133451+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:23:20.565701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"38'4143","reported_seq":4250,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-17T15:26:21.183420+0000","last_change":"2024-09-17T15:26:20.170902+0000","last_active":"2024-09-17T15:26:20.170902+0000","last_peered":"2024-09-17T15:26:20.164921+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:15.139446+0000","last_became_peered":"2024-09-17T15:26:15.139446+0000","last_unstale":"2024-09-17T15:26:21.183420+0000","last_undegraded":"2024-09-17T15:26:21.183420+0000","last_fullsized":"2024-09-17T15:26:21.183420+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":4143,"log_dups_size":0,"ondisk_log_size":4143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16945152,"num_objects":4137,"num_object_clones":0,"num_object_copies":8274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1905,"num_write_kb":7620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"38'4127","reported_seq":4247,"reported_epoch":38,"state":"active+clean+remapped","last_fresh":"2024-09-17T15:26:19.928097+0000","last_change":"2024-09-17T15:26:15.154611+0000","last_active":"2024-09-17T15:26:19.928097+0000","last_peered":"2024-09-17T15:26:19.928097+0000","last_clean":"2024-09-17T15:26:19.928097+0000","last_became_active":"2024-09-17T15:26:15.142706+0000","last_became_peered":"2024-09-17T15:26:15.142706+0000","last_unstale":"2024-09-17T15:26:19.928097+0000","last_undegraded":"2024-09-17T15:26:19.928097+0000","last_fullsized":"2024-09-17T15:26:19.928097+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":4127,"log_dups_size":0,"ondisk_log_size":4127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:54:33.867193+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16896000,"num_objects":4125,"num_object_clones":0,"num_object_copies":8250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4125,"num_objects_unfound":0,"num_objects_dirty":4125,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":675,"num_write_kb":2700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":4125}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":86,"reported_epoch":38,"state":"active+clean+remapped","last_fresh":"2024-09-17T15:26:19.171015+0000","last_change":"2024-09-17T15:26:15.137379+0000","last_active":"2024-09-17T15:26:19.171015+0000","last_peered":"2024-09-17T15:26:19.171015+0000","last_clean":"2024-09-17T15:26:19.171015+0000","last_became_active":"2024-09-17T15:26:15.135848+0000","last_became_peered":"2024-09-17T15:26:15.135848+0000","last_unstale":"2024-09-17T15:26:19.171015+0000","last_undegraded":"2024-09-17T15:26:19.171015+0000","last_fullsized":"2024-09-17T15:26:19.171015+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:39:08.366449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"38'4150","reported_seq":4277,"reported_epoch":38,"state":"active+clean+remapped","last_fresh":"2024-09-17T15:26:19.896686+0000","last_change":"2024-09-17T15:26:15.145078+0000","last_active":"2024-09-17T15:26:19.896686+0000","last_peered":"2024-09-17T15:26:19.896686+0000","last_clean":"2024-09-17T15:26:19.896686+0000","last_became_active":"2024-09-17T15:26:15.135701+0000","last_became_peered":"2024-09-17T15:26:15.135701+0000","last_unstale":"2024-09-17T15:26:19.896686+0000","last_undegraded":"2024-09-17T15:26:19.896686+0000","last_fullsized":"2024-09-17T15:26:19.896686+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":4150,"log_dups_size":0,"ondisk_log_size":4150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:58:40.063370+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16965632,"num_objects":4142,"num_object_clones":0,"num_object_copies":8284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4142,"num_objects_unfound":0,"num_objects_dirty":4142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":437,"num_write_kb":1748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":4142}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":62,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T15:26:21.184340+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:21.184340+0000","last_peered":"2024-09-17T15:26:21.184340+0000","last_clean":"2024-09-17T15:26:21.184340+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:21.184340+0000","last_undegraded":"2024-09-17T15:26:21.184340+0000","last_fullsized":"2024-09-17T15:26:21.184340+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"38'4047","reported_seq":4166,"reported_epoch":38,"state":"active+clean+remapped","last_fresh":"2024-09-17T15:26:19.914342+0000","last_change":"2024-09-17T15:26:15.148287+0000","last_active":"2024-09-17T15:26:19.914342+0000","last_peered":"2024-09-17T15:26:19.914342+0000","last_clean":"2024-09-17T15:26:19.914342+0000","last_became_active":"2024-09-17T15:26:15.135645+0000","last_became_peered":"2024-09-17T15:26:15.135645+0000","last_unstale":"2024-09-17T15:26:19.914342+0000","last_undegraded":"2024-09-17T15:26:19.914342+0000","last_fullsized":"2024-09-17T15:26:19.914342+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":4047,"log_dups_size":0,"ondisk_log_size":4047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:11:51.064560+0000","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16560128,"num_objects":4043,"num_object_clones":0,"num_object_copies":8086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4043,"num_objects_unfound":0,"num_objects_dirty":4043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":834,"num_write_kb":3336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":4043}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":86,"reported_epoch":38,"state":"active+clean+remapped","last_fresh":"2024-09-17T15:26:19.170957+0000","last_change":"2024-09-17T15:26:15.136697+0000","last_active":"2024-09-17T15:26:19.170957+0000","last_peered":"2024-09-17T15:26:19.170957+0000","last_clean":"2024-09-17T15:26:19.170957+0000","last_became_active":"2024-09-17T15:26:15.135718+0000","last_became_peered":"2024-09-17T15:26:15.135718+0000","last_unstale":"2024-09-17T15:26:19.170957+0000","last_undegraded":"2024-09-17T15:26:19.170957+0000","last_fullsized":"2024-09-17T15:26:19.170957+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:33:45.454249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":86,"reported_epoch":37,"state":"active+clean+remapped","last_fresh":"2024-09-17T15:26:15.151327+0000","last_change":"2024-09-17T15:26:15.151202+0000","last_active":"2024-09-17T15:26:15.151327+0000","last_peered":"2024-09-17T15:26:15.151327+0000","last_clean":"2024-09-17T15:26:15.151327+0000","last_became_active":"2024-09-17T15:26:15.150841+0000","last_became_peered":"2024-09-17T15:26:15.150841+0000","last_unstale":"2024-09-17T15:26:15.151327+0000","last_undegraded":"2024-09-17T15:26:15.151327+0000","last_fullsized":"2024-09-17T15:26:15.151327+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:13:40.584905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"38'4096","reported_seq":4224,"reported_epoch":38,"state":"active+clean+remapped","last_fresh":"2024-09-17T15:26:19.905281+0000","last_change":"2024-09-17T15:26:15.154318+0000","last_active":"2024-09-17T15:26:19.905281+0000","last_peered":"2024-09-17T15:26:19.905281+0000","last_clean":"2024-09-17T15:26:19.905281+0000","last_became_active":"2024-09-17T15:26:15.136292+0000","last_became_peered":"2024-09-17T15:26:15.136292+0000","last_unstale":"2024-09-17T15:26:19.905281+0000","last_undegraded":"2024-09-17T15:26:19.905281+0000","last_fullsized":"2024-09-17T15:26:19.905281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":4096,"log_dups_size":0,"ondisk_log_size":4096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:11:42.118386+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16729030,"num_objects":4086,"num_object_clones":0,"num_object_copies":8172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4086,"num_objects_unfound":0,"num_objects_dirty":4086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":538,"num_write_kb":2152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":4086}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":268321502,"num_objects":65517,"num_object_clones":0,"num_object_copies":131034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":32831,"num_objects_unfound":0,"num_objects_dirty":65517,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":25624,"num_write_kb":102496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":537157632,"data_stored":536984208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65589,"ondisk_log_size":65589,"up":24,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":10,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051600,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":411476,"kb_used_data":263160,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":93960364,"statfs":{"total":96636764160,"available":96215412736,"internally_reserved":0,"allocated":269475840,"data_stored":269210760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":151842476},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,9],"upper_bound":1024},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":0,"seq":1,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":316,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":323584,"data_stored":73212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":28176043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247828,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":400664,"kb_used_data":260740,"kb_used_omap":4,"kb_used_meta":139899,"kb_avail":93971176,"statfs":{"total":96636764160,"available":96226484224,"internally_reserved":0,"allocated":266997760,"data_stored":266742166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":143257256},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,1,3,3,4,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":267907072,"data_stored":267870942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":269135872,"data_stored":269099742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:26:25.462 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:26:26.022 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:26:26.025 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:26:26.049 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":95,"stamp":"2024-09-17T15:26:25.593105+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":272634609,"num_objects":66572,"num_object_clones":0,"num_object_copies":133144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33002,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66572,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":26679,"num_write_kb":106710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66645,"ondisk_log_size":66645,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":64,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":870716,"kb_used_data":532732,"kb_used_omap":13,"kb_used_meta":337906,"kb_avail":282244804,"statfs":{"total":289910292480,"available":289018679296,"internally_reserved":0,"allocated":545517568,"data_stored":544665019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":346016767},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,2,2,8,4,3,0,0,9],"upper_bound":1024},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27312128,"num_objects":6668,"num_object_clones":0,"num_object_copies":13336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33002,"num_objects_misplaced":-15364,"num_objects_unfound":0,"num_objects_dirty":6668,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6668,"num_write_kb":26672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001159"},"pg_stats":[{"pgid":"2.f","version":"41'4155","reported_seq":4227,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:24.894738+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:26:24.894738+0000","last_peered":"2024-09-17T15:26:24.894738+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:26:24.894738+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":4155,"log_dups_size":0,"ondisk_log_size":4155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17018880,"num_objects":4155,"num_object_clones":0,"num_object_copies":8310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4155,"num_write_kb":16620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":85},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'4077","reported_seq":4172,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T15:26:21.183706+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:26:21.183706+0000","last_peered":"2024-09-17T15:26:21.183706+0000","last_clean":"2024-09-17T15:26:21.183706+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:26:21.183706+0000","last_undegraded":"2024-09-17T15:26:21.183706+0000","last_fullsized":"2024-09-17T15:26:21.183706+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":4077,"log_dups_size":0,"ondisk_log_size":4077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16675782,"num_objects":4073,"num_object_clones":0,"num_object_copies":8146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4073,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2434,"num_write_kb":9736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"38'4172","reported_seq":4286,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-17T15:26:21.184781+0000","last_change":"2024-09-17T15:26:20.172232+0000","last_active":"2024-09-17T15:26:20.172232+0000","last_peered":"2024-09-17T15:26:20.158176+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:15.138987+0000","last_became_peered":"2024-09-17T15:26:15.138987+0000","last_unstale":"2024-09-17T15:26:21.184781+0000","last_undegraded":"2024-09-17T15:26:21.184781+0000","last_fullsized":"2024-09-17T15:26:21.184781+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":4172,"log_dups_size":0,"ondisk_log_size":4172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17072128,"num_objects":4168,"num_object_clones":0,"num_object_copies":8336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2329,"num_write_kb":9316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'4064","reported_seq":4126,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T15:26:21.184921+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:26:21.184921+0000","last_peered":"2024-09-17T15:26:21.184921+0000","last_clean":"2024-09-17T15:26:21.184921+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:26:21.184921+0000","last_undegraded":"2024-09-17T15:26:21.184921+0000","last_fullsized":"2024-09-17T15:26:21.184921+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":4064,"log_dups_size":0,"ondisk_log_size":4064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16646144,"num_objects":4064,"num_object_clones":0,"num_object_copies":8128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4064,"num_write_kb":16256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'4071","reported_seq":4194,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:25.062817+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:26:25.062817+0000","last_peered":"2024-09-17T15:26:25.062817+0000","last_clean":"2024-09-17T15:26:25.062817+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:26:25.062817+0000","last_undegraded":"2024-09-17T15:26:25.062817+0000","last_fullsized":"2024-09-17T15:26:25.062817+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":4071,"log_dups_size":0,"ondisk_log_size":4071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16646627,"num_objects":4065,"num_object_clones":0,"num_object_copies":8130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4065,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1053,"num_write_kb":4212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'4350","reported_seq":4479,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:24.894513+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:26:24.894513+0000","last_peered":"2024-09-17T15:26:24.894513+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:26:24.894513+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":4350,"log_dups_size":0,"ondisk_log_size":4350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17797603,"num_objects":4346,"num_object_clones":0,"num_object_copies":8692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4346,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":849,"num_write_kb":3396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":76},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'4277","reported_seq":4410,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:25.065029+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:26:25.065029+0000","last_peered":"2024-09-17T15:26:25.065029+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:26:25.065029+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":4277,"log_dups_size":0,"ondisk_log_size":4277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17494982,"num_objects":4273,"num_object_clones":0,"num_object_copies":8546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4273,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1046,"num_write_kb":4184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":99},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'4240","reported_seq":4361,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:24.907054+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:26:24.907054+0000","last_peered":"2024-09-17T15:26:24.907054+0000","last_clean":"2024-09-17T15:26:24.907054+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:26:24.907054+0000","last_undegraded":"2024-09-17T15:26:24.907054+0000","last_fullsized":"2024-09-17T15:26:24.907054+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":4240,"log_dups_size":0,"ondisk_log_size":4240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17358848,"num_objects":4238,"num_object_clones":0,"num_object_copies":8476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1194,"num_write_kb":4776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"38'4077","reported_seq":4193,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-17T15:26:21.183272+0000","last_change":"2024-09-17T15:26:20.172430+0000","last_active":"2024-09-17T15:26:20.172430+0000","last_peered":"2024-09-17T15:26:20.155646+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:15.132800+0000","last_became_peered":"2024-09-17T15:26:15.132800+0000","last_unstale":"2024-09-17T15:26:21.183272+0000","last_undegraded":"2024-09-17T15:26:21.183272+0000","last_fullsized":"2024-09-17T15:26:21.183272+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":4077,"log_dups_size":0,"ondisk_log_size":4077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16671203,"num_objects":4071,"num_object_clones":0,"num_object_copies":8142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4071,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2077,"num_write_kb":8308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":62,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T15:26:21.185047+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:21.185047+0000","last_peered":"2024-09-17T15:26:21.185047+0000","last_clean":"2024-09-17T15:26:21.185047+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:21.185047+0000","last_undegraded":"2024-09-17T15:26:21.185047+0000","last_fullsized":"2024-09-17T15:26:21.185047+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'4107","reported_seq":4230,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:25.065881+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:26:25.065881+0000","last_peered":"2024-09-17T15:26:25.065881+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:26:25.065881+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":4107,"log_dups_size":0,"ondisk_log_size":4107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16789504,"num_objects":4099,"num_object_clones":0,"num_object_copies":8198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4099,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":703,"num_write_kb":2812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":89},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":97,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412553+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412553+0000","last_peered":"2024-09-17T15:26:22.412553+0000","last_clean":"2024-09-17T15:26:22.412553+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412553+0000","last_undegraded":"2024-09-17T15:26:22.412553+0000","last_fullsized":"2024-09-17T15:26:22.412553+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"38'4081","reported_seq":4190,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-17T15:26:21.184771+0000","last_change":"2024-09-17T15:26:20.169734+0000","last_active":"2024-09-17T15:26:20.169734+0000","last_peered":"2024-09-17T15:26:20.152702+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:15.137344+0000","last_became_peered":"2024-09-17T15:26:15.137344+0000","last_unstale":"2024-09-17T15:26:21.184771+0000","last_undegraded":"2024-09-17T15:26:21.184771+0000","last_fullsized":"2024-09-17T15:26:21.184771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":4081,"log_dups_size":0,"ondisk_log_size":4081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16695779,"num_objects":4077,"num_object_clones":0,"num_object_copies":8154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4077,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1976,"num_write_kb":7904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"38'4143","reported_seq":4250,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-17T15:26:21.183420+0000","last_change":"2024-09-17T15:26:20.170902+0000","last_active":"2024-09-17T15:26:20.170902+0000","last_peered":"2024-09-17T15:26:20.164921+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:15.139446+0000","last_became_peered":"2024-09-17T15:26:15.139446+0000","last_unstale":"2024-09-17T15:26:21.183420+0000","last_undegraded":"2024-09-17T15:26:21.183420+0000","last_fullsized":"2024-09-17T15:26:21.183420+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":4143,"log_dups_size":0,"ondisk_log_size":4143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16945152,"num_objects":4137,"num_object_clones":0,"num_object_copies":8274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1905,"num_write_kb":7620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'4235","reported_seq":4365,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:24.935123+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:26:24.935123+0000","last_peered":"2024-09-17T15:26:24.935123+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:26:24.935123+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":4235,"log_dups_size":0,"ondisk_log_size":4235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17338368,"num_objects":4233,"num_object_clones":0,"num_object_copies":8466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":783,"num_write_kb":3132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":95},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'4235","reported_seq":4371,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:24.831300+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:26:24.831300+0000","last_peered":"2024-09-17T15:26:24.831300+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:26:24.831300+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":4235,"log_dups_size":0,"ondisk_log_size":4235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17313792,"num_objects":4227,"num_object_clones":0,"num_object_copies":8454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":522,"num_write_kb":2088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":70},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":62,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T15:26:21.184340+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:21.184340+0000","last_peered":"2024-09-17T15:26:21.184340+0000","last_clean":"2024-09-17T15:26:21.184340+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:21.184340+0000","last_undegraded":"2024-09-17T15:26:21.184340+0000","last_fullsized":"2024-09-17T15:26:21.184340+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'4154","reported_seq":4280,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:24.930653+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:26:24.930653+0000","last_peered":"2024-09-17T15:26:24.930653+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:26:24.930653+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":4154,"log_dups_size":0,"ondisk_log_size":4154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16998400,"num_objects":4150,"num_object_clones":0,"num_object_copies":8300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":941,"num_write_kb":3764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":87},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'4204","reported_seq":4341,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:24.825144+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:26:24.825144+0000","last_peered":"2024-09-17T15:26:24.825144+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:26:24.825144+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":4204,"log_dups_size":0,"ondisk_log_size":4204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17171398,"num_objects":4194,"num_object_clones":0,"num_object_copies":8388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":646,"num_write_kb":2584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":85},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":272634590,"num_objects":66570,"num_object_clones":0,"num_object_copies":133140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33002,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66570,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":26677,"num_write_kb":106708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":545865728,"data_stored":545710369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66642,"ondisk_log_size":66642,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051600,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":411476,"kb_used_data":263160,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":93960364,"statfs":{"total":96636764160,"available":96215412736,"internally_reserved":0,"allocated":269475840,"data_stored":269210760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":151842476},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,9],"upper_bound":1024},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724546,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52360,"kb_used_data":2612,"kb_used_omap":4,"kb_used_meta":49723,"kb_avail":94319480,"statfs":{"total":96636764160,"available":96583147520,"internally_reserved":0,"allocated":2674688,"data_stored":2357469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":50917035},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,4,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247829,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":406880,"kb_used_data":266960,"kb_used_omap":4,"kb_used_meta":139899,"kb_avail":93964960,"statfs":{"total":96636764160,"available":96220119040,"internally_reserved":0,"allocated":273367040,"data_stored":273096790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":143257256},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2,4,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":273518592,"data_stored":273482462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3211264,"data_stored":3128165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":269135872,"data_stored":269099742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:26:26.050 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:26:26.050 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'4155", 'reported_seq': 4227, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:24.894738+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:26:24.894738+0000', 'last_peered': '2024-09-17T15:26:24.894738+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:26:24.894738+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 4155, 'log_dups_size': 0, 'ondisk_log_size': 4155, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17018880, 'num_objects': 4155, 'num_object_clones': 0, 'num_object_copies': 8310, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4070, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4155, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4155, 'num_write_kb': 16620, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 85}, {'shards': '0,2', 'objects': 4070}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:26.051 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:26:26.051 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "38'4172", 'reported_seq': 4286, 'reported_epoch': 40, 'state': 'remapped+peering', 'last_fresh': '2024-09-17T15:26:21.184781+0000', 'last_change': '2024-09-17T15:26:20.172232+0000', 'last_active': '2024-09-17T15:26:20.172232+0000', 'last_peered': '2024-09-17T15:26:20.158176+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:15.138987+0000', 'last_became_peered': '2024-09-17T15:26:15.138987+0000', 'last_unstale': '2024-09-17T15:26:21.184781+0000', 'last_undegraded': '2024-09-17T15:26:21.184781+0000', 'last_fullsized': '2024-09-17T15:26:21.184781+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 4172, 'log_dups_size': 0, 'ondisk_log_size': 4172, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17072128, 'num_objects': 4168, 'num_object_clones': 0, 'num_object_copies': 8336, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4168, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2329, 'num_write_kb': 9316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:26:26.051 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:26:26.051 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'4350", 'reported_seq': 4479, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:24.894513+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:26:24.894513+0000', 'last_peered': '2024-09-17T15:26:24.894513+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:26:24.894513+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 4350, 'log_dups_size': 0, 'ondisk_log_size': 4350, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17797603, 'num_objects': 4346, 'num_object_clones': 0, 'num_object_copies': 8692, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4271, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4346, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 849, 'num_write_kb': 3396, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 76}, {'shards': '0,2', 'objects': 4268}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:26.051 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:26:26.051 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'4277", 'reported_seq': 4410, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:25.065029+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:26:25.065029+0000', 'last_peered': '2024-09-17T15:26:25.065029+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:26:25.065029+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 4277, 'log_dups_size': 0, 'ondisk_log_size': 4277, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17494982, 'num_objects': 4273, 'num_object_clones': 0, 'num_object_copies': 8546, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4174, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4273, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 1046, 'num_write_kb': 4184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 99}, {'shards': '0,2', 'objects': 4174}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:26.051 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-17T15:26:26.051 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "38'4077", 'reported_seq': 4193, 'reported_epoch': 40, 'state': 'remapped+peering', 'last_fresh': '2024-09-17T15:26:21.183272+0000', 'last_change': '2024-09-17T15:26:20.172430+0000', 'last_active': '2024-09-17T15:26:20.172430+0000', 'last_peered': '2024-09-17T15:26:20.155646+0000', 'last_clean': '2024-09-17T15:26:20.155646+0000', 'last_became_active': '2024-09-17T15:26:15.132800+0000', 'last_became_peered': '2024-09-17T15:26:15.132800+0000', 'last_unstale': '2024-09-17T15:26:21.183272+0000', 'last_undegraded': '2024-09-17T15:26:21.183272+0000', 'last_fullsized': '2024-09-17T15:26:21.183272+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2016", 'last_scrub_stamp': '2024-09-17T15:25:53.959011+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:53.959011+0000', 'objects_scrubbed': 1994, 'log_size': 4077, 'log_dups_size': 0, 'ondisk_log_size': 4077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 568, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16671203, 'num_objects': 4071, 'num_object_clones': 0, 'num_object_copies': 8142, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4071, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2077, 'num_write_kb': 8308, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:26:26.052 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:26:26.052 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'4107", 'reported_seq': 4230, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:25.065881+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:26:25.065881+0000', 'last_peered': '2024-09-17T15:26:25.065881+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:26:25.065881+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 4107, 'log_dups_size': 0, 'ondisk_log_size': 4107, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16789504, 'num_objects': 4099, 'num_object_clones': 0, 'num_object_copies': 8198, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4012, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4099, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 703, 'num_write_kb': 2812, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 89}, {'shards': '0,2', 'objects': 4010}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:26.052 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:26:26.052 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "38'4081", 'reported_seq': 4190, 'reported_epoch': 40, 'state': 'remapped+peering', 'last_fresh': '2024-09-17T15:26:21.184771+0000', 'last_change': '2024-09-17T15:26:20.169734+0000', 'last_active': '2024-09-17T15:26:20.169734+0000', 'last_peered': '2024-09-17T15:26:20.152702+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:15.137344+0000', 'last_became_peered': '2024-09-17T15:26:15.137344+0000', 'last_unstale': '2024-09-17T15:26:21.184771+0000', 'last_undegraded': '2024-09-17T15:26:21.184771+0000', 'last_fullsized': '2024-09-17T15:26:21.184771+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 4081, 'log_dups_size': 0, 'ondisk_log_size': 4081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16695779, 'num_objects': 4077, 'num_object_clones': 0, 'num_object_copies': 8154, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4077, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 1976, 'num_write_kb': 7904, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:26:26.052 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:26:26.052 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "38'4143", 'reported_seq': 4250, 'reported_epoch': 40, 'state': 'remapped+peering', 'last_fresh': '2024-09-17T15:26:21.183420+0000', 'last_change': '2024-09-17T15:26:20.170902+0000', 'last_active': '2024-09-17T15:26:20.170902+0000', 'last_peered': '2024-09-17T15:26:20.164921+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:15.139446+0000', 'last_became_peered': '2024-09-17T15:26:15.139446+0000', 'last_unstale': '2024-09-17T15:26:21.183420+0000', 'last_undegraded': '2024-09-17T15:26:21.183420+0000', 'last_fullsized': '2024-09-17T15:26:21.183420+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 4143, 'log_dups_size': 0, 'ondisk_log_size': 4143, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16945152, 'num_objects': 4137, 'num_object_clones': 0, 'num_object_copies': 8274, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4137, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1905, 'num_write_kb': 7620, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:26:26.052 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:26:26.052 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'4235", 'reported_seq': 4365, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:24.935123+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:26:24.935123+0000', 'last_peered': '2024-09-17T15:26:24.935123+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:26:24.935123+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 4235, 'log_dups_size': 0, 'ondisk_log_size': 4235, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17338368, 'num_objects': 4233, 'num_object_clones': 0, 'num_object_copies': 8466, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4233, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 783, 'num_write_kb': 3132, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 95}, {'shards': '0,2', 'objects': 4138}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:26.052 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:26:26.052 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'4235", 'reported_seq': 4371, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:24.831300+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:26:24.831300+0000', 'last_peered': '2024-09-17T15:26:24.831300+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:26:24.831300+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 4235, 'log_dups_size': 0, 'ondisk_log_size': 4235, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17313792, 'num_objects': 4227, 'num_object_clones': 0, 'num_object_copies': 8454, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4160, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4227, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 522, 'num_write_kb': 2088, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 70}, {'shards': '0,2', 'objects': 4157}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:26.053 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:26:26.053 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'4154", 'reported_seq': 4280, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:24.930653+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:26:24.930653+0000', 'last_peered': '2024-09-17T15:26:24.930653+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:26:24.930653+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 4154, 'log_dups_size': 0, 'ondisk_log_size': 4154, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16998400, 'num_objects': 4150, 'num_object_clones': 0, 'num_object_copies': 8300, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4065, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4150, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 941, 'num_write_kb': 3764, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 87}, {'shards': '0,2', 'objects': 4063}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:26.053 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:26:26.053 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'4204", 'reported_seq': 4341, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:24.825144+0000', 'last_change': '2024-09-17T15:26:22.654521+0000', 'last_active': '2024-09-17T15:26:24.825144+0000', 'last_peered': '2024-09-17T15:26:24.825144+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:26:24.825144+0000', 'last_undegraded': '2024-09-17T15:26:22.307795+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 4204, 'log_dups_size': 0, 'ondisk_log_size': 4204, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17171398, 'num_objects': 4194, 'num_object_clones': 0, 'num_object_copies': 8388, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4111, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4194, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 646, 'num_write_kb': 2584, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 85}, {'shards': '0,2', 'objects': 4109}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:26.053 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:26:26.292 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 47 20 67858 67838 5.63707 2.32031 0.0573718 0.0138506 2024-09-17T15:26:26.736 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:26:26.736 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33002/133144 objects degraded (24.787%), 8 pgs degraded","count":8},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":123,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"remapped+peering","count":4}],"num_pgs":24,"num_pools":2,"num_objects":66572,"data_bytes":272634609,"bytes_used":891613184,"bytes_avail":289018679296,"bytes_total":289910292480,"inactive_pgs_ratio":0.1666666716337204,"degraded_objects":33002,"degraded_total":133144,"degraded_ratio":0.24786697109896053,"write_bytes_sec":4551141,"read_op_per_sec":0,"write_op_per_sec":1111},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-17T15:26:23.592815+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-17T15:26:26.765 INFO:tasks.lost_unfound.ceph_manager:no progress seen, keeping timeout for now 2024-09-17T15:26:26.765 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:26:27.292 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 48 20 68445 68425 5.5674 2.29297 0.0475453 0.0140229 2024-09-17T15:26:27.332 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:26:27.332 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:26:27.356 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":95,"stamp":"2024-09-17T15:26:25.593105+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":272634609,"num_objects":66572,"num_object_clones":0,"num_object_copies":133144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33002,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66572,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":26679,"num_write_kb":106710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66645,"ondisk_log_size":66645,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":64,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":870716,"kb_used_data":532732,"kb_used_omap":13,"kb_used_meta":337906,"kb_avail":282244804,"statfs":{"total":289910292480,"available":289018679296,"internally_reserved":0,"allocated":545517568,"data_stored":544665019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":346016767},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,2,2,8,4,3,0,0,9],"upper_bound":1024},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27312128,"num_objects":6668,"num_object_clones":0,"num_object_copies":13336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33002,"num_objects_misplaced":-15364,"num_objects_unfound":0,"num_objects_dirty":6668,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6668,"num_write_kb":26672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001159"},"pg_stats":[{"pgid":"2.f","version":"41'4155","reported_seq":4227,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:24.894738+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:26:24.894738+0000","last_peered":"2024-09-17T15:26:24.894738+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:26:24.894738+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":4155,"log_dups_size":0,"ondisk_log_size":4155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17018880,"num_objects":4155,"num_object_clones":0,"num_object_copies":8310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4155,"num_write_kb":16620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":85},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'4077","reported_seq":4172,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T15:26:21.183706+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:26:21.183706+0000","last_peered":"2024-09-17T15:26:21.183706+0000","last_clean":"2024-09-17T15:26:21.183706+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:26:21.183706+0000","last_undegraded":"2024-09-17T15:26:21.183706+0000","last_fullsized":"2024-09-17T15:26:21.183706+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":4077,"log_dups_size":0,"ondisk_log_size":4077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16675782,"num_objects":4073,"num_object_clones":0,"num_object_copies":8146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4073,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2434,"num_write_kb":9736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"38'4172","reported_seq":4286,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-17T15:26:21.184781+0000","last_change":"2024-09-17T15:26:20.172232+0000","last_active":"2024-09-17T15:26:20.172232+0000","last_peered":"2024-09-17T15:26:20.158176+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:15.138987+0000","last_became_peered":"2024-09-17T15:26:15.138987+0000","last_unstale":"2024-09-17T15:26:21.184781+0000","last_undegraded":"2024-09-17T15:26:21.184781+0000","last_fullsized":"2024-09-17T15:26:21.184781+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":4172,"log_dups_size":0,"ondisk_log_size":4172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17072128,"num_objects":4168,"num_object_clones":0,"num_object_copies":8336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2329,"num_write_kb":9316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'4064","reported_seq":4126,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T15:26:21.184921+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:26:21.184921+0000","last_peered":"2024-09-17T15:26:21.184921+0000","last_clean":"2024-09-17T15:26:21.184921+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:26:21.184921+0000","last_undegraded":"2024-09-17T15:26:21.184921+0000","last_fullsized":"2024-09-17T15:26:21.184921+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":4064,"log_dups_size":0,"ondisk_log_size":4064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16646144,"num_objects":4064,"num_object_clones":0,"num_object_copies":8128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4064,"num_write_kb":16256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'4071","reported_seq":4194,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:25.062817+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:26:25.062817+0000","last_peered":"2024-09-17T15:26:25.062817+0000","last_clean":"2024-09-17T15:26:25.062817+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:26:25.062817+0000","last_undegraded":"2024-09-17T15:26:25.062817+0000","last_fullsized":"2024-09-17T15:26:25.062817+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":4071,"log_dups_size":0,"ondisk_log_size":4071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16646627,"num_objects":4065,"num_object_clones":0,"num_object_copies":8130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4065,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1053,"num_write_kb":4212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'4350","reported_seq":4479,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:24.894513+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:26:24.894513+0000","last_peered":"2024-09-17T15:26:24.894513+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:26:24.894513+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":4350,"log_dups_size":0,"ondisk_log_size":4350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17797603,"num_objects":4346,"num_object_clones":0,"num_object_copies":8692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4346,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":849,"num_write_kb":3396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":76},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'4277","reported_seq":4410,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:25.065029+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:26:25.065029+0000","last_peered":"2024-09-17T15:26:25.065029+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:26:25.065029+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":4277,"log_dups_size":0,"ondisk_log_size":4277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17494982,"num_objects":4273,"num_object_clones":0,"num_object_copies":8546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4273,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1046,"num_write_kb":4184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":99},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'4240","reported_seq":4361,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:24.907054+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:26:24.907054+0000","last_peered":"2024-09-17T15:26:24.907054+0000","last_clean":"2024-09-17T15:26:24.907054+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:26:24.907054+0000","last_undegraded":"2024-09-17T15:26:24.907054+0000","last_fullsized":"2024-09-17T15:26:24.907054+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":4240,"log_dups_size":0,"ondisk_log_size":4240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17358848,"num_objects":4238,"num_object_clones":0,"num_object_copies":8476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1194,"num_write_kb":4776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"38'4077","reported_seq":4193,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-17T15:26:21.183272+0000","last_change":"2024-09-17T15:26:20.172430+0000","last_active":"2024-09-17T15:26:20.172430+0000","last_peered":"2024-09-17T15:26:20.155646+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:15.132800+0000","last_became_peered":"2024-09-17T15:26:15.132800+0000","last_unstale":"2024-09-17T15:26:21.183272+0000","last_undegraded":"2024-09-17T15:26:21.183272+0000","last_fullsized":"2024-09-17T15:26:21.183272+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":4077,"log_dups_size":0,"ondisk_log_size":4077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16671203,"num_objects":4071,"num_object_clones":0,"num_object_copies":8142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4071,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2077,"num_write_kb":8308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":62,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T15:26:21.185047+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:21.185047+0000","last_peered":"2024-09-17T15:26:21.185047+0000","last_clean":"2024-09-17T15:26:21.185047+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:21.185047+0000","last_undegraded":"2024-09-17T15:26:21.185047+0000","last_fullsized":"2024-09-17T15:26:21.185047+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'4107","reported_seq":4230,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:25.065881+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:26:25.065881+0000","last_peered":"2024-09-17T15:26:25.065881+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:26:25.065881+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":4107,"log_dups_size":0,"ondisk_log_size":4107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16789504,"num_objects":4099,"num_object_clones":0,"num_object_copies":8198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4099,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":703,"num_write_kb":2812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":89},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":97,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412553+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412553+0000","last_peered":"2024-09-17T15:26:22.412553+0000","last_clean":"2024-09-17T15:26:22.412553+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412553+0000","last_undegraded":"2024-09-17T15:26:22.412553+0000","last_fullsized":"2024-09-17T15:26:22.412553+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"38'4081","reported_seq":4190,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-17T15:26:21.184771+0000","last_change":"2024-09-17T15:26:20.169734+0000","last_active":"2024-09-17T15:26:20.169734+0000","last_peered":"2024-09-17T15:26:20.152702+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:15.137344+0000","last_became_peered":"2024-09-17T15:26:15.137344+0000","last_unstale":"2024-09-17T15:26:21.184771+0000","last_undegraded":"2024-09-17T15:26:21.184771+0000","last_fullsized":"2024-09-17T15:26:21.184771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":4081,"log_dups_size":0,"ondisk_log_size":4081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16695779,"num_objects":4077,"num_object_clones":0,"num_object_copies":8154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4077,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1976,"num_write_kb":7904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"38'4143","reported_seq":4250,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-17T15:26:21.183420+0000","last_change":"2024-09-17T15:26:20.170902+0000","last_active":"2024-09-17T15:26:20.170902+0000","last_peered":"2024-09-17T15:26:20.164921+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:15.139446+0000","last_became_peered":"2024-09-17T15:26:15.139446+0000","last_unstale":"2024-09-17T15:26:21.183420+0000","last_undegraded":"2024-09-17T15:26:21.183420+0000","last_fullsized":"2024-09-17T15:26:21.183420+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":4143,"log_dups_size":0,"ondisk_log_size":4143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16945152,"num_objects":4137,"num_object_clones":0,"num_object_copies":8274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1905,"num_write_kb":7620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'4235","reported_seq":4365,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:24.935123+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:26:24.935123+0000","last_peered":"2024-09-17T15:26:24.935123+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:26:24.935123+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":4235,"log_dups_size":0,"ondisk_log_size":4235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17338368,"num_objects":4233,"num_object_clones":0,"num_object_copies":8466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":783,"num_write_kb":3132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":95},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'4235","reported_seq":4371,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:24.831300+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:26:24.831300+0000","last_peered":"2024-09-17T15:26:24.831300+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:26:24.831300+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":4235,"log_dups_size":0,"ondisk_log_size":4235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17313792,"num_objects":4227,"num_object_clones":0,"num_object_copies":8454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":522,"num_write_kb":2088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":70},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":62,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T15:26:21.184340+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:21.184340+0000","last_peered":"2024-09-17T15:26:21.184340+0000","last_clean":"2024-09-17T15:26:21.184340+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:21.184340+0000","last_undegraded":"2024-09-17T15:26:21.184340+0000","last_fullsized":"2024-09-17T15:26:21.184340+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'4154","reported_seq":4280,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:24.930653+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:26:24.930653+0000","last_peered":"2024-09-17T15:26:24.930653+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:26:24.930653+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":4154,"log_dups_size":0,"ondisk_log_size":4154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16998400,"num_objects":4150,"num_object_clones":0,"num_object_copies":8300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":941,"num_write_kb":3764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":87},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'4204","reported_seq":4341,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:24.825144+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:26:24.825144+0000","last_peered":"2024-09-17T15:26:24.825144+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:26:24.825144+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":4204,"log_dups_size":0,"ondisk_log_size":4204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17171398,"num_objects":4194,"num_object_clones":0,"num_object_copies":8388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":646,"num_write_kb":2584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":85},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":272634590,"num_objects":66570,"num_object_clones":0,"num_object_copies":133140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33002,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66570,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":26677,"num_write_kb":106708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":545865728,"data_stored":545710369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66642,"ondisk_log_size":66642,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051600,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":411476,"kb_used_data":263160,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":93960364,"statfs":{"total":96636764160,"available":96215412736,"internally_reserved":0,"allocated":269475840,"data_stored":269210760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":151842476},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,9],"upper_bound":1024},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724546,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52360,"kb_used_data":2612,"kb_used_omap":4,"kb_used_meta":49723,"kb_avail":94319480,"statfs":{"total":96636764160,"available":96583147520,"internally_reserved":0,"allocated":2674688,"data_stored":2357469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":50917035},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,4,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247829,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":406880,"kb_used_data":266960,"kb_used_omap":4,"kb_used_meta":139899,"kb_avail":93964960,"statfs":{"total":96636764160,"available":96220119040,"internally_reserved":0,"allocated":273367040,"data_stored":273096790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":143257256},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2,4,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":273518592,"data_stored":273482462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3211264,"data_stored":3128165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":269135872,"data_stored":269099742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:26:28.292 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 49 20 69121 69101 5.50767 2.64062 0.0197334 0.0141762 2024-09-17T15:26:29.292 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 50 20 69792 69772 5.44994 2.62109 0.0223223 0.0143264 2024-09-17T15:26:30.292 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 51 20 70342 70322 5.38521 2.14844 0.0243082 0.0144998 2024-09-17T15:26:30.359 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:26:30.932 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:26:30.933 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:26:30.959 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":97,"stamp":"2024-09-17T15:26:29.594011+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":276435697,"num_objects":67500,"num_object_clones":0,"num_object_copies":135000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":3956,"num_objects_unfound":0,"num_objects_dirty":67500,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":27607,"num_write_kb":110422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":471059,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":67573,"ondisk_log_size":67573,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":876400,"kb_used_data":546752,"kb_used_omap":13,"kb_used_meta":329586,"kb_avail":282239120,"statfs":{"total":289910292480,"available":289012858880,"internally_reserved":0,"allocated":559874048,"data_stored":559020406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":337497087},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,3,7,12,6,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":14987264,"num_objects":3659,"num_object_clones":0,"num_object_copies":7318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":-44240,"num_objects_unfound":0,"num_objects_dirty":3659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3659,"num_write_kb":14636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":471040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.415986"},"pg_stats":[{"pgid":"2.f","version":"41'4155","reported_seq":4227,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:24.894738+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:26:24.894738+0000","last_peered":"2024-09-17T15:26:24.894738+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:26:24.894738+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":4155,"log_dups_size":0,"ondisk_log_size":4155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17018880,"num_objects":4155,"num_object_clones":0,"num_object_copies":8310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4155,"num_write_kb":16620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":85},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'4224","reported_seq":4321,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:26.699779+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:26:26.699779+0000","last_peered":"2024-09-17T15:26:26.699779+0000","last_clean":"2024-09-17T15:26:26.699779+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:26:26.699779+0000","last_undegraded":"2024-09-17T15:26:26.699779+0000","last_fullsized":"2024-09-17T15:26:26.699779+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":4224,"log_dups_size":0,"ondisk_log_size":4224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17277894,"num_objects":4220,"num_object_clones":0,"num_object_copies":8440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4220,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2581,"num_write_kb":10324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'4333","reported_seq":4453,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:26.649473+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:26:26.649473+0000","last_peered":"2024-09-17T15:26:26.649473+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:26:26.649473+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":4333,"log_dups_size":0,"ondisk_log_size":4333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17731584,"num_objects":4329,"num_object_clones":0,"num_object_copies":8658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2490,"num_write_kb":9960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":161}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'4243","reported_seq":4307,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:26.688426+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:26:26.688426+0000","last_peered":"2024-09-17T15:26:26.688426+0000","last_clean":"2024-09-17T15:26:26.688426+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:26:26.688426+0000","last_undegraded":"2024-09-17T15:26:26.688426+0000","last_fullsized":"2024-09-17T15:26:26.688426+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":4243,"log_dups_size":0,"ondisk_log_size":4243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17379328,"num_objects":4243,"num_object_clones":0,"num_object_copies":8486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4243,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4243,"num_write_kb":16972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'4071","reported_seq":4194,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:25.062817+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:26:25.062817+0000","last_peered":"2024-09-17T15:26:25.062817+0000","last_clean":"2024-09-17T15:26:25.062817+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:26:25.062817+0000","last_undegraded":"2024-09-17T15:26:25.062817+0000","last_fullsized":"2024-09-17T15:26:25.062817+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":4071,"log_dups_size":0,"ondisk_log_size":4071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16646627,"num_objects":4065,"num_object_clones":0,"num_object_copies":8130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4065,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1053,"num_write_kb":4212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'4350","reported_seq":4479,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:24.894513+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:26:24.894513+0000","last_peered":"2024-09-17T15:26:24.894513+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:26:24.894513+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":4350,"log_dups_size":0,"ondisk_log_size":4350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17797603,"num_objects":4346,"num_object_clones":0,"num_object_copies":8692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4346,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":849,"num_write_kb":3396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":76},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'4277","reported_seq":4410,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:25.065029+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:26:25.065029+0000","last_peered":"2024-09-17T15:26:25.065029+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:26:25.065029+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":4277,"log_dups_size":0,"ondisk_log_size":4277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17494982,"num_objects":4273,"num_object_clones":0,"num_object_copies":8546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4273,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1046,"num_write_kb":4184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":99},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'4240","reported_seq":4361,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:24.907054+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:26:24.907054+0000","last_peered":"2024-09-17T15:26:24.907054+0000","last_clean":"2024-09-17T15:26:24.907054+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:26:24.907054+0000","last_undegraded":"2024-09-17T15:26:24.907054+0000","last_fullsized":"2024-09-17T15:26:24.907054+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":4240,"log_dups_size":0,"ondisk_log_size":4240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17358848,"num_objects":4238,"num_object_clones":0,"num_object_copies":8476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1194,"num_write_kb":4776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'4224","reported_seq":4577,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:26:26.706796+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:26:26.706796+0000","last_peered":"2024-09-17T15:26:26.706796+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:26:26.706796+0000","last_undegraded":"2024-09-17T15:26:26.706796+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":4224,"log_dups_size":0,"ondisk_log_size":4224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17273315,"num_objects":4218,"num_object_clones":0,"num_object_copies":8436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3956,"num_objects_unfound":0,"num_objects_dirty":4218,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2224,"num_write_kb":8896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":471040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3956},{"shards":"2","objects":262}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233552+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233552+0000","last_peered":"2024-09-17T15:26:22.233552+0000","last_clean":"2024-09-17T15:26:22.233552+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233552+0000","last_undegraded":"2024-09-17T15:26:22.233552+0000","last_fullsized":"2024-09-17T15:26:22.233552+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'4107","reported_seq":4230,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:25.065881+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:26:25.065881+0000","last_peered":"2024-09-17T15:26:25.065881+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:26:25.065881+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":4107,"log_dups_size":0,"ondisk_log_size":4107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16789504,"num_objects":4099,"num_object_clones":0,"num_object_copies":8198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4099,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":703,"num_write_kb":2812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":89},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":97,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412553+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412553+0000","last_peered":"2024-09-17T15:26:22.412553+0000","last_clean":"2024-09-17T15:26:22.412553+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412553+0000","last_undegraded":"2024-09-17T15:26:22.412553+0000","last_fullsized":"2024-09-17T15:26:22.412553+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'4218","reported_seq":4333,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:26.678751+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:26:26.678751+0000","last_peered":"2024-09-17T15:26:26.678751+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:26:26.678751+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":4218,"log_dups_size":0,"ondisk_log_size":4218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17256931,"num_objects":4214,"num_object_clones":0,"num_object_copies":8428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4214,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2113,"num_write_kb":8452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":137}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'4300","reported_seq":4413,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:26.657448+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:26:26.657448+0000","last_peered":"2024-09-17T15:26:26.657448+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:26:26.657448+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":4300,"log_dups_size":0,"ondisk_log_size":4300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17588224,"num_objects":4294,"num_object_clones":0,"num_object_copies":8588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2062,"num_write_kb":8248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":157}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'4235","reported_seq":4365,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:24.935123+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:26:24.935123+0000","last_peered":"2024-09-17T15:26:24.935123+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:26:24.935123+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":4235,"log_dups_size":0,"ondisk_log_size":4235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17338368,"num_objects":4233,"num_object_clones":0,"num_object_copies":8466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":783,"num_write_kb":3132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":95},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'4235","reported_seq":4371,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:24.831300+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:26:24.831300+0000","last_peered":"2024-09-17T15:26:24.831300+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:26:24.831300+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":4235,"log_dups_size":0,"ondisk_log_size":4235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17313792,"num_objects":4227,"num_object_clones":0,"num_object_copies":8454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":522,"num_write_kb":2088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":70},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229013+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229013+0000","last_peered":"2024-09-17T15:26:22.229013+0000","last_clean":"2024-09-17T15:26:22.229013+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229013+0000","last_undegraded":"2024-09-17T15:26:22.229013+0000","last_fullsized":"2024-09-17T15:26:22.229013+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'4154","reported_seq":4280,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:24.930653+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:26:24.930653+0000","last_peered":"2024-09-17T15:26:24.930653+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:26:24.930653+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":4154,"log_dups_size":0,"ondisk_log_size":4154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16998400,"num_objects":4150,"num_object_clones":0,"num_object_copies":8300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":941,"num_write_kb":3764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":87},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'4204","reported_seq":4341,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:24.825144+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:26:24.825144+0000","last_peered":"2024-09-17T15:26:24.825144+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:26:24.825144+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":4204,"log_dups_size":0,"ondisk_log_size":4204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17171398,"num_objects":4194,"num_object_clones":0,"num_object_copies":8388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":646,"num_write_kb":2584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":85},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":276435678,"num_objects":67498,"num_object_clones":0,"num_object_copies":134996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":3956,"num_objects_unfound":0,"num_objects_dirty":67498,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":27605,"num_write_kb":110420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":471040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":560885760,"data_stored":560730401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":67570,"ondisk_log_size":67570,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051601,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":403632,"kb_used_data":267808,"kb_used_omap":4,"kb_used_meta":135803,"kb_avail":93968208,"statfs":{"total":96636764160,"available":96223444992,"internally_reserved":0,"allocated":274235392,"data_stored":273969219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":139062956},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,0,6,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724547,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65888,"kb_used_data":11984,"kb_used_omap":4,"kb_used_meta":53883,"kb_avail":94305952,"statfs":{"total":96636764160,"available":96569294848,"internally_reserved":0,"allocated":12271616,"data_stored":11954397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":55176875},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,5,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247829,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":406880,"kb_used_data":266960,"kb_used_omap":4,"kb_used_meta":139899,"kb_avail":93964960,"statfs":{"total":96636764160,"available":96220119040,"internally_reserved":0,"allocated":273367040,"data_stored":273096790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":143257256},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2,4,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":273518592,"data_stored":273482462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":13205504,"data_stored":13122405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":274161664,"data_stored":274125534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:26:30.960 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:26:31.292 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 52 20 70934 70914 5.32612 2.3125 0.0264182 0.0146593 2024-09-17T15:26:31.505 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:26:31.506 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:26:31.537 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":97,"stamp":"2024-09-17T15:26:29.594011+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":276435697,"num_objects":67500,"num_object_clones":0,"num_object_copies":135000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":3956,"num_objects_unfound":0,"num_objects_dirty":67500,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":27607,"num_write_kb":110422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":471059,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":67573,"ondisk_log_size":67573,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":876400,"kb_used_data":546752,"kb_used_omap":13,"kb_used_meta":329586,"kb_avail":282239120,"statfs":{"total":289910292480,"available":289012858880,"internally_reserved":0,"allocated":559874048,"data_stored":559020406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":337497087},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,3,7,12,6,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":14987264,"num_objects":3659,"num_object_clones":0,"num_object_copies":7318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":-44240,"num_objects_unfound":0,"num_objects_dirty":3659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3659,"num_write_kb":14636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":471040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.415986"},"pg_stats":[{"pgid":"2.f","version":"41'4155","reported_seq":4227,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:24.894738+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:26:24.894738+0000","last_peered":"2024-09-17T15:26:24.894738+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:26:24.894738+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":4155,"log_dups_size":0,"ondisk_log_size":4155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17018880,"num_objects":4155,"num_object_clones":0,"num_object_copies":8310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4155,"num_write_kb":16620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":85},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'4224","reported_seq":4321,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:26.699779+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:26:26.699779+0000","last_peered":"2024-09-17T15:26:26.699779+0000","last_clean":"2024-09-17T15:26:26.699779+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:26:26.699779+0000","last_undegraded":"2024-09-17T15:26:26.699779+0000","last_fullsized":"2024-09-17T15:26:26.699779+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":4224,"log_dups_size":0,"ondisk_log_size":4224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17277894,"num_objects":4220,"num_object_clones":0,"num_object_copies":8440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4220,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2581,"num_write_kb":10324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'4333","reported_seq":4453,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:26.649473+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:26:26.649473+0000","last_peered":"2024-09-17T15:26:26.649473+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:26:26.649473+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":4333,"log_dups_size":0,"ondisk_log_size":4333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17731584,"num_objects":4329,"num_object_clones":0,"num_object_copies":8658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2490,"num_write_kb":9960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":161}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'4243","reported_seq":4307,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:26.688426+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:26:26.688426+0000","last_peered":"2024-09-17T15:26:26.688426+0000","last_clean":"2024-09-17T15:26:26.688426+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:26:26.688426+0000","last_undegraded":"2024-09-17T15:26:26.688426+0000","last_fullsized":"2024-09-17T15:26:26.688426+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":4243,"log_dups_size":0,"ondisk_log_size":4243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17379328,"num_objects":4243,"num_object_clones":0,"num_object_copies":8486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4243,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4243,"num_write_kb":16972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'4071","reported_seq":4194,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:25.062817+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:26:25.062817+0000","last_peered":"2024-09-17T15:26:25.062817+0000","last_clean":"2024-09-17T15:26:25.062817+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:26:25.062817+0000","last_undegraded":"2024-09-17T15:26:25.062817+0000","last_fullsized":"2024-09-17T15:26:25.062817+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":4071,"log_dups_size":0,"ondisk_log_size":4071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16646627,"num_objects":4065,"num_object_clones":0,"num_object_copies":8130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4065,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1053,"num_write_kb":4212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'4350","reported_seq":4479,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:24.894513+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:26:24.894513+0000","last_peered":"2024-09-17T15:26:24.894513+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:26:24.894513+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":4350,"log_dups_size":0,"ondisk_log_size":4350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17797603,"num_objects":4346,"num_object_clones":0,"num_object_copies":8692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4346,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":849,"num_write_kb":3396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":76},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'4277","reported_seq":4410,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:25.065029+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:26:25.065029+0000","last_peered":"2024-09-17T15:26:25.065029+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:26:25.065029+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":4277,"log_dups_size":0,"ondisk_log_size":4277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17494982,"num_objects":4273,"num_object_clones":0,"num_object_copies":8546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4273,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1046,"num_write_kb":4184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":99},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'4240","reported_seq":4361,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:24.907054+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:26:24.907054+0000","last_peered":"2024-09-17T15:26:24.907054+0000","last_clean":"2024-09-17T15:26:24.907054+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:26:24.907054+0000","last_undegraded":"2024-09-17T15:26:24.907054+0000","last_fullsized":"2024-09-17T15:26:24.907054+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":4240,"log_dups_size":0,"ondisk_log_size":4240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17358848,"num_objects":4238,"num_object_clones":0,"num_object_copies":8476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1194,"num_write_kb":4776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'4224","reported_seq":4577,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:26:26.706796+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:26:26.706796+0000","last_peered":"2024-09-17T15:26:26.706796+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:26:26.706796+0000","last_undegraded":"2024-09-17T15:26:26.706796+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":4224,"log_dups_size":0,"ondisk_log_size":4224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17273315,"num_objects":4218,"num_object_clones":0,"num_object_copies":8436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3956,"num_objects_unfound":0,"num_objects_dirty":4218,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2224,"num_write_kb":8896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":471040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3956},{"shards":"2","objects":262}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233552+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233552+0000","last_peered":"2024-09-17T15:26:22.233552+0000","last_clean":"2024-09-17T15:26:22.233552+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233552+0000","last_undegraded":"2024-09-17T15:26:22.233552+0000","last_fullsized":"2024-09-17T15:26:22.233552+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'4107","reported_seq":4230,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:25.065881+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:26:25.065881+0000","last_peered":"2024-09-17T15:26:25.065881+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:26:25.065881+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":4107,"log_dups_size":0,"ondisk_log_size":4107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16789504,"num_objects":4099,"num_object_clones":0,"num_object_copies":8198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4099,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":703,"num_write_kb":2812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":89},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":97,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412553+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412553+0000","last_peered":"2024-09-17T15:26:22.412553+0000","last_clean":"2024-09-17T15:26:22.412553+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412553+0000","last_undegraded":"2024-09-17T15:26:22.412553+0000","last_fullsized":"2024-09-17T15:26:22.412553+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'4218","reported_seq":4333,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:26.678751+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:26:26.678751+0000","last_peered":"2024-09-17T15:26:26.678751+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:26:26.678751+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":4218,"log_dups_size":0,"ondisk_log_size":4218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17256931,"num_objects":4214,"num_object_clones":0,"num_object_copies":8428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4214,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2113,"num_write_kb":8452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":137}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'4300","reported_seq":4413,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:26.657448+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:26:26.657448+0000","last_peered":"2024-09-17T15:26:26.657448+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:26:26.657448+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":4300,"log_dups_size":0,"ondisk_log_size":4300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17588224,"num_objects":4294,"num_object_clones":0,"num_object_copies":8588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2062,"num_write_kb":8248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":157}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'4235","reported_seq":4365,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:24.935123+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:26:24.935123+0000","last_peered":"2024-09-17T15:26:24.935123+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:26:24.935123+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":4235,"log_dups_size":0,"ondisk_log_size":4235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17338368,"num_objects":4233,"num_object_clones":0,"num_object_copies":8466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":783,"num_write_kb":3132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":95},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'4235","reported_seq":4371,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:24.831300+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:26:24.831300+0000","last_peered":"2024-09-17T15:26:24.831300+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:26:24.831300+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":4235,"log_dups_size":0,"ondisk_log_size":4235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17313792,"num_objects":4227,"num_object_clones":0,"num_object_copies":8454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":522,"num_write_kb":2088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":70},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229013+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229013+0000","last_peered":"2024-09-17T15:26:22.229013+0000","last_clean":"2024-09-17T15:26:22.229013+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229013+0000","last_undegraded":"2024-09-17T15:26:22.229013+0000","last_fullsized":"2024-09-17T15:26:22.229013+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'4154","reported_seq":4280,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:24.930653+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:26:24.930653+0000","last_peered":"2024-09-17T15:26:24.930653+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:26:24.930653+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":4154,"log_dups_size":0,"ondisk_log_size":4154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16998400,"num_objects":4150,"num_object_clones":0,"num_object_copies":8300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":941,"num_write_kb":3764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":87},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'4204","reported_seq":4341,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:24.825144+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:26:24.825144+0000","last_peered":"2024-09-17T15:26:24.825144+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:26:24.825144+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":4204,"log_dups_size":0,"ondisk_log_size":4204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17171398,"num_objects":4194,"num_object_clones":0,"num_object_copies":8388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":646,"num_write_kb":2584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":85},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":276435678,"num_objects":67498,"num_object_clones":0,"num_object_copies":134996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":3956,"num_objects_unfound":0,"num_objects_dirty":67498,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":27605,"num_write_kb":110420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":471040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":560885760,"data_stored":560730401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":67570,"ondisk_log_size":67570,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051601,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":403632,"kb_used_data":267808,"kb_used_omap":4,"kb_used_meta":135803,"kb_avail":93968208,"statfs":{"total":96636764160,"available":96223444992,"internally_reserved":0,"allocated":274235392,"data_stored":273969219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":139062956},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,0,6,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724547,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65888,"kb_used_data":11984,"kb_used_omap":4,"kb_used_meta":53883,"kb_avail":94305952,"statfs":{"total":96636764160,"available":96569294848,"internally_reserved":0,"allocated":12271616,"data_stored":11954397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":55176875},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,5,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247829,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":406880,"kb_used_data":266960,"kb_used_omap":4,"kb_used_meta":139899,"kb_avail":93964960,"statfs":{"total":96636764160,"available":96220119040,"internally_reserved":0,"allocated":273367040,"data_stored":273096790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":143257256},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2,4,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":273518592,"data_stored":273482462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":13205504,"data_stored":13122405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":274161664,"data_stored":274125534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:26:31.538 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:26:31.538 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'4155", 'reported_seq': 4227, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:24.894738+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:26:24.894738+0000', 'last_peered': '2024-09-17T15:26:24.894738+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:26:24.894738+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 4155, 'log_dups_size': 0, 'ondisk_log_size': 4155, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17018880, 'num_objects': 4155, 'num_object_clones': 0, 'num_object_copies': 8310, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4070, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4155, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4155, 'num_write_kb': 16620, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 85}, {'shards': '0,2', 'objects': 4070}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:31.538 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:26:31.538 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'4333", 'reported_seq': 4453, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:26.649473+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:26:26.649473+0000', 'last_peered': '2024-09-17T15:26:26.649473+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:26:26.649473+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 4333, 'log_dups_size': 0, 'ondisk_log_size': 4333, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17731584, 'num_objects': 4329, 'num_object_clones': 0, 'num_object_copies': 8658, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4169, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4329, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2490, 'num_write_kb': 9960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4168}, {'shards': '2', 'objects': 161}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:26:31.538 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:26:31.538 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'4350", 'reported_seq': 4479, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:24.894513+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:26:24.894513+0000', 'last_peered': '2024-09-17T15:26:24.894513+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:26:24.894513+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 4350, 'log_dups_size': 0, 'ondisk_log_size': 4350, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17797603, 'num_objects': 4346, 'num_object_clones': 0, 'num_object_copies': 8692, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4271, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4346, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 849, 'num_write_kb': 3396, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 76}, {'shards': '0,2', 'objects': 4268}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:31.538 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:26:31.539 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'4277", 'reported_seq': 4410, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:25.065029+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:26:25.065029+0000', 'last_peered': '2024-09-17T15:26:25.065029+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:26:25.065029+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 4277, 'log_dups_size': 0, 'ondisk_log_size': 4277, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17494982, 'num_objects': 4273, 'num_object_clones': 0, 'num_object_copies': 8546, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4174, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4273, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 1046, 'num_write_kb': 4184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 99}, {'shards': '0,2', 'objects': 4174}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:31.539 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-17T15:26:31.539 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "41'4224", 'reported_seq': 4577, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:26:26.706796+0000', 'last_change': '2024-09-17T15:26:22.588242+0000', 'last_active': '2024-09-17T15:26:26.706796+0000', 'last_peered': '2024-09-17T15:26:26.706796+0000', 'last_clean': '2024-09-17T15:26:20.155646+0000', 'last_became_active': '2024-09-17T15:26:22.557742+0000', 'last_became_peered': '2024-09-17T15:26:22.557742+0000', 'last_unstale': '2024-09-17T15:26:26.706796+0000', 'last_undegraded': '2024-09-17T15:26:26.706796+0000', 'last_fullsized': '2024-09-17T15:26:22.188850+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2016", 'last_scrub_stamp': '2024-09-17T15:25:53.959011+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:53.959011+0000', 'objects_scrubbed': 1994, 'log_size': 4224, 'log_dups_size': 0, 'ondisk_log_size': 4224, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 568, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17273315, 'num_objects': 4218, 'num_object_clones': 0, 'num_object_copies': 8436, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3956, 'num_objects_unfound': 0, 'num_objects_dirty': 4218, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2224, 'num_write_kb': 8896, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 115, 'num_bytes_recovered': 471040, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3956}, {'shards': '2', 'objects': 262}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:26:31.539 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:26:31.539 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'4107", 'reported_seq': 4230, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:25.065881+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:26:25.065881+0000', 'last_peered': '2024-09-17T15:26:25.065881+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:26:25.065881+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 4107, 'log_dups_size': 0, 'ondisk_log_size': 4107, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16789504, 'num_objects': 4099, 'num_object_clones': 0, 'num_object_copies': 8198, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4012, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4099, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 703, 'num_write_kb': 2812, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 89}, {'shards': '0,2', 'objects': 4010}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:31.539 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:26:31.539 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'4218", 'reported_seq': 4333, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:26.678751+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:26:26.678751+0000', 'last_peered': '2024-09-17T15:26:26.678751+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:26:26.678751+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 4218, 'log_dups_size': 0, 'ondisk_log_size': 4218, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17256931, 'num_objects': 4214, 'num_object_clones': 0, 'num_object_copies': 8428, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4078, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4214, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2113, 'num_write_kb': 8452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4077}, {'shards': '2', 'objects': 137}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:26:31.539 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:26:31.539 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'4300", 'reported_seq': 4413, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:26.657448+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:26:26.657448+0000', 'last_peered': '2024-09-17T15:26:26.657448+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:26:26.657448+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 4300, 'log_dups_size': 0, 'ondisk_log_size': 4300, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17588224, 'num_objects': 4294, 'num_object_clones': 0, 'num_object_copies': 8588, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4294, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2062, 'num_write_kb': 8248, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4137}, {'shards': '2', 'objects': 157}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:26:31.539 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:26:31.540 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'4235", 'reported_seq': 4365, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:24.935123+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:26:24.935123+0000', 'last_peered': '2024-09-17T15:26:24.935123+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:26:24.935123+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 4235, 'log_dups_size': 0, 'ondisk_log_size': 4235, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17338368, 'num_objects': 4233, 'num_object_clones': 0, 'num_object_copies': 8466, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4233, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 783, 'num_write_kb': 3132, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 95}, {'shards': '0,2', 'objects': 4138}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:31.540 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:26:31.540 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'4235", 'reported_seq': 4371, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:24.831300+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:26:24.831300+0000', 'last_peered': '2024-09-17T15:26:24.831300+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:26:24.831300+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 4235, 'log_dups_size': 0, 'ondisk_log_size': 4235, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17313792, 'num_objects': 4227, 'num_object_clones': 0, 'num_object_copies': 8454, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4160, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4227, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 522, 'num_write_kb': 2088, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 70}, {'shards': '0,2', 'objects': 4157}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:31.540 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:26:31.540 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'4154", 'reported_seq': 4280, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:24.930653+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:26:24.930653+0000', 'last_peered': '2024-09-17T15:26:24.930653+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:26:24.930653+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 4154, 'log_dups_size': 0, 'ondisk_log_size': 4154, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16998400, 'num_objects': 4150, 'num_object_clones': 0, 'num_object_copies': 8300, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4065, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4150, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 941, 'num_write_kb': 3764, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 87}, {'shards': '0,2', 'objects': 4063}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:31.540 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:26:31.540 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'4204", 'reported_seq': 4341, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:24.825144+0000', 'last_change': '2024-09-17T15:26:22.654521+0000', 'last_active': '2024-09-17T15:26:24.825144+0000', 'last_peered': '2024-09-17T15:26:24.825144+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:26:24.825144+0000', 'last_undegraded': '2024-09-17T15:26:22.307795+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 4204, 'log_dups_size': 0, 'ondisk_log_size': 4204, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17171398, 'num_objects': 4194, 'num_object_clones': 0, 'num_object_copies': 8388, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4111, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4194, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 646, 'num_write_kb': 2584, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 85}, {'shards': '0,2', 'objects': 4109}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:31.540 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:26:32.238 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:26:32.238 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45388/135000 objects degraded (33.621%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":129,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":67500,"data_bytes":276435697,"bytes_used":897433600,"bytes_avail":289012858880,"bytes_total":289910292480,"degraded_objects":45388,"degraded_total":135000,"degraded_ratio":0.33620740740740745,"misplaced_objects":3956,"misplaced_total":135000,"misplaced_ratio":0.029303703703703703,"recovering_objects_per_sec":13,"recovering_bytes_per_sec":55969,"recovering_keys_per_sec":0,"num_objects_recovered":115,"num_bytes_recovered":471040,"num_keys_recovered":0,"write_bytes_sec":1780808,"read_op_per_sec":0,"write_op_per_sec":434},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-17T15:26:23.592815+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-17T15:26:32.269 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:26:32.269 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:26:32.292 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 53 20 71517 71497 5.2686 2.27734 0.0697501 0.0148184 2024-09-17T15:26:32.821 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:26:32.821 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:26:32.847 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":98,"stamp":"2024-09-17T15:26:31.594705+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":284545777,"num_objects":69480,"num_object_clones":0,"num_object_copies":138960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":3956,"num_objects_unfound":0,"num_objects_dirty":69480,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":29587,"num_write_kb":118342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":471059,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69553,"ondisk_log_size":69553,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":894200,"kb_used_data":556232,"kb_used_omap":13,"kb_used_meta":337906,"kb_avail":282221320,"statfs":{"total":289910292480,"available":288994631680,"internally_reserved":0,"allocated":569581568,"data_stored":568727926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":346016767},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,3,8,9,9,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23097344,"num_objects":5639,"num_object_clones":0,"num_object_copies":11278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":-44240,"num_objects_unfound":0,"num_objects_dirty":5639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5639,"num_write_kb":22556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":471040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002363"},"pg_stats":[{"pgid":"2.f","version":"41'4359","reported_seq":4431,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:29.829931+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:26:29.829931+0000","last_peered":"2024-09-17T15:26:29.829931+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:26:29.829931+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":4359,"log_dups_size":0,"ondisk_log_size":4359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17854464,"num_objects":4359,"num_object_clones":0,"num_object_copies":8718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4359,"num_write_kb":17436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":289},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'4224","reported_seq":4321,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:26.699779+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:26:26.699779+0000","last_peered":"2024-09-17T15:26:26.699779+0000","last_clean":"2024-09-17T15:26:26.699779+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:26:26.699779+0000","last_undegraded":"2024-09-17T15:26:26.699779+0000","last_fullsized":"2024-09-17T15:26:26.699779+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":4224,"log_dups_size":0,"ondisk_log_size":4224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17277894,"num_objects":4220,"num_object_clones":0,"num_object_copies":8440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4220,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2581,"num_write_kb":10324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'4333","reported_seq":4453,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:26.649473+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:26:26.649473+0000","last_peered":"2024-09-17T15:26:26.649473+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:26:26.649473+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":4333,"log_dups_size":0,"ondisk_log_size":4333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17731584,"num_objects":4329,"num_object_clones":0,"num_object_copies":8658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2490,"num_write_kb":9960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":161}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'4243","reported_seq":4307,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:26.688426+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:26:26.688426+0000","last_peered":"2024-09-17T15:26:26.688426+0000","last_clean":"2024-09-17T15:26:26.688426+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:26:26.688426+0000","last_undegraded":"2024-09-17T15:26:26.688426+0000","last_fullsized":"2024-09-17T15:26:26.688426+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":4243,"log_dups_size":0,"ondisk_log_size":4243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17379328,"num_objects":4243,"num_object_clones":0,"num_object_copies":8486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4243,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4243,"num_write_kb":16972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'4274","reported_seq":4398,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:29.880259+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:26:29.880259+0000","last_peered":"2024-09-17T15:26:29.880259+0000","last_clean":"2024-09-17T15:26:29.880259+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:26:29.880259+0000","last_undegraded":"2024-09-17T15:26:29.880259+0000","last_fullsized":"2024-09-17T15:26:29.880259+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":4274,"log_dups_size":0,"ondisk_log_size":4274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17478115,"num_objects":4268,"num_object_clones":0,"num_object_copies":8536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4268,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1256,"num_write_kb":5024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'4571","reported_seq":4700,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:29.853133+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:26:29.853133+0000","last_peered":"2024-09-17T15:26:29.853133+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:26:29.853133+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":4571,"log_dups_size":0,"ondisk_log_size":4571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18702819,"num_objects":4567,"num_object_clones":0,"num_object_copies":9134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4567,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1070,"num_write_kb":4280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":297},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'4468","reported_seq":4601,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:29.848025+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:26:29.848025+0000","last_peered":"2024-09-17T15:26:29.848025+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:26:29.848025+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":4468,"log_dups_size":0,"ondisk_log_size":4468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18277318,"num_objects":4464,"num_object_clones":0,"num_object_copies":8928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4464,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1237,"num_write_kb":4948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":290},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'4439","reported_seq":4561,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:29.879955+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:26:29.879955+0000","last_peered":"2024-09-17T15:26:29.879955+0000","last_clean":"2024-09-17T15:26:29.879955+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:26:29.879955+0000","last_undegraded":"2024-09-17T15:26:29.879955+0000","last_fullsized":"2024-09-17T15:26:29.879955+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":4439,"log_dups_size":0,"ondisk_log_size":4439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18173952,"num_objects":4437,"num_object_clones":0,"num_object_copies":8874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1393,"num_write_kb":5572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'4224","reported_seq":4577,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:26:26.706796+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:26:26.706796+0000","last_peered":"2024-09-17T15:26:26.706796+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:26:26.706796+0000","last_undegraded":"2024-09-17T15:26:26.706796+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":4224,"log_dups_size":0,"ondisk_log_size":4224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17273315,"num_objects":4218,"num_object_clones":0,"num_object_copies":8436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3956,"num_objects_unfound":0,"num_objects_dirty":4218,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2224,"num_write_kb":8896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":471040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3956},{"shards":"2","objects":262}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233552+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233552+0000","last_peered":"2024-09-17T15:26:22.233552+0000","last_clean":"2024-09-17T15:26:22.233552+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233552+0000","last_undegraded":"2024-09-17T15:26:22.233552+0000","last_fullsized":"2024-09-17T15:26:22.233552+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'4287","reported_seq":4412,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:29.859384+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:26:29.859384+0000","last_peered":"2024-09-17T15:26:29.859384+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:26:29.859384+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":4287,"log_dups_size":0,"ondisk_log_size":4287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17526784,"num_objects":4279,"num_object_clones":0,"num_object_copies":8558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":883,"num_write_kb":3532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":269},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":97,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412553+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412553+0000","last_peered":"2024-09-17T15:26:22.412553+0000","last_clean":"2024-09-17T15:26:22.412553+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412553+0000","last_undegraded":"2024-09-17T15:26:22.412553+0000","last_fullsized":"2024-09-17T15:26:22.412553+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'4218","reported_seq":4333,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:26.678751+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:26:26.678751+0000","last_peered":"2024-09-17T15:26:26.678751+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:26:26.678751+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":4218,"log_dups_size":0,"ondisk_log_size":4218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17256931,"num_objects":4214,"num_object_clones":0,"num_object_copies":8428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4214,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2113,"num_write_kb":8452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":137}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'4300","reported_seq":4413,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:26.657448+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:26:26.657448+0000","last_peered":"2024-09-17T15:26:26.657448+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:26:26.657448+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":4300,"log_dups_size":0,"ondisk_log_size":4300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17588224,"num_objects":4294,"num_object_clones":0,"num_object_copies":8588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2062,"num_write_kb":8248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":157}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'4425","reported_seq":4555,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:30.089765+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:26:30.089765+0000","last_peered":"2024-09-17T15:26:30.089765+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:26:30.089765+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":4425,"log_dups_size":0,"ondisk_log_size":4425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18116608,"num_objects":4423,"num_object_clones":0,"num_object_copies":8846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4423,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":973,"num_write_kb":3892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":285},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'4407","reported_seq":4539,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:30.100544+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:26:30.100544+0000","last_peered":"2024-09-17T15:26:30.100544+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:26:30.100544+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":4407,"log_dups_size":0,"ondisk_log_size":4407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18018304,"num_objects":4399,"num_object_clones":0,"num_object_copies":8798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":694,"num_write_kb":2776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":242},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229013+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229013+0000","last_peered":"2024-09-17T15:26:22.229013+0000","last_clean":"2024-09-17T15:26:22.229013+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229013+0000","last_undegraded":"2024-09-17T15:26:22.229013+0000","last_fullsized":"2024-09-17T15:26:22.229013+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'4378","reported_seq":4499,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:30.064008+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:26:30.064008+0000","last_peered":"2024-09-17T15:26:30.064008+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:26:30.064008+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":4378,"log_dups_size":0,"ondisk_log_size":4378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17915904,"num_objects":4374,"num_object_clones":0,"num_object_copies":8748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4374,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1165,"num_write_kb":4660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":311},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'4400","reported_seq":4537,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:30.098114+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:26:30.098114+0000","last_peered":"2024-09-17T15:26:30.098114+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:26:30.098114+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":4400,"log_dups_size":0,"ondisk_log_size":4400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17974214,"num_objects":4390,"num_object_clones":0,"num_object_copies":8780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":842,"num_write_kb":3368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":281},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":284545758,"num_objects":69478,"num_object_clones":0,"num_object_copies":138956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":3956,"num_objects_unfound":0,"num_objects_dirty":69478,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":29585,"num_write_kb":118340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":471040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":570589184,"data_stored":570433825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69550,"ondisk_log_size":69550,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051601,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":403632,"kb_used_data":267808,"kb_used_omap":4,"kb_used_meta":135803,"kb_avail":93968208,"statfs":{"total":96636764160,"available":96223444992,"internally_reserved":0,"allocated":274235392,"data_stored":273969219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":139062956},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,0,6,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724547,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65888,"kb_used_data":11984,"kb_used_omap":4,"kb_used_meta":53883,"kb_avail":94305952,"statfs":{"total":96636764160,"available":96569294848,"internally_reserved":0,"allocated":12271616,"data_stored":11954397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":55176875},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,5,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247830,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":424680,"kb_used_data":276440,"kb_used_omap":4,"kb_used_meta":148219,"kb_avail":93947160,"statfs":{"total":96636764160,"available":96201891840,"internally_reserved":0,"allocated":283074560,"data_stored":282804310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":151776936},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,1,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":283222016,"data_stored":283185886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":13205504,"data_stored":13122405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":274161664,"data_stored":274125534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:26:33.292 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 54 19 72140 72121 5.21617 2.4375 0.0385202 0.0149698 2024-09-17T15:26:34.295 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 55 20 72806 72786 5.16856 2.59766 0.0277453 0.0151067 2024-09-17T15:26:35.295 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 56 20 73457 73437 5.12148 2.54297 0.0140632 0.0152456 2024-09-17T15:26:35.849 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:26:36.295 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 57 20 74089 74069 5.07494 2.46875 0.0450527 0.0153853 2024-09-17T15:26:36.417 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:26:36.418 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:26:36.443 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2024-09-17T15:26:35.595533+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":297190129,"num_objects":72567,"num_object_clones":0,"num_object_copies":145134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":3799,"num_objects_unfound":0,"num_objects_dirty":72567,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":32674,"num_write_kb":130690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":273,"num_bytes_recovered":1114131,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72640,"ondisk_log_size":72640,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":902308,"kb_used_data":580980,"kb_used_omap":13,"kb_used_meta":321266,"kb_avail":282213212,"statfs":{"total":289910292480,"available":288986329088,"internally_reserved":0,"allocated":594923520,"data_stored":594069878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":328977407},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,4,7,16,6,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28868608,"num_objects":7048,"num_object_clones":0,"num_object_copies":14096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":-29034,"num_objects_unfound":0,"num_objects_dirty":7048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7048,"num_write_kb":28192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":1114112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002751"},"pg_stats":[{"pgid":"2.f","version":"41'4551","reported_seq":4623,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:35.046639+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:26:35.046639+0000","last_peered":"2024-09-17T15:26:35.046639+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:26:35.046639+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":4551,"log_dups_size":0,"ondisk_log_size":4551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18640896,"num_objects":4551,"num_object_clones":0,"num_object_copies":9102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4551,"num_write_kb":18204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":481},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'4409","reported_seq":4505,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:31.723801+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:26:31.723801+0000","last_peered":"2024-09-17T15:26:31.723801+0000","last_clean":"2024-09-17T15:26:31.723801+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:26:31.723801+0000","last_undegraded":"2024-09-17T15:26:31.723801+0000","last_fullsized":"2024-09-17T15:26:31.723801+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":4409,"log_dups_size":0,"ondisk_log_size":4409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18035654,"num_objects":4405,"num_object_clones":0,"num_object_copies":8810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4405,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2766,"num_write_kb":11064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'4548","reported_seq":4665,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:31.692519+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:26:31.692519+0000","last_peered":"2024-09-17T15:26:31.692519+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:26:31.692519+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":4548,"log_dups_size":0,"ondisk_log_size":4548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18612224,"num_objects":4544,"num_object_clones":0,"num_object_copies":9088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2705,"num_write_kb":10820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":376}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'4442","reported_seq":4506,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:31.647390+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:26:31.647390+0000","last_peered":"2024-09-17T15:26:31.647390+0000","last_clean":"2024-09-17T15:26:31.647390+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:26:31.647390+0000","last_undegraded":"2024-09-17T15:26:31.647390+0000","last_fullsized":"2024-09-17T15:26:31.647390+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":4442,"log_dups_size":0,"ondisk_log_size":4442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18194432,"num_objects":4442,"num_object_clones":0,"num_object_copies":8884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4442,"num_write_kb":17768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'4483","reported_seq":4606,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:35.129078+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:26:35.129078+0000","last_peered":"2024-09-17T15:26:35.129078+0000","last_clean":"2024-09-17T15:26:35.129078+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:26:35.129078+0000","last_undegraded":"2024-09-17T15:26:35.129078+0000","last_fullsized":"2024-09-17T15:26:35.129078+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":4483,"log_dups_size":0,"ondisk_log_size":4483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18334179,"num_objects":4477,"num_object_clones":0,"num_object_copies":8954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4477,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1465,"num_write_kb":5860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'4766","reported_seq":4895,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:35.108815+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:26:35.108815+0000","last_peered":"2024-09-17T15:26:35.108815+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:26:35.108815+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":4766,"log_dups_size":0,"ondisk_log_size":4766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19501539,"num_objects":4762,"num_object_clones":0,"num_object_copies":9524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4762,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1265,"num_write_kb":5060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":492},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'4640","reported_seq":4771,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:35.049809+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:26:35.049809+0000","last_peered":"2024-09-17T15:26:35.049809+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:26:35.049809+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":4640,"log_dups_size":0,"ondisk_log_size":4640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18981830,"num_objects":4636,"num_object_clones":0,"num_object_copies":9272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4636,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1409,"num_write_kb":5636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":462},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'4631","reported_seq":4753,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:35.124823+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:26:35.124823+0000","last_peered":"2024-09-17T15:26:35.124823+0000","last_clean":"2024-09-17T15:26:35.124823+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:26:35.124823+0000","last_undegraded":"2024-09-17T15:26:35.124823+0000","last_fullsized":"2024-09-17T15:26:35.124823+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":4631,"log_dups_size":0,"ondisk_log_size":4631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18960384,"num_objects":4629,"num_object_clones":0,"num_object_copies":9258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1585,"num_write_kb":6340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'4413","reported_seq":5079,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:26:31.727199+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:26:31.727199+0000","last_peered":"2024-09-17T15:26:31.727199+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:26:31.727199+0000","last_undegraded":"2024-09-17T15:26:31.727199+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":4413,"log_dups_size":0,"ondisk_log_size":4413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18047459,"num_objects":4407,"num_object_clones":0,"num_object_copies":8814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3799,"num_objects_unfound":0,"num_objects_dirty":4407,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2413,"num_write_kb":9652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":1114112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3799},{"shards":"2","objects":608}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233552+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233552+0000","last_peered":"2024-09-17T15:26:22.233552+0000","last_clean":"2024-09-17T15:26:22.233552+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233552+0000","last_undegraded":"2024-09-17T15:26:22.233552+0000","last_fullsized":"2024-09-17T15:26:22.233552+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'4482","reported_seq":4607,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:35.114030+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:26:35.114030+0000","last_peered":"2024-09-17T15:26:35.114030+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:26:35.114030+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":4482,"log_dups_size":0,"ondisk_log_size":4482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18325504,"num_objects":4474,"num_object_clones":0,"num_object_copies":8948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1078,"num_write_kb":4312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":464},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":97,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412553+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412553+0000","last_peered":"2024-09-17T15:26:22.412553+0000","last_clean":"2024-09-17T15:26:22.412553+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412553+0000","last_undegraded":"2024-09-17T15:26:22.412553+0000","last_fullsized":"2024-09-17T15:26:22.412553+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'4381","reported_seq":4493,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:31.706019+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:26:31.706019+0000","last_peered":"2024-09-17T15:26:31.706019+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:26:31.706019+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":4381,"log_dups_size":0,"ondisk_log_size":4381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17924579,"num_objects":4377,"num_object_clones":0,"num_object_copies":8754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4377,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2276,"num_write_kb":9104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":300}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'4468","reported_seq":4581,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:31.676677+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:26:31.676677+0000","last_peered":"2024-09-17T15:26:31.676677+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:26:31.676677+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":4468,"log_dups_size":0,"ondisk_log_size":4468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18276352,"num_objects":4462,"num_object_clones":0,"num_object_copies":8924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2230,"num_write_kb":8920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":325}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'4638","reported_seq":4768,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:35.110011+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:26:35.110011+0000","last_peered":"2024-09-17T15:26:35.110011+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:26:35.110011+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":4638,"log_dups_size":0,"ondisk_log_size":4638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18989056,"num_objects":4636,"num_object_clones":0,"num_object_copies":9272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1186,"num_write_kb":4744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":498},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'4617","reported_seq":4753,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:35.124516+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:26:35.124516+0000","last_peered":"2024-09-17T15:26:35.124516+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:26:35.124516+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":4617,"log_dups_size":0,"ondisk_log_size":4617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18878464,"num_objects":4609,"num_object_clones":0,"num_object_copies":9218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":904,"num_write_kb":3616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":452},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229013+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229013+0000","last_peered":"2024-09-17T15:26:22.229013+0000","last_clean":"2024-09-17T15:26:22.229013+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229013+0000","last_undegraded":"2024-09-17T15:26:22.229013+0000","last_fullsized":"2024-09-17T15:26:22.229013+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'4575","reported_seq":4701,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:35.062617+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:26:35.062617+0000","last_peered":"2024-09-17T15:26:35.062617+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:26:35.062617+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":4575,"log_dups_size":0,"ondisk_log_size":4575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18722816,"num_objects":4571,"num_object_clones":0,"num_object_copies":9142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1362,"num_write_kb":5448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":508},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'4593","reported_seq":4730,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:35.108835+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:26:35.108835+0000","last_peered":"2024-09-17T15:26:35.108835+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:26:35.108835+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":4593,"log_dups_size":0,"ondisk_log_size":4593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18764742,"num_objects":4583,"num_object_clones":0,"num_object_copies":9166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1035,"num_write_kb":4140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":474},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":297190110,"num_objects":72565,"num_object_clones":0,"num_object_copies":145130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":3799,"num_objects_unfound":0,"num_objects_dirty":72565,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":32672,"num_write_kb":130688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":1114112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":596353024,"data_stored":596197665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72637,"ondisk_log_size":72637,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051602,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":413808,"kb_used_data":273824,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93958032,"statfs":{"total":96636764160,"available":96213024768,"internally_reserved":0,"allocated":280395776,"data_stored":280129603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":143322796},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724548,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67252,"kb_used_data":21668,"kb_used_omap":4,"kb_used_meta":45563,"kb_avail":94304588,"statfs":{"total":96636764160,"available":96567898112,"internally_reserved":0,"allocated":22188032,"data_stored":21870813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":46657195},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,5,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247831,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":421248,"kb_used_data":285488,"kb_used_omap":4,"kb_used_meta":135739,"kb_avail":93950592,"statfs":{"total":96636764160,"available":96205406208,"internally_reserved":0,"allocated":292339712,"data_stored":292069462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":138997416},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,4,1,11,3,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":292884480,"data_stored":292848350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":23076864,"data_stored":22993765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":280391680,"data_stored":280355550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:26:36.444 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:26:37.005 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:26:37.006 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:26:37.030 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2024-09-17T15:26:35.595533+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":297190129,"num_objects":72567,"num_object_clones":0,"num_object_copies":145134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":3799,"num_objects_unfound":0,"num_objects_dirty":72567,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":32674,"num_write_kb":130690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":273,"num_bytes_recovered":1114131,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72640,"ondisk_log_size":72640,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":902308,"kb_used_data":580980,"kb_used_omap":13,"kb_used_meta":321266,"kb_avail":282213212,"statfs":{"total":289910292480,"available":288986329088,"internally_reserved":0,"allocated":594923520,"data_stored":594069878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":328977407},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,4,7,16,6,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28868608,"num_objects":7048,"num_object_clones":0,"num_object_copies":14096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":-29034,"num_objects_unfound":0,"num_objects_dirty":7048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7048,"num_write_kb":28192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":1114112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002751"},"pg_stats":[{"pgid":"2.f","version":"41'4551","reported_seq":4623,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:35.046639+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:26:35.046639+0000","last_peered":"2024-09-17T15:26:35.046639+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:26:35.046639+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":4551,"log_dups_size":0,"ondisk_log_size":4551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18640896,"num_objects":4551,"num_object_clones":0,"num_object_copies":9102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4551,"num_write_kb":18204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":481},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'4409","reported_seq":4505,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:31.723801+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:26:31.723801+0000","last_peered":"2024-09-17T15:26:31.723801+0000","last_clean":"2024-09-17T15:26:31.723801+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:26:31.723801+0000","last_undegraded":"2024-09-17T15:26:31.723801+0000","last_fullsized":"2024-09-17T15:26:31.723801+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":4409,"log_dups_size":0,"ondisk_log_size":4409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18035654,"num_objects":4405,"num_object_clones":0,"num_object_copies":8810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4405,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2766,"num_write_kb":11064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'4548","reported_seq":4665,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:31.692519+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:26:31.692519+0000","last_peered":"2024-09-17T15:26:31.692519+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:26:31.692519+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":4548,"log_dups_size":0,"ondisk_log_size":4548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18612224,"num_objects":4544,"num_object_clones":0,"num_object_copies":9088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2705,"num_write_kb":10820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":376}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'4442","reported_seq":4506,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:31.647390+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:26:31.647390+0000","last_peered":"2024-09-17T15:26:31.647390+0000","last_clean":"2024-09-17T15:26:31.647390+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:26:31.647390+0000","last_undegraded":"2024-09-17T15:26:31.647390+0000","last_fullsized":"2024-09-17T15:26:31.647390+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":4442,"log_dups_size":0,"ondisk_log_size":4442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18194432,"num_objects":4442,"num_object_clones":0,"num_object_copies":8884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4442,"num_write_kb":17768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'4483","reported_seq":4606,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:35.129078+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:26:35.129078+0000","last_peered":"2024-09-17T15:26:35.129078+0000","last_clean":"2024-09-17T15:26:35.129078+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:26:35.129078+0000","last_undegraded":"2024-09-17T15:26:35.129078+0000","last_fullsized":"2024-09-17T15:26:35.129078+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":4483,"log_dups_size":0,"ondisk_log_size":4483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18334179,"num_objects":4477,"num_object_clones":0,"num_object_copies":8954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4477,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1465,"num_write_kb":5860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'4766","reported_seq":4895,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:35.108815+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:26:35.108815+0000","last_peered":"2024-09-17T15:26:35.108815+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:26:35.108815+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":4766,"log_dups_size":0,"ondisk_log_size":4766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19501539,"num_objects":4762,"num_object_clones":0,"num_object_copies":9524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4762,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1265,"num_write_kb":5060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":492},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'4640","reported_seq":4771,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:35.049809+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:26:35.049809+0000","last_peered":"2024-09-17T15:26:35.049809+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:26:35.049809+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":4640,"log_dups_size":0,"ondisk_log_size":4640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18981830,"num_objects":4636,"num_object_clones":0,"num_object_copies":9272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4636,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1409,"num_write_kb":5636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":462},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'4631","reported_seq":4753,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:35.124823+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:26:35.124823+0000","last_peered":"2024-09-17T15:26:35.124823+0000","last_clean":"2024-09-17T15:26:35.124823+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:26:35.124823+0000","last_undegraded":"2024-09-17T15:26:35.124823+0000","last_fullsized":"2024-09-17T15:26:35.124823+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":4631,"log_dups_size":0,"ondisk_log_size":4631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18960384,"num_objects":4629,"num_object_clones":0,"num_object_copies":9258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1585,"num_write_kb":6340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'4413","reported_seq":5079,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:26:31.727199+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:26:31.727199+0000","last_peered":"2024-09-17T15:26:31.727199+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:26:31.727199+0000","last_undegraded":"2024-09-17T15:26:31.727199+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":4413,"log_dups_size":0,"ondisk_log_size":4413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18047459,"num_objects":4407,"num_object_clones":0,"num_object_copies":8814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3799,"num_objects_unfound":0,"num_objects_dirty":4407,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2413,"num_write_kb":9652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":1114112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3799},{"shards":"2","objects":608}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233552+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233552+0000","last_peered":"2024-09-17T15:26:22.233552+0000","last_clean":"2024-09-17T15:26:22.233552+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233552+0000","last_undegraded":"2024-09-17T15:26:22.233552+0000","last_fullsized":"2024-09-17T15:26:22.233552+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'4482","reported_seq":4607,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:35.114030+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:26:35.114030+0000","last_peered":"2024-09-17T15:26:35.114030+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:26:35.114030+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":4482,"log_dups_size":0,"ondisk_log_size":4482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18325504,"num_objects":4474,"num_object_clones":0,"num_object_copies":8948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1078,"num_write_kb":4312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":464},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":97,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412553+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412553+0000","last_peered":"2024-09-17T15:26:22.412553+0000","last_clean":"2024-09-17T15:26:22.412553+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412553+0000","last_undegraded":"2024-09-17T15:26:22.412553+0000","last_fullsized":"2024-09-17T15:26:22.412553+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'4381","reported_seq":4493,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:31.706019+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:26:31.706019+0000","last_peered":"2024-09-17T15:26:31.706019+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:26:31.706019+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":4381,"log_dups_size":0,"ondisk_log_size":4381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17924579,"num_objects":4377,"num_object_clones":0,"num_object_copies":8754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4377,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2276,"num_write_kb":9104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":300}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'4468","reported_seq":4581,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:31.676677+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:26:31.676677+0000","last_peered":"2024-09-17T15:26:31.676677+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:26:31.676677+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":4468,"log_dups_size":0,"ondisk_log_size":4468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18276352,"num_objects":4462,"num_object_clones":0,"num_object_copies":8924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2230,"num_write_kb":8920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":325}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'4638","reported_seq":4768,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:35.110011+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:26:35.110011+0000","last_peered":"2024-09-17T15:26:35.110011+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:26:35.110011+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":4638,"log_dups_size":0,"ondisk_log_size":4638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18989056,"num_objects":4636,"num_object_clones":0,"num_object_copies":9272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1186,"num_write_kb":4744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":498},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'4617","reported_seq":4753,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:35.124516+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:26:35.124516+0000","last_peered":"2024-09-17T15:26:35.124516+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:26:35.124516+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":4617,"log_dups_size":0,"ondisk_log_size":4617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18878464,"num_objects":4609,"num_object_clones":0,"num_object_copies":9218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":904,"num_write_kb":3616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":452},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229013+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229013+0000","last_peered":"2024-09-17T15:26:22.229013+0000","last_clean":"2024-09-17T15:26:22.229013+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229013+0000","last_undegraded":"2024-09-17T15:26:22.229013+0000","last_fullsized":"2024-09-17T15:26:22.229013+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'4575","reported_seq":4701,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:35.062617+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:26:35.062617+0000","last_peered":"2024-09-17T15:26:35.062617+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:26:35.062617+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":4575,"log_dups_size":0,"ondisk_log_size":4575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18722816,"num_objects":4571,"num_object_clones":0,"num_object_copies":9142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1362,"num_write_kb":5448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":508},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'4593","reported_seq":4730,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:35.108835+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:26:35.108835+0000","last_peered":"2024-09-17T15:26:35.108835+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:26:35.108835+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":4593,"log_dups_size":0,"ondisk_log_size":4593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18764742,"num_objects":4583,"num_object_clones":0,"num_object_copies":9166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1035,"num_write_kb":4140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":474},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":297190110,"num_objects":72565,"num_object_clones":0,"num_object_copies":145130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":3799,"num_objects_unfound":0,"num_objects_dirty":72565,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":32672,"num_write_kb":130688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":1114112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":596353024,"data_stored":596197665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72637,"ondisk_log_size":72637,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051602,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":413808,"kb_used_data":273824,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93958032,"statfs":{"total":96636764160,"available":96213024768,"internally_reserved":0,"allocated":280395776,"data_stored":280129603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":143322796},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724548,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67252,"kb_used_data":21668,"kb_used_omap":4,"kb_used_meta":45563,"kb_avail":94304588,"statfs":{"total":96636764160,"available":96567898112,"internally_reserved":0,"allocated":22188032,"data_stored":21870813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":46657195},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,5,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247831,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":421248,"kb_used_data":285488,"kb_used_omap":4,"kb_used_meta":135739,"kb_avail":93950592,"statfs":{"total":96636764160,"available":96205406208,"internally_reserved":0,"allocated":292339712,"data_stored":292069462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":138997416},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,4,1,11,3,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":292884480,"data_stored":292848350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":23076864,"data_stored":22993765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":280391680,"data_stored":280355550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:26:37.031 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:26:37.031 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'4551", 'reported_seq': 4623, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:35.046639+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:26:35.046639+0000', 'last_peered': '2024-09-17T15:26:35.046639+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:26:35.046639+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 4551, 'log_dups_size': 0, 'ondisk_log_size': 4551, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18640896, 'num_objects': 4551, 'num_object_clones': 0, 'num_object_copies': 9102, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4070, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4551, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4551, 'num_write_kb': 18204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 481}, {'shards': '0,2', 'objects': 4070}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:37.031 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:26:37.032 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'4548", 'reported_seq': 4665, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:31.692519+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:26:31.692519+0000', 'last_peered': '2024-09-17T15:26:31.692519+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:26:31.692519+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 4548, 'log_dups_size': 0, 'ondisk_log_size': 4548, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18612224, 'num_objects': 4544, 'num_object_clones': 0, 'num_object_copies': 9088, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4169, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4544, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2705, 'num_write_kb': 10820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4168}, {'shards': '2', 'objects': 376}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:26:37.032 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:26:37.032 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'4766", 'reported_seq': 4895, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:35.108815+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:26:35.108815+0000', 'last_peered': '2024-09-17T15:26:35.108815+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:26:35.108815+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 4766, 'log_dups_size': 0, 'ondisk_log_size': 4766, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19501539, 'num_objects': 4762, 'num_object_clones': 0, 'num_object_copies': 9524, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4271, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4762, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 1265, 'num_write_kb': 5060, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 492}, {'shards': '0,2', 'objects': 4268}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:37.032 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:26:37.032 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'4640", 'reported_seq': 4771, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:35.049809+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:26:35.049809+0000', 'last_peered': '2024-09-17T15:26:35.049809+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:26:35.049809+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 4640, 'log_dups_size': 0, 'ondisk_log_size': 4640, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18981830, 'num_objects': 4636, 'num_object_clones': 0, 'num_object_copies': 9272, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4174, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4636, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 1409, 'num_write_kb': 5636, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 462}, {'shards': '0,2', 'objects': 4174}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:37.032 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-17T15:26:37.032 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "41'4413", 'reported_seq': 5079, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:26:31.727199+0000', 'last_change': '2024-09-17T15:26:22.588242+0000', 'last_active': '2024-09-17T15:26:31.727199+0000', 'last_peered': '2024-09-17T15:26:31.727199+0000', 'last_clean': '2024-09-17T15:26:20.155646+0000', 'last_became_active': '2024-09-17T15:26:22.557742+0000', 'last_became_peered': '2024-09-17T15:26:22.557742+0000', 'last_unstale': '2024-09-17T15:26:31.727199+0000', 'last_undegraded': '2024-09-17T15:26:31.727199+0000', 'last_fullsized': '2024-09-17T15:26:22.188850+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2016", 'last_scrub_stamp': '2024-09-17T15:25:53.959011+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:53.959011+0000', 'objects_scrubbed': 1994, 'log_size': 4413, 'log_dups_size': 0, 'ondisk_log_size': 4413, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 568, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18047459, 'num_objects': 4407, 'num_object_clones': 0, 'num_object_copies': 8814, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3799, 'num_objects_unfound': 0, 'num_objects_dirty': 4407, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2413, 'num_write_kb': 9652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 272, 'num_bytes_recovered': 1114112, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3799}, {'shards': '2', 'objects': 608}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:26:37.032 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:26:37.032 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'4482", 'reported_seq': 4607, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:35.114030+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:26:35.114030+0000', 'last_peered': '2024-09-17T15:26:35.114030+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:26:35.114030+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 4482, 'log_dups_size': 0, 'ondisk_log_size': 4482, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18325504, 'num_objects': 4474, 'num_object_clones': 0, 'num_object_copies': 8948, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4012, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4474, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1078, 'num_write_kb': 4312, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 464}, {'shards': '0,2', 'objects': 4010}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:37.032 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:26:37.033 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'4381", 'reported_seq': 4493, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:31.706019+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:26:31.706019+0000', 'last_peered': '2024-09-17T15:26:31.706019+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:26:31.706019+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 4381, 'log_dups_size': 0, 'ondisk_log_size': 4381, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17924579, 'num_objects': 4377, 'num_object_clones': 0, 'num_object_copies': 8754, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4078, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4377, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2276, 'num_write_kb': 9104, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4077}, {'shards': '2', 'objects': 300}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:26:37.033 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:26:37.033 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'4468", 'reported_seq': 4581, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:31.676677+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:26:31.676677+0000', 'last_peered': '2024-09-17T15:26:31.676677+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:26:31.676677+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 4468, 'log_dups_size': 0, 'ondisk_log_size': 4468, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18276352, 'num_objects': 4462, 'num_object_clones': 0, 'num_object_copies': 8924, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4462, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2230, 'num_write_kb': 8920, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4137}, {'shards': '2', 'objects': 325}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:26:37.033 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:26:37.033 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'4638", 'reported_seq': 4768, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:35.110011+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:26:35.110011+0000', 'last_peered': '2024-09-17T15:26:35.110011+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:26:35.110011+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 4638, 'log_dups_size': 0, 'ondisk_log_size': 4638, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18989056, 'num_objects': 4636, 'num_object_clones': 0, 'num_object_copies': 9272, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4636, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1186, 'num_write_kb': 4744, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 498}, {'shards': '0,2', 'objects': 4138}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:37.033 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:26:37.033 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'4617", 'reported_seq': 4753, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:35.124516+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:26:35.124516+0000', 'last_peered': '2024-09-17T15:26:35.124516+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:26:35.124516+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 4617, 'log_dups_size': 0, 'ondisk_log_size': 4617, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18878464, 'num_objects': 4609, 'num_object_clones': 0, 'num_object_copies': 9218, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4160, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4609, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 904, 'num_write_kb': 3616, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 452}, {'shards': '0,2', 'objects': 4157}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:37.033 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:26:37.033 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'4575", 'reported_seq': 4701, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:35.062617+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:26:35.062617+0000', 'last_peered': '2024-09-17T15:26:35.062617+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:26:35.062617+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 4575, 'log_dups_size': 0, 'ondisk_log_size': 4575, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18722816, 'num_objects': 4571, 'num_object_clones': 0, 'num_object_copies': 9142, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4065, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4571, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1362, 'num_write_kb': 5448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 508}, {'shards': '0,2', 'objects': 4063}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:37.033 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:26:37.034 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'4593", 'reported_seq': 4730, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:35.108835+0000', 'last_change': '2024-09-17T15:26:22.654521+0000', 'last_active': '2024-09-17T15:26:35.108835+0000', 'last_peered': '2024-09-17T15:26:35.108835+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:26:35.108835+0000', 'last_undegraded': '2024-09-17T15:26:22.307795+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 4593, 'log_dups_size': 0, 'ondisk_log_size': 4593, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18764742, 'num_objects': 4583, 'num_object_clones': 0, 'num_object_copies': 9166, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4111, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4583, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1035, 'num_write_kb': 4140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 474}, {'shards': '0,2', 'objects': 4109}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:37.034 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:26:37.295 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 58 20 74657 74637 5.0257 2.21875 0.0184319 0.015536 2024-09-17T15:26:37.740 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:26:37.740 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45388/145134 objects degraded (31.273%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":134,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":72567,"data_bytes":297190129,"bytes_used":923963392,"bytes_avail":288986329088,"bytes_total":289910292480,"degraded_objects":45388,"degraded_total":145134,"degraded_ratio":0.31273168244518856,"misplaced_objects":3799,"misplaced_total":145134,"misplaced_ratio":0.026175809941157827,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":92821,"recovering_keys_per_sec":0,"num_objects_recovered":272,"num_bytes_recovered":1114112,"num_keys_recovered":0,"write_bytes_sec":2405165,"read_op_per_sec":0,"write_op_per_sec":587},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-17T15:26:23.592815+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-17T15:26:37.761 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:26:37.762 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:26:38.295 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 59 20 75223 75203 4.97799 2.21094 0.0296745 0.0156854 2024-09-17T15:26:38.306 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:26:38.306 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:26:38.334 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":101,"stamp":"2024-09-17T15:26:37.595987+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":302068465,"num_objects":73758,"num_object_clones":0,"num_object_copies":147516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":3667,"num_objects_unfound":0,"num_objects_dirty":73758,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":33865,"num_write_kb":135454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":405,"num_bytes_recovered":1654803,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":73831,"ondisk_log_size":73831,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":912900,"kb_used_data":587408,"kb_used_omap":13,"kb_used_meta":325426,"kb_avail":282202620,"statfs":{"total":289910292480,"available":288975482880,"internally_reserved":0,"allocated":601505792,"data_stored":600652150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":333237247},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,5,7,19,4,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29433856,"num_objects":7186,"num_object_clones":0,"num_object_copies":14372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12386,"num_objects_misplaced":3667,"num_objects_unfound":0,"num_objects_dirty":7186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7186,"num_write_kb":28744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":404,"num_bytes_recovered":1654784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002882"},"pg_stats":[{"pgid":"2.f","version":"41'4551","reported_seq":4623,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:35.046639+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:26:35.046639+0000","last_peered":"2024-09-17T15:26:35.046639+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:26:35.046639+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":4551,"log_dups_size":0,"ondisk_log_size":4551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18640896,"num_objects":4551,"num_object_clones":0,"num_object_copies":9102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4551,"num_write_kb":18204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":481},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'4601","reported_seq":4698,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:36.743022+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:26:36.743022+0000","last_peered":"2024-09-17T15:26:36.743022+0000","last_clean":"2024-09-17T15:26:36.743022+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:26:36.743022+0000","last_undegraded":"2024-09-17T15:26:36.743022+0000","last_fullsized":"2024-09-17T15:26:36.743022+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":4601,"log_dups_size":0,"ondisk_log_size":4601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18822086,"num_objects":4597,"num_object_clones":0,"num_object_copies":9194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4597,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2958,"num_write_kb":11832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'4739","reported_seq":4858,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:36.718115+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:26:36.718115+0000","last_peered":"2024-09-17T15:26:36.718115+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:26:36.718115+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":4739,"log_dups_size":0,"ondisk_log_size":4739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19394560,"num_objects":4735,"num_object_clones":0,"num_object_copies":9470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2896,"num_write_kb":11584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":567}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'4656","reported_seq":4720,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:36.742782+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:26:36.742782+0000","last_peered":"2024-09-17T15:26:36.742782+0000","last_clean":"2024-09-17T15:26:36.742782+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:26:36.742782+0000","last_undegraded":"2024-09-17T15:26:36.742782+0000","last_fullsized":"2024-09-17T15:26:36.742782+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":4656,"log_dups_size":0,"ondisk_log_size":4656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19070976,"num_objects":4656,"num_object_clones":0,"num_object_copies":9312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4656,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4656,"num_write_kb":18624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'4483","reported_seq":4606,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:35.129078+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:26:35.129078+0000","last_peered":"2024-09-17T15:26:35.129078+0000","last_clean":"2024-09-17T15:26:35.129078+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:26:35.129078+0000","last_undegraded":"2024-09-17T15:26:35.129078+0000","last_fullsized":"2024-09-17T15:26:35.129078+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":4483,"log_dups_size":0,"ondisk_log_size":4483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18334179,"num_objects":4477,"num_object_clones":0,"num_object_copies":8954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4477,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1465,"num_write_kb":5860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'4766","reported_seq":4895,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:35.108815+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:26:35.108815+0000","last_peered":"2024-09-17T15:26:35.108815+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:26:35.108815+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":4766,"log_dups_size":0,"ondisk_log_size":4766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19501539,"num_objects":4762,"num_object_clones":0,"num_object_copies":9524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4762,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1265,"num_write_kb":5060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":492},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'4640","reported_seq":4771,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:35.049809+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:26:35.049809+0000","last_peered":"2024-09-17T15:26:35.049809+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:26:35.049809+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":4640,"log_dups_size":0,"ondisk_log_size":4640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18981830,"num_objects":4636,"num_object_clones":0,"num_object_copies":9272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4636,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1409,"num_write_kb":5636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":462},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'4631","reported_seq":4753,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:35.124823+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:26:35.124823+0000","last_peered":"2024-09-17T15:26:35.124823+0000","last_clean":"2024-09-17T15:26:35.124823+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:26:35.124823+0000","last_undegraded":"2024-09-17T15:26:35.124823+0000","last_fullsized":"2024-09-17T15:26:35.124823+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":4631,"log_dups_size":0,"ondisk_log_size":4631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18960384,"num_objects":4629,"num_object_clones":0,"num_object_copies":9258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1585,"num_write_kb":6340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'4614","reported_seq":5545,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:26:36.742739+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:26:36.742739+0000","last_peered":"2024-09-17T15:26:36.742739+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:26:36.742739+0000","last_undegraded":"2024-09-17T15:26:36.742739+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":4614,"log_dups_size":0,"ondisk_log_size":4614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18870755,"num_objects":4608,"num_object_clones":0,"num_object_copies":9216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3667,"num_objects_unfound":0,"num_objects_dirty":4608,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2614,"num_write_kb":10456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":404,"num_bytes_recovered":1654784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3667},{"shards":"2","objects":941}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233552+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233552+0000","last_peered":"2024-09-17T15:26:22.233552+0000","last_clean":"2024-09-17T15:26:22.233552+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233552+0000","last_undegraded":"2024-09-17T15:26:22.233552+0000","last_fullsized":"2024-09-17T15:26:22.233552+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'4482","reported_seq":4607,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:35.114030+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:26:35.114030+0000","last_peered":"2024-09-17T15:26:35.114030+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:26:35.114030+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":4482,"log_dups_size":0,"ondisk_log_size":4482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18325504,"num_objects":4474,"num_object_clones":0,"num_object_copies":8948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1078,"num_write_kb":4312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":464},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":97,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412553+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412553+0000","last_peered":"2024-09-17T15:26:22.412553+0000","last_clean":"2024-09-17T15:26:22.412553+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412553+0000","last_undegraded":"2024-09-17T15:26:22.412553+0000","last_fullsized":"2024-09-17T15:26:22.412553+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'4584","reported_seq":4699,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:36.718250+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:26:36.718250+0000","last_peered":"2024-09-17T15:26:36.718250+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:26:36.718250+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":4584,"log_dups_size":0,"ondisk_log_size":4584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18756067,"num_objects":4580,"num_object_clones":0,"num_object_copies":9160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4580,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2479,"num_write_kb":9916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":503}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'4658","reported_seq":4771,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:36.705278+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:26:36.705278+0000","last_peered":"2024-09-17T15:26:36.705278+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:26:36.705278+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":4658,"log_dups_size":0,"ondisk_log_size":4658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19054592,"num_objects":4652,"num_object_clones":0,"num_object_copies":9304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2420,"num_write_kb":9680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":515}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'4638","reported_seq":4768,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:35.110011+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:26:35.110011+0000","last_peered":"2024-09-17T15:26:35.110011+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:26:35.110011+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":4638,"log_dups_size":0,"ondisk_log_size":4638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18989056,"num_objects":4636,"num_object_clones":0,"num_object_copies":9272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1186,"num_write_kb":4744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":498},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'4617","reported_seq":4753,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:35.124516+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:26:35.124516+0000","last_peered":"2024-09-17T15:26:35.124516+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:26:35.124516+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":4617,"log_dups_size":0,"ondisk_log_size":4617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18878464,"num_objects":4609,"num_object_clones":0,"num_object_copies":9218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":904,"num_write_kb":3616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":452},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229013+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229013+0000","last_peered":"2024-09-17T15:26:22.229013+0000","last_clean":"2024-09-17T15:26:22.229013+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229013+0000","last_undegraded":"2024-09-17T15:26:22.229013+0000","last_fullsized":"2024-09-17T15:26:22.229013+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'4575","reported_seq":4701,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:35.062617+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:26:35.062617+0000","last_peered":"2024-09-17T15:26:35.062617+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:26:35.062617+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":4575,"log_dups_size":0,"ondisk_log_size":4575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18722816,"num_objects":4571,"num_object_clones":0,"num_object_copies":9142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1362,"num_write_kb":5448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":508},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'4593","reported_seq":4730,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:35.108835+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:26:35.108835+0000","last_peered":"2024-09-17T15:26:35.108835+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:26:35.108835+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":4593,"log_dups_size":0,"ondisk_log_size":4593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18764742,"num_objects":4583,"num_object_clones":0,"num_object_copies":9166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1035,"num_write_kb":4140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":474},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":302068446,"num_objects":73756,"num_object_clones":0,"num_object_copies":147512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":3667,"num_objects_unfound":0,"num_objects_dirty":73756,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":33863,"num_write_kb":135452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":404,"num_bytes_recovered":1654784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":602816512,"data_stored":602661153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":73828,"ondisk_log_size":73828,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051603,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":424400,"kb_used_data":280252,"kb_used_omap":4,"kb_used_meta":144123,"kb_avail":93947440,"statfs":{"total":96636764160,"available":96202178560,"internally_reserved":0,"allocated":286978048,"data_stored":286711875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":147582636},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,6],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724548,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67252,"kb_used_data":21668,"kb_used_omap":4,"kb_used_meta":45563,"kb_avail":94304588,"statfs":{"total":96636764160,"available":96567898112,"internally_reserved":0,"allocated":22188032,"data_stored":21870813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":46657195},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,5,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247831,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":421248,"kb_used_data":285488,"kb_used_omap":4,"kb_used_meta":135739,"kb_avail":93950592,"statfs":{"total":96636764160,"available":96205406208,"internally_reserved":0,"allocated":292339712,"data_stored":292069462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":138997416},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,4,1,11,3,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":292884480,"data_stored":292848350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":23076864,"data_stored":22993765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":286855168,"data_stored":286819038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:26:39.295 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout:2024-09-17T15:26:39.295767+0000 min lat: 0.00187689 max lat: 2.64156 avg lat: 0.0158175 2024-09-17T15:26:39.295 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-17T15:26:39.295 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 60 20 75859 75839 4.93643 2.48438 0.0611908 0.0158175 2024-09-17T15:26:40.295 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 61 20 76521 76501 4.8979 2.58594 0.0194404 0.0159413 2024-09-17T15:26:41.295 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 62 20 77172 77152 4.85992 2.54297 0.0262754 0.0160685 2024-09-17T15:26:41.336 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:26:41.914 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:26:41.916 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:26:41.943 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":103,"stamp":"2024-09-17T15:26:41.596900+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":309875441,"num_objects":75664,"num_object_clones":0,"num_object_copies":151328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":3667,"num_objects_unfound":0,"num_objects_dirty":75664,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":35771,"num_write_kb":143078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":405,"num_bytes_recovered":1654803,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75737,"ondisk_log_size":75737,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":994200,"kb_used_data":606240,"kb_used_omap":13,"kb_used_meta":387890,"kb_avail":282121320,"statfs":{"total":289910292480,"available":288892231680,"internally_reserved":0,"allocated":620789760,"data_stored":619936118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":397200383},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,7,1,13,7,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":33439744,"num_objects":8164,"num_object_clones":0,"num_object_copies":16328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-289,"num_objects_unfound":0,"num_objects_dirty":8164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8164,"num_write_kb":32656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":289,"num_bytes_recovered":1183744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002888"},"pg_stats":[{"pgid":"2.f","version":"41'4728","reported_seq":4800,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:40.106771+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:26:40.106771+0000","last_peered":"2024-09-17T15:26:40.106771+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:26:40.106771+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":4728,"log_dups_size":0,"ondisk_log_size":4728,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19365888,"num_objects":4728,"num_object_clones":0,"num_object_copies":9456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4728,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4728,"num_write_kb":18912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":658},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'4601","reported_seq":4698,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:36.743022+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:26:36.743022+0000","last_peered":"2024-09-17T15:26:36.743022+0000","last_clean":"2024-09-17T15:26:36.743022+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:26:36.743022+0000","last_undegraded":"2024-09-17T15:26:36.743022+0000","last_fullsized":"2024-09-17T15:26:36.743022+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":4601,"log_dups_size":0,"ondisk_log_size":4601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18822086,"num_objects":4597,"num_object_clones":0,"num_object_copies":9194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4597,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2958,"num_write_kb":11832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'4739","reported_seq":4858,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:36.718115+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:26:36.718115+0000","last_peered":"2024-09-17T15:26:36.718115+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:26:36.718115+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":4739,"log_dups_size":0,"ondisk_log_size":4739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19394560,"num_objects":4735,"num_object_clones":0,"num_object_copies":9470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2896,"num_write_kb":11584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":567}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'4656","reported_seq":4720,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:36.742782+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:26:36.742782+0000","last_peered":"2024-09-17T15:26:36.742782+0000","last_clean":"2024-09-17T15:26:36.742782+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:26:36.742782+0000","last_undegraded":"2024-09-17T15:26:36.742782+0000","last_fullsized":"2024-09-17T15:26:36.742782+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":4656,"log_dups_size":0,"ondisk_log_size":4656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19070976,"num_objects":4656,"num_object_clones":0,"num_object_copies":9312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4656,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4656,"num_write_kb":18624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'4665","reported_seq":4789,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:40.164311+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:26:40.164311+0000","last_peered":"2024-09-17T15:26:40.164311+0000","last_clean":"2024-09-17T15:26:40.164311+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:26:40.164311+0000","last_undegraded":"2024-09-17T15:26:40.164311+0000","last_fullsized":"2024-09-17T15:26:40.164311+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":4665,"log_dups_size":0,"ondisk_log_size":4665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19079651,"num_objects":4659,"num_object_clones":0,"num_object_copies":9318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4659,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1647,"num_write_kb":6588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'4958","reported_seq":5087,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:40.137264+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:26:40.137264+0000","last_peered":"2024-09-17T15:26:40.137264+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:26:40.137264+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":4958,"log_dups_size":0,"ondisk_log_size":4958,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20287971,"num_objects":4954,"num_object_clones":0,"num_object_copies":9908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4954,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1457,"num_write_kb":5828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":684},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'4848","reported_seq":4981,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:40.127012+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:26:40.127012+0000","last_peered":"2024-09-17T15:26:40.127012+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:26:40.127012+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":4848,"log_dups_size":0,"ondisk_log_size":4848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19833798,"num_objects":4844,"num_object_clones":0,"num_object_copies":9688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4844,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1617,"num_write_kb":6468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":670},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'4834","reported_seq":4956,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:40.117220+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:26:40.117220+0000","last_peered":"2024-09-17T15:26:40.117220+0000","last_clean":"2024-09-17T15:26:40.117220+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:26:40.117220+0000","last_undegraded":"2024-09-17T15:26:40.117220+0000","last_fullsized":"2024-09-17T15:26:40.117220+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":4834,"log_dups_size":0,"ondisk_log_size":4834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19791872,"num_objects":4832,"num_object_clones":0,"num_object_copies":9664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1788,"num_write_kb":7152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'4614","reported_seq":5545,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:26:36.742739+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:26:36.742739+0000","last_peered":"2024-09-17T15:26:36.742739+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:26:36.742739+0000","last_undegraded":"2024-09-17T15:26:36.742739+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":4614,"log_dups_size":0,"ondisk_log_size":4614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18870755,"num_objects":4608,"num_object_clones":0,"num_object_copies":9216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3667,"num_objects_unfound":0,"num_objects_dirty":4608,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2614,"num_write_kb":10456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":404,"num_bytes_recovered":1654784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3667},{"shards":"2","objects":941}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233552+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233552+0000","last_peered":"2024-09-17T15:26:22.233552+0000","last_clean":"2024-09-17T15:26:22.233552+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233552+0000","last_undegraded":"2024-09-17T15:26:22.233552+0000","last_fullsized":"2024-09-17T15:26:22.233552+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'4689","reported_seq":4813,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:40.143773+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:26:40.143773+0000","last_peered":"2024-09-17T15:26:40.143773+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:26:40.143773+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":4689,"log_dups_size":0,"ondisk_log_size":4689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19173376,"num_objects":4681,"num_object_clones":0,"num_object_copies":9362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4681,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1285,"num_write_kb":5140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":671},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":97,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412553+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412553+0000","last_peered":"2024-09-17T15:26:22.412553+0000","last_clean":"2024-09-17T15:26:22.412553+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412553+0000","last_undegraded":"2024-09-17T15:26:22.412553+0000","last_fullsized":"2024-09-17T15:26:22.412553+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'4584","reported_seq":4699,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:36.718250+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:26:36.718250+0000","last_peered":"2024-09-17T15:26:36.718250+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:26:36.718250+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":4584,"log_dups_size":0,"ondisk_log_size":4584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18756067,"num_objects":4580,"num_object_clones":0,"num_object_copies":9160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4580,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2479,"num_write_kb":9916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":503}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'4658","reported_seq":4771,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:36.705278+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:26:36.705278+0000","last_peered":"2024-09-17T15:26:36.705278+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:26:36.705278+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":4658,"log_dups_size":0,"ondisk_log_size":4658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19054592,"num_objects":4652,"num_object_clones":0,"num_object_copies":9304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2420,"num_write_kb":9680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":515}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'4840","reported_seq":4970,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:40.027161+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:26:40.027161+0000","last_peered":"2024-09-17T15:26:40.027161+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:26:40.027161+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":4840,"log_dups_size":0,"ondisk_log_size":4840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19816448,"num_objects":4838,"num_object_clones":0,"num_object_copies":9676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4838,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1388,"num_write_kb":5552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":700},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'4789","reported_seq":4924,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:40.134411+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:26:40.134411+0000","last_peered":"2024-09-17T15:26:40.134411+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:26:40.134411+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":4789,"log_dups_size":0,"ondisk_log_size":4789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19582976,"num_objects":4781,"num_object_clones":0,"num_object_copies":9562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1076,"num_write_kb":4304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":624},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229013+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229013+0000","last_peered":"2024-09-17T15:26:22.229013+0000","last_clean":"2024-09-17T15:26:22.229013+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229013+0000","last_undegraded":"2024-09-17T15:26:22.229013+0000","last_fullsized":"2024-09-17T15:26:22.229013+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'4750","reported_seq":4876,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:40.081115+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:26:40.081115+0000","last_peered":"2024-09-17T15:26:40.081115+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:26:40.081115+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":4750,"log_dups_size":0,"ondisk_log_size":4750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19439616,"num_objects":4746,"num_object_clones":0,"num_object_copies":9492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4746,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1537,"num_write_kb":6148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":683},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'4781","reported_seq":4918,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:40.167584+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:26:40.167584+0000","last_peered":"2024-09-17T15:26:40.167584+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:26:40.167584+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":4781,"log_dups_size":0,"ondisk_log_size":4781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19534790,"num_objects":4771,"num_object_clones":0,"num_object_copies":9542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1223,"num_write_kb":4892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":662},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":309875422,"num_objects":75662,"num_object_clones":0,"num_object_copies":151324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":3667,"num_objects_unfound":0,"num_objects_dirty":75662,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":35769,"num_write_kb":143076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":404,"num_bytes_recovered":1654784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":622354432,"data_stored":622199073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75734,"ondisk_log_size":75734,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051603,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":424400,"kb_used_data":280252,"kb_used_omap":4,"kb_used_meta":144123,"kb_avail":93947440,"statfs":{"total":96636764160,"available":96202178560,"internally_reserved":0,"allocated":286978048,"data_stored":286711875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":147582636},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,6],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724549,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117368,"kb_used_data":31528,"kb_used_omap":4,"kb_used_meta":85819,"kb_avail":94254472,"statfs":{"total":96636764160,"available":96516579328,"internally_reserved":0,"allocated":32284672,"data_stored":31967453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":87879339},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247832,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":452432,"kb_used_data":294460,"kb_used_omap":4,"kb_used_meta":157947,"kb_avail":93919408,"statfs":{"total":96636764160,"available":96173473792,"internally_reserved":0,"allocated":301527040,"data_stored":301256790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":161738408},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,6,0,6,7,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":302292992,"data_stored":302256862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33206272,"data_stored":33123173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":286855168,"data_stored":286819038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:26:41.944 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:26:42.295 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 63 20 77728 77708 4.81725 2.17188 0.00835734 0.0162101 2024-09-17T15:26:42.476 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:26:42.477 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:26:42.498 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":103,"stamp":"2024-09-17T15:26:41.596900+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":309875441,"num_objects":75664,"num_object_clones":0,"num_object_copies":151328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":3667,"num_objects_unfound":0,"num_objects_dirty":75664,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":35771,"num_write_kb":143078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":405,"num_bytes_recovered":1654803,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75737,"ondisk_log_size":75737,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":994200,"kb_used_data":606240,"kb_used_omap":13,"kb_used_meta":387890,"kb_avail":282121320,"statfs":{"total":289910292480,"available":288892231680,"internally_reserved":0,"allocated":620789760,"data_stored":619936118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":397200383},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,7,1,13,7,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":33439744,"num_objects":8164,"num_object_clones":0,"num_object_copies":16328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-289,"num_objects_unfound":0,"num_objects_dirty":8164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8164,"num_write_kb":32656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":289,"num_bytes_recovered":1183744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002888"},"pg_stats":[{"pgid":"2.f","version":"41'4728","reported_seq":4800,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:40.106771+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:26:40.106771+0000","last_peered":"2024-09-17T15:26:40.106771+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:26:40.106771+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":4728,"log_dups_size":0,"ondisk_log_size":4728,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19365888,"num_objects":4728,"num_object_clones":0,"num_object_copies":9456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4728,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4728,"num_write_kb":18912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":658},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'4601","reported_seq":4698,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:36.743022+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:26:36.743022+0000","last_peered":"2024-09-17T15:26:36.743022+0000","last_clean":"2024-09-17T15:26:36.743022+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:26:36.743022+0000","last_undegraded":"2024-09-17T15:26:36.743022+0000","last_fullsized":"2024-09-17T15:26:36.743022+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":4601,"log_dups_size":0,"ondisk_log_size":4601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18822086,"num_objects":4597,"num_object_clones":0,"num_object_copies":9194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4597,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2958,"num_write_kb":11832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'4739","reported_seq":4858,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:36.718115+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:26:36.718115+0000","last_peered":"2024-09-17T15:26:36.718115+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:26:36.718115+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":4739,"log_dups_size":0,"ondisk_log_size":4739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19394560,"num_objects":4735,"num_object_clones":0,"num_object_copies":9470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2896,"num_write_kb":11584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":567}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'4656","reported_seq":4720,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:36.742782+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:26:36.742782+0000","last_peered":"2024-09-17T15:26:36.742782+0000","last_clean":"2024-09-17T15:26:36.742782+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:26:36.742782+0000","last_undegraded":"2024-09-17T15:26:36.742782+0000","last_fullsized":"2024-09-17T15:26:36.742782+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":4656,"log_dups_size":0,"ondisk_log_size":4656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19070976,"num_objects":4656,"num_object_clones":0,"num_object_copies":9312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4656,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4656,"num_write_kb":18624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'4665","reported_seq":4789,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:40.164311+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:26:40.164311+0000","last_peered":"2024-09-17T15:26:40.164311+0000","last_clean":"2024-09-17T15:26:40.164311+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:26:40.164311+0000","last_undegraded":"2024-09-17T15:26:40.164311+0000","last_fullsized":"2024-09-17T15:26:40.164311+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":4665,"log_dups_size":0,"ondisk_log_size":4665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19079651,"num_objects":4659,"num_object_clones":0,"num_object_copies":9318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4659,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1647,"num_write_kb":6588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'4958","reported_seq":5087,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:40.137264+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:26:40.137264+0000","last_peered":"2024-09-17T15:26:40.137264+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:26:40.137264+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":4958,"log_dups_size":0,"ondisk_log_size":4958,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20287971,"num_objects":4954,"num_object_clones":0,"num_object_copies":9908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4954,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1457,"num_write_kb":5828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":684},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'4848","reported_seq":4981,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:40.127012+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:26:40.127012+0000","last_peered":"2024-09-17T15:26:40.127012+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:26:40.127012+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":4848,"log_dups_size":0,"ondisk_log_size":4848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19833798,"num_objects":4844,"num_object_clones":0,"num_object_copies":9688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4844,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1617,"num_write_kb":6468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":670},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'4834","reported_seq":4956,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:40.117220+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:26:40.117220+0000","last_peered":"2024-09-17T15:26:40.117220+0000","last_clean":"2024-09-17T15:26:40.117220+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:26:40.117220+0000","last_undegraded":"2024-09-17T15:26:40.117220+0000","last_fullsized":"2024-09-17T15:26:40.117220+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":4834,"log_dups_size":0,"ondisk_log_size":4834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19791872,"num_objects":4832,"num_object_clones":0,"num_object_copies":9664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1788,"num_write_kb":7152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'4614","reported_seq":5545,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:26:36.742739+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:26:36.742739+0000","last_peered":"2024-09-17T15:26:36.742739+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:26:36.742739+0000","last_undegraded":"2024-09-17T15:26:36.742739+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":4614,"log_dups_size":0,"ondisk_log_size":4614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18870755,"num_objects":4608,"num_object_clones":0,"num_object_copies":9216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3667,"num_objects_unfound":0,"num_objects_dirty":4608,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2614,"num_write_kb":10456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":404,"num_bytes_recovered":1654784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3667},{"shards":"2","objects":941}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233552+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233552+0000","last_peered":"2024-09-17T15:26:22.233552+0000","last_clean":"2024-09-17T15:26:22.233552+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233552+0000","last_undegraded":"2024-09-17T15:26:22.233552+0000","last_fullsized":"2024-09-17T15:26:22.233552+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'4689","reported_seq":4813,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:40.143773+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:26:40.143773+0000","last_peered":"2024-09-17T15:26:40.143773+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:26:40.143773+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":4689,"log_dups_size":0,"ondisk_log_size":4689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19173376,"num_objects":4681,"num_object_clones":0,"num_object_copies":9362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4681,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1285,"num_write_kb":5140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":671},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":97,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412553+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412553+0000","last_peered":"2024-09-17T15:26:22.412553+0000","last_clean":"2024-09-17T15:26:22.412553+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412553+0000","last_undegraded":"2024-09-17T15:26:22.412553+0000","last_fullsized":"2024-09-17T15:26:22.412553+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'4584","reported_seq":4699,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:36.718250+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:26:36.718250+0000","last_peered":"2024-09-17T15:26:36.718250+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:26:36.718250+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":4584,"log_dups_size":0,"ondisk_log_size":4584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18756067,"num_objects":4580,"num_object_clones":0,"num_object_copies":9160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4580,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2479,"num_write_kb":9916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":503}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'4658","reported_seq":4771,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:36.705278+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:26:36.705278+0000","last_peered":"2024-09-17T15:26:36.705278+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:26:36.705278+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":4658,"log_dups_size":0,"ondisk_log_size":4658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19054592,"num_objects":4652,"num_object_clones":0,"num_object_copies":9304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2420,"num_write_kb":9680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":515}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'4840","reported_seq":4970,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:40.027161+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:26:40.027161+0000","last_peered":"2024-09-17T15:26:40.027161+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:26:40.027161+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":4840,"log_dups_size":0,"ondisk_log_size":4840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19816448,"num_objects":4838,"num_object_clones":0,"num_object_copies":9676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4838,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1388,"num_write_kb":5552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":700},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'4789","reported_seq":4924,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:40.134411+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:26:40.134411+0000","last_peered":"2024-09-17T15:26:40.134411+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:26:40.134411+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":4789,"log_dups_size":0,"ondisk_log_size":4789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19582976,"num_objects":4781,"num_object_clones":0,"num_object_copies":9562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1076,"num_write_kb":4304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":624},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229013+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229013+0000","last_peered":"2024-09-17T15:26:22.229013+0000","last_clean":"2024-09-17T15:26:22.229013+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229013+0000","last_undegraded":"2024-09-17T15:26:22.229013+0000","last_fullsized":"2024-09-17T15:26:22.229013+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'4750","reported_seq":4876,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:40.081115+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:26:40.081115+0000","last_peered":"2024-09-17T15:26:40.081115+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:26:40.081115+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":4750,"log_dups_size":0,"ondisk_log_size":4750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19439616,"num_objects":4746,"num_object_clones":0,"num_object_copies":9492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4746,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1537,"num_write_kb":6148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":683},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'4781","reported_seq":4918,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:40.167584+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:26:40.167584+0000","last_peered":"2024-09-17T15:26:40.167584+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:26:40.167584+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":4781,"log_dups_size":0,"ondisk_log_size":4781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19534790,"num_objects":4771,"num_object_clones":0,"num_object_copies":9542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1223,"num_write_kb":4892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":662},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":309875422,"num_objects":75662,"num_object_clones":0,"num_object_copies":151324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":3667,"num_objects_unfound":0,"num_objects_dirty":75662,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":35769,"num_write_kb":143076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":404,"num_bytes_recovered":1654784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":622354432,"data_stored":622199073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75734,"ondisk_log_size":75734,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051603,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":424400,"kb_used_data":280252,"kb_used_omap":4,"kb_used_meta":144123,"kb_avail":93947440,"statfs":{"total":96636764160,"available":96202178560,"internally_reserved":0,"allocated":286978048,"data_stored":286711875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":147582636},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,6],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724549,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117368,"kb_used_data":31528,"kb_used_omap":4,"kb_used_meta":85819,"kb_avail":94254472,"statfs":{"total":96636764160,"available":96516579328,"internally_reserved":0,"allocated":32284672,"data_stored":31967453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":87879339},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247832,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":452432,"kb_used_data":294460,"kb_used_omap":4,"kb_used_meta":157947,"kb_avail":93919408,"statfs":{"total":96636764160,"available":96173473792,"internally_reserved":0,"allocated":301527040,"data_stored":301256790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":161738408},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,6,0,6,7,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":302292992,"data_stored":302256862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33206272,"data_stored":33123173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":286855168,"data_stored":286819038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:26:42.500 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:26:42.500 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'4728", 'reported_seq': 4800, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:40.106771+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:26:40.106771+0000', 'last_peered': '2024-09-17T15:26:40.106771+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:26:40.106771+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 4728, 'log_dups_size': 0, 'ondisk_log_size': 4728, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19365888, 'num_objects': 4728, 'num_object_clones': 0, 'num_object_copies': 9456, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4070, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4728, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4728, 'num_write_kb': 18912, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 658}, {'shards': '0,2', 'objects': 4070}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:42.500 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:26:42.500 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'4739", 'reported_seq': 4858, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:36.718115+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:26:36.718115+0000', 'last_peered': '2024-09-17T15:26:36.718115+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:26:36.718115+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 4739, 'log_dups_size': 0, 'ondisk_log_size': 4739, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19394560, 'num_objects': 4735, 'num_object_clones': 0, 'num_object_copies': 9470, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4169, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4735, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2896, 'num_write_kb': 11584, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4168}, {'shards': '2', 'objects': 567}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:26:42.501 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:26:42.501 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'4958", 'reported_seq': 5087, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:40.137264+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:26:40.137264+0000', 'last_peered': '2024-09-17T15:26:40.137264+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:26:40.137264+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 4958, 'log_dups_size': 0, 'ondisk_log_size': 4958, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20287971, 'num_objects': 4954, 'num_object_clones': 0, 'num_object_copies': 9908, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4271, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4954, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 1457, 'num_write_kb': 5828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 684}, {'shards': '0,2', 'objects': 4268}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:42.501 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:26:42.501 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'4848", 'reported_seq': 4981, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:40.127012+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:26:40.127012+0000', 'last_peered': '2024-09-17T15:26:40.127012+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:26:40.127012+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 4848, 'log_dups_size': 0, 'ondisk_log_size': 4848, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19833798, 'num_objects': 4844, 'num_object_clones': 0, 'num_object_copies': 9688, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4174, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4844, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 1617, 'num_write_kb': 6468, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 670}, {'shards': '0,2', 'objects': 4174}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:42.501 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-17T15:26:42.501 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "41'4614", 'reported_seq': 5545, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:26:36.742739+0000', 'last_change': '2024-09-17T15:26:22.588242+0000', 'last_active': '2024-09-17T15:26:36.742739+0000', 'last_peered': '2024-09-17T15:26:36.742739+0000', 'last_clean': '2024-09-17T15:26:20.155646+0000', 'last_became_active': '2024-09-17T15:26:22.557742+0000', 'last_became_peered': '2024-09-17T15:26:22.557742+0000', 'last_unstale': '2024-09-17T15:26:36.742739+0000', 'last_undegraded': '2024-09-17T15:26:36.742739+0000', 'last_fullsized': '2024-09-17T15:26:22.188850+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2016", 'last_scrub_stamp': '2024-09-17T15:25:53.959011+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:53.959011+0000', 'objects_scrubbed': 1994, 'log_size': 4614, 'log_dups_size': 0, 'ondisk_log_size': 4614, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 568, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18870755, 'num_objects': 4608, 'num_object_clones': 0, 'num_object_copies': 9216, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3667, 'num_objects_unfound': 0, 'num_objects_dirty': 4608, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2614, 'num_write_kb': 10456, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 404, 'num_bytes_recovered': 1654784, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3667}, {'shards': '2', 'objects': 941}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:26:42.501 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:26:42.501 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'4689", 'reported_seq': 4813, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:40.143773+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:26:40.143773+0000', 'last_peered': '2024-09-17T15:26:40.143773+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:26:40.143773+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 4689, 'log_dups_size': 0, 'ondisk_log_size': 4689, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19173376, 'num_objects': 4681, 'num_object_clones': 0, 'num_object_copies': 9362, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4012, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4681, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1285, 'num_write_kb': 5140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 671}, {'shards': '0,2', 'objects': 4010}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:42.501 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:26:42.501 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'4584", 'reported_seq': 4699, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:36.718250+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:26:36.718250+0000', 'last_peered': '2024-09-17T15:26:36.718250+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:26:36.718250+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 4584, 'log_dups_size': 0, 'ondisk_log_size': 4584, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18756067, 'num_objects': 4580, 'num_object_clones': 0, 'num_object_copies': 9160, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4078, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4580, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2479, 'num_write_kb': 9916, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4077}, {'shards': '2', 'objects': 503}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:26:42.502 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:26:42.502 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'4658", 'reported_seq': 4771, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:36.705278+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:26:36.705278+0000', 'last_peered': '2024-09-17T15:26:36.705278+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:26:36.705278+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 4658, 'log_dups_size': 0, 'ondisk_log_size': 4658, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19054592, 'num_objects': 4652, 'num_object_clones': 0, 'num_object_copies': 9304, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4652, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2420, 'num_write_kb': 9680, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4137}, {'shards': '2', 'objects': 515}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:26:42.502 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:26:42.502 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'4840", 'reported_seq': 4970, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:40.027161+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:26:40.027161+0000', 'last_peered': '2024-09-17T15:26:40.027161+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:26:40.027161+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 4840, 'log_dups_size': 0, 'ondisk_log_size': 4840, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19816448, 'num_objects': 4838, 'num_object_clones': 0, 'num_object_copies': 9676, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4838, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1388, 'num_write_kb': 5552, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 700}, {'shards': '0,2', 'objects': 4138}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:42.502 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:26:42.502 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'4789", 'reported_seq': 4924, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:40.134411+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:26:40.134411+0000', 'last_peered': '2024-09-17T15:26:40.134411+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:26:40.134411+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 4789, 'log_dups_size': 0, 'ondisk_log_size': 4789, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19582976, 'num_objects': 4781, 'num_object_clones': 0, 'num_object_copies': 9562, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4160, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4781, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1076, 'num_write_kb': 4304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 624}, {'shards': '0,2', 'objects': 4157}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:42.502 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:26:42.502 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'4750", 'reported_seq': 4876, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:40.081115+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:26:40.081115+0000', 'last_peered': '2024-09-17T15:26:40.081115+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:26:40.081115+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 4750, 'log_dups_size': 0, 'ondisk_log_size': 4750, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19439616, 'num_objects': 4746, 'num_object_clones': 0, 'num_object_copies': 9492, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4065, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4746, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1537, 'num_write_kb': 6148, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 683}, {'shards': '0,2', 'objects': 4063}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:42.502 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:26:42.502 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'4781", 'reported_seq': 4918, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:40.167584+0000', 'last_change': '2024-09-17T15:26:22.654521+0000', 'last_active': '2024-09-17T15:26:40.167584+0000', 'last_peered': '2024-09-17T15:26:40.167584+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:26:40.167584+0000', 'last_undegraded': '2024-09-17T15:26:22.307795+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 4781, 'log_dups_size': 0, 'ondisk_log_size': 4781, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19534790, 'num_objects': 4771, 'num_object_clones': 0, 'num_object_copies': 9542, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4111, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4771, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1223, 'num_write_kb': 4892, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 662}, {'shards': '0,2', 'objects': 4109}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:42.503 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:26:43.214 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:26:43.214 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45388/151328 objects degraded (29.993%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":140,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":75664,"data_bytes":309875441,"bytes_used":1018060800,"bytes_avail":288892231680,"bytes_total":289910292480,"degraded_objects":45388,"degraded_total":151328,"degraded_ratio":0.29993127511101714,"misplaced_objects":3667,"misplaced_total":151328,"misplaced_ratio":0.024232131528864453,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":98621,"recovering_keys_per_sec":0,"num_objects_recovered":289,"num_bytes_recovered":1183744,"num_keys_recovered":0,"write_bytes_sec":2785974,"read_op_per_sec":0,"write_op_per_sec":680},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-17T15:26:23.592815+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-17T15:26:43.239 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:26:43.239 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:26:43.295 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 64 20 78282 78262 4.7758 2.16406 0.0275796 0.0163506 2024-09-17T15:26:43.817 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:26:43.817 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:26:43.847 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2024-09-17T15:26:43.597364+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":314548977,"num_objects":76805,"num_object_clones":0,"num_object_copies":153610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":3529,"num_objects_unfound":0,"num_objects_dirty":76805,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":36912,"num_write_kb":147642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":543,"num_bytes_recovered":2220051,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76878,"ondisk_log_size":76878,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1008708,"kb_used_data":612432,"kb_used_omap":13,"kb_used_meta":396210,"kb_avail":282106812,"statfs":{"total":289910292480,"available":288877375488,"internally_reserved":0,"allocated":627130368,"data_stored":626276726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":405720063},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,4,8,1,14,9,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30003200,"num_objects":7325,"num_object_clones":0,"num_object_copies":14650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-427,"num_objects_unfound":0,"num_objects_dirty":7325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7325,"num_write_kb":29300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":427,"num_bytes_recovered":1748992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002658"},"pg_stats":[{"pgid":"2.f","version":"41'4728","reported_seq":4800,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:40.106771+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:26:40.106771+0000","last_peered":"2024-09-17T15:26:40.106771+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:26:40.106771+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":4728,"log_dups_size":0,"ondisk_log_size":4728,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19365888,"num_objects":4728,"num_object_clones":0,"num_object_copies":9456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4728,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4728,"num_write_kb":18912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":658},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'4790","reported_seq":4887,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:41.743279+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:26:41.743279+0000","last_peered":"2024-09-17T15:26:41.743279+0000","last_clean":"2024-09-17T15:26:41.743279+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:26:41.743279+0000","last_undegraded":"2024-09-17T15:26:41.743279+0000","last_fullsized":"2024-09-17T15:26:41.743279+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":4790,"log_dups_size":0,"ondisk_log_size":4790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19596230,"num_objects":4786,"num_object_clones":0,"num_object_copies":9572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4786,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3147,"num_write_kb":12588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'4937","reported_seq":5057,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:41.614481+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:26:41.614481+0000","last_peered":"2024-09-17T15:26:41.614481+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:26:41.614481+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":4937,"log_dups_size":0,"ondisk_log_size":4937,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20205568,"num_objects":4933,"num_object_clones":0,"num_object_copies":9866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4933,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3094,"num_write_kb":12376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":765}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'4833","reported_seq":4897,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:41.747788+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:26:41.747788+0000","last_peered":"2024-09-17T15:26:41.747788+0000","last_clean":"2024-09-17T15:26:41.747788+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:26:41.747788+0000","last_undegraded":"2024-09-17T15:26:41.747788+0000","last_fullsized":"2024-09-17T15:26:41.747788+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":4833,"log_dups_size":0,"ondisk_log_size":4833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19795968,"num_objects":4833,"num_object_clones":0,"num_object_copies":9666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4833,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4833,"num_write_kb":19332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'4665","reported_seq":4789,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:40.164311+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:26:40.164311+0000","last_peered":"2024-09-17T15:26:40.164311+0000","last_clean":"2024-09-17T15:26:40.164311+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:26:40.164311+0000","last_undegraded":"2024-09-17T15:26:40.164311+0000","last_fullsized":"2024-09-17T15:26:40.164311+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":4665,"log_dups_size":0,"ondisk_log_size":4665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19079651,"num_objects":4659,"num_object_clones":0,"num_object_copies":9318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4659,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1647,"num_write_kb":6588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'4958","reported_seq":5087,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:40.137264+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:26:40.137264+0000","last_peered":"2024-09-17T15:26:40.137264+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:26:40.137264+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":4958,"log_dups_size":0,"ondisk_log_size":4958,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20287971,"num_objects":4954,"num_object_clones":0,"num_object_copies":9908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4954,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1457,"num_write_kb":5828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":684},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'4848","reported_seq":4981,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:40.127012+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:26:40.127012+0000","last_peered":"2024-09-17T15:26:40.127012+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:26:40.127012+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":4848,"log_dups_size":0,"ondisk_log_size":4848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19833798,"num_objects":4844,"num_object_clones":0,"num_object_copies":9688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4844,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1617,"num_write_kb":6468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":670},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'4834","reported_seq":4956,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:40.117220+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:26:40.117220+0000","last_peered":"2024-09-17T15:26:40.117220+0000","last_clean":"2024-09-17T15:26:40.117220+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:26:40.117220+0000","last_undegraded":"2024-09-17T15:26:40.117220+0000","last_fullsized":"2024-09-17T15:26:40.117220+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":4834,"log_dups_size":0,"ondisk_log_size":4834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19791872,"num_objects":4832,"num_object_clones":0,"num_object_copies":9664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1788,"num_write_kb":7152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'4810","reported_seq":6017,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:26:41.734750+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:26:41.734750+0000","last_peered":"2024-09-17T15:26:41.734750+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:26:41.734750+0000","last_undegraded":"2024-09-17T15:26:41.734750+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":4810,"log_dups_size":0,"ondisk_log_size":4810,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19673571,"num_objects":4804,"num_object_clones":0,"num_object_copies":9608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3529,"num_objects_unfound":0,"num_objects_dirty":4804,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2810,"num_write_kb":11240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":542,"num_bytes_recovered":2220032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3529},{"shards":"2","objects":1275}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233552+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233552+0000","last_peered":"2024-09-17T15:26:22.233552+0000","last_clean":"2024-09-17T15:26:22.233552+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233552+0000","last_undegraded":"2024-09-17T15:26:22.233552+0000","last_fullsized":"2024-09-17T15:26:22.233552+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'4689","reported_seq":4813,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:40.143773+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:26:40.143773+0000","last_peered":"2024-09-17T15:26:40.143773+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:26:40.143773+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":4689,"log_dups_size":0,"ondisk_log_size":4689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19173376,"num_objects":4681,"num_object_clones":0,"num_object_copies":9362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4681,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1285,"num_write_kb":5140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":671},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":97,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412553+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412553+0000","last_peered":"2024-09-17T15:26:22.412553+0000","last_clean":"2024-09-17T15:26:22.412553+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412553+0000","last_undegraded":"2024-09-17T15:26:22.412553+0000","last_fullsized":"2024-09-17T15:26:22.412553+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'4785","reported_seq":4900,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:41.705999+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:26:41.705999+0000","last_peered":"2024-09-17T15:26:41.705999+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:26:41.705999+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":4785,"log_dups_size":0,"ondisk_log_size":4785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19579363,"num_objects":4781,"num_object_clones":0,"num_object_copies":9562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4781,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2680,"num_write_kb":10720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":704}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'4838","reported_seq":4951,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:41.758858+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:26:41.758858+0000","last_peered":"2024-09-17T15:26:41.758858+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:26:41.758858+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":4838,"log_dups_size":0,"ondisk_log_size":4838,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19791872,"num_objects":4832,"num_object_clones":0,"num_object_copies":9664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2600,"num_write_kb":10400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":695}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'4840","reported_seq":4970,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:40.027161+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:26:40.027161+0000","last_peered":"2024-09-17T15:26:40.027161+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:26:40.027161+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":4840,"log_dups_size":0,"ondisk_log_size":4840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19816448,"num_objects":4838,"num_object_clones":0,"num_object_copies":9676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4838,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1388,"num_write_kb":5552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":700},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'4789","reported_seq":4924,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:40.134411+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:26:40.134411+0000","last_peered":"2024-09-17T15:26:40.134411+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:26:40.134411+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":4789,"log_dups_size":0,"ondisk_log_size":4789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19582976,"num_objects":4781,"num_object_clones":0,"num_object_copies":9562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1076,"num_write_kb":4304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":624},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229013+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229013+0000","last_peered":"2024-09-17T15:26:22.229013+0000","last_clean":"2024-09-17T15:26:22.229013+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229013+0000","last_undegraded":"2024-09-17T15:26:22.229013+0000","last_fullsized":"2024-09-17T15:26:22.229013+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'4750","reported_seq":4876,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:40.081115+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:26:40.081115+0000","last_peered":"2024-09-17T15:26:40.081115+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:26:40.081115+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":4750,"log_dups_size":0,"ondisk_log_size":4750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19439616,"num_objects":4746,"num_object_clones":0,"num_object_copies":9492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4746,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1537,"num_write_kb":6148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":683},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'4781","reported_seq":4918,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:40.167584+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:26:40.167584+0000","last_peered":"2024-09-17T15:26:40.167584+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:26:40.167584+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":4781,"log_dups_size":0,"ondisk_log_size":4781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19534790,"num_objects":4771,"num_object_clones":0,"num_object_copies":9542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1223,"num_write_kb":4892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":662},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":314548958,"num_objects":76803,"num_object_clones":0,"num_object_copies":153606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":3529,"num_objects_unfound":0,"num_objects_dirty":76803,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":36910,"num_write_kb":147640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":542,"num_bytes_recovered":2220032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":628670464,"data_stored":628515105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76875,"ondisk_log_size":76875,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051604,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":438908,"kb_used_data":286444,"kb_used_omap":4,"kb_used_meta":152443,"kb_avail":93932932,"statfs":{"total":96636764160,"available":96187322368,"internally_reserved":0,"allocated":293318656,"data_stored":293052483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":156102316},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1,7,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724549,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117368,"kb_used_data":31528,"kb_used_omap":4,"kb_used_meta":85819,"kb_avail":94254472,"statfs":{"total":96636764160,"available":96516579328,"internally_reserved":0,"allocated":32284672,"data_stored":31967453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":87879339},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247832,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":452432,"kb_used_data":294460,"kb_used_omap":4,"kb_used_meta":157947,"kb_avail":93919408,"statfs":{"total":96636764160,"available":96173473792,"internally_reserved":0,"allocated":301527040,"data_stored":301256790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":161738408},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,6,0,6,7,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":302292992,"data_stored":302256862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33206272,"data_stored":33123173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":293171200,"data_stored":293135070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:26:44.296 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 65 20 78870 78850 4.73766 2.29688 0.0326135 0.0164814 2024-09-17T15:26:45.296 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 66 20 79511 79491 4.70382 2.50391 0.0374777 0.0166009 2024-09-17T15:26:46.296 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 67 20 80137 80117 4.67011 2.44531 0.027618 0.0167214 2024-09-17T15:26:46.849 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:26:47.296 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 68 20 80728 80708 4.63538 2.30859 0.0212276 0.0168445 2024-09-17T15:26:47.412 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:26:47.414 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:26:47.437 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":105,"stamp":"2024-09-17T15:26:45.597836+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":322335473,"num_objects":78706,"num_object_clones":0,"num_object_copies":157412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":3529,"num_objects_unfound":0,"num_objects_dirty":78706,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":38813,"num_write_kb":155246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":543,"num_bytes_recovered":2220051,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78779,"ondisk_log_size":78779,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1043996,"kb_used_data":631084,"kb_used_omap":13,"kb_used_meta":412850,"kb_avail":282071524,"statfs":{"total":289910292480,"available":288841240576,"internally_reserved":0,"allocated":646230016,"data_stored":645376374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":422759423},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,3,8,9,15,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":33206272,"num_objects":8107,"num_object_clones":0,"num_object_copies":16214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-270,"num_objects_unfound":0,"num_objects_dirty":8107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8107,"num_write_kb":32428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":270,"num_bytes_recovered":1105920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002728"},"pg_stats":[{"pgid":"2.f","version":"41'4916","reported_seq":4988,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:45.154480+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:26:45.154480+0000","last_peered":"2024-09-17T15:26:45.154480+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:26:45.154480+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":4916,"log_dups_size":0,"ondisk_log_size":4916,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20135936,"num_objects":4916,"num_object_clones":0,"num_object_copies":9832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4916,"num_write_kb":19664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":846},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'4790","reported_seq":4887,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:41.743279+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:26:41.743279+0000","last_peered":"2024-09-17T15:26:41.743279+0000","last_clean":"2024-09-17T15:26:41.743279+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:26:41.743279+0000","last_undegraded":"2024-09-17T15:26:41.743279+0000","last_fullsized":"2024-09-17T15:26:41.743279+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":4790,"log_dups_size":0,"ondisk_log_size":4790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19596230,"num_objects":4786,"num_object_clones":0,"num_object_copies":9572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4786,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3147,"num_write_kb":12588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'4937","reported_seq":5057,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:41.614481+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:26:41.614481+0000","last_peered":"2024-09-17T15:26:41.614481+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:26:41.614481+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":4937,"log_dups_size":0,"ondisk_log_size":4937,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20205568,"num_objects":4933,"num_object_clones":0,"num_object_copies":9866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4933,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3094,"num_write_kb":12376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":765}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'4833","reported_seq":4897,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:41.747788+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:26:41.747788+0000","last_peered":"2024-09-17T15:26:41.747788+0000","last_clean":"2024-09-17T15:26:41.747788+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:26:41.747788+0000","last_undegraded":"2024-09-17T15:26:41.747788+0000","last_fullsized":"2024-09-17T15:26:41.747788+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":4833,"log_dups_size":0,"ondisk_log_size":4833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19795968,"num_objects":4833,"num_object_clones":0,"num_object_copies":9666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4833,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4833,"num_write_kb":19332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'4833","reported_seq":4957,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:45.175904+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:26:45.175904+0000","last_peered":"2024-09-17T15:26:45.175904+0000","last_clean":"2024-09-17T15:26:45.175904+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:26:45.175904+0000","last_undegraded":"2024-09-17T15:26:45.175904+0000","last_fullsized":"2024-09-17T15:26:45.175904+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":4833,"log_dups_size":0,"ondisk_log_size":4833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19767779,"num_objects":4827,"num_object_clones":0,"num_object_copies":9654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4827,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1815,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'5154","reported_seq":5283,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:45.122535+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:26:45.122535+0000","last_peered":"2024-09-17T15:26:45.122535+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:26:45.122535+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":5154,"log_dups_size":0,"ondisk_log_size":5154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21090787,"num_objects":5150,"num_object_clones":0,"num_object_copies":10300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5150,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1653,"num_write_kb":6612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":880},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'5069","reported_seq":5202,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:45.154619+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:26:45.154619+0000","last_peered":"2024-09-17T15:26:45.154619+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:26:45.154619+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":5069,"log_dups_size":0,"ondisk_log_size":5069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20739014,"num_objects":5065,"num_object_clones":0,"num_object_copies":10130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5065,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1838,"num_write_kb":7352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":891},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'5014","reported_seq":5136,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:45.147127+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:26:45.147127+0000","last_peered":"2024-09-17T15:26:45.147127+0000","last_clean":"2024-09-17T15:26:45.147127+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:26:45.147127+0000","last_undegraded":"2024-09-17T15:26:45.147127+0000","last_fullsized":"2024-09-17T15:26:45.147127+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":5014,"log_dups_size":0,"ondisk_log_size":5014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20529152,"num_objects":5012,"num_object_clones":0,"num_object_copies":10024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1968,"num_write_kb":7872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'4810","reported_seq":6017,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:26:41.734750+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:26:41.734750+0000","last_peered":"2024-09-17T15:26:41.734750+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:26:41.734750+0000","last_undegraded":"2024-09-17T15:26:41.734750+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":4810,"log_dups_size":0,"ondisk_log_size":4810,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19673571,"num_objects":4804,"num_object_clones":0,"num_object_copies":9608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3529,"num_objects_unfound":0,"num_objects_dirty":4804,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2810,"num_write_kb":11240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":542,"num_bytes_recovered":2220032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3529},{"shards":"2","objects":1275}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233552+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233552+0000","last_peered":"2024-09-17T15:26:22.233552+0000","last_clean":"2024-09-17T15:26:22.233552+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233552+0000","last_undegraded":"2024-09-17T15:26:22.233552+0000","last_fullsized":"2024-09-17T15:26:22.233552+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'4852","reported_seq":4975,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:45.098797+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:26:45.098797+0000","last_peered":"2024-09-17T15:26:45.098797+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:26:45.098797+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":4852,"log_dups_size":0,"ondisk_log_size":4852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19841024,"num_objects":4844,"num_object_clones":0,"num_object_copies":9688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1448,"num_write_kb":5792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":834},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":97,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412553+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412553+0000","last_peered":"2024-09-17T15:26:22.412553+0000","last_clean":"2024-09-17T15:26:22.412553+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412553+0000","last_undegraded":"2024-09-17T15:26:22.412553+0000","last_fullsized":"2024-09-17T15:26:22.412553+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'4785","reported_seq":4900,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:41.705999+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:26:41.705999+0000","last_peered":"2024-09-17T15:26:41.705999+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:26:41.705999+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":4785,"log_dups_size":0,"ondisk_log_size":4785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19579363,"num_objects":4781,"num_object_clones":0,"num_object_copies":9562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4781,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2680,"num_write_kb":10720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":704}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'4838","reported_seq":4951,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:41.758858+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:26:41.758858+0000","last_peered":"2024-09-17T15:26:41.758858+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:26:41.758858+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":4838,"log_dups_size":0,"ondisk_log_size":4838,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19791872,"num_objects":4832,"num_object_clones":0,"num_object_copies":9664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2600,"num_write_kb":10400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":695}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'5042","reported_seq":5172,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:45.187269+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:26:45.187269+0000","last_peered":"2024-09-17T15:26:45.187269+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:26:45.187269+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":5042,"log_dups_size":0,"ondisk_log_size":5042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20643840,"num_objects":5040,"num_object_clones":0,"num_object_copies":10080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1590,"num_write_kb":6360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":902},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'4974","reported_seq":5110,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:45.158690+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:26:45.158690+0000","last_peered":"2024-09-17T15:26:45.158690+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:26:45.158690+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":4974,"log_dups_size":0,"ondisk_log_size":4974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20340736,"num_objects":4966,"num_object_clones":0,"num_object_copies":9932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1261,"num_write_kb":5044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":809},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229013+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229013+0000","last_peered":"2024-09-17T15:26:22.229013+0000","last_clean":"2024-09-17T15:26:22.229013+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229013+0000","last_undegraded":"2024-09-17T15:26:22.229013+0000","last_fullsized":"2024-09-17T15:26:22.229013+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'4956","reported_seq":5082,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:45.138531+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:26:45.138531+0000","last_peered":"2024-09-17T15:26:45.138531+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:26:45.138531+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":4956,"log_dups_size":0,"ondisk_log_size":4956,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20283392,"num_objects":4952,"num_object_clones":0,"num_object_copies":9904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1743,"num_write_kb":6972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":889},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'4973","reported_seq":5110,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:45.126647+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:26:45.126647+0000","last_peered":"2024-09-17T15:26:45.126647+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:26:45.126647+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":4973,"log_dups_size":0,"ondisk_log_size":4973,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20321222,"num_objects":4963,"num_object_clones":0,"num_object_copies":9926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1415,"num_write_kb":5660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":854},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":322335454,"num_objects":78704,"num_object_clones":0,"num_object_copies":157408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":3529,"num_objects_unfound":0,"num_objects_dirty":78704,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":38811,"num_write_kb":155244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":542,"num_bytes_recovered":2220032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":647872512,"data_stored":647717153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78776,"ondisk_log_size":78776,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051604,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":438908,"kb_used_data":286444,"kb_used_omap":4,"kb_used_meta":152443,"kb_avail":93932932,"statfs":{"total":96636764160,"available":96187322368,"internally_reserved":0,"allocated":293318656,"data_stored":293052483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":156102316},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1,7,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724550,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":135372,"kb_used_data":41212,"kb_used_omap":4,"kb_used_meta":94139,"kb_avail":94236468,"statfs":{"total":96636764160,"available":96498143232,"internally_reserved":0,"allocated":42201088,"data_stored":41883869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":96399019},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,4,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247833,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469716,"kb_used_data":303428,"kb_used_omap":4,"kb_used_meta":166267,"kb_avail":93902124,"statfs":{"total":96636764160,"available":96155774976,"internally_reserved":0,"allocated":310710272,"data_stored":310440022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":170258088},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,1,10,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":311541760,"data_stored":311505630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":43159552,"data_stored":43076453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":293171200,"data_stored":293135070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:26:47.438 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:26:47.995 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:26:47.998 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:26:48.019 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":106,"stamp":"2024-09-17T15:26:47.598278+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":326836977,"num_objects":79805,"num_object_clones":0,"num_object_copies":159610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":3385,"num_objects_unfound":0,"num_objects_dirty":79805,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":39912,"num_write_kb":159642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":687,"num_bytes_recovered":2809875,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79878,"ondisk_log_size":79878,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1037204,"kb_used_data":636772,"kb_used_omap":13,"kb_used_meta":400370,"kb_avail":282078316,"statfs":{"total":289910292480,"available":288848195584,"internally_reserved":0,"allocated":652054528,"data_stored":651200886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":409979903},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,1,8,4,16,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29646848,"num_objects":7238,"num_object_clones":0,"num_object_copies":14476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-414,"num_objects_unfound":0,"num_objects_dirty":7238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7238,"num_write_kb":28952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":414,"num_bytes_recovered":1695744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002744"},"pg_stats":[{"pgid":"2.f","version":"41'4916","reported_seq":4988,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:45.154480+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:26:45.154480+0000","last_peered":"2024-09-17T15:26:45.154480+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:26:45.154480+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":4916,"log_dups_size":0,"ondisk_log_size":4916,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20135936,"num_objects":4916,"num_object_clones":0,"num_object_copies":9832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4916,"num_write_kb":19664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":846},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'4976","reported_seq":5073,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:46.743572+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:26:46.743572+0000","last_peered":"2024-09-17T15:26:46.743572+0000","last_clean":"2024-09-17T15:26:46.743572+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:26:46.743572+0000","last_undegraded":"2024-09-17T15:26:46.743572+0000","last_fullsized":"2024-09-17T15:26:46.743572+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":4976,"log_dups_size":0,"ondisk_log_size":4976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20358086,"num_objects":4972,"num_object_clones":0,"num_object_copies":9944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4972,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3333,"num_write_kb":13332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'5125","reported_seq":5245,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:46.790561+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:26:46.790561+0000","last_peered":"2024-09-17T15:26:46.790561+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:26:46.790561+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":5125,"log_dups_size":0,"ondisk_log_size":5125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20975616,"num_objects":5121,"num_object_clones":0,"num_object_copies":10242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3282,"num_write_kb":13128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":953}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'5022","reported_seq":5085,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:46.752155+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:26:46.752155+0000","last_peered":"2024-09-17T15:26:46.752155+0000","last_clean":"2024-09-17T15:26:46.752155+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:26:46.752155+0000","last_undegraded":"2024-09-17T15:26:46.752155+0000","last_fullsized":"2024-09-17T15:26:46.752155+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":5022,"log_dups_size":0,"ondisk_log_size":5022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20570112,"num_objects":5022,"num_object_clones":0,"num_object_copies":10044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5022,"num_write_kb":20088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'4833","reported_seq":4957,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:45.175904+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:26:45.175904+0000","last_peered":"2024-09-17T15:26:45.175904+0000","last_clean":"2024-09-17T15:26:45.175904+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:26:45.175904+0000","last_undegraded":"2024-09-17T15:26:45.175904+0000","last_fullsized":"2024-09-17T15:26:45.175904+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":4833,"log_dups_size":0,"ondisk_log_size":4833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19767779,"num_objects":4827,"num_object_clones":0,"num_object_copies":9654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4827,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1815,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'5154","reported_seq":5283,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:45.122535+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:26:45.122535+0000","last_peered":"2024-09-17T15:26:45.122535+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:26:45.122535+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":5154,"log_dups_size":0,"ondisk_log_size":5154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21090787,"num_objects":5150,"num_object_clones":0,"num_object_copies":10300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5150,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1653,"num_write_kb":6612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":880},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'5069","reported_seq":5202,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:45.154619+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:26:45.154619+0000","last_peered":"2024-09-17T15:26:45.154619+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:26:45.154619+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":5069,"log_dups_size":0,"ondisk_log_size":5069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20739014,"num_objects":5065,"num_object_clones":0,"num_object_copies":10130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5065,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1838,"num_write_kb":7352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":891},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'5014","reported_seq":5136,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:45.147127+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:26:45.147127+0000","last_peered":"2024-09-17T15:26:45.147127+0000","last_clean":"2024-09-17T15:26:45.147127+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:26:45.147127+0000","last_undegraded":"2024-09-17T15:26:45.147127+0000","last_fullsized":"2024-09-17T15:26:45.147127+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":5014,"log_dups_size":0,"ondisk_log_size":5014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20529152,"num_objects":5012,"num_object_clones":0,"num_object_copies":10024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1968,"num_write_kb":7872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'4986","reported_seq":6481,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:26:46.782930+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:26:46.782930+0000","last_peered":"2024-09-17T15:26:46.782930+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:26:46.782930+0000","last_undegraded":"2024-09-17T15:26:46.782930+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":4986,"log_dups_size":0,"ondisk_log_size":4986,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20394467,"num_objects":4980,"num_object_clones":0,"num_object_copies":9960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3385,"num_objects_unfound":0,"num_objects_dirty":4980,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2986,"num_write_kb":11944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":686,"num_bytes_recovered":2809856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3385},{"shards":"2","objects":1595}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233552+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233552+0000","last_peered":"2024-09-17T15:26:22.233552+0000","last_clean":"2024-09-17T15:26:22.233552+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233552+0000","last_undegraded":"2024-09-17T15:26:22.233552+0000","last_fullsized":"2024-09-17T15:26:22.233552+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'4852","reported_seq":4975,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:45.098797+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:26:45.098797+0000","last_peered":"2024-09-17T15:26:45.098797+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:26:45.098797+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":4852,"log_dups_size":0,"ondisk_log_size":4852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19841024,"num_objects":4844,"num_object_clones":0,"num_object_copies":9688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1448,"num_write_kb":5792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":834},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":97,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412553+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412553+0000","last_peered":"2024-09-17T15:26:22.412553+0000","last_clean":"2024-09-17T15:26:22.412553+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412553+0000","last_undegraded":"2024-09-17T15:26:22.412553+0000","last_fullsized":"2024-09-17T15:26:22.412553+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'4972","reported_seq":5087,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:46.746182+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:26:46.746182+0000","last_peered":"2024-09-17T15:26:46.746182+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:26:46.746182+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":4972,"log_dups_size":0,"ondisk_log_size":4972,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20345315,"num_objects":4968,"num_object_clones":0,"num_object_copies":9936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4968,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2867,"num_write_kb":11468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":891}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'5011","reported_seq":5124,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:46.759504+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:26:46.759504+0000","last_peered":"2024-09-17T15:26:46.759504+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:26:46.759504+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":5011,"log_dups_size":0,"ondisk_log_size":5011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20500480,"num_objects":5005,"num_object_clones":0,"num_object_copies":10010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2773,"num_write_kb":11092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":868}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'5042","reported_seq":5172,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:45.187269+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:26:45.187269+0000","last_peered":"2024-09-17T15:26:45.187269+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:26:45.187269+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":5042,"log_dups_size":0,"ondisk_log_size":5042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20643840,"num_objects":5040,"num_object_clones":0,"num_object_copies":10080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1590,"num_write_kb":6360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":902},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'4974","reported_seq":5110,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:45.158690+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:26:45.158690+0000","last_peered":"2024-09-17T15:26:45.158690+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:26:45.158690+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":4974,"log_dups_size":0,"ondisk_log_size":4974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20340736,"num_objects":4966,"num_object_clones":0,"num_object_copies":9932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1261,"num_write_kb":5044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":809},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229013+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229013+0000","last_peered":"2024-09-17T15:26:22.229013+0000","last_clean":"2024-09-17T15:26:22.229013+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229013+0000","last_undegraded":"2024-09-17T15:26:22.229013+0000","last_fullsized":"2024-09-17T15:26:22.229013+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'4956","reported_seq":5082,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:45.138531+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:26:45.138531+0000","last_peered":"2024-09-17T15:26:45.138531+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:26:45.138531+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":4956,"log_dups_size":0,"ondisk_log_size":4956,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20283392,"num_objects":4952,"num_object_clones":0,"num_object_copies":9904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1743,"num_write_kb":6972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":889},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'4973","reported_seq":5110,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:45.126647+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:26:45.126647+0000","last_peered":"2024-09-17T15:26:45.126647+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:26:45.126647+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":4973,"log_dups_size":0,"ondisk_log_size":4973,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20321222,"num_objects":4963,"num_object_clones":0,"num_object_copies":9926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1415,"num_write_kb":5660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":854},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":326836958,"num_objects":79803,"num_object_clones":0,"num_object_copies":159606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":3385,"num_objects_unfound":0,"num_objects_dirty":79803,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":39910,"num_write_kb":159640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":686,"num_bytes_recovered":2809856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":653787136,"data_stored":653631777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79875,"ondisk_log_size":79875,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051605,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432116,"kb_used_data":292132,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93939724,"statfs":{"total":96636764160,"available":96194277376,"internally_reserved":0,"allocated":299143168,"data_stored":298876995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":143322796},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,1,2,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724550,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":135372,"kb_used_data":41212,"kb_used_omap":4,"kb_used_meta":94139,"kb_avail":94236468,"statfs":{"total":96636764160,"available":96498143232,"internally_reserved":0,"allocated":42201088,"data_stored":41883869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":96399019},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,4,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247833,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469716,"kb_used_data":303428,"kb_used_omap":4,"kb_used_meta":166267,"kb_avail":93902124,"statfs":{"total":96636764160,"available":96155774976,"internally_reserved":0,"allocated":310710272,"data_stored":310440022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":170258088},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,1,10,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":311541760,"data_stored":311505630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":43159552,"data_stored":43076453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":299085824,"data_stored":299049694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:26:48.020 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:26:48.020 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'4916", 'reported_seq': 4988, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:45.154480+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:26:45.154480+0000', 'last_peered': '2024-09-17T15:26:45.154480+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:26:45.154480+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 4916, 'log_dups_size': 0, 'ondisk_log_size': 4916, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20135936, 'num_objects': 4916, 'num_object_clones': 0, 'num_object_copies': 9832, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4070, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4916, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4916, 'num_write_kb': 19664, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 846}, {'shards': '0,2', 'objects': 4070}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:48.020 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:26:48.020 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'5125", 'reported_seq': 5245, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:46.790561+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:26:46.790561+0000', 'last_peered': '2024-09-17T15:26:46.790561+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:26:46.790561+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 5125, 'log_dups_size': 0, 'ondisk_log_size': 5125, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20975616, 'num_objects': 5121, 'num_object_clones': 0, 'num_object_copies': 10242, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4169, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5121, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3282, 'num_write_kb': 13128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4168}, {'shards': '2', 'objects': 953}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:26:48.020 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:26:48.021 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'5154", 'reported_seq': 5283, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:45.122535+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:26:45.122535+0000', 'last_peered': '2024-09-17T15:26:45.122535+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:26:45.122535+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 5154, 'log_dups_size': 0, 'ondisk_log_size': 5154, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21090787, 'num_objects': 5150, 'num_object_clones': 0, 'num_object_copies': 10300, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4271, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5150, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 1653, 'num_write_kb': 6612, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 880}, {'shards': '0,2', 'objects': 4268}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:48.021 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:26:48.021 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'5069", 'reported_seq': 5202, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:45.154619+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:26:45.154619+0000', 'last_peered': '2024-09-17T15:26:45.154619+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:26:45.154619+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 5069, 'log_dups_size': 0, 'ondisk_log_size': 5069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20739014, 'num_objects': 5065, 'num_object_clones': 0, 'num_object_copies': 10130, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4174, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5065, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 1838, 'num_write_kb': 7352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 891}, {'shards': '0,2', 'objects': 4174}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:48.021 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-17T15:26:48.021 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "41'4986", 'reported_seq': 6481, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:26:46.782930+0000', 'last_change': '2024-09-17T15:26:22.588242+0000', 'last_active': '2024-09-17T15:26:46.782930+0000', 'last_peered': '2024-09-17T15:26:46.782930+0000', 'last_clean': '2024-09-17T15:26:20.155646+0000', 'last_became_active': '2024-09-17T15:26:22.557742+0000', 'last_became_peered': '2024-09-17T15:26:22.557742+0000', 'last_unstale': '2024-09-17T15:26:46.782930+0000', 'last_undegraded': '2024-09-17T15:26:46.782930+0000', 'last_fullsized': '2024-09-17T15:26:22.188850+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2016", 'last_scrub_stamp': '2024-09-17T15:25:53.959011+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:53.959011+0000', 'objects_scrubbed': 1994, 'log_size': 4986, 'log_dups_size': 0, 'ondisk_log_size': 4986, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 568, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20394467, 'num_objects': 4980, 'num_object_clones': 0, 'num_object_copies': 9960, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3385, 'num_objects_unfound': 0, 'num_objects_dirty': 4980, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2986, 'num_write_kb': 11944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 686, 'num_bytes_recovered': 2809856, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3385}, {'shards': '2', 'objects': 1595}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:26:48.021 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:26:48.021 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'4852", 'reported_seq': 4975, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:45.098797+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:26:45.098797+0000', 'last_peered': '2024-09-17T15:26:45.098797+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:26:45.098797+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 4852, 'log_dups_size': 0, 'ondisk_log_size': 4852, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19841024, 'num_objects': 4844, 'num_object_clones': 0, 'num_object_copies': 9688, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4012, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4844, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1448, 'num_write_kb': 5792, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 834}, {'shards': '0,2', 'objects': 4010}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:48.021 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:26:48.021 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'4972", 'reported_seq': 5087, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:46.746182+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:26:46.746182+0000', 'last_peered': '2024-09-17T15:26:46.746182+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:26:46.746182+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 4972, 'log_dups_size': 0, 'ondisk_log_size': 4972, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20345315, 'num_objects': 4968, 'num_object_clones': 0, 'num_object_copies': 9936, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4078, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4968, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2867, 'num_write_kb': 11468, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4077}, {'shards': '2', 'objects': 891}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:26:48.021 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:26:48.021 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'5011", 'reported_seq': 5124, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:46.759504+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:26:46.759504+0000', 'last_peered': '2024-09-17T15:26:46.759504+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:26:46.759504+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 5011, 'log_dups_size': 0, 'ondisk_log_size': 5011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20500480, 'num_objects': 5005, 'num_object_clones': 0, 'num_object_copies': 10010, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5005, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2773, 'num_write_kb': 11092, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4137}, {'shards': '2', 'objects': 868}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:26:48.022 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:26:48.022 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'5042", 'reported_seq': 5172, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:45.187269+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:26:45.187269+0000', 'last_peered': '2024-09-17T15:26:45.187269+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:26:45.187269+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 5042, 'log_dups_size': 0, 'ondisk_log_size': 5042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20643840, 'num_objects': 5040, 'num_object_clones': 0, 'num_object_copies': 10080, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5040, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1590, 'num_write_kb': 6360, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 902}, {'shards': '0,2', 'objects': 4138}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:48.022 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:26:48.022 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'4974", 'reported_seq': 5110, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:45.158690+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:26:45.158690+0000', 'last_peered': '2024-09-17T15:26:45.158690+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:26:45.158690+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 4974, 'log_dups_size': 0, 'ondisk_log_size': 4974, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20340736, 'num_objects': 4966, 'num_object_clones': 0, 'num_object_copies': 9932, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4160, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4966, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1261, 'num_write_kb': 5044, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 809}, {'shards': '0,2', 'objects': 4157}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:48.022 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:26:48.022 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'4956", 'reported_seq': 5082, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:45.138531+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:26:45.138531+0000', 'last_peered': '2024-09-17T15:26:45.138531+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:26:45.138531+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 4956, 'log_dups_size': 0, 'ondisk_log_size': 4956, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20283392, 'num_objects': 4952, 'num_object_clones': 0, 'num_object_copies': 9904, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4065, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4952, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1743, 'num_write_kb': 6972, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 889}, {'shards': '0,2', 'objects': 4063}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:48.022 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:26:48.022 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'4973", 'reported_seq': 5110, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:45.126647+0000', 'last_change': '2024-09-17T15:26:22.654521+0000', 'last_active': '2024-09-17T15:26:45.126647+0000', 'last_peered': '2024-09-17T15:26:45.126647+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:26:45.126647+0000', 'last_undegraded': '2024-09-17T15:26:22.307795+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 4973, 'log_dups_size': 0, 'ondisk_log_size': 4973, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20321222, 'num_objects': 4963, 'num_object_clones': 0, 'num_object_copies': 9926, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4111, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4963, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1415, 'num_write_kb': 5660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 854}, {'shards': '0,2', 'objects': 4109}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:48.023 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:26:48.296 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 69 20 81283 81263 4.59962 2.16797 0.0201651 0.0169744 2024-09-17T15:26:48.727 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:26:48.727 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45388/159610 objects degraded (28.437%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":145,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":79805,"data_bytes":326836977,"bytes_used":1062096896,"bytes_avail":288848195584,"bytes_total":289910292480,"degraded_objects":45388,"degraded_total":159610,"degraded_ratio":0.28436814735918803,"misplaced_objects":3385,"misplaced_total":159610,"misplaced_ratio":0.0212079443643882,"recovering_objects_per_sec":34,"recovering_bytes_per_sec":141279,"recovering_keys_per_sec":0,"num_objects_recovered":414,"num_bytes_recovered":1695744,"num_keys_recovered":0,"write_bytes_sec":2470005,"read_op_per_sec":0,"write_op_per_sec":603},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-17T15:26:23.592815+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-17T15:26:48.751 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:26:48.752 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:26:49.296 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 70 20 81840 81820 4.56499 2.17578 0.0502749 0.0171065 2024-09-17T15:26:49.318 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:26:49.318 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:26:49.346 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":106,"stamp":"2024-09-17T15:26:47.598278+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":326836977,"num_objects":79805,"num_object_clones":0,"num_object_copies":159610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":3385,"num_objects_unfound":0,"num_objects_dirty":79805,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":39912,"num_write_kb":159642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":687,"num_bytes_recovered":2809875,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79878,"ondisk_log_size":79878,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1037204,"kb_used_data":636772,"kb_used_omap":13,"kb_used_meta":400370,"kb_avail":282078316,"statfs":{"total":289910292480,"available":288848195584,"internally_reserved":0,"allocated":652054528,"data_stored":651200886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":409979903},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,1,8,4,16,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29646848,"num_objects":7238,"num_object_clones":0,"num_object_copies":14476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-414,"num_objects_unfound":0,"num_objects_dirty":7238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7238,"num_write_kb":28952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":414,"num_bytes_recovered":1695744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002744"},"pg_stats":[{"pgid":"2.f","version":"41'4916","reported_seq":4988,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:45.154480+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:26:45.154480+0000","last_peered":"2024-09-17T15:26:45.154480+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:26:45.154480+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":4916,"log_dups_size":0,"ondisk_log_size":4916,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20135936,"num_objects":4916,"num_object_clones":0,"num_object_copies":9832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4916,"num_write_kb":19664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":846},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'4976","reported_seq":5073,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:46.743572+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:26:46.743572+0000","last_peered":"2024-09-17T15:26:46.743572+0000","last_clean":"2024-09-17T15:26:46.743572+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:26:46.743572+0000","last_undegraded":"2024-09-17T15:26:46.743572+0000","last_fullsized":"2024-09-17T15:26:46.743572+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":4976,"log_dups_size":0,"ondisk_log_size":4976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20358086,"num_objects":4972,"num_object_clones":0,"num_object_copies":9944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4972,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3333,"num_write_kb":13332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'5125","reported_seq":5245,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:46.790561+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:26:46.790561+0000","last_peered":"2024-09-17T15:26:46.790561+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:26:46.790561+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":5125,"log_dups_size":0,"ondisk_log_size":5125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20975616,"num_objects":5121,"num_object_clones":0,"num_object_copies":10242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3282,"num_write_kb":13128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":953}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'5022","reported_seq":5085,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:46.752155+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:26:46.752155+0000","last_peered":"2024-09-17T15:26:46.752155+0000","last_clean":"2024-09-17T15:26:46.752155+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:26:46.752155+0000","last_undegraded":"2024-09-17T15:26:46.752155+0000","last_fullsized":"2024-09-17T15:26:46.752155+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":5022,"log_dups_size":0,"ondisk_log_size":5022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20570112,"num_objects":5022,"num_object_clones":0,"num_object_copies":10044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5022,"num_write_kb":20088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'4833","reported_seq":4957,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:45.175904+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:26:45.175904+0000","last_peered":"2024-09-17T15:26:45.175904+0000","last_clean":"2024-09-17T15:26:45.175904+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:26:45.175904+0000","last_undegraded":"2024-09-17T15:26:45.175904+0000","last_fullsized":"2024-09-17T15:26:45.175904+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":4833,"log_dups_size":0,"ondisk_log_size":4833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19767779,"num_objects":4827,"num_object_clones":0,"num_object_copies":9654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4827,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1815,"num_write_kb":7260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'5154","reported_seq":5283,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:45.122535+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:26:45.122535+0000","last_peered":"2024-09-17T15:26:45.122535+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:26:45.122535+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":5154,"log_dups_size":0,"ondisk_log_size":5154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21090787,"num_objects":5150,"num_object_clones":0,"num_object_copies":10300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5150,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1653,"num_write_kb":6612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":880},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'5069","reported_seq":5202,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:45.154619+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:26:45.154619+0000","last_peered":"2024-09-17T15:26:45.154619+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:26:45.154619+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":5069,"log_dups_size":0,"ondisk_log_size":5069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20739014,"num_objects":5065,"num_object_clones":0,"num_object_copies":10130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5065,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1838,"num_write_kb":7352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":891},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'5014","reported_seq":5136,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:45.147127+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:26:45.147127+0000","last_peered":"2024-09-17T15:26:45.147127+0000","last_clean":"2024-09-17T15:26:45.147127+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:26:45.147127+0000","last_undegraded":"2024-09-17T15:26:45.147127+0000","last_fullsized":"2024-09-17T15:26:45.147127+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":5014,"log_dups_size":0,"ondisk_log_size":5014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20529152,"num_objects":5012,"num_object_clones":0,"num_object_copies":10024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1968,"num_write_kb":7872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'4986","reported_seq":6481,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:26:46.782930+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:26:46.782930+0000","last_peered":"2024-09-17T15:26:46.782930+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:26:46.782930+0000","last_undegraded":"2024-09-17T15:26:46.782930+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":4986,"log_dups_size":0,"ondisk_log_size":4986,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20394467,"num_objects":4980,"num_object_clones":0,"num_object_copies":9960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3385,"num_objects_unfound":0,"num_objects_dirty":4980,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2986,"num_write_kb":11944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":686,"num_bytes_recovered":2809856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3385},{"shards":"2","objects":1595}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233552+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233552+0000","last_peered":"2024-09-17T15:26:22.233552+0000","last_clean":"2024-09-17T15:26:22.233552+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233552+0000","last_undegraded":"2024-09-17T15:26:22.233552+0000","last_fullsized":"2024-09-17T15:26:22.233552+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'4852","reported_seq":4975,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:45.098797+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:26:45.098797+0000","last_peered":"2024-09-17T15:26:45.098797+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:26:45.098797+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":4852,"log_dups_size":0,"ondisk_log_size":4852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19841024,"num_objects":4844,"num_object_clones":0,"num_object_copies":9688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1448,"num_write_kb":5792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":834},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":97,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412553+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412553+0000","last_peered":"2024-09-17T15:26:22.412553+0000","last_clean":"2024-09-17T15:26:22.412553+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412553+0000","last_undegraded":"2024-09-17T15:26:22.412553+0000","last_fullsized":"2024-09-17T15:26:22.412553+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'4972","reported_seq":5087,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:46.746182+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:26:46.746182+0000","last_peered":"2024-09-17T15:26:46.746182+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:26:46.746182+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":4972,"log_dups_size":0,"ondisk_log_size":4972,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20345315,"num_objects":4968,"num_object_clones":0,"num_object_copies":9936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4968,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2867,"num_write_kb":11468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":891}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'5011","reported_seq":5124,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:46.759504+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:26:46.759504+0000","last_peered":"2024-09-17T15:26:46.759504+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:26:46.759504+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":5011,"log_dups_size":0,"ondisk_log_size":5011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20500480,"num_objects":5005,"num_object_clones":0,"num_object_copies":10010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2773,"num_write_kb":11092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":868}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'5042","reported_seq":5172,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:45.187269+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:26:45.187269+0000","last_peered":"2024-09-17T15:26:45.187269+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:26:45.187269+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":5042,"log_dups_size":0,"ondisk_log_size":5042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20643840,"num_objects":5040,"num_object_clones":0,"num_object_copies":10080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1590,"num_write_kb":6360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":902},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'4974","reported_seq":5110,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:45.158690+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:26:45.158690+0000","last_peered":"2024-09-17T15:26:45.158690+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:26:45.158690+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":4974,"log_dups_size":0,"ondisk_log_size":4974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20340736,"num_objects":4966,"num_object_clones":0,"num_object_copies":9932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1261,"num_write_kb":5044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":809},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229013+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229013+0000","last_peered":"2024-09-17T15:26:22.229013+0000","last_clean":"2024-09-17T15:26:22.229013+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229013+0000","last_undegraded":"2024-09-17T15:26:22.229013+0000","last_fullsized":"2024-09-17T15:26:22.229013+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'4956","reported_seq":5082,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:45.138531+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:26:45.138531+0000","last_peered":"2024-09-17T15:26:45.138531+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:26:45.138531+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":4956,"log_dups_size":0,"ondisk_log_size":4956,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20283392,"num_objects":4952,"num_object_clones":0,"num_object_copies":9904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1743,"num_write_kb":6972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":889},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'4973","reported_seq":5110,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:45.126647+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:26:45.126647+0000","last_peered":"2024-09-17T15:26:45.126647+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:26:45.126647+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":4973,"log_dups_size":0,"ondisk_log_size":4973,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20321222,"num_objects":4963,"num_object_clones":0,"num_object_copies":9926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1415,"num_write_kb":5660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":854},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":326836958,"num_objects":79803,"num_object_clones":0,"num_object_copies":159606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":3385,"num_objects_unfound":0,"num_objects_dirty":79803,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":39910,"num_write_kb":159640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":686,"num_bytes_recovered":2809856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":653787136,"data_stored":653631777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79875,"ondisk_log_size":79875,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051605,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432116,"kb_used_data":292132,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93939724,"statfs":{"total":96636764160,"available":96194277376,"internally_reserved":0,"allocated":299143168,"data_stored":298876995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":143322796},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,1,2,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724550,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":135372,"kb_used_data":41212,"kb_used_omap":4,"kb_used_meta":94139,"kb_avail":94236468,"statfs":{"total":96636764160,"available":96498143232,"internally_reserved":0,"allocated":42201088,"data_stored":41883869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":96399019},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,4,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247833,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469716,"kb_used_data":303428,"kb_used_omap":4,"kb_used_meta":166267,"kb_avail":93902124,"statfs":{"total":96636764160,"available":96155774976,"internally_reserved":0,"allocated":310710272,"data_stored":310440022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":170258088},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,1,10,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":311541760,"data_stored":311505630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":43159552,"data_stored":43076453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":299085824,"data_stored":299049694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:26:50.296 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 71 20 82472 82452 4.53547 2.46875 0.0419365 0.0172176 2024-09-17T15:26:51.296 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 72 20 83096 83076 4.50633 2.4375 0.0166521 0.0173262 2024-09-17T15:26:52.297 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 73 20 83729 83709 4.47847 2.47266 0.0588638 0.0174357 2024-09-17T15:26:52.348 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:26:52.908 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:26:52.908 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:26:52.935 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":108,"stamp":"2024-09-17T15:26:51.599481+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":334500593,"num_objects":81676,"num_object_clones":0,"num_object_copies":163352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":3385,"num_objects_unfound":0,"num_objects_dirty":81676,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":41783,"num_write_kb":167126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":687,"num_bytes_recovered":2809875,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81749,"ondisk_log_size":81749,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1030492,"kb_used_data":655016,"kb_used_omap":13,"kb_used_meta":375410,"kb_avail":282085028,"statfs":{"total":289910292480,"available":288855068672,"internally_reserved":0,"allocated":670736384,"data_stored":669882742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":384420863},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,0,7,8,11,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32432128,"num_objects":7918,"num_object_clones":0,"num_object_copies":15836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-282,"num_objects_unfound":0,"num_objects_dirty":7918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7918,"num_write_kb":31672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":1155072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003015"},"pg_stats":[{"pgid":"2.f","version":"41'5097","reported_seq":5169,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:50.207460+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:26:50.207460+0000","last_peered":"2024-09-17T15:26:50.207460+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:26:50.207460+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":5097,"log_dups_size":0,"ondisk_log_size":5097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20877312,"num_objects":5097,"num_object_clones":0,"num_object_copies":10194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5097,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5097,"num_write_kb":20388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1027},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'4976","reported_seq":5073,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:46.743572+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:26:46.743572+0000","last_peered":"2024-09-17T15:26:46.743572+0000","last_clean":"2024-09-17T15:26:46.743572+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:26:46.743572+0000","last_undegraded":"2024-09-17T15:26:46.743572+0000","last_fullsized":"2024-09-17T15:26:46.743572+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":4976,"log_dups_size":0,"ondisk_log_size":4976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20358086,"num_objects":4972,"num_object_clones":0,"num_object_copies":9944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4972,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3333,"num_write_kb":13332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'5125","reported_seq":5245,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:46.790561+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:26:46.790561+0000","last_peered":"2024-09-17T15:26:46.790561+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:26:46.790561+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":5125,"log_dups_size":0,"ondisk_log_size":5125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20975616,"num_objects":5121,"num_object_clones":0,"num_object_copies":10242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3282,"num_write_kb":13128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":953}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'5022","reported_seq":5085,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:46.752155+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:26:46.752155+0000","last_peered":"2024-09-17T15:26:46.752155+0000","last_clean":"2024-09-17T15:26:46.752155+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:26:46.752155+0000","last_undegraded":"2024-09-17T15:26:46.752155+0000","last_fullsized":"2024-09-17T15:26:46.752155+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":5022,"log_dups_size":0,"ondisk_log_size":5022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20570112,"num_objects":5022,"num_object_clones":0,"num_object_copies":10044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5022,"num_write_kb":20088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'5028","reported_seq":5152,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:50.104289+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:26:50.104289+0000","last_peered":"2024-09-17T15:26:50.104289+0000","last_clean":"2024-09-17T15:26:50.104289+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:26:50.104289+0000","last_undegraded":"2024-09-17T15:26:50.104289+0000","last_fullsized":"2024-09-17T15:26:50.104289+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":5028,"log_dups_size":0,"ondisk_log_size":5028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20566499,"num_objects":5022,"num_object_clones":0,"num_object_copies":10044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5022,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2010,"num_write_kb":8040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'5349","reported_seq":5478,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:50.182340+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:26:50.182340+0000","last_peered":"2024-09-17T15:26:50.182340+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:26:50.182340+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":5349,"log_dups_size":0,"ondisk_log_size":5349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21889507,"num_objects":5345,"num_object_clones":0,"num_object_copies":10690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5345,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1848,"num_write_kb":7392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1075},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'5230","reported_seq":5363,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:50.170821+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:26:50.170821+0000","last_peered":"2024-09-17T15:26:50.170821+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:26:50.170821+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":5230,"log_dups_size":0,"ondisk_log_size":5230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21398470,"num_objects":5226,"num_object_clones":0,"num_object_copies":10452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5226,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1999,"num_write_kb":7996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1052},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'5193","reported_seq":5315,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:50.211031+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:26:50.211031+0000","last_peered":"2024-09-17T15:26:50.211031+0000","last_clean":"2024-09-17T15:26:50.211031+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:26:50.211031+0000","last_undegraded":"2024-09-17T15:26:50.211031+0000","last_fullsized":"2024-09-17T15:26:50.211031+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":5193,"log_dups_size":0,"ondisk_log_size":5193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21262336,"num_objects":5191,"num_object_clones":0,"num_object_copies":10382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2147,"num_write_kb":8588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'4986","reported_seq":6481,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:26:46.782930+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:26:46.782930+0000","last_peered":"2024-09-17T15:26:46.782930+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:26:46.782930+0000","last_undegraded":"2024-09-17T15:26:46.782930+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":4986,"log_dups_size":0,"ondisk_log_size":4986,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20394467,"num_objects":4980,"num_object_clones":0,"num_object_copies":9960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3385,"num_objects_unfound":0,"num_objects_dirty":4980,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2986,"num_write_kb":11944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":686,"num_bytes_recovered":2809856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3385},{"shards":"2","objects":1595}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233552+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233552+0000","last_peered":"2024-09-17T15:26:22.233552+0000","last_clean":"2024-09-17T15:26:22.233552+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233552+0000","last_undegraded":"2024-09-17T15:26:22.233552+0000","last_fullsized":"2024-09-17T15:26:22.233552+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'5027","reported_seq":5152,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:50.175271+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:26:50.175271+0000","last_peered":"2024-09-17T15:26:50.175271+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:26:50.175271+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":5027,"log_dups_size":0,"ondisk_log_size":5027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20557824,"num_objects":5019,"num_object_clones":0,"num_object_copies":10038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1623,"num_write_kb":6492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1009},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":97,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412553+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412553+0000","last_peered":"2024-09-17T15:26:22.412553+0000","last_clean":"2024-09-17T15:26:22.412553+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412553+0000","last_undegraded":"2024-09-17T15:26:22.412553+0000","last_fullsized":"2024-09-17T15:26:22.412553+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'4972","reported_seq":5087,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:46.746182+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:26:46.746182+0000","last_peered":"2024-09-17T15:26:46.746182+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:26:46.746182+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":4972,"log_dups_size":0,"ondisk_log_size":4972,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20345315,"num_objects":4968,"num_object_clones":0,"num_object_copies":9936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4968,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2867,"num_write_kb":11468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":891}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'5011","reported_seq":5124,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:46.759504+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:26:46.759504+0000","last_peered":"2024-09-17T15:26:46.759504+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:26:46.759504+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":5011,"log_dups_size":0,"ondisk_log_size":5011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20500480,"num_objects":5005,"num_object_clones":0,"num_object_copies":10010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2773,"num_write_kb":11092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":868}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'5236","reported_seq":5366,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:50.136408+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:26:50.136408+0000","last_peered":"2024-09-17T15:26:50.136408+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:26:50.136408+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":5236,"log_dups_size":0,"ondisk_log_size":5236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21438464,"num_objects":5234,"num_object_clones":0,"num_object_copies":10468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1784,"num_write_kb":7136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1096},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'5175","reported_seq":5311,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:50.191230+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:26:50.191230+0000","last_peered":"2024-09-17T15:26:50.191230+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:26:50.191230+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":5175,"log_dups_size":0,"ondisk_log_size":5175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21164032,"num_objects":5167,"num_object_clones":0,"num_object_copies":10334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1462,"num_write_kb":5848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1010},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229013+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229013+0000","last_peered":"2024-09-17T15:26:22.229013+0000","last_clean":"2024-09-17T15:26:22.229013+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229013+0000","last_undegraded":"2024-09-17T15:26:22.229013+0000","last_fullsized":"2024-09-17T15:26:22.229013+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'5161","reported_seq":5287,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:50.181168+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:26:50.181168+0000","last_peered":"2024-09-17T15:26:50.181168+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:26:50.181168+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":5161,"log_dups_size":0,"ondisk_log_size":5161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21123072,"num_objects":5157,"num_object_clones":0,"num_object_copies":10314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1948,"num_write_kb":7792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1094},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'5158","reported_seq":5295,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:50.164588+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:26:50.164588+0000","last_peered":"2024-09-17T15:26:50.164588+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:26:50.164588+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":5158,"log_dups_size":0,"ondisk_log_size":5158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21078982,"num_objects":5148,"num_object_clones":0,"num_object_copies":10296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1600,"num_write_kb":6400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1039},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":334500574,"num_objects":81674,"num_object_clones":0,"num_object_copies":163348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":3385,"num_objects_unfound":0,"num_objects_dirty":81674,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":41781,"num_write_kb":167124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":686,"num_bytes_recovered":2809856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":672866304,"data_stored":672710945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81746,"ondisk_log_size":81746,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051605,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432116,"kb_used_data":292132,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93939724,"statfs":{"total":96636764160,"available":96194277376,"internally_reserved":0,"allocated":299143168,"data_stored":298876995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":143322796},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,1,2,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724551,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":132460,"kb_used_data":50780,"kb_used_omap":4,"kb_used_meta":81659,"kb_avail":94239380,"statfs":{"total":96636764160,"available":96501125120,"internally_reserved":0,"allocated":51998720,"data_stored":51681501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":83619499},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,4,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247834,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465916,"kb_used_data":312104,"kb_used_omap":4,"kb_used_meta":153787,"kb_avail":93905924,"statfs":{"total":96636764160,"available":96159666176,"internally_reserved":0,"allocated":319594496,"data_stored":319324246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":157478568},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,5,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":320806912,"data_stored":320770782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":52973568,"data_stored":52890469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":299085824,"data_stored":299049694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:26:52.936 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:26:53.296 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 74 20 84282 84262 4.44714 2.16016 0.0249385 0.0175576 2024-09-17T15:26:53.515 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:26:53.516 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:26:53.542 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":108,"stamp":"2024-09-17T15:26:51.599481+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":334500593,"num_objects":81676,"num_object_clones":0,"num_object_copies":163352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":3385,"num_objects_unfound":0,"num_objects_dirty":81676,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":41783,"num_write_kb":167126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":687,"num_bytes_recovered":2809875,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81749,"ondisk_log_size":81749,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1030492,"kb_used_data":655016,"kb_used_omap":13,"kb_used_meta":375410,"kb_avail":282085028,"statfs":{"total":289910292480,"available":288855068672,"internally_reserved":0,"allocated":670736384,"data_stored":669882742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":384420863},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,0,7,8,11,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32432128,"num_objects":7918,"num_object_clones":0,"num_object_copies":15836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-282,"num_objects_unfound":0,"num_objects_dirty":7918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7918,"num_write_kb":31672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":1155072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003015"},"pg_stats":[{"pgid":"2.f","version":"41'5097","reported_seq":5169,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:50.207460+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:26:50.207460+0000","last_peered":"2024-09-17T15:26:50.207460+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:26:50.207460+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":5097,"log_dups_size":0,"ondisk_log_size":5097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20877312,"num_objects":5097,"num_object_clones":0,"num_object_copies":10194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5097,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5097,"num_write_kb":20388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1027},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'4976","reported_seq":5073,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:46.743572+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:26:46.743572+0000","last_peered":"2024-09-17T15:26:46.743572+0000","last_clean":"2024-09-17T15:26:46.743572+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:26:46.743572+0000","last_undegraded":"2024-09-17T15:26:46.743572+0000","last_fullsized":"2024-09-17T15:26:46.743572+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":4976,"log_dups_size":0,"ondisk_log_size":4976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20358086,"num_objects":4972,"num_object_clones":0,"num_object_copies":9944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4972,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3333,"num_write_kb":13332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'5125","reported_seq":5245,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:46.790561+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:26:46.790561+0000","last_peered":"2024-09-17T15:26:46.790561+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:26:46.790561+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":5125,"log_dups_size":0,"ondisk_log_size":5125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20975616,"num_objects":5121,"num_object_clones":0,"num_object_copies":10242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3282,"num_write_kb":13128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":953}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'5022","reported_seq":5085,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:46.752155+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:26:46.752155+0000","last_peered":"2024-09-17T15:26:46.752155+0000","last_clean":"2024-09-17T15:26:46.752155+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:26:46.752155+0000","last_undegraded":"2024-09-17T15:26:46.752155+0000","last_fullsized":"2024-09-17T15:26:46.752155+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":5022,"log_dups_size":0,"ondisk_log_size":5022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20570112,"num_objects":5022,"num_object_clones":0,"num_object_copies":10044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5022,"num_write_kb":20088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'5028","reported_seq":5152,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:50.104289+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:26:50.104289+0000","last_peered":"2024-09-17T15:26:50.104289+0000","last_clean":"2024-09-17T15:26:50.104289+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:26:50.104289+0000","last_undegraded":"2024-09-17T15:26:50.104289+0000","last_fullsized":"2024-09-17T15:26:50.104289+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":5028,"log_dups_size":0,"ondisk_log_size":5028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20566499,"num_objects":5022,"num_object_clones":0,"num_object_copies":10044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5022,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2010,"num_write_kb":8040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'5349","reported_seq":5478,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:50.182340+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:26:50.182340+0000","last_peered":"2024-09-17T15:26:50.182340+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:26:50.182340+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":5349,"log_dups_size":0,"ondisk_log_size":5349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21889507,"num_objects":5345,"num_object_clones":0,"num_object_copies":10690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5345,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1848,"num_write_kb":7392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1075},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'5230","reported_seq":5363,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:50.170821+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:26:50.170821+0000","last_peered":"2024-09-17T15:26:50.170821+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:26:50.170821+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":5230,"log_dups_size":0,"ondisk_log_size":5230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21398470,"num_objects":5226,"num_object_clones":0,"num_object_copies":10452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5226,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1999,"num_write_kb":7996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1052},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'5193","reported_seq":5315,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:50.211031+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:26:50.211031+0000","last_peered":"2024-09-17T15:26:50.211031+0000","last_clean":"2024-09-17T15:26:50.211031+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:26:50.211031+0000","last_undegraded":"2024-09-17T15:26:50.211031+0000","last_fullsized":"2024-09-17T15:26:50.211031+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":5193,"log_dups_size":0,"ondisk_log_size":5193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21262336,"num_objects":5191,"num_object_clones":0,"num_object_copies":10382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2147,"num_write_kb":8588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'4986","reported_seq":6481,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:26:46.782930+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:26:46.782930+0000","last_peered":"2024-09-17T15:26:46.782930+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:26:46.782930+0000","last_undegraded":"2024-09-17T15:26:46.782930+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":4986,"log_dups_size":0,"ondisk_log_size":4986,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20394467,"num_objects":4980,"num_object_clones":0,"num_object_copies":9960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3385,"num_objects_unfound":0,"num_objects_dirty":4980,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2986,"num_write_kb":11944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":686,"num_bytes_recovered":2809856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3385},{"shards":"2","objects":1595}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233552+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233552+0000","last_peered":"2024-09-17T15:26:22.233552+0000","last_clean":"2024-09-17T15:26:22.233552+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233552+0000","last_undegraded":"2024-09-17T15:26:22.233552+0000","last_fullsized":"2024-09-17T15:26:22.233552+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'5027","reported_seq":5152,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:50.175271+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:26:50.175271+0000","last_peered":"2024-09-17T15:26:50.175271+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:26:50.175271+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":5027,"log_dups_size":0,"ondisk_log_size":5027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20557824,"num_objects":5019,"num_object_clones":0,"num_object_copies":10038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1623,"num_write_kb":6492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1009},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":97,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412553+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412553+0000","last_peered":"2024-09-17T15:26:22.412553+0000","last_clean":"2024-09-17T15:26:22.412553+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412553+0000","last_undegraded":"2024-09-17T15:26:22.412553+0000","last_fullsized":"2024-09-17T15:26:22.412553+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'4972","reported_seq":5087,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:46.746182+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:26:46.746182+0000","last_peered":"2024-09-17T15:26:46.746182+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:26:46.746182+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":4972,"log_dups_size":0,"ondisk_log_size":4972,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20345315,"num_objects":4968,"num_object_clones":0,"num_object_copies":9936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4968,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2867,"num_write_kb":11468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":891}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'5011","reported_seq":5124,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:46.759504+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:26:46.759504+0000","last_peered":"2024-09-17T15:26:46.759504+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:26:46.759504+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":5011,"log_dups_size":0,"ondisk_log_size":5011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20500480,"num_objects":5005,"num_object_clones":0,"num_object_copies":10010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2773,"num_write_kb":11092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":868}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'5236","reported_seq":5366,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:50.136408+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:26:50.136408+0000","last_peered":"2024-09-17T15:26:50.136408+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:26:50.136408+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":5236,"log_dups_size":0,"ondisk_log_size":5236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21438464,"num_objects":5234,"num_object_clones":0,"num_object_copies":10468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1784,"num_write_kb":7136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1096},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'5175","reported_seq":5311,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:50.191230+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:26:50.191230+0000","last_peered":"2024-09-17T15:26:50.191230+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:26:50.191230+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":5175,"log_dups_size":0,"ondisk_log_size":5175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21164032,"num_objects":5167,"num_object_clones":0,"num_object_copies":10334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1462,"num_write_kb":5848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1010},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229013+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229013+0000","last_peered":"2024-09-17T15:26:22.229013+0000","last_clean":"2024-09-17T15:26:22.229013+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229013+0000","last_undegraded":"2024-09-17T15:26:22.229013+0000","last_fullsized":"2024-09-17T15:26:22.229013+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'5161","reported_seq":5287,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:50.181168+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:26:50.181168+0000","last_peered":"2024-09-17T15:26:50.181168+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:26:50.181168+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":5161,"log_dups_size":0,"ondisk_log_size":5161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21123072,"num_objects":5157,"num_object_clones":0,"num_object_copies":10314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1948,"num_write_kb":7792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1094},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'5158","reported_seq":5295,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:50.164588+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:26:50.164588+0000","last_peered":"2024-09-17T15:26:50.164588+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:26:50.164588+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":5158,"log_dups_size":0,"ondisk_log_size":5158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21078982,"num_objects":5148,"num_object_clones":0,"num_object_copies":10296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1600,"num_write_kb":6400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1039},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":334500574,"num_objects":81674,"num_object_clones":0,"num_object_copies":163348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":3385,"num_objects_unfound":0,"num_objects_dirty":81674,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":41781,"num_write_kb":167124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":686,"num_bytes_recovered":2809856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":672866304,"data_stored":672710945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81746,"ondisk_log_size":81746,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051605,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432116,"kb_used_data":292132,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93939724,"statfs":{"total":96636764160,"available":96194277376,"internally_reserved":0,"allocated":299143168,"data_stored":298876995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":143322796},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,1,2,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724551,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":132460,"kb_used_data":50780,"kb_used_omap":4,"kb_used_meta":81659,"kb_avail":94239380,"statfs":{"total":96636764160,"available":96501125120,"internally_reserved":0,"allocated":51998720,"data_stored":51681501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":83619499},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,4,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247834,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465916,"kb_used_data":312104,"kb_used_omap":4,"kb_used_meta":153787,"kb_avail":93905924,"statfs":{"total":96636764160,"available":96159666176,"internally_reserved":0,"allocated":319594496,"data_stored":319324246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":157478568},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,5,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":320806912,"data_stored":320770782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":52973568,"data_stored":52890469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":299085824,"data_stored":299049694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:26:53.543 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:26:53.543 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'5097", 'reported_seq': 5169, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:50.207460+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:26:50.207460+0000', 'last_peered': '2024-09-17T15:26:50.207460+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:26:50.207460+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 5097, 'log_dups_size': 0, 'ondisk_log_size': 5097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20877312, 'num_objects': 5097, 'num_object_clones': 0, 'num_object_copies': 10194, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4070, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5097, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5097, 'num_write_kb': 20388, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1027}, {'shards': '0,2', 'objects': 4070}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:53.543 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:26:53.543 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'5125", 'reported_seq': 5245, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:46.790561+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:26:46.790561+0000', 'last_peered': '2024-09-17T15:26:46.790561+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:26:46.790561+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 5125, 'log_dups_size': 0, 'ondisk_log_size': 5125, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20975616, 'num_objects': 5121, 'num_object_clones': 0, 'num_object_copies': 10242, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4169, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5121, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3282, 'num_write_kb': 13128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4168}, {'shards': '2', 'objects': 953}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:26:53.543 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:26:53.543 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'5349", 'reported_seq': 5478, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:50.182340+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:26:50.182340+0000', 'last_peered': '2024-09-17T15:26:50.182340+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:26:50.182340+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 5349, 'log_dups_size': 0, 'ondisk_log_size': 5349, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21889507, 'num_objects': 5345, 'num_object_clones': 0, 'num_object_copies': 10690, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4271, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5345, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 1848, 'num_write_kb': 7392, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1075}, {'shards': '0,2', 'objects': 4268}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:53.544 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:26:53.544 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'5230", 'reported_seq': 5363, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:50.170821+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:26:50.170821+0000', 'last_peered': '2024-09-17T15:26:50.170821+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:26:50.170821+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 5230, 'log_dups_size': 0, 'ondisk_log_size': 5230, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21398470, 'num_objects': 5226, 'num_object_clones': 0, 'num_object_copies': 10452, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4174, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5226, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 1999, 'num_write_kb': 7996, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1052}, {'shards': '0,2', 'objects': 4174}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:53.544 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-17T15:26:53.544 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "41'4986", 'reported_seq': 6481, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:26:46.782930+0000', 'last_change': '2024-09-17T15:26:22.588242+0000', 'last_active': '2024-09-17T15:26:46.782930+0000', 'last_peered': '2024-09-17T15:26:46.782930+0000', 'last_clean': '2024-09-17T15:26:20.155646+0000', 'last_became_active': '2024-09-17T15:26:22.557742+0000', 'last_became_peered': '2024-09-17T15:26:22.557742+0000', 'last_unstale': '2024-09-17T15:26:46.782930+0000', 'last_undegraded': '2024-09-17T15:26:46.782930+0000', 'last_fullsized': '2024-09-17T15:26:22.188850+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2016", 'last_scrub_stamp': '2024-09-17T15:25:53.959011+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:53.959011+0000', 'objects_scrubbed': 1994, 'log_size': 4986, 'log_dups_size': 0, 'ondisk_log_size': 4986, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 568, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20394467, 'num_objects': 4980, 'num_object_clones': 0, 'num_object_copies': 9960, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3385, 'num_objects_unfound': 0, 'num_objects_dirty': 4980, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2986, 'num_write_kb': 11944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 686, 'num_bytes_recovered': 2809856, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3385}, {'shards': '2', 'objects': 1595}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:26:53.544 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:26:53.544 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'5027", 'reported_seq': 5152, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:50.175271+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:26:50.175271+0000', 'last_peered': '2024-09-17T15:26:50.175271+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:26:50.175271+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 5027, 'log_dups_size': 0, 'ondisk_log_size': 5027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20557824, 'num_objects': 5019, 'num_object_clones': 0, 'num_object_copies': 10038, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4012, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5019, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1623, 'num_write_kb': 6492, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1009}, {'shards': '0,2', 'objects': 4010}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:53.544 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:26:53.544 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'4972", 'reported_seq': 5087, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:46.746182+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:26:46.746182+0000', 'last_peered': '2024-09-17T15:26:46.746182+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:26:46.746182+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 4972, 'log_dups_size': 0, 'ondisk_log_size': 4972, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20345315, 'num_objects': 4968, 'num_object_clones': 0, 'num_object_copies': 9936, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4078, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4968, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2867, 'num_write_kb': 11468, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4077}, {'shards': '2', 'objects': 891}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:26:53.544 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:26:53.545 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'5011", 'reported_seq': 5124, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:46.759504+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:26:46.759504+0000', 'last_peered': '2024-09-17T15:26:46.759504+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:26:46.759504+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 5011, 'log_dups_size': 0, 'ondisk_log_size': 5011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20500480, 'num_objects': 5005, 'num_object_clones': 0, 'num_object_copies': 10010, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5005, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2773, 'num_write_kb': 11092, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4137}, {'shards': '2', 'objects': 868}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:26:53.545 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:26:53.545 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'5236", 'reported_seq': 5366, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:50.136408+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:26:50.136408+0000', 'last_peered': '2024-09-17T15:26:50.136408+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:26:50.136408+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 5236, 'log_dups_size': 0, 'ondisk_log_size': 5236, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21438464, 'num_objects': 5234, 'num_object_clones': 0, 'num_object_copies': 10468, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5234, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1784, 'num_write_kb': 7136, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1096}, {'shards': '0,2', 'objects': 4138}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:53.545 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:26:53.545 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'5175", 'reported_seq': 5311, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:50.191230+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:26:50.191230+0000', 'last_peered': '2024-09-17T15:26:50.191230+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:26:50.191230+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 5175, 'log_dups_size': 0, 'ondisk_log_size': 5175, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21164032, 'num_objects': 5167, 'num_object_clones': 0, 'num_object_copies': 10334, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4160, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5167, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1462, 'num_write_kb': 5848, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1010}, {'shards': '0,2', 'objects': 4157}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:53.545 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:26:53.545 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'5161", 'reported_seq': 5287, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:50.181168+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:26:50.181168+0000', 'last_peered': '2024-09-17T15:26:50.181168+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:26:50.181168+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 5161, 'log_dups_size': 0, 'ondisk_log_size': 5161, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21123072, 'num_objects': 5157, 'num_object_clones': 0, 'num_object_copies': 10314, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4065, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5157, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1948, 'num_write_kb': 7792, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1094}, {'shards': '0,2', 'objects': 4063}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:53.545 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:26:53.545 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'5158", 'reported_seq': 5295, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:50.164588+0000', 'last_change': '2024-09-17T15:26:22.654521+0000', 'last_active': '2024-09-17T15:26:50.164588+0000', 'last_peered': '2024-09-17T15:26:50.164588+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:26:50.164588+0000', 'last_undegraded': '2024-09-17T15:26:22.307795+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 5158, 'log_dups_size': 0, 'ondisk_log_size': 5158, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21078982, 'num_objects': 5148, 'num_object_clones': 0, 'num_object_copies': 10296, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4111, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5148, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1600, 'num_write_kb': 6400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1039}, {'shards': '0,2', 'objects': 4109}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:53.546 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:26:54.225 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:26:54.225 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45388/163352 objects degraded (27.785%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":151,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":81676,"data_bytes":334500593,"bytes_used":1055223808,"bytes_avail":288855068672,"bytes_total":289910292480,"degraded_objects":45388,"degraded_total":163352,"degraded_ratio":0.27785395954748027,"misplaced_objects":3385,"misplaced_total":163352,"misplaced_ratio":0.020722121553455115,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":96231,"recovering_keys_per_sec":0,"num_objects_recovered":282,"num_bytes_recovered":1155072,"num_keys_recovered":0,"write_bytes_sec":2701998,"read_op_per_sec":0,"write_op_per_sec":659},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-17T15:26:23.592815+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-17T15:26:54.252 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:26:54.252 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:26:54.296 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 75 20 84819 84799 4.41581 2.09766 0.0680579 0.017684 2024-09-17T15:26:54.803 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:26:54.807 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:26:54.834 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":109,"stamp":"2024-09-17T15:26:53.603100+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":339190513,"num_objects":82821,"num_object_clones":0,"num_object_copies":165642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":3250,"num_objects_unfound":0,"num_objects_dirty":82821,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":42928,"num_write_kb":171706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":822,"num_bytes_recovered":3362835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82894,"ondisk_log_size":82894,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1058640,"kb_used_data":660956,"kb_used_omap":13,"kb_used_meta":397618,"kb_avail":282056880,"statfs":{"total":289910292480,"available":288826245120,"internally_reserved":0,"allocated":676818944,"data_stored":675965302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":407161855},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,8,9,10,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29315072,"num_objects":7157,"num_object_clones":0,"num_object_copies":14314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-417,"num_objects_unfound":0,"num_objects_dirty":7157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7157,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":417,"num_bytes_recovered":1708032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006200"},"pg_stats":[{"pgid":"2.f","version":"41'5097","reported_seq":5169,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:50.207460+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:26:50.207460+0000","last_peered":"2024-09-17T15:26:50.207460+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:26:50.207460+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":5097,"log_dups_size":0,"ondisk_log_size":5097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20877312,"num_objects":5097,"num_object_clones":0,"num_object_copies":10194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5097,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5097,"num_write_kb":20388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1027},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'5167","reported_seq":5264,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:51.753796+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:26:51.753796+0000","last_peered":"2024-09-17T15:26:51.753796+0000","last_clean":"2024-09-17T15:26:51.753796+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:26:51.753796+0000","last_undegraded":"2024-09-17T15:26:51.753796+0000","last_fullsized":"2024-09-17T15:26:51.753796+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":5167,"log_dups_size":0,"ondisk_log_size":5167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21140422,"num_objects":5163,"num_object_clones":0,"num_object_copies":10326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5163,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3524,"num_write_kb":14096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'5306","reported_seq":5426,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:51.723401+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:26:51.723401+0000","last_peered":"2024-09-17T15:26:51.723401+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:26:51.723401+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":5306,"log_dups_size":0,"ondisk_log_size":5306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21716992,"num_objects":5302,"num_object_clones":0,"num_object_copies":10604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3463,"num_write_kb":13852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":1134}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'5222","reported_seq":5286,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:51.749188+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:26:51.749188+0000","last_peered":"2024-09-17T15:26:51.749188+0000","last_clean":"2024-09-17T15:26:51.749188+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:26:51.749188+0000","last_undegraded":"2024-09-17T15:26:51.749188+0000","last_fullsized":"2024-09-17T15:26:51.749188+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":5222,"log_dups_size":0,"ondisk_log_size":5222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21389312,"num_objects":5222,"num_object_clones":0,"num_object_copies":10444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5222,"num_write_kb":20888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'5028","reported_seq":5152,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:50.104289+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:26:50.104289+0000","last_peered":"2024-09-17T15:26:50.104289+0000","last_clean":"2024-09-17T15:26:50.104289+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:26:50.104289+0000","last_undegraded":"2024-09-17T15:26:50.104289+0000","last_fullsized":"2024-09-17T15:26:50.104289+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":5028,"log_dups_size":0,"ondisk_log_size":5028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20566499,"num_objects":5022,"num_object_clones":0,"num_object_copies":10044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5022,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2010,"num_write_kb":8040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'5349","reported_seq":5478,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:50.182340+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:26:50.182340+0000","last_peered":"2024-09-17T15:26:50.182340+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:26:50.182340+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":5349,"log_dups_size":0,"ondisk_log_size":5349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21889507,"num_objects":5345,"num_object_clones":0,"num_object_copies":10690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5345,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1848,"num_write_kb":7392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1075},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'5230","reported_seq":5363,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:50.170821+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:26:50.170821+0000","last_peered":"2024-09-17T15:26:50.170821+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:26:50.170821+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":5230,"log_dups_size":0,"ondisk_log_size":5230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21398470,"num_objects":5226,"num_object_clones":0,"num_object_copies":10452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5226,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1999,"num_write_kb":7996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1052},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'5193","reported_seq":5315,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:50.211031+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:26:50.211031+0000","last_peered":"2024-09-17T15:26:50.211031+0000","last_clean":"2024-09-17T15:26:50.211031+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:26:50.211031+0000","last_undegraded":"2024-09-17T15:26:50.211031+0000","last_fullsized":"2024-09-17T15:26:50.211031+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":5193,"log_dups_size":0,"ondisk_log_size":5193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21262336,"num_objects":5191,"num_object_clones":0,"num_object_copies":10382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2147,"num_write_kb":8588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'5176","reported_seq":6940,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:26:51.792555+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:26:51.792555+0000","last_peered":"2024-09-17T15:26:51.792555+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:26:51.792555+0000","last_undegraded":"2024-09-17T15:26:51.792555+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":5176,"log_dups_size":0,"ondisk_log_size":5176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21172707,"num_objects":5170,"num_object_clones":0,"num_object_copies":10340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3250,"num_objects_unfound":0,"num_objects_dirty":5170,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3176,"num_write_kb":12704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":821,"num_bytes_recovered":3362816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3250},{"shards":"2","objects":1920}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233552+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233552+0000","last_peered":"2024-09-17T15:26:22.233552+0000","last_clean":"2024-09-17T15:26:22.233552+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233552+0000","last_undegraded":"2024-09-17T15:26:22.233552+0000","last_fullsized":"2024-09-17T15:26:22.233552+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'5027","reported_seq":5152,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:50.175271+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:26:50.175271+0000","last_peered":"2024-09-17T15:26:50.175271+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:26:50.175271+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":5027,"log_dups_size":0,"ondisk_log_size":5027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20557824,"num_objects":5019,"num_object_clones":0,"num_object_copies":10038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1623,"num_write_kb":6492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1009},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":97,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412553+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412553+0000","last_peered":"2024-09-17T15:26:22.412553+0000","last_clean":"2024-09-17T15:26:22.412553+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412553+0000","last_undegraded":"2024-09-17T15:26:22.412553+0000","last_fullsized":"2024-09-17T15:26:22.412553+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'5149","reported_seq":5264,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:51.778785+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:26:51.778785+0000","last_peered":"2024-09-17T15:26:51.778785+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:26:51.778785+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":5149,"log_dups_size":0,"ondisk_log_size":5149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21070307,"num_objects":5145,"num_object_clones":0,"num_object_copies":10290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5145,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3044,"num_write_kb":12176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":1068}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'5217","reported_seq":5330,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:51.795257+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:26:51.795257+0000","last_peered":"2024-09-17T15:26:51.795257+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:26:51.795257+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":5217,"log_dups_size":0,"ondisk_log_size":5217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21344256,"num_objects":5211,"num_object_clones":0,"num_object_copies":10422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5211,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2979,"num_write_kb":11916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":1074}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'5236","reported_seq":5366,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:50.136408+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:26:50.136408+0000","last_peered":"2024-09-17T15:26:50.136408+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:26:50.136408+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":5236,"log_dups_size":0,"ondisk_log_size":5236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21438464,"num_objects":5234,"num_object_clones":0,"num_object_copies":10468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1784,"num_write_kb":7136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1096},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'5175","reported_seq":5311,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:50.191230+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:26:50.191230+0000","last_peered":"2024-09-17T15:26:50.191230+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:26:50.191230+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":5175,"log_dups_size":0,"ondisk_log_size":5175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21164032,"num_objects":5167,"num_object_clones":0,"num_object_copies":10334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1462,"num_write_kb":5848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1010},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229013+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229013+0000","last_peered":"2024-09-17T15:26:22.229013+0000","last_clean":"2024-09-17T15:26:22.229013+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229013+0000","last_undegraded":"2024-09-17T15:26:22.229013+0000","last_fullsized":"2024-09-17T15:26:22.229013+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'5161","reported_seq":5287,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:50.181168+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:26:50.181168+0000","last_peered":"2024-09-17T15:26:50.181168+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:26:50.181168+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":5161,"log_dups_size":0,"ondisk_log_size":5161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21123072,"num_objects":5157,"num_object_clones":0,"num_object_copies":10314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1948,"num_write_kb":7792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1094},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'5158","reported_seq":5295,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:50.164588+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:26:50.164588+0000","last_peered":"2024-09-17T15:26:50.164588+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:26:50.164588+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":5158,"log_dups_size":0,"ondisk_log_size":5158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21078982,"num_objects":5148,"num_object_clones":0,"num_object_copies":10296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1600,"num_write_kb":6400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1039},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":339190494,"num_objects":82819,"num_object_clones":0,"num_object_copies":165638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":3250,"num_objects_unfound":0,"num_objects_dirty":82819,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":42926,"num_write_kb":171704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":821,"num_bytes_recovered":3362816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":679104512,"data_stored":678949153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82891,"ondisk_log_size":82891,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051606,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":460264,"kb_used_data":298072,"kb_used_omap":4,"kb_used_meta":162171,"kb_avail":93911576,"statfs":{"total":96636764160,"available":96165453824,"internally_reserved":0,"allocated":305225728,"data_stored":304959555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":166063788},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724551,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":132460,"kb_used_data":50780,"kb_used_omap":4,"kb_used_meta":81659,"kb_avail":94239380,"statfs":{"total":96636764160,"available":96501125120,"internally_reserved":0,"allocated":51998720,"data_stored":51681501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":83619499},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,4,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247834,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465916,"kb_used_data":312104,"kb_used_omap":4,"kb_used_meta":153787,"kb_avail":93905924,"statfs":{"total":96636764160,"available":96159666176,"internally_reserved":0,"allocated":319594496,"data_stored":319324246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":157478568},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,5,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":320806912,"data_stored":320770782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":52973568,"data_stored":52890469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":305324032,"data_stored":305287902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:26:55.297 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 76 20 85383 85363 4.3867 2.20312 0.0433095 0.0178014 2024-09-17T15:26:56.297 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 77 20 86007 85987 4.36138 2.4375 0.0461832 0.0179051 2024-09-17T15:26:57.297 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 78 20 86625 86605 4.33642 2.41406 0.0475527 0.0180071 2024-09-17T15:26:57.836 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:26:58.297 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 79 20 87206 87186 4.31025 2.26953 0.0277391 0.0181168 2024-09-17T15:26:58.379 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:26:58.379 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:26:58.407 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2024-09-17T15:26:57.604135+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":351150833,"num_objects":85741,"num_object_clones":0,"num_object_copies":171482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":3102,"num_objects_unfound":0,"num_objects_dirty":85741,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":45848,"num_write_kb":183386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":970,"num_bytes_recovered":3969043,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85814,"ondisk_log_size":85814,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1117332,"kb_used_data":684964,"kb_used_omap":13,"kb_used_meta":432306,"kb_avail":281998188,"statfs":{"total":289910292480,"available":288766144512,"internally_reserved":0,"allocated":701403136,"data_stored":700549494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":442682367},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,5,0,2,3,14,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28815360,"num_objects":7035,"num_object_clones":0,"num_object_copies":14070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-427,"num_objects_unfound":0,"num_objects_dirty":7035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7035,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":427,"num_bytes_recovered":1748992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006298"},"pg_stats":[{"pgid":"2.f","version":"41'5287","reported_seq":5359,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:55.253719+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:26:55.253719+0000","last_peered":"2024-09-17T15:26:55.253719+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:26:55.253719+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":5287,"log_dups_size":0,"ondisk_log_size":5287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21655552,"num_objects":5287,"num_object_clones":0,"num_object_copies":10574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5287,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5287,"num_write_kb":21148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1217},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'5369","reported_seq":5464,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:56.830840+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:26:56.830840+0000","last_peered":"2024-09-17T15:26:56.830840+0000","last_clean":"2024-09-17T15:26:56.830840+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:26:56.830840+0000","last_undegraded":"2024-09-17T15:26:56.830840+0000","last_fullsized":"2024-09-17T15:26:56.830840+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":5369,"log_dups_size":0,"ondisk_log_size":5369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21967814,"num_objects":5365,"num_object_clones":0,"num_object_copies":10730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5365,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3726,"num_write_kb":14904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'5478","reported_seq":5597,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:56.834200+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:26:56.834200+0000","last_peered":"2024-09-17T15:26:56.834200+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:26:56.834200+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":5478,"log_dups_size":0,"ondisk_log_size":5478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22421504,"num_objects":5474,"num_object_clones":0,"num_object_copies":10948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3635,"num_write_kb":14540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":1306}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'5423","reported_seq":5487,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:56.809390+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:26:56.809390+0000","last_peered":"2024-09-17T15:26:56.809390+0000","last_clean":"2024-09-17T15:26:56.809390+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:26:56.809390+0000","last_undegraded":"2024-09-17T15:26:56.809390+0000","last_fullsized":"2024-09-17T15:26:56.809390+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":5423,"log_dups_size":0,"ondisk_log_size":5423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22212608,"num_objects":5423,"num_object_clones":0,"num_object_copies":10846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5423,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5423,"num_write_kb":21692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'5206","reported_seq":5330,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:55.233097+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:26:55.233097+0000","last_peered":"2024-09-17T15:26:55.233097+0000","last_clean":"2024-09-17T15:26:55.233097+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:26:55.233097+0000","last_undegraded":"2024-09-17T15:26:55.233097+0000","last_fullsized":"2024-09-17T15:26:55.233097+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":5206,"log_dups_size":0,"ondisk_log_size":5206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21295587,"num_objects":5200,"num_object_clones":0,"num_object_copies":10400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5200,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2188,"num_write_kb":8752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'5526","reported_seq":5655,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:55.064898+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:26:55.064898+0000","last_peered":"2024-09-17T15:26:55.064898+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:26:55.064898+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":5526,"log_dups_size":0,"ondisk_log_size":5526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22614499,"num_objects":5522,"num_object_clones":0,"num_object_copies":11044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5522,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2025,"num_write_kb":8100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1252},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'5419","reported_seq":5552,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:55.263031+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:26:55.263031+0000","last_peered":"2024-09-17T15:26:55.263031+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:26:55.263031+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":5419,"log_dups_size":0,"ondisk_log_size":5419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22172614,"num_objects":5415,"num_object_clones":0,"num_object_copies":10830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5415,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2188,"num_write_kb":8752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1241},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'5379","reported_seq":5501,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:55.187917+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:26:55.187917+0000","last_peered":"2024-09-17T15:26:55.187917+0000","last_clean":"2024-09-17T15:26:55.187917+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:26:55.187917+0000","last_undegraded":"2024-09-17T15:26:55.187917+0000","last_fullsized":"2024-09-17T15:26:55.187917+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":5379,"log_dups_size":0,"ondisk_log_size":5379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22024192,"num_objects":5377,"num_object_clones":0,"num_object_copies":10754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2333,"num_write_kb":9332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'5334","reported_seq":7395,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:26:56.804336+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:26:56.804336+0000","last_peered":"2024-09-17T15:26:56.804336+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:26:56.804336+0000","last_undegraded":"2024-09-17T15:26:56.804336+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":5334,"log_dups_size":0,"ondisk_log_size":5334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21819875,"num_objects":5328,"num_object_clones":0,"num_object_copies":10656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3102,"num_objects_unfound":0,"num_objects_dirty":5328,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3334,"num_write_kb":13336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":969,"num_bytes_recovered":3969024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3102},{"shards":"2","objects":2226}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233552+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233552+0000","last_peered":"2024-09-17T15:26:22.233552+0000","last_clean":"2024-09-17T15:26:22.233552+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233552+0000","last_undegraded":"2024-09-17T15:26:22.233552+0000","last_fullsized":"2024-09-17T15:26:22.233552+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'5214","reported_seq":5337,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:55.271216+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:26:55.271216+0000","last_peered":"2024-09-17T15:26:55.271216+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:26:55.271216+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":5214,"log_dups_size":0,"ondisk_log_size":5214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21323776,"num_objects":5206,"num_object_clones":0,"num_object_copies":10412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5206,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1810,"num_write_kb":7240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1196},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":97,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412553+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412553+0000","last_peered":"2024-09-17T15:26:22.412553+0000","last_clean":"2024-09-17T15:26:22.412553+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412553+0000","last_undegraded":"2024-09-17T15:26:22.412553+0000","last_fullsized":"2024-09-17T15:26:22.412553+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'5342","reported_seq":5457,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:56.816142+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:26:56.816142+0000","last_peered":"2024-09-17T15:26:56.816142+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:26:56.816142+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":5342,"log_dups_size":0,"ondisk_log_size":5342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21860835,"num_objects":5338,"num_object_clones":0,"num_object_copies":10676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5338,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3237,"num_write_kb":12948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":1261}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'5392","reported_seq":5505,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:56.794775+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:26:56.794775+0000","last_peered":"2024-09-17T15:26:56.794775+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:26:56.794775+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":5392,"log_dups_size":0,"ondisk_log_size":5392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22061056,"num_objects":5386,"num_object_clones":0,"num_object_copies":10772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3154,"num_write_kb":12616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":1249}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'5407","reported_seq":5537,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:55.250198+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:26:55.250198+0000","last_peered":"2024-09-17T15:26:55.250198+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:26:55.250198+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":5407,"log_dups_size":0,"ondisk_log_size":5407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22138880,"num_objects":5405,"num_object_clones":0,"num_object_copies":10810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1955,"num_write_kb":7820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1267},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'5346","reported_seq":5482,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:55.287247+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:26:55.287247+0000","last_peered":"2024-09-17T15:26:55.287247+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:26:55.287247+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":5346,"log_dups_size":0,"ondisk_log_size":5346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21864448,"num_objects":5338,"num_object_clones":0,"num_object_copies":10676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1633,"num_write_kb":6532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1181},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229013+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229013+0000","last_peered":"2024-09-17T15:26:22.229013+0000","last_clean":"2024-09-17T15:26:22.229013+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229013+0000","last_undegraded":"2024-09-17T15:26:22.229013+0000","last_fullsized":"2024-09-17T15:26:22.229013+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'5344","reported_seq":5470,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:55.227400+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:26:55.227400+0000","last_peered":"2024-09-17T15:26:55.227400+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:26:55.227400+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":5344,"log_dups_size":0,"ondisk_log_size":5344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21872640,"num_objects":5340,"num_object_clones":0,"num_object_copies":10680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2131,"num_write_kb":8524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1277},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'5345","reported_seq":5481,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:55.272163+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:26:55.272163+0000","last_peered":"2024-09-17T15:26:55.272163+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:26:55.272163+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":5345,"log_dups_size":0,"ondisk_log_size":5345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21844934,"num_objects":5335,"num_object_clones":0,"num_object_copies":10670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1787,"num_write_kb":7148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1226},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":351150814,"num_objects":85739,"num_object_clones":0,"num_object_copies":171478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":3102,"num_objects_unfound":0,"num_objects_dirty":85739,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":45846,"num_write_kb":183384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":969,"num_bytes_recovered":3969024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":703852544,"data_stored":703697185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85811,"ondisk_log_size":85811,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051607,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":470300,"kb_used_data":303948,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":93901540,"statfs":{"total":96636764160,"available":96155176960,"internally_reserved":0,"allocated":311242752,"data_stored":310976579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":170323628},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,1,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724552,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149984,"kb_used_data":59984,"kb_used_omap":4,"kb_used_meta":89979,"kb_avail":94221856,"statfs":{"total":96636764160,"available":96483180544,"internally_reserved":0,"allocated":61423616,"data_stored":61106397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":92139179},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,1,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247835,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497048,"kb_used_data":321032,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93874792,"statfs":{"total":96636764160,"available":96127787008,"internally_reserved":0,"allocated":328736768,"data_stored":328466518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":180219560},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,2,1,8,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":329887744,"data_stored":329851614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":62619648,"data_stored":62536549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":311345152,"data_stored":311309022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:26:58.408 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:26:58.987 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:26:58.988 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:26:59.013 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2024-09-17T15:26:57.604135+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":351150833,"num_objects":85741,"num_object_clones":0,"num_object_copies":171482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":3102,"num_objects_unfound":0,"num_objects_dirty":85741,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":45848,"num_write_kb":183386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":970,"num_bytes_recovered":3969043,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85814,"ondisk_log_size":85814,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1117332,"kb_used_data":684964,"kb_used_omap":13,"kb_used_meta":432306,"kb_avail":281998188,"statfs":{"total":289910292480,"available":288766144512,"internally_reserved":0,"allocated":701403136,"data_stored":700549494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":442682367},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,5,0,2,3,14,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28815360,"num_objects":7035,"num_object_clones":0,"num_object_copies":14070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-427,"num_objects_unfound":0,"num_objects_dirty":7035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7035,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":427,"num_bytes_recovered":1748992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006298"},"pg_stats":[{"pgid":"2.f","version":"41'5287","reported_seq":5359,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:55.253719+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:26:55.253719+0000","last_peered":"2024-09-17T15:26:55.253719+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:26:55.253719+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":5287,"log_dups_size":0,"ondisk_log_size":5287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21655552,"num_objects":5287,"num_object_clones":0,"num_object_copies":10574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5287,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5287,"num_write_kb":21148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1217},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'5369","reported_seq":5464,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:56.830840+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:26:56.830840+0000","last_peered":"2024-09-17T15:26:56.830840+0000","last_clean":"2024-09-17T15:26:56.830840+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:26:56.830840+0000","last_undegraded":"2024-09-17T15:26:56.830840+0000","last_fullsized":"2024-09-17T15:26:56.830840+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":5369,"log_dups_size":0,"ondisk_log_size":5369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21967814,"num_objects":5365,"num_object_clones":0,"num_object_copies":10730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5365,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3726,"num_write_kb":14904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'5478","reported_seq":5597,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:56.834200+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:26:56.834200+0000","last_peered":"2024-09-17T15:26:56.834200+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:26:56.834200+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":5478,"log_dups_size":0,"ondisk_log_size":5478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22421504,"num_objects":5474,"num_object_clones":0,"num_object_copies":10948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3635,"num_write_kb":14540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":1306}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'5423","reported_seq":5487,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:56.809390+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:26:56.809390+0000","last_peered":"2024-09-17T15:26:56.809390+0000","last_clean":"2024-09-17T15:26:56.809390+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:26:56.809390+0000","last_undegraded":"2024-09-17T15:26:56.809390+0000","last_fullsized":"2024-09-17T15:26:56.809390+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":5423,"log_dups_size":0,"ondisk_log_size":5423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22212608,"num_objects":5423,"num_object_clones":0,"num_object_copies":10846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5423,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5423,"num_write_kb":21692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'5206","reported_seq":5330,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:55.233097+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:26:55.233097+0000","last_peered":"2024-09-17T15:26:55.233097+0000","last_clean":"2024-09-17T15:26:55.233097+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:26:55.233097+0000","last_undegraded":"2024-09-17T15:26:55.233097+0000","last_fullsized":"2024-09-17T15:26:55.233097+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":5206,"log_dups_size":0,"ondisk_log_size":5206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21295587,"num_objects":5200,"num_object_clones":0,"num_object_copies":10400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5200,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2188,"num_write_kb":8752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'5526","reported_seq":5655,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:55.064898+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:26:55.064898+0000","last_peered":"2024-09-17T15:26:55.064898+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:26:55.064898+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":5526,"log_dups_size":0,"ondisk_log_size":5526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22614499,"num_objects":5522,"num_object_clones":0,"num_object_copies":11044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5522,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2025,"num_write_kb":8100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1252},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'5419","reported_seq":5552,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:55.263031+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:26:55.263031+0000","last_peered":"2024-09-17T15:26:55.263031+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:26:55.263031+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":5419,"log_dups_size":0,"ondisk_log_size":5419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22172614,"num_objects":5415,"num_object_clones":0,"num_object_copies":10830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5415,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2188,"num_write_kb":8752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1241},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'5379","reported_seq":5501,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:55.187917+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:26:55.187917+0000","last_peered":"2024-09-17T15:26:55.187917+0000","last_clean":"2024-09-17T15:26:55.187917+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:26:55.187917+0000","last_undegraded":"2024-09-17T15:26:55.187917+0000","last_fullsized":"2024-09-17T15:26:55.187917+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":5379,"log_dups_size":0,"ondisk_log_size":5379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22024192,"num_objects":5377,"num_object_clones":0,"num_object_copies":10754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2333,"num_write_kb":9332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'5334","reported_seq":7395,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:26:56.804336+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:26:56.804336+0000","last_peered":"2024-09-17T15:26:56.804336+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:26:56.804336+0000","last_undegraded":"2024-09-17T15:26:56.804336+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":5334,"log_dups_size":0,"ondisk_log_size":5334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21819875,"num_objects":5328,"num_object_clones":0,"num_object_copies":10656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3102,"num_objects_unfound":0,"num_objects_dirty":5328,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3334,"num_write_kb":13336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":969,"num_bytes_recovered":3969024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3102},{"shards":"2","objects":2226}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233552+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233552+0000","last_peered":"2024-09-17T15:26:22.233552+0000","last_clean":"2024-09-17T15:26:22.233552+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233552+0000","last_undegraded":"2024-09-17T15:26:22.233552+0000","last_fullsized":"2024-09-17T15:26:22.233552+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'5214","reported_seq":5337,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:55.271216+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:26:55.271216+0000","last_peered":"2024-09-17T15:26:55.271216+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:26:55.271216+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":5214,"log_dups_size":0,"ondisk_log_size":5214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21323776,"num_objects":5206,"num_object_clones":0,"num_object_copies":10412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5206,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1810,"num_write_kb":7240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1196},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":97,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412553+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412553+0000","last_peered":"2024-09-17T15:26:22.412553+0000","last_clean":"2024-09-17T15:26:22.412553+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412553+0000","last_undegraded":"2024-09-17T15:26:22.412553+0000","last_fullsized":"2024-09-17T15:26:22.412553+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'5342","reported_seq":5457,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:56.816142+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:26:56.816142+0000","last_peered":"2024-09-17T15:26:56.816142+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:26:56.816142+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":5342,"log_dups_size":0,"ondisk_log_size":5342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21860835,"num_objects":5338,"num_object_clones":0,"num_object_copies":10676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5338,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3237,"num_write_kb":12948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":1261}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'5392","reported_seq":5505,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:56.794775+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:26:56.794775+0000","last_peered":"2024-09-17T15:26:56.794775+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:26:56.794775+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":5392,"log_dups_size":0,"ondisk_log_size":5392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22061056,"num_objects":5386,"num_object_clones":0,"num_object_copies":10772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3154,"num_write_kb":12616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":1249}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'5407","reported_seq":5537,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:55.250198+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:26:55.250198+0000","last_peered":"2024-09-17T15:26:55.250198+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:26:55.250198+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":5407,"log_dups_size":0,"ondisk_log_size":5407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22138880,"num_objects":5405,"num_object_clones":0,"num_object_copies":10810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1955,"num_write_kb":7820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1267},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'5346","reported_seq":5482,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:55.287247+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:26:55.287247+0000","last_peered":"2024-09-17T15:26:55.287247+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:26:55.287247+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":5346,"log_dups_size":0,"ondisk_log_size":5346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21864448,"num_objects":5338,"num_object_clones":0,"num_object_copies":10676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1633,"num_write_kb":6532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1181},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229013+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229013+0000","last_peered":"2024-09-17T15:26:22.229013+0000","last_clean":"2024-09-17T15:26:22.229013+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229013+0000","last_undegraded":"2024-09-17T15:26:22.229013+0000","last_fullsized":"2024-09-17T15:26:22.229013+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'5344","reported_seq":5470,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:55.227400+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:26:55.227400+0000","last_peered":"2024-09-17T15:26:55.227400+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:26:55.227400+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":5344,"log_dups_size":0,"ondisk_log_size":5344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21872640,"num_objects":5340,"num_object_clones":0,"num_object_copies":10680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2131,"num_write_kb":8524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1277},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'5345","reported_seq":5481,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:55.272163+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:26:55.272163+0000","last_peered":"2024-09-17T15:26:55.272163+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:26:55.272163+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":5345,"log_dups_size":0,"ondisk_log_size":5345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21844934,"num_objects":5335,"num_object_clones":0,"num_object_copies":10670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1787,"num_write_kb":7148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1226},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":351150814,"num_objects":85739,"num_object_clones":0,"num_object_copies":171478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":3102,"num_objects_unfound":0,"num_objects_dirty":85739,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":45846,"num_write_kb":183384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":969,"num_bytes_recovered":3969024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":703852544,"data_stored":703697185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85811,"ondisk_log_size":85811,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051607,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":470300,"kb_used_data":303948,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":93901540,"statfs":{"total":96636764160,"available":96155176960,"internally_reserved":0,"allocated":311242752,"data_stored":310976579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":170323628},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,1,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724552,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149984,"kb_used_data":59984,"kb_used_omap":4,"kb_used_meta":89979,"kb_avail":94221856,"statfs":{"total":96636764160,"available":96483180544,"internally_reserved":0,"allocated":61423616,"data_stored":61106397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":92139179},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,1,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247835,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497048,"kb_used_data":321032,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93874792,"statfs":{"total":96636764160,"available":96127787008,"internally_reserved":0,"allocated":328736768,"data_stored":328466518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":180219560},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,2,1,8,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":329887744,"data_stored":329851614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":62619648,"data_stored":62536549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":311345152,"data_stored":311309022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:26:59.014 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:26:59.014 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'5287", 'reported_seq': 5359, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:55.253719+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:26:55.253719+0000', 'last_peered': '2024-09-17T15:26:55.253719+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:26:55.253719+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 5287, 'log_dups_size': 0, 'ondisk_log_size': 5287, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21655552, 'num_objects': 5287, 'num_object_clones': 0, 'num_object_copies': 10574, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4070, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5287, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5287, 'num_write_kb': 21148, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1217}, {'shards': '0,2', 'objects': 4070}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:59.014 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:26:59.014 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'5478", 'reported_seq': 5597, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:56.834200+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:26:56.834200+0000', 'last_peered': '2024-09-17T15:26:56.834200+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:26:56.834200+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 5478, 'log_dups_size': 0, 'ondisk_log_size': 5478, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22421504, 'num_objects': 5474, 'num_object_clones': 0, 'num_object_copies': 10948, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4169, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5474, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3635, 'num_write_kb': 14540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4168}, {'shards': '2', 'objects': 1306}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:26:59.014 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:26:59.015 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'5526", 'reported_seq': 5655, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:55.064898+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:26:55.064898+0000', 'last_peered': '2024-09-17T15:26:55.064898+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:26:55.064898+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 5526, 'log_dups_size': 0, 'ondisk_log_size': 5526, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22614499, 'num_objects': 5522, 'num_object_clones': 0, 'num_object_copies': 11044, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4271, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5522, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2025, 'num_write_kb': 8100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1252}, {'shards': '0,2', 'objects': 4268}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:59.015 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:26:59.015 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'5419", 'reported_seq': 5552, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:55.263031+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:26:55.263031+0000', 'last_peered': '2024-09-17T15:26:55.263031+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:26:55.263031+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 5419, 'log_dups_size': 0, 'ondisk_log_size': 5419, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22172614, 'num_objects': 5415, 'num_object_clones': 0, 'num_object_copies': 10830, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4174, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5415, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 2188, 'num_write_kb': 8752, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1241}, {'shards': '0,2', 'objects': 4174}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:59.015 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-17T15:26:59.015 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "41'5334", 'reported_seq': 7395, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:26:56.804336+0000', 'last_change': '2024-09-17T15:26:22.588242+0000', 'last_active': '2024-09-17T15:26:56.804336+0000', 'last_peered': '2024-09-17T15:26:56.804336+0000', 'last_clean': '2024-09-17T15:26:20.155646+0000', 'last_became_active': '2024-09-17T15:26:22.557742+0000', 'last_became_peered': '2024-09-17T15:26:22.557742+0000', 'last_unstale': '2024-09-17T15:26:56.804336+0000', 'last_undegraded': '2024-09-17T15:26:56.804336+0000', 'last_fullsized': '2024-09-17T15:26:22.188850+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2016", 'last_scrub_stamp': '2024-09-17T15:25:53.959011+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:53.959011+0000', 'objects_scrubbed': 1994, 'log_size': 5334, 'log_dups_size': 0, 'ondisk_log_size': 5334, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 568, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21819875, 'num_objects': 5328, 'num_object_clones': 0, 'num_object_copies': 10656, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3102, 'num_objects_unfound': 0, 'num_objects_dirty': 5328, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3334, 'num_write_kb': 13336, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 969, 'num_bytes_recovered': 3969024, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3102}, {'shards': '2', 'objects': 2226}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:26:59.015 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:26:59.015 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'5214", 'reported_seq': 5337, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:55.271216+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:26:55.271216+0000', 'last_peered': '2024-09-17T15:26:55.271216+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:26:55.271216+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 5214, 'log_dups_size': 0, 'ondisk_log_size': 5214, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21323776, 'num_objects': 5206, 'num_object_clones': 0, 'num_object_copies': 10412, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4012, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5206, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1810, 'num_write_kb': 7240, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1196}, {'shards': '0,2', 'objects': 4010}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:59.015 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:26:59.015 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'5342", 'reported_seq': 5457, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:56.816142+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:26:56.816142+0000', 'last_peered': '2024-09-17T15:26:56.816142+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:26:56.816142+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 5342, 'log_dups_size': 0, 'ondisk_log_size': 5342, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21860835, 'num_objects': 5338, 'num_object_clones': 0, 'num_object_copies': 10676, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4078, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5338, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3237, 'num_write_kb': 12948, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4077}, {'shards': '2', 'objects': 1261}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:26:59.015 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:26:59.015 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'5392", 'reported_seq': 5505, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:56.794775+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:26:56.794775+0000', 'last_peered': '2024-09-17T15:26:56.794775+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:26:56.794775+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 5392, 'log_dups_size': 0, 'ondisk_log_size': 5392, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22061056, 'num_objects': 5386, 'num_object_clones': 0, 'num_object_copies': 10772, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5386, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3154, 'num_write_kb': 12616, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4137}, {'shards': '2', 'objects': 1249}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:26:59.016 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:26:59.016 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'5407", 'reported_seq': 5537, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:55.250198+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:26:55.250198+0000', 'last_peered': '2024-09-17T15:26:55.250198+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:26:55.250198+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 5407, 'log_dups_size': 0, 'ondisk_log_size': 5407, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22138880, 'num_objects': 5405, 'num_object_clones': 0, 'num_object_copies': 10810, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5405, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1955, 'num_write_kb': 7820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1267}, {'shards': '0,2', 'objects': 4138}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:59.016 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:26:59.016 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'5346", 'reported_seq': 5482, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:55.287247+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:26:55.287247+0000', 'last_peered': '2024-09-17T15:26:55.287247+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:26:55.287247+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 5346, 'log_dups_size': 0, 'ondisk_log_size': 5346, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21864448, 'num_objects': 5338, 'num_object_clones': 0, 'num_object_copies': 10676, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4160, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5338, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1633, 'num_write_kb': 6532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1181}, {'shards': '0,2', 'objects': 4157}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:59.016 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:26:59.016 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'5344", 'reported_seq': 5470, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:55.227400+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:26:55.227400+0000', 'last_peered': '2024-09-17T15:26:55.227400+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:26:55.227400+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 5344, 'log_dups_size': 0, 'ondisk_log_size': 5344, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21872640, 'num_objects': 5340, 'num_object_clones': 0, 'num_object_copies': 10680, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4065, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5340, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2131, 'num_write_kb': 8524, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1277}, {'shards': '0,2', 'objects': 4063}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:59.016 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:26:59.016 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'5345", 'reported_seq': 5481, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:26:55.272163+0000', 'last_change': '2024-09-17T15:26:22.654521+0000', 'last_active': '2024-09-17T15:26:55.272163+0000', 'last_peered': '2024-09-17T15:26:55.272163+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:26:55.272163+0000', 'last_undegraded': '2024-09-17T15:26:22.307795+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 5345, 'log_dups_size': 0, 'ondisk_log_size': 5345, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21844934, 'num_objects': 5335, 'num_object_clones': 0, 'num_object_copies': 10670, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4111, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5335, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1787, 'num_write_kb': 7148, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1226}, {'shards': '0,2', 'objects': 4109}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:26:59.017 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:26:59.297 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout:2024-09-17T15:26:59.297712+0000 min lat: 0.00187689 max lat: 2.64156 avg lat: 0.0182324 2024-09-17T15:26:59.297 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-17T15:26:59.297 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 80 20 87745 87725 4.28269 2.10547 0.0233505 0.0182324 2024-09-17T15:26:59.695 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:26:59.696 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45388/171482 objects degraded (26.468%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":156,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":85741,"data_bytes":351150833,"bytes_used":1158250496,"bytes_avail":288752041984,"bytes_total":289910292480,"degraded_objects":45388,"degraded_total":171482,"degraded_ratio":0.26468084113784535,"misplaced_objects":3102,"misplaced_total":171482,"misplaced_ratio":0.018089362148796959,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":96544,"recovering_keys_per_sec":0,"num_objects_recovered":283,"num_bytes_recovered":1159168,"num_keys_recovered":0,"write_bytes_sec":2025037,"read_op_per_sec":0,"write_op_per_sec":494},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-17T15:26:23.592815+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-17T15:26:59.729 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:26:59.729 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:27:00.297 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 81 20 88279 88259 4.25557 2.08594 0.0448516 0.0183497 2024-09-17T15:27:00.315 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:27:00.316 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:27:00.338 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":112,"stamp":"2024-09-17T15:26:59.604898+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":351150833,"num_objects":85741,"num_object_clones":0,"num_object_copies":171482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":3102,"num_objects_unfound":0,"num_objects_dirty":85741,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":45848,"num_write_kb":183386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":970,"num_bytes_recovered":3969043,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85814,"ondisk_log_size":85814,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1131104,"kb_used_data":694576,"kb_used_omap":13,"kb_used_meta":436466,"kb_avail":281984416,"statfs":{"total":289910292480,"available":288752041984,"internally_reserved":0,"allocated":711245824,"data_stored":710392182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":446942207},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,6,1,3,5,15,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24313856,"num_objects":5936,"num_object_clones":0,"num_object_copies":11872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-283,"num_objects_unfound":0,"num_objects_dirty":5936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5936,"num_write_kb":23744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":283,"num_bytes_recovered":1159168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006620"},"pg_stats":[{"pgid":"2.f","version":"41'5287","reported_seq":5359,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:55.253719+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:26:55.253719+0000","last_peered":"2024-09-17T15:26:55.253719+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:26:55.253719+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":5287,"log_dups_size":0,"ondisk_log_size":5287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21655552,"num_objects":5287,"num_object_clones":0,"num_object_copies":10574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5287,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5287,"num_write_kb":21148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1217},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'5369","reported_seq":5464,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:56.830840+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:26:56.830840+0000","last_peered":"2024-09-17T15:26:56.830840+0000","last_clean":"2024-09-17T15:26:56.830840+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:26:56.830840+0000","last_undegraded":"2024-09-17T15:26:56.830840+0000","last_fullsized":"2024-09-17T15:26:56.830840+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":5369,"log_dups_size":0,"ondisk_log_size":5369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21967814,"num_objects":5365,"num_object_clones":0,"num_object_copies":10730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5365,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3726,"num_write_kb":14904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'5478","reported_seq":5597,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:56.834200+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:26:56.834200+0000","last_peered":"2024-09-17T15:26:56.834200+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:26:56.834200+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":5478,"log_dups_size":0,"ondisk_log_size":5478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22421504,"num_objects":5474,"num_object_clones":0,"num_object_copies":10948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3635,"num_write_kb":14540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":1306}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'5423","reported_seq":5487,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:56.809390+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:26:56.809390+0000","last_peered":"2024-09-17T15:26:56.809390+0000","last_clean":"2024-09-17T15:26:56.809390+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:26:56.809390+0000","last_undegraded":"2024-09-17T15:26:56.809390+0000","last_fullsized":"2024-09-17T15:26:56.809390+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":5423,"log_dups_size":0,"ondisk_log_size":5423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22212608,"num_objects":5423,"num_object_clones":0,"num_object_copies":10846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5423,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5423,"num_write_kb":21692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'5206","reported_seq":5330,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:55.233097+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:26:55.233097+0000","last_peered":"2024-09-17T15:26:55.233097+0000","last_clean":"2024-09-17T15:26:55.233097+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:26:55.233097+0000","last_undegraded":"2024-09-17T15:26:55.233097+0000","last_fullsized":"2024-09-17T15:26:55.233097+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":5206,"log_dups_size":0,"ondisk_log_size":5206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21295587,"num_objects":5200,"num_object_clones":0,"num_object_copies":10400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5200,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2188,"num_write_kb":8752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'5526","reported_seq":5655,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:55.064898+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:26:55.064898+0000","last_peered":"2024-09-17T15:26:55.064898+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:26:55.064898+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":5526,"log_dups_size":0,"ondisk_log_size":5526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22614499,"num_objects":5522,"num_object_clones":0,"num_object_copies":11044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5522,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2025,"num_write_kb":8100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1252},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'5419","reported_seq":5552,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:55.263031+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:26:55.263031+0000","last_peered":"2024-09-17T15:26:55.263031+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:26:55.263031+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":5419,"log_dups_size":0,"ondisk_log_size":5419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22172614,"num_objects":5415,"num_object_clones":0,"num_object_copies":10830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5415,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2188,"num_write_kb":8752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1241},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'5379","reported_seq":5501,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:55.187917+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:26:55.187917+0000","last_peered":"2024-09-17T15:26:55.187917+0000","last_clean":"2024-09-17T15:26:55.187917+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:26:55.187917+0000","last_undegraded":"2024-09-17T15:26:55.187917+0000","last_fullsized":"2024-09-17T15:26:55.187917+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":5379,"log_dups_size":0,"ondisk_log_size":5379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22024192,"num_objects":5377,"num_object_clones":0,"num_object_copies":10754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2333,"num_write_kb":9332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'5334","reported_seq":7395,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:26:56.804336+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:26:56.804336+0000","last_peered":"2024-09-17T15:26:56.804336+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:26:56.804336+0000","last_undegraded":"2024-09-17T15:26:56.804336+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":5334,"log_dups_size":0,"ondisk_log_size":5334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21819875,"num_objects":5328,"num_object_clones":0,"num_object_copies":10656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3102,"num_objects_unfound":0,"num_objects_dirty":5328,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3334,"num_write_kb":13336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":969,"num_bytes_recovered":3969024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3102},{"shards":"2","objects":2226}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233552+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233552+0000","last_peered":"2024-09-17T15:26:22.233552+0000","last_clean":"2024-09-17T15:26:22.233552+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233552+0000","last_undegraded":"2024-09-17T15:26:22.233552+0000","last_fullsized":"2024-09-17T15:26:22.233552+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'5214","reported_seq":5337,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:55.271216+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:26:55.271216+0000","last_peered":"2024-09-17T15:26:55.271216+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:26:55.271216+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":5214,"log_dups_size":0,"ondisk_log_size":5214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21323776,"num_objects":5206,"num_object_clones":0,"num_object_copies":10412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5206,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1810,"num_write_kb":7240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1196},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":97,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412553+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412553+0000","last_peered":"2024-09-17T15:26:22.412553+0000","last_clean":"2024-09-17T15:26:22.412553+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412553+0000","last_undegraded":"2024-09-17T15:26:22.412553+0000","last_fullsized":"2024-09-17T15:26:22.412553+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'5342","reported_seq":5457,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:56.816142+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:26:56.816142+0000","last_peered":"2024-09-17T15:26:56.816142+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:26:56.816142+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":5342,"log_dups_size":0,"ondisk_log_size":5342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21860835,"num_objects":5338,"num_object_clones":0,"num_object_copies":10676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5338,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3237,"num_write_kb":12948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":1261}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'5392","reported_seq":5505,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:56.794775+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:26:56.794775+0000","last_peered":"2024-09-17T15:26:56.794775+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:26:56.794775+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":5392,"log_dups_size":0,"ondisk_log_size":5392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22061056,"num_objects":5386,"num_object_clones":0,"num_object_copies":10772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3154,"num_write_kb":12616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":1249}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'5407","reported_seq":5537,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:55.250198+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:26:55.250198+0000","last_peered":"2024-09-17T15:26:55.250198+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:26:55.250198+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":5407,"log_dups_size":0,"ondisk_log_size":5407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22138880,"num_objects":5405,"num_object_clones":0,"num_object_copies":10810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1955,"num_write_kb":7820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1267},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'5346","reported_seq":5482,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:55.287247+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:26:55.287247+0000","last_peered":"2024-09-17T15:26:55.287247+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:26:55.287247+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":5346,"log_dups_size":0,"ondisk_log_size":5346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21864448,"num_objects":5338,"num_object_clones":0,"num_object_copies":10676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1633,"num_write_kb":6532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1181},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229013+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229013+0000","last_peered":"2024-09-17T15:26:22.229013+0000","last_clean":"2024-09-17T15:26:22.229013+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229013+0000","last_undegraded":"2024-09-17T15:26:22.229013+0000","last_fullsized":"2024-09-17T15:26:22.229013+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'5344","reported_seq":5470,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:55.227400+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:26:55.227400+0000","last_peered":"2024-09-17T15:26:55.227400+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:26:55.227400+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":5344,"log_dups_size":0,"ondisk_log_size":5344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21872640,"num_objects":5340,"num_object_clones":0,"num_object_copies":10680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2131,"num_write_kb":8524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1277},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'5345","reported_seq":5481,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:26:55.272163+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:26:55.272163+0000","last_peered":"2024-09-17T15:26:55.272163+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:26:55.272163+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":5345,"log_dups_size":0,"ondisk_log_size":5345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21844934,"num_objects":5335,"num_object_clones":0,"num_object_copies":10670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1787,"num_write_kb":7148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1226},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":351150814,"num_objects":85739,"num_object_clones":0,"num_object_copies":171478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":3102,"num_objects_unfound":0,"num_objects_dirty":85739,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":45846,"num_write_kb":183384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":969,"num_bytes_recovered":3969024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":713486336,"data_stored":713330977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85811,"ondisk_log_size":85811,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051607,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":470300,"kb_used_data":303948,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":93901540,"statfs":{"total":96636764160,"available":96155176960,"internally_reserved":0,"allocated":311242752,"data_stored":310976579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":170323628},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,1,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724553,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":163756,"kb_used_data":69596,"kb_used_omap":4,"kb_used_meta":94139,"kb_avail":94208084,"statfs":{"total":96636764160,"available":96469078016,"internally_reserved":0,"allocated":71266304,"data_stored":70949085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":96399019},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,1,3,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247835,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497048,"kb_used_data":321032,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93874792,"statfs":{"total":96636764160,"available":96127787008,"internally_reserved":0,"allocated":328736768,"data_stored":328466518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":180219560},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,2,1,8,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":329887744,"data_stored":329851614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":72253440,"data_stored":72170341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":311345152,"data_stored":311309022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:27:01.297 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 82 20 88901 88881 4.2333 2.42969 0.0379422 0.018448 2024-09-17T15:27:02.297 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 83 20 89509 89489 4.21091 2.375 0.0203786 0.0185444 2024-09-17T15:27:03.298 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 84 20 90111 90091 4.18878 2.35156 0.0309022 0.018643 2024-09-17T15:27:03.340 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:27:03.879 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:27:03.882 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:27:03.901 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":114,"stamp":"2024-09-17T15:27:03.610844+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":363037425,"num_objects":88643,"num_object_clones":0,"num_object_copies":177286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":2969,"num_objects_unfound":0,"num_objects_dirty":88643,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":48750,"num_write_kb":194994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1103,"num_bytes_recovered":4513811,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88716,"ondisk_log_size":88716,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1141492,"kb_used_data":709124,"kb_used_omap":13,"kb_used_meta":432306,"kb_avail":281974028,"statfs":{"total":289910292480,"available":288741404672,"internally_reserved":0,"allocated":726142976,"data_stored":725289334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":442682367},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,4,2,3,10,9,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28536832,"num_objects":6967,"num_object_clones":0,"num_object_copies":13934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-416,"num_objects_unfound":0,"num_objects_dirty":6967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6967,"num_write_kb":27868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":416,"num_bytes_recovered":1703936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.011362"},"pg_stats":[{"pgid":"2.f","version":"41'5471","reported_seq":5543,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:00.270777+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:27:00.270777+0000","last_peered":"2024-09-17T15:27:00.270777+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:27:00.270777+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":5471,"log_dups_size":0,"ondisk_log_size":5471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22409216,"num_objects":5471,"num_object_clones":0,"num_object_copies":10942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5471,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5471,"num_write_kb":21884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1401},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'5529","reported_seq":5626,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:01.817804+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:27:01.817804+0000","last_peered":"2024-09-17T15:27:01.817804+0000","last_clean":"2024-09-17T15:27:01.817804+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:27:01.817804+0000","last_undegraded":"2024-09-17T15:27:01.817804+0000","last_fullsized":"2024-09-17T15:27:01.817804+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":5529,"log_dups_size":0,"ondisk_log_size":5529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22623174,"num_objects":5525,"num_object_clones":0,"num_object_copies":11050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5525,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3886,"num_write_kb":15544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'5640","reported_seq":5760,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:01.728386+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:27:01.728386+0000","last_peered":"2024-09-17T15:27:01.728386+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:27:01.728386+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":5640,"log_dups_size":0,"ondisk_log_size":5640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23085056,"num_objects":5636,"num_object_clones":0,"num_object_copies":11272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3797,"num_write_kb":15188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":1468}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'5602","reported_seq":5666,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:01.815957+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:27:01.815957+0000","last_peered":"2024-09-17T15:27:01.815957+0000","last_clean":"2024-09-17T15:27:01.815957+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:27:01.815957+0000","last_undegraded":"2024-09-17T15:27:01.815957+0000","last_fullsized":"2024-09-17T15:27:01.815957+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":5602,"log_dups_size":0,"ondisk_log_size":5602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22945792,"num_objects":5602,"num_object_clones":0,"num_object_copies":11204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5602,"num_write_kb":22408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'5387","reported_seq":5511,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:00.267040+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:27:00.267040+0000","last_peered":"2024-09-17T15:27:00.267040+0000","last_clean":"2024-09-17T15:27:00.267040+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:27:00.267040+0000","last_undegraded":"2024-09-17T15:27:00.267040+0000","last_fullsized":"2024-09-17T15:27:00.267040+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":5387,"log_dups_size":0,"ondisk_log_size":5387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22036963,"num_objects":5381,"num_object_clones":0,"num_object_copies":10762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5381,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2369,"num_write_kb":9476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'5679","reported_seq":5808,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:00.189213+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:27:00.189213+0000","last_peered":"2024-09-17T15:27:00.189213+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:27:00.189213+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":5679,"log_dups_size":0,"ondisk_log_size":5679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23241187,"num_objects":5675,"num_object_clones":0,"num_object_copies":11350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5675,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2178,"num_write_kb":8712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1405},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'5592","reported_seq":5725,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:00.282125+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:27:00.282125+0000","last_peered":"2024-09-17T15:27:00.282125+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:27:00.282125+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":5592,"log_dups_size":0,"ondisk_log_size":5592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22881222,"num_objects":5588,"num_object_clones":0,"num_object_copies":11176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5588,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2361,"num_write_kb":9444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1414},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'5554","reported_seq":5676,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:00.280536+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:27:00.280536+0000","last_peered":"2024-09-17T15:27:00.280536+0000","last_clean":"2024-09-17T15:27:00.280536+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:27:00.280536+0000","last_undegraded":"2024-09-17T15:27:00.280536+0000","last_fullsized":"2024-09-17T15:27:00.280536+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":5554,"log_dups_size":0,"ondisk_log_size":5554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22740992,"num_objects":5552,"num_object_clones":0,"num_object_copies":11104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2508,"num_write_kb":10032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'5531","reported_seq":7858,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:27:01.806008+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:27:01.806008+0000","last_peered":"2024-09-17T15:27:01.806008+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:27:01.806008+0000","last_undegraded":"2024-09-17T15:27:01.806008+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":5531,"log_dups_size":0,"ondisk_log_size":5531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22626787,"num_objects":5525,"num_object_clones":0,"num_object_copies":11050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2969,"num_objects_unfound":0,"num_objects_dirty":5525,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3531,"num_write_kb":14124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1102,"num_bytes_recovered":4513792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2969},{"shards":"2","objects":2556}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233552+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233552+0000","last_peered":"2024-09-17T15:26:22.233552+0000","last_clean":"2024-09-17T15:26:22.233552+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233552+0000","last_undegraded":"2024-09-17T15:26:22.233552+0000","last_fullsized":"2024-09-17T15:26:22.233552+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'5412","reported_seq":5535,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:00.295398+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:27:00.295398+0000","last_peered":"2024-09-17T15:27:00.295398+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:27:00.295398+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":5412,"log_dups_size":0,"ondisk_log_size":5412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22134784,"num_objects":5404,"num_object_clones":0,"num_object_copies":10808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2008,"num_write_kb":8032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1394},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":97,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412553+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412553+0000","last_peered":"2024-09-17T15:26:22.412553+0000","last_clean":"2024-09-17T15:26:22.412553+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412553+0000","last_undegraded":"2024-09-17T15:26:22.412553+0000","last_fullsized":"2024-09-17T15:26:22.412553+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'5510","reported_seq":5625,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:01.779701+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:27:01.779701+0000","last_peered":"2024-09-17T15:27:01.779701+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:27:01.779701+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":5510,"log_dups_size":0,"ondisk_log_size":5510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22548963,"num_objects":5506,"num_object_clones":0,"num_object_copies":11012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5506,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3405,"num_write_kb":13620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":1429}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'5591","reported_seq":5704,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:01.804664+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:27:01.804664+0000","last_peered":"2024-09-17T15:27:01.804664+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:27:01.804664+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":5591,"log_dups_size":0,"ondisk_log_size":5591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22876160,"num_objects":5585,"num_object_clones":0,"num_object_copies":11170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5585,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3353,"num_write_kb":13412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":1448}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'5585","reported_seq":5715,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:00.279702+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:27:00.279702+0000","last_peered":"2024-09-17T15:27:00.279702+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:27:00.279702+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":5585,"log_dups_size":0,"ondisk_log_size":5585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22867968,"num_objects":5583,"num_object_clones":0,"num_object_copies":11166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2133,"num_write_kb":8532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1445},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'5538","reported_seq":5674,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:00.157428+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:27:00.157428+0000","last_peered":"2024-09-17T15:27:00.157428+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:27:00.157428+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":5538,"log_dups_size":0,"ondisk_log_size":5538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22650880,"num_objects":5530,"num_object_clones":0,"num_object_copies":11060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1825,"num_write_kb":7300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1373},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229013+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229013+0000","last_peered":"2024-09-17T15:26:22.229013+0000","last_clean":"2024-09-17T15:26:22.229013+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229013+0000","last_undegraded":"2024-09-17T15:26:22.229013+0000","last_fullsized":"2024-09-17T15:26:22.229013+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'5549","reported_seq":5674,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:00.326008+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:27:00.326008+0000","last_peered":"2024-09-17T15:27:00.326008+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:27:00.326008+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":5549,"log_dups_size":0,"ondisk_log_size":5549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22712320,"num_objects":5545,"num_object_clones":0,"num_object_copies":11090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2336,"num_write_kb":9344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1482},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'5543","reported_seq":5677,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:00.303129+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:27:00.303129+0000","last_peered":"2024-09-17T15:27:00.303129+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:27:00.303129+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":5543,"log_dups_size":0,"ondisk_log_size":5543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22655942,"num_objects":5533,"num_object_clones":0,"num_object_copies":11066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1985,"num_write_kb":7940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1424},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":363037406,"num_objects":88641,"num_object_clones":0,"num_object_copies":177282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":2969,"num_objects_unfound":0,"num_objects_dirty":88641,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":48748,"num_write_kb":194992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1102,"num_bytes_recovered":4513792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":728342528,"data_stored":728187169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88713,"ondisk_log_size":88713,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051608,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":463604,"kb_used_data":309732,"kb_used_omap":4,"kb_used_meta":153851,"kb_avail":93908236,"statfs":{"total":96636764160,"available":96162033664,"internally_reserved":0,"allocated":317165568,"data_stored":316899395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":157544108},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,0,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724553,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":163756,"kb_used_data":69596,"kb_used_omap":4,"kb_used_meta":94139,"kb_avail":94208084,"statfs":{"total":96636764160,"available":96469078016,"internally_reserved":0,"allocated":71266304,"data_stored":70949085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":96399019},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,1,3,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247836,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":514132,"kb_used_data":329796,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93857708,"statfs":{"total":96636764160,"available":96110292992,"internally_reserved":0,"allocated":337711104,"data_stored":337440854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":188739240},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,7,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":338911232,"data_stored":338875102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":72253440,"data_stored":72170341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":317177856,"data_stored":317141726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:27:03.902 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:27:04.297 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 85 20 90620 90600 4.16289 1.98828 0.0250928 0.0187585 2024-09-17T15:27:04.436 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:27:04.439 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:27:04.463 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":114,"stamp":"2024-09-17T15:27:03.610844+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":363037425,"num_objects":88643,"num_object_clones":0,"num_object_copies":177286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":2969,"num_objects_unfound":0,"num_objects_dirty":88643,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":48750,"num_write_kb":194994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1103,"num_bytes_recovered":4513811,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88716,"ondisk_log_size":88716,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1141492,"kb_used_data":709124,"kb_used_omap":13,"kb_used_meta":432306,"kb_avail":281974028,"statfs":{"total":289910292480,"available":288741404672,"internally_reserved":0,"allocated":726142976,"data_stored":725289334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":442682367},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,4,2,3,10,9,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28536832,"num_objects":6967,"num_object_clones":0,"num_object_copies":13934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-416,"num_objects_unfound":0,"num_objects_dirty":6967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6967,"num_write_kb":27868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":416,"num_bytes_recovered":1703936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.011362"},"pg_stats":[{"pgid":"2.f","version":"41'5471","reported_seq":5543,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:00.270777+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:27:00.270777+0000","last_peered":"2024-09-17T15:27:00.270777+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:27:00.270777+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":5471,"log_dups_size":0,"ondisk_log_size":5471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22409216,"num_objects":5471,"num_object_clones":0,"num_object_copies":10942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5471,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5471,"num_write_kb":21884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1401},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'5529","reported_seq":5626,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:01.817804+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:27:01.817804+0000","last_peered":"2024-09-17T15:27:01.817804+0000","last_clean":"2024-09-17T15:27:01.817804+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:27:01.817804+0000","last_undegraded":"2024-09-17T15:27:01.817804+0000","last_fullsized":"2024-09-17T15:27:01.817804+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":5529,"log_dups_size":0,"ondisk_log_size":5529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22623174,"num_objects":5525,"num_object_clones":0,"num_object_copies":11050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5525,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3886,"num_write_kb":15544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'5640","reported_seq":5760,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:01.728386+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:27:01.728386+0000","last_peered":"2024-09-17T15:27:01.728386+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:27:01.728386+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":5640,"log_dups_size":0,"ondisk_log_size":5640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23085056,"num_objects":5636,"num_object_clones":0,"num_object_copies":11272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3797,"num_write_kb":15188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":1468}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'5602","reported_seq":5666,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:01.815957+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:27:01.815957+0000","last_peered":"2024-09-17T15:27:01.815957+0000","last_clean":"2024-09-17T15:27:01.815957+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:27:01.815957+0000","last_undegraded":"2024-09-17T15:27:01.815957+0000","last_fullsized":"2024-09-17T15:27:01.815957+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":5602,"log_dups_size":0,"ondisk_log_size":5602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22945792,"num_objects":5602,"num_object_clones":0,"num_object_copies":11204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5602,"num_write_kb":22408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'5387","reported_seq":5511,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:00.267040+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:27:00.267040+0000","last_peered":"2024-09-17T15:27:00.267040+0000","last_clean":"2024-09-17T15:27:00.267040+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:27:00.267040+0000","last_undegraded":"2024-09-17T15:27:00.267040+0000","last_fullsized":"2024-09-17T15:27:00.267040+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":5387,"log_dups_size":0,"ondisk_log_size":5387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22036963,"num_objects":5381,"num_object_clones":0,"num_object_copies":10762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5381,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2369,"num_write_kb":9476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'5679","reported_seq":5808,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:00.189213+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:27:00.189213+0000","last_peered":"2024-09-17T15:27:00.189213+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:27:00.189213+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":5679,"log_dups_size":0,"ondisk_log_size":5679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23241187,"num_objects":5675,"num_object_clones":0,"num_object_copies":11350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5675,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2178,"num_write_kb":8712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1405},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'5592","reported_seq":5725,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:00.282125+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:27:00.282125+0000","last_peered":"2024-09-17T15:27:00.282125+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:27:00.282125+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":5592,"log_dups_size":0,"ondisk_log_size":5592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22881222,"num_objects":5588,"num_object_clones":0,"num_object_copies":11176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5588,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2361,"num_write_kb":9444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1414},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'5554","reported_seq":5676,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:00.280536+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:27:00.280536+0000","last_peered":"2024-09-17T15:27:00.280536+0000","last_clean":"2024-09-17T15:27:00.280536+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:27:00.280536+0000","last_undegraded":"2024-09-17T15:27:00.280536+0000","last_fullsized":"2024-09-17T15:27:00.280536+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":5554,"log_dups_size":0,"ondisk_log_size":5554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22740992,"num_objects":5552,"num_object_clones":0,"num_object_copies":11104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2508,"num_write_kb":10032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'5531","reported_seq":7858,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:27:01.806008+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:27:01.806008+0000","last_peered":"2024-09-17T15:27:01.806008+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:27:01.806008+0000","last_undegraded":"2024-09-17T15:27:01.806008+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":5531,"log_dups_size":0,"ondisk_log_size":5531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22626787,"num_objects":5525,"num_object_clones":0,"num_object_copies":11050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2969,"num_objects_unfound":0,"num_objects_dirty":5525,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3531,"num_write_kb":14124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1102,"num_bytes_recovered":4513792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2969},{"shards":"2","objects":2556}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233552+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233552+0000","last_peered":"2024-09-17T15:26:22.233552+0000","last_clean":"2024-09-17T15:26:22.233552+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233552+0000","last_undegraded":"2024-09-17T15:26:22.233552+0000","last_fullsized":"2024-09-17T15:26:22.233552+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'5412","reported_seq":5535,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:00.295398+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:27:00.295398+0000","last_peered":"2024-09-17T15:27:00.295398+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:27:00.295398+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":5412,"log_dups_size":0,"ondisk_log_size":5412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22134784,"num_objects":5404,"num_object_clones":0,"num_object_copies":10808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2008,"num_write_kb":8032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1394},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":97,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412553+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412553+0000","last_peered":"2024-09-17T15:26:22.412553+0000","last_clean":"2024-09-17T15:26:22.412553+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412553+0000","last_undegraded":"2024-09-17T15:26:22.412553+0000","last_fullsized":"2024-09-17T15:26:22.412553+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'5510","reported_seq":5625,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:01.779701+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:27:01.779701+0000","last_peered":"2024-09-17T15:27:01.779701+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:27:01.779701+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":5510,"log_dups_size":0,"ondisk_log_size":5510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22548963,"num_objects":5506,"num_object_clones":0,"num_object_copies":11012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5506,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3405,"num_write_kb":13620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":1429}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'5591","reported_seq":5704,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:01.804664+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:27:01.804664+0000","last_peered":"2024-09-17T15:27:01.804664+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:27:01.804664+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":5591,"log_dups_size":0,"ondisk_log_size":5591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22876160,"num_objects":5585,"num_object_clones":0,"num_object_copies":11170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5585,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3353,"num_write_kb":13412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":1448}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'5585","reported_seq":5715,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:00.279702+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:27:00.279702+0000","last_peered":"2024-09-17T15:27:00.279702+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:27:00.279702+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":5585,"log_dups_size":0,"ondisk_log_size":5585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22867968,"num_objects":5583,"num_object_clones":0,"num_object_copies":11166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2133,"num_write_kb":8532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1445},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'5538","reported_seq":5674,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:00.157428+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:27:00.157428+0000","last_peered":"2024-09-17T15:27:00.157428+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:27:00.157428+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":5538,"log_dups_size":0,"ondisk_log_size":5538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22650880,"num_objects":5530,"num_object_clones":0,"num_object_copies":11060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1825,"num_write_kb":7300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1373},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229013+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229013+0000","last_peered":"2024-09-17T15:26:22.229013+0000","last_clean":"2024-09-17T15:26:22.229013+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229013+0000","last_undegraded":"2024-09-17T15:26:22.229013+0000","last_fullsized":"2024-09-17T15:26:22.229013+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'5549","reported_seq":5674,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:00.326008+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:27:00.326008+0000","last_peered":"2024-09-17T15:27:00.326008+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:27:00.326008+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":5549,"log_dups_size":0,"ondisk_log_size":5549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22712320,"num_objects":5545,"num_object_clones":0,"num_object_copies":11090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2336,"num_write_kb":9344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1482},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'5543","reported_seq":5677,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:00.303129+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:27:00.303129+0000","last_peered":"2024-09-17T15:27:00.303129+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:27:00.303129+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":5543,"log_dups_size":0,"ondisk_log_size":5543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22655942,"num_objects":5533,"num_object_clones":0,"num_object_copies":11066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1985,"num_write_kb":7940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1424},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":363037406,"num_objects":88641,"num_object_clones":0,"num_object_copies":177282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":2969,"num_objects_unfound":0,"num_objects_dirty":88641,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":48748,"num_write_kb":194992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1102,"num_bytes_recovered":4513792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":728342528,"data_stored":728187169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88713,"ondisk_log_size":88713,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051608,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":463604,"kb_used_data":309732,"kb_used_omap":4,"kb_used_meta":153851,"kb_avail":93908236,"statfs":{"total":96636764160,"available":96162033664,"internally_reserved":0,"allocated":317165568,"data_stored":316899395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":157544108},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,0,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724553,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":163756,"kb_used_data":69596,"kb_used_omap":4,"kb_used_meta":94139,"kb_avail":94208084,"statfs":{"total":96636764160,"available":96469078016,"internally_reserved":0,"allocated":71266304,"data_stored":70949085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":96399019},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,1,3,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247836,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":514132,"kb_used_data":329796,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93857708,"statfs":{"total":96636764160,"available":96110292992,"internally_reserved":0,"allocated":337711104,"data_stored":337440854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":188739240},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,7,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":338911232,"data_stored":338875102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":72253440,"data_stored":72170341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":317177856,"data_stored":317141726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:27:04.464 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:27:04.464 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'5471", 'reported_seq': 5543, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:00.270777+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:27:00.270777+0000', 'last_peered': '2024-09-17T15:27:00.270777+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:27:00.270777+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 5471, 'log_dups_size': 0, 'ondisk_log_size': 5471, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22409216, 'num_objects': 5471, 'num_object_clones': 0, 'num_object_copies': 10942, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4070, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5471, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5471, 'num_write_kb': 21884, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1401}, {'shards': '0,2', 'objects': 4070}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:04.464 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:27:04.464 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'5640", 'reported_seq': 5760, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:01.728386+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:27:01.728386+0000', 'last_peered': '2024-09-17T15:27:01.728386+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:27:01.728386+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 5640, 'log_dups_size': 0, 'ondisk_log_size': 5640, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23085056, 'num_objects': 5636, 'num_object_clones': 0, 'num_object_copies': 11272, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4169, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5636, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3797, 'num_write_kb': 15188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4168}, {'shards': '2', 'objects': 1468}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:27:04.465 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:27:04.465 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'5679", 'reported_seq': 5808, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:00.189213+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:27:00.189213+0000', 'last_peered': '2024-09-17T15:27:00.189213+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:27:00.189213+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 5679, 'log_dups_size': 0, 'ondisk_log_size': 5679, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23241187, 'num_objects': 5675, 'num_object_clones': 0, 'num_object_copies': 11350, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4271, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5675, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2178, 'num_write_kb': 8712, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1405}, {'shards': '0,2', 'objects': 4268}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:04.465 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:27:04.465 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'5592", 'reported_seq': 5725, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:00.282125+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:27:00.282125+0000', 'last_peered': '2024-09-17T15:27:00.282125+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:27:00.282125+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 5592, 'log_dups_size': 0, 'ondisk_log_size': 5592, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22881222, 'num_objects': 5588, 'num_object_clones': 0, 'num_object_copies': 11176, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4174, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5588, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 2361, 'num_write_kb': 9444, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1414}, {'shards': '0,2', 'objects': 4174}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:04.465 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-17T15:27:04.465 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "41'5531", 'reported_seq': 7858, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:27:01.806008+0000', 'last_change': '2024-09-17T15:26:22.588242+0000', 'last_active': '2024-09-17T15:27:01.806008+0000', 'last_peered': '2024-09-17T15:27:01.806008+0000', 'last_clean': '2024-09-17T15:26:20.155646+0000', 'last_became_active': '2024-09-17T15:26:22.557742+0000', 'last_became_peered': '2024-09-17T15:26:22.557742+0000', 'last_unstale': '2024-09-17T15:27:01.806008+0000', 'last_undegraded': '2024-09-17T15:27:01.806008+0000', 'last_fullsized': '2024-09-17T15:26:22.188850+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2016", 'last_scrub_stamp': '2024-09-17T15:25:53.959011+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:53.959011+0000', 'objects_scrubbed': 1994, 'log_size': 5531, 'log_dups_size': 0, 'ondisk_log_size': 5531, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 568, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22626787, 'num_objects': 5525, 'num_object_clones': 0, 'num_object_copies': 11050, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2969, 'num_objects_unfound': 0, 'num_objects_dirty': 5525, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3531, 'num_write_kb': 14124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1102, 'num_bytes_recovered': 4513792, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2969}, {'shards': '2', 'objects': 2556}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:27:04.465 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:27:04.465 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'5412", 'reported_seq': 5535, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:00.295398+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:27:00.295398+0000', 'last_peered': '2024-09-17T15:27:00.295398+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:27:00.295398+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 5412, 'log_dups_size': 0, 'ondisk_log_size': 5412, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22134784, 'num_objects': 5404, 'num_object_clones': 0, 'num_object_copies': 10808, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4012, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5404, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2008, 'num_write_kb': 8032, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1394}, {'shards': '0,2', 'objects': 4010}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:04.465 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:27:04.465 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'5510", 'reported_seq': 5625, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:01.779701+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:27:01.779701+0000', 'last_peered': '2024-09-17T15:27:01.779701+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:27:01.779701+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 5510, 'log_dups_size': 0, 'ondisk_log_size': 5510, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22548963, 'num_objects': 5506, 'num_object_clones': 0, 'num_object_copies': 11012, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4078, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5506, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3405, 'num_write_kb': 13620, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4077}, {'shards': '2', 'objects': 1429}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:27:04.465 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:27:04.466 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'5591", 'reported_seq': 5704, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:01.804664+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:27:01.804664+0000', 'last_peered': '2024-09-17T15:27:01.804664+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:27:01.804664+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 5591, 'log_dups_size': 0, 'ondisk_log_size': 5591, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22876160, 'num_objects': 5585, 'num_object_clones': 0, 'num_object_copies': 11170, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5585, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3353, 'num_write_kb': 13412, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4137}, {'shards': '2', 'objects': 1448}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:27:04.466 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:27:04.466 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'5585", 'reported_seq': 5715, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:00.279702+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:27:00.279702+0000', 'last_peered': '2024-09-17T15:27:00.279702+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:27:00.279702+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 5585, 'log_dups_size': 0, 'ondisk_log_size': 5585, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22867968, 'num_objects': 5583, 'num_object_clones': 0, 'num_object_copies': 11166, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5583, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2133, 'num_write_kb': 8532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1445}, {'shards': '0,2', 'objects': 4138}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:04.466 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:27:04.466 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'5538", 'reported_seq': 5674, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:00.157428+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:27:00.157428+0000', 'last_peered': '2024-09-17T15:27:00.157428+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:27:00.157428+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 5538, 'log_dups_size': 0, 'ondisk_log_size': 5538, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22650880, 'num_objects': 5530, 'num_object_clones': 0, 'num_object_copies': 11060, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4160, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5530, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1825, 'num_write_kb': 7300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1373}, {'shards': '0,2', 'objects': 4157}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:04.466 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:27:04.466 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'5549", 'reported_seq': 5674, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:00.326008+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:27:00.326008+0000', 'last_peered': '2024-09-17T15:27:00.326008+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:27:00.326008+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 5549, 'log_dups_size': 0, 'ondisk_log_size': 5549, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22712320, 'num_objects': 5545, 'num_object_clones': 0, 'num_object_copies': 11090, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4065, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5545, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2336, 'num_write_kb': 9344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1482}, {'shards': '0,2', 'objects': 4063}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:04.466 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:27:04.466 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'5543", 'reported_seq': 5677, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:00.303129+0000', 'last_change': '2024-09-17T15:26:22.654521+0000', 'last_active': '2024-09-17T15:27:00.303129+0000', 'last_peered': '2024-09-17T15:27:00.303129+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:27:00.303129+0000', 'last_undegraded': '2024-09-17T15:26:22.307795+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 5543, 'log_dups_size': 0, 'ondisk_log_size': 5543, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22655942, 'num_objects': 5533, 'num_object_clones': 0, 'num_object_copies': 11066, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4111, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5533, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1985, 'num_write_kb': 7940, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1424}, {'shards': '0,2', 'objects': 4109}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:04.467 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:27:05.161 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:27:05.161 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45388/177286 objects degraded (25.602%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":162,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":88643,"data_bytes":363037425,"bytes_used":1168887808,"bytes_avail":288741404672,"bytes_total":289910292480,"degraded_objects":45388,"degraded_total":177286,"degraded_ratio":0.25601570343963992,"misplaced_objects":2969,"misplaced_total":177286,"misplaced_ratio":0.016746951253906118,"recovering_objects_per_sec":34,"recovering_bytes_per_sec":141860,"recovering_keys_per_sec":0,"num_objects_recovered":416,"num_bytes_recovered":1703936,"num_keys_recovered":0,"write_bytes_sec":2375819,"read_op_per_sec":0,"write_op_per_sec":580},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-17T15:26:23.592815+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-17T15:27:05.185 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:27:05.186 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:27:05.299 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 86 20 91152 91132 4.13856 2.07812 0.030521 0.0188694 2024-09-17T15:27:05.773 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:27:05.773 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:27:05.801 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":115,"stamp":"2024-09-17T15:27:05.611422+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":370352881,"num_objects":90429,"num_object_clones":0,"num_object_copies":180858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":2969,"num_objects_unfound":0,"num_objects_dirty":90429,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":50536,"num_write_kb":202138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1103,"num_bytes_recovered":4513811,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":90502,"ondisk_log_size":90502,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1156856,"kb_used_data":727240,"kb_used_omap":13,"kb_used_meta":429554,"kb_avail":281958664,"statfs":{"total":289910292480,"available":288725671936,"internally_reserved":0,"allocated":744693760,"data_stored":743840118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":439864319},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,7,0,2,7,9,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":31162368,"num_objects":7608,"num_object_clones":0,"num_object_copies":15216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-281,"num_objects_unfound":0,"num_objects_dirty":7608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7608,"num_write_kb":30432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":281,"num_bytes_recovered":1150976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008321"},"pg_stats":[{"pgid":"2.f","version":"41'5643","reported_seq":5715,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:05.284328+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:27:05.284328+0000","last_peered":"2024-09-17T15:27:05.284328+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:27:05.284328+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":5643,"log_dups_size":0,"ondisk_log_size":5643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23113728,"num_objects":5643,"num_object_clones":0,"num_object_copies":11286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5643,"num_write_kb":22572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1573},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'5529","reported_seq":5626,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:01.817804+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:27:01.817804+0000","last_peered":"2024-09-17T15:27:01.817804+0000","last_clean":"2024-09-17T15:27:01.817804+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:27:01.817804+0000","last_undegraded":"2024-09-17T15:27:01.817804+0000","last_fullsized":"2024-09-17T15:27:01.817804+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":5529,"log_dups_size":0,"ondisk_log_size":5529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22623174,"num_objects":5525,"num_object_clones":0,"num_object_copies":11050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5525,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3886,"num_write_kb":15544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'5640","reported_seq":5760,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:01.728386+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:27:01.728386+0000","last_peered":"2024-09-17T15:27:01.728386+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:27:01.728386+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":5640,"log_dups_size":0,"ondisk_log_size":5640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23085056,"num_objects":5636,"num_object_clones":0,"num_object_copies":11272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3797,"num_write_kb":15188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":1468}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'5602","reported_seq":5666,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:01.815957+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:27:01.815957+0000","last_peered":"2024-09-17T15:27:01.815957+0000","last_clean":"2024-09-17T15:27:01.815957+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:27:01.815957+0000","last_undegraded":"2024-09-17T15:27:01.815957+0000","last_fullsized":"2024-09-17T15:27:01.815957+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":5602,"log_dups_size":0,"ondisk_log_size":5602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22945792,"num_objects":5602,"num_object_clones":0,"num_object_copies":11204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5602,"num_write_kb":22408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'5595","reported_seq":5719,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:05.319199+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:27:05.319199+0000","last_peered":"2024-09-17T15:27:05.319199+0000","last_clean":"2024-09-17T15:27:05.319199+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:27:05.319199+0000","last_undegraded":"2024-09-17T15:27:05.319199+0000","last_fullsized":"2024-09-17T15:27:05.319199+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":5595,"log_dups_size":0,"ondisk_log_size":5595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22888931,"num_objects":5589,"num_object_clones":0,"num_object_copies":11178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5589,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2577,"num_write_kb":10308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'5845","reported_seq":5974,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:05.278811+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:27:05.278811+0000","last_peered":"2024-09-17T15:27:05.278811+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:27:05.278811+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":5845,"log_dups_size":0,"ondisk_log_size":5845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23921123,"num_objects":5841,"num_object_clones":0,"num_object_copies":11682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5841,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2344,"num_write_kb":9376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1571},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'5777","reported_seq":5910,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:05.238201+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:27:05.238201+0000","last_peered":"2024-09-17T15:27:05.238201+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:27:05.238201+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":5777,"log_dups_size":0,"ondisk_log_size":5777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23638982,"num_objects":5773,"num_object_clones":0,"num_object_copies":11546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5773,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2546,"num_write_kb":10184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1599},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'5720","reported_seq":5842,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:05.241282+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:27:05.241282+0000","last_peered":"2024-09-17T15:27:05.241282+0000","last_clean":"2024-09-17T15:27:05.241282+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:27:05.241282+0000","last_undegraded":"2024-09-17T15:27:05.241282+0000","last_fullsized":"2024-09-17T15:27:05.241282+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":5720,"log_dups_size":0,"ondisk_log_size":5720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23420928,"num_objects":5718,"num_object_clones":0,"num_object_copies":11436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2674,"num_write_kb":10696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'5531","reported_seq":7858,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:27:01.806008+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:27:01.806008+0000","last_peered":"2024-09-17T15:27:01.806008+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:27:01.806008+0000","last_undegraded":"2024-09-17T15:27:01.806008+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":5531,"log_dups_size":0,"ondisk_log_size":5531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22626787,"num_objects":5525,"num_object_clones":0,"num_object_copies":11050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2969,"num_objects_unfound":0,"num_objects_dirty":5525,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3531,"num_write_kb":14124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1102,"num_bytes_recovered":4513792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2969},{"shards":"2","objects":2556}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233552+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233552+0000","last_peered":"2024-09-17T15:26:22.233552+0000","last_clean":"2024-09-17T15:26:22.233552+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233552+0000","last_undegraded":"2024-09-17T15:26:22.233552+0000","last_fullsized":"2024-09-17T15:26:22.233552+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'5586","reported_seq":5711,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:05.267397+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:27:05.267397+0000","last_peered":"2024-09-17T15:27:05.267397+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:27:05.267397+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":5586,"log_dups_size":0,"ondisk_log_size":5586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22847488,"num_objects":5578,"num_object_clones":0,"num_object_copies":11156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5578,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2182,"num_write_kb":8728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1568},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":97,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412553+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412553+0000","last_peered":"2024-09-17T15:26:22.412553+0000","last_clean":"2024-09-17T15:26:22.412553+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412553+0000","last_undegraded":"2024-09-17T15:26:22.412553+0000","last_fullsized":"2024-09-17T15:26:22.412553+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'5510","reported_seq":5625,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:01.779701+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:27:01.779701+0000","last_peered":"2024-09-17T15:27:01.779701+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:27:01.779701+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":5510,"log_dups_size":0,"ondisk_log_size":5510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22548963,"num_objects":5506,"num_object_clones":0,"num_object_copies":11012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5506,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3405,"num_write_kb":13620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":1429}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'5591","reported_seq":5704,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:01.804664+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:27:01.804664+0000","last_peered":"2024-09-17T15:27:01.804664+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:27:01.804664+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":5591,"log_dups_size":0,"ondisk_log_size":5591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22876160,"num_objects":5585,"num_object_clones":0,"num_object_copies":11170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5585,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3353,"num_write_kb":13412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":1448}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'5765","reported_seq":5895,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:05.167785+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:27:05.167785+0000","last_peered":"2024-09-17T15:27:05.167785+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:27:05.167785+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":5765,"log_dups_size":0,"ondisk_log_size":5765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23605248,"num_objects":5763,"num_object_clones":0,"num_object_copies":11526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2313,"num_write_kb":9252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1625},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'5726","reported_seq":5862,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:05.221310+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:27:05.221310+0000","last_peered":"2024-09-17T15:27:05.221310+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:27:05.221310+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":5726,"log_dups_size":0,"ondisk_log_size":5726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23420928,"num_objects":5718,"num_object_clones":0,"num_object_copies":11436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2013,"num_write_kb":8052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1561},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229013+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229013+0000","last_peered":"2024-09-17T15:26:22.229013+0000","last_clean":"2024-09-17T15:26:22.229013+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229013+0000","last_undegraded":"2024-09-17T15:26:22.229013+0000","last_fullsized":"2024-09-17T15:26:22.229013+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'5730","reported_seq":5856,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:05.241189+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:27:05.241189+0000","last_peered":"2024-09-17T15:27:05.241189+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:27:05.241189+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":5730,"log_dups_size":0,"ondisk_log_size":5730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23453696,"num_objects":5726,"num_object_clones":0,"num_object_copies":11452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2517,"num_write_kb":10068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1663},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'5709","reported_seq":5845,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:05.331930+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:27:05.331930+0000","last_peered":"2024-09-17T15:27:05.331930+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:27:05.331930+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":5709,"log_dups_size":0,"ondisk_log_size":5709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23335878,"num_objects":5699,"num_object_clones":0,"num_object_copies":11398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5699,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2151,"num_write_kb":8604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1590},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":370352862,"num_objects":90427,"num_object_clones":0,"num_object_copies":180854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":2969,"num_objects_unfound":0,"num_objects_dirty":90427,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":50534,"num_write_kb":202136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1102,"num_bytes_recovered":4513792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":746516480,"data_stored":746361121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":90499,"ondisk_log_size":90499,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051608,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":463604,"kb_used_data":309732,"kb_used_omap":4,"kb_used_meta":153851,"kb_avail":93908236,"statfs":{"total":96636764160,"available":96162033664,"internally_reserved":0,"allocated":317165568,"data_stored":316899395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":157544108},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,0,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724554,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":182864,"kb_used_data":78976,"kb_used_omap":4,"kb_used_meta":103867,"kb_avail":94188976,"statfs":{"total":96636764160,"available":96449511424,"internally_reserved":0,"allocated":80871424,"data_stored":80554205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":106360491},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,1,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247837,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":510388,"kb_used_data":338532,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93861452,"statfs":{"total":96636764160,"available":96114126848,"internally_reserved":0,"allocated":346656768,"data_stored":346386518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":175959720},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,1,5,6,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":347648000,"data_stored":347611870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81690624,"data_stored":81607525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":317177856,"data_stored":317141726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:27:06.301 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 87 20 91721 91701 4.11646 2.22266 0.0082839 0.0189688 2024-09-17T15:27:07.301 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 88 20 92342 92322 4.09725 2.42578 0.0275424 0.0190607 2024-09-17T15:27:08.302 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 89 20 92938 92918 4.07737 2.32812 0.0485516 0.0191536 2024-09-17T15:27:08.803 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:27:09.301 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 90 20 93499 93479 4.05641 2.19141 0.0132884 0.0192511 2024-09-17T15:27:09.350 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:27:09.351 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:27:09.375 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":116,"stamp":"2024-09-17T15:27:07.612068+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":374702833,"num_objects":91491,"num_object_clones":0,"num_object_copies":182982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":2821,"num_objects_unfound":0,"num_objects_dirty":91491,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":51598,"num_write_kb":206386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1251,"num_bytes_recovered":5120019,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91564,"ondisk_log_size":91564,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1166776,"kb_used_data":733000,"kb_used_omap":13,"kb_used_meta":433714,"kb_avail":281948744,"statfs":{"total":289910292480,"available":288715513856,"internally_reserved":0,"allocated":750592000,"data_stored":749738358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":444124159},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,5,1,2,11,14,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28061696,"num_objects":6851,"num_object_clones":0,"num_object_copies":13702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-429,"num_objects_unfound":0,"num_objects_dirty":6851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6851,"num_write_kb":27404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":429,"num_bytes_recovered":1757184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008531"},"pg_stats":[{"pgid":"2.f","version":"41'5643","reported_seq":5715,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:05.284328+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:27:05.284328+0000","last_peered":"2024-09-17T15:27:05.284328+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:27:05.284328+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":5643,"log_dups_size":0,"ondisk_log_size":5643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23113728,"num_objects":5643,"num_object_clones":0,"num_object_copies":11286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5643,"num_write_kb":22572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1573},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'5704","reported_seq":5801,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:06.833189+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:27:06.833189+0000","last_peered":"2024-09-17T15:27:06.833189+0000","last_clean":"2024-09-17T15:27:06.833189+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:27:06.833189+0000","last_undegraded":"2024-09-17T15:27:06.833189+0000","last_fullsized":"2024-09-17T15:27:06.833189+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":5704,"log_dups_size":0,"ondisk_log_size":5704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23339974,"num_objects":5700,"num_object_clones":0,"num_object_copies":11400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5700,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4061,"num_write_kb":16244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'5809","reported_seq":5927,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:06.845856+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:27:06.845856+0000","last_peered":"2024-09-17T15:27:06.845856+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:27:06.845856+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":5809,"log_dups_size":0,"ondisk_log_size":5809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23777280,"num_objects":5805,"num_object_clones":0,"num_object_copies":11610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3966,"num_write_kb":15864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":1637}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'5793","reported_seq":5857,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:06.842626+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:27:06.842626+0000","last_peered":"2024-09-17T15:27:06.842626+0000","last_clean":"2024-09-17T15:27:06.842626+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:27:06.842626+0000","last_undegraded":"2024-09-17T15:27:06.842626+0000","last_fullsized":"2024-09-17T15:27:06.842626+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":5793,"log_dups_size":0,"ondisk_log_size":5793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23728128,"num_objects":5793,"num_object_clones":0,"num_object_copies":11586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5793,"num_write_kb":23172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'5595","reported_seq":5719,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:05.319199+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:27:05.319199+0000","last_peered":"2024-09-17T15:27:05.319199+0000","last_clean":"2024-09-17T15:27:05.319199+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:27:05.319199+0000","last_undegraded":"2024-09-17T15:27:05.319199+0000","last_fullsized":"2024-09-17T15:27:05.319199+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":5595,"log_dups_size":0,"ondisk_log_size":5595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22888931,"num_objects":5589,"num_object_clones":0,"num_object_copies":11178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5589,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2577,"num_write_kb":10308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'5845","reported_seq":5974,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:05.278811+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:27:05.278811+0000","last_peered":"2024-09-17T15:27:05.278811+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:27:05.278811+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":5845,"log_dups_size":0,"ondisk_log_size":5845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23921123,"num_objects":5841,"num_object_clones":0,"num_object_copies":11682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5841,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2344,"num_write_kb":9376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1571},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'5777","reported_seq":5910,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:05.238201+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:27:05.238201+0000","last_peered":"2024-09-17T15:27:05.238201+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:27:05.238201+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":5777,"log_dups_size":0,"ondisk_log_size":5777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23638982,"num_objects":5773,"num_object_clones":0,"num_object_copies":11546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5773,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2546,"num_write_kb":10184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1599},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'5720","reported_seq":5842,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:05.241282+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:27:05.241282+0000","last_peered":"2024-09-17T15:27:05.241282+0000","last_clean":"2024-09-17T15:27:05.241282+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:27:05.241282+0000","last_undegraded":"2024-09-17T15:27:05.241282+0000","last_fullsized":"2024-09-17T15:27:05.241282+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":5720,"log_dups_size":0,"ondisk_log_size":5720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23420928,"num_objects":5718,"num_object_clones":0,"num_object_copies":11436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2674,"num_write_kb":10696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'5695","reported_seq":8318,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:27:06.841750+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:27:06.841750+0000","last_peered":"2024-09-17T15:27:06.841750+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:27:06.841750+0000","last_undegraded":"2024-09-17T15:27:06.841750+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":5695,"log_dups_size":0,"ondisk_log_size":5695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23298531,"num_objects":5689,"num_object_clones":0,"num_object_copies":11378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2821,"num_objects_unfound":0,"num_objects_dirty":5689,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3695,"num_write_kb":14780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1250,"num_bytes_recovered":5120000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2821},{"shards":"2","objects":2868}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233552+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233552+0000","last_peered":"2024-09-17T15:26:22.233552+0000","last_clean":"2024-09-17T15:26:22.233552+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233552+0000","last_undegraded":"2024-09-17T15:26:22.233552+0000","last_fullsized":"2024-09-17T15:26:22.233552+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'5586","reported_seq":5711,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:05.267397+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:27:05.267397+0000","last_peered":"2024-09-17T15:27:05.267397+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:27:05.267397+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":5586,"log_dups_size":0,"ondisk_log_size":5586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22847488,"num_objects":5578,"num_object_clones":0,"num_object_copies":11156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5578,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2182,"num_write_kb":8728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1568},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":97,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412553+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412553+0000","last_peered":"2024-09-17T15:26:22.412553+0000","last_clean":"2024-09-17T15:26:22.412553+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412553+0000","last_undegraded":"2024-09-17T15:26:22.412553+0000","last_fullsized":"2024-09-17T15:26:22.412553+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'5698","reported_seq":5812,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:06.820685+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:27:06.820685+0000","last_peered":"2024-09-17T15:27:06.820685+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:27:06.820685+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":5698,"log_dups_size":0,"ondisk_log_size":5698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23319011,"num_objects":5694,"num_object_clones":0,"num_object_copies":11388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5694,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3593,"num_write_kb":14372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":1617}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'5766","reported_seq":5879,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:06.799508+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:27:06.799508+0000","last_peered":"2024-09-17T15:27:06.799508+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:27:06.799508+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":5766,"log_dups_size":0,"ondisk_log_size":5766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23592960,"num_objects":5760,"num_object_clones":0,"num_object_copies":11520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5760,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3528,"num_write_kb":14112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":1623}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'5765","reported_seq":5895,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:05.167785+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:27:05.167785+0000","last_peered":"2024-09-17T15:27:05.167785+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:27:05.167785+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":5765,"log_dups_size":0,"ondisk_log_size":5765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23605248,"num_objects":5763,"num_object_clones":0,"num_object_copies":11526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2313,"num_write_kb":9252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1625},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'5726","reported_seq":5862,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:05.221310+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:27:05.221310+0000","last_peered":"2024-09-17T15:27:05.221310+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:27:05.221310+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":5726,"log_dups_size":0,"ondisk_log_size":5726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23420928,"num_objects":5718,"num_object_clones":0,"num_object_copies":11436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2013,"num_write_kb":8052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1561},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229013+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229013+0000","last_peered":"2024-09-17T15:26:22.229013+0000","last_clean":"2024-09-17T15:26:22.229013+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229013+0000","last_undegraded":"2024-09-17T15:26:22.229013+0000","last_fullsized":"2024-09-17T15:26:22.229013+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'5730","reported_seq":5856,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:05.241189+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:27:05.241189+0000","last_peered":"2024-09-17T15:27:05.241189+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:27:05.241189+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":5730,"log_dups_size":0,"ondisk_log_size":5730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23453696,"num_objects":5726,"num_object_clones":0,"num_object_copies":11452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2517,"num_write_kb":10068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1663},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'5709","reported_seq":5845,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:05.331930+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:27:05.331930+0000","last_peered":"2024-09-17T15:27:05.331930+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:27:05.331930+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":5709,"log_dups_size":0,"ondisk_log_size":5709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23335878,"num_objects":5699,"num_object_clones":0,"num_object_copies":11398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5699,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2151,"num_write_kb":8604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1590},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":374702814,"num_objects":91489,"num_object_clones":0,"num_object_copies":182978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":2821,"num_objects_unfound":0,"num_objects_dirty":91489,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":51596,"num_write_kb":206384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1250,"num_bytes_recovered":5120000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":752349184,"data_stored":752193825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91561,"ondisk_log_size":91561,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051609,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473524,"kb_used_data":315492,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":93898316,"statfs":{"total":96636764160,"available":96151875584,"internally_reserved":0,"allocated":323063808,"data_stored":322797635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":161803948},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,4,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724554,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":182864,"kb_used_data":78976,"kb_used_omap":4,"kb_used_meta":103867,"kb_avail":94188976,"statfs":{"total":96636764160,"available":96449511424,"internally_reserved":0,"allocated":80871424,"data_stored":80554205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":106360491},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,1,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247837,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":510388,"kb_used_data":338532,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93861452,"statfs":{"total":96636764160,"available":96114126848,"internally_reserved":0,"allocated":346656768,"data_stored":346386518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":175959720},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,1,5,6,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":347648000,"data_stored":347611870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81690624,"data_stored":81607525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":323010560,"data_stored":322974430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:27:09.377 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:27:09.949 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:27:09.949 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:27:09.973 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":117,"stamp":"2024-09-17T15:27:09.612841+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":374702833,"num_objects":91491,"num_object_clones":0,"num_object_copies":182982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":2821,"num_objects_unfound":0,"num_objects_dirty":91491,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":51598,"num_write_kb":206386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1251,"num_bytes_recovered":5120019,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91564,"ondisk_log_size":91564,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1166776,"kb_used_data":733000,"kb_used_omap":13,"kb_used_meta":433714,"kb_avail":281948744,"statfs":{"total":289910292480,"available":288715513856,"internally_reserved":0,"allocated":750592000,"data_stored":749738358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":444124159},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,5,1,2,11,14,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23552000,"num_objects":5750,"num_object_clones":0,"num_object_copies":11500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-281,"num_objects_unfound":0,"num_objects_dirty":5750,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5750,"num_write_kb":23000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":281,"num_bytes_recovered":1150976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008706"},"pg_stats":[{"pgid":"2.f","version":"41'5643","reported_seq":5715,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:05.284328+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:27:05.284328+0000","last_peered":"2024-09-17T15:27:05.284328+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:27:05.284328+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":5643,"log_dups_size":0,"ondisk_log_size":5643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23113728,"num_objects":5643,"num_object_clones":0,"num_object_copies":11286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5643,"num_write_kb":22572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1573},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'5704","reported_seq":5801,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:06.833189+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:27:06.833189+0000","last_peered":"2024-09-17T15:27:06.833189+0000","last_clean":"2024-09-17T15:27:06.833189+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:27:06.833189+0000","last_undegraded":"2024-09-17T15:27:06.833189+0000","last_fullsized":"2024-09-17T15:27:06.833189+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":5704,"log_dups_size":0,"ondisk_log_size":5704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23339974,"num_objects":5700,"num_object_clones":0,"num_object_copies":11400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5700,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4061,"num_write_kb":16244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'5809","reported_seq":5927,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:06.845856+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:27:06.845856+0000","last_peered":"2024-09-17T15:27:06.845856+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:27:06.845856+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":5809,"log_dups_size":0,"ondisk_log_size":5809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23777280,"num_objects":5805,"num_object_clones":0,"num_object_copies":11610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3966,"num_write_kb":15864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":1637}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'5793","reported_seq":5857,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:06.842626+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:27:06.842626+0000","last_peered":"2024-09-17T15:27:06.842626+0000","last_clean":"2024-09-17T15:27:06.842626+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:27:06.842626+0000","last_undegraded":"2024-09-17T15:27:06.842626+0000","last_fullsized":"2024-09-17T15:27:06.842626+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":5793,"log_dups_size":0,"ondisk_log_size":5793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23728128,"num_objects":5793,"num_object_clones":0,"num_object_copies":11586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5793,"num_write_kb":23172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'5595","reported_seq":5719,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:05.319199+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:27:05.319199+0000","last_peered":"2024-09-17T15:27:05.319199+0000","last_clean":"2024-09-17T15:27:05.319199+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:27:05.319199+0000","last_undegraded":"2024-09-17T15:27:05.319199+0000","last_fullsized":"2024-09-17T15:27:05.319199+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":5595,"log_dups_size":0,"ondisk_log_size":5595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22888931,"num_objects":5589,"num_object_clones":0,"num_object_copies":11178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5589,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2577,"num_write_kb":10308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'5845","reported_seq":5974,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:05.278811+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:27:05.278811+0000","last_peered":"2024-09-17T15:27:05.278811+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:27:05.278811+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":5845,"log_dups_size":0,"ondisk_log_size":5845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23921123,"num_objects":5841,"num_object_clones":0,"num_object_copies":11682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5841,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2344,"num_write_kb":9376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1571},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'5777","reported_seq":5910,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:05.238201+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:27:05.238201+0000","last_peered":"2024-09-17T15:27:05.238201+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:27:05.238201+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":5777,"log_dups_size":0,"ondisk_log_size":5777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23638982,"num_objects":5773,"num_object_clones":0,"num_object_copies":11546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5773,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2546,"num_write_kb":10184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1599},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'5720","reported_seq":5842,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:05.241282+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:27:05.241282+0000","last_peered":"2024-09-17T15:27:05.241282+0000","last_clean":"2024-09-17T15:27:05.241282+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:27:05.241282+0000","last_undegraded":"2024-09-17T15:27:05.241282+0000","last_fullsized":"2024-09-17T15:27:05.241282+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":5720,"log_dups_size":0,"ondisk_log_size":5720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23420928,"num_objects":5718,"num_object_clones":0,"num_object_copies":11436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2674,"num_write_kb":10696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'5695","reported_seq":8318,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:27:06.841750+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:27:06.841750+0000","last_peered":"2024-09-17T15:27:06.841750+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:27:06.841750+0000","last_undegraded":"2024-09-17T15:27:06.841750+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":5695,"log_dups_size":0,"ondisk_log_size":5695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23298531,"num_objects":5689,"num_object_clones":0,"num_object_copies":11378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2821,"num_objects_unfound":0,"num_objects_dirty":5689,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3695,"num_write_kb":14780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1250,"num_bytes_recovered":5120000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2821},{"shards":"2","objects":2868}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233552+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233552+0000","last_peered":"2024-09-17T15:26:22.233552+0000","last_clean":"2024-09-17T15:26:22.233552+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233552+0000","last_undegraded":"2024-09-17T15:26:22.233552+0000","last_fullsized":"2024-09-17T15:26:22.233552+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'5586","reported_seq":5711,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:05.267397+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:27:05.267397+0000","last_peered":"2024-09-17T15:27:05.267397+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:27:05.267397+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":5586,"log_dups_size":0,"ondisk_log_size":5586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22847488,"num_objects":5578,"num_object_clones":0,"num_object_copies":11156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5578,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2182,"num_write_kb":8728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1568},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":97,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412553+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412553+0000","last_peered":"2024-09-17T15:26:22.412553+0000","last_clean":"2024-09-17T15:26:22.412553+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412553+0000","last_undegraded":"2024-09-17T15:26:22.412553+0000","last_fullsized":"2024-09-17T15:26:22.412553+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'5698","reported_seq":5812,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:06.820685+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:27:06.820685+0000","last_peered":"2024-09-17T15:27:06.820685+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:27:06.820685+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":5698,"log_dups_size":0,"ondisk_log_size":5698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23319011,"num_objects":5694,"num_object_clones":0,"num_object_copies":11388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5694,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3593,"num_write_kb":14372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":1617}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'5766","reported_seq":5879,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:06.799508+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:27:06.799508+0000","last_peered":"2024-09-17T15:27:06.799508+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:27:06.799508+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":5766,"log_dups_size":0,"ondisk_log_size":5766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23592960,"num_objects":5760,"num_object_clones":0,"num_object_copies":11520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5760,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3528,"num_write_kb":14112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":1623}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'5765","reported_seq":5895,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:05.167785+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:27:05.167785+0000","last_peered":"2024-09-17T15:27:05.167785+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:27:05.167785+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":5765,"log_dups_size":0,"ondisk_log_size":5765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23605248,"num_objects":5763,"num_object_clones":0,"num_object_copies":11526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2313,"num_write_kb":9252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1625},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'5726","reported_seq":5862,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:05.221310+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:27:05.221310+0000","last_peered":"2024-09-17T15:27:05.221310+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:27:05.221310+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":5726,"log_dups_size":0,"ondisk_log_size":5726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23420928,"num_objects":5718,"num_object_clones":0,"num_object_copies":11436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2013,"num_write_kb":8052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1561},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229013+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229013+0000","last_peered":"2024-09-17T15:26:22.229013+0000","last_clean":"2024-09-17T15:26:22.229013+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229013+0000","last_undegraded":"2024-09-17T15:26:22.229013+0000","last_fullsized":"2024-09-17T15:26:22.229013+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'5730","reported_seq":5856,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:05.241189+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:27:05.241189+0000","last_peered":"2024-09-17T15:27:05.241189+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:27:05.241189+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":5730,"log_dups_size":0,"ondisk_log_size":5730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23453696,"num_objects":5726,"num_object_clones":0,"num_object_copies":11452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2517,"num_write_kb":10068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1663},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'5709","reported_seq":5845,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:05.331930+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:27:05.331930+0000","last_peered":"2024-09-17T15:27:05.331930+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:27:05.331930+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":5709,"log_dups_size":0,"ondisk_log_size":5709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23335878,"num_objects":5699,"num_object_clones":0,"num_object_copies":11398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5699,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2151,"num_write_kb":8604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1590},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":374702814,"num_objects":91489,"num_object_clones":0,"num_object_copies":182978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":2821,"num_objects_unfound":0,"num_objects_dirty":91489,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":51596,"num_write_kb":206384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1250,"num_bytes_recovered":5120000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":752349184,"data_stored":752193825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91561,"ondisk_log_size":91561,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051609,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473524,"kb_used_data":315492,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":93898316,"statfs":{"total":96636764160,"available":96151875584,"internally_reserved":0,"allocated":323063808,"data_stored":322797635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":161803948},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,4,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724554,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":182864,"kb_used_data":78976,"kb_used_omap":4,"kb_used_meta":103867,"kb_avail":94188976,"statfs":{"total":96636764160,"available":96449511424,"internally_reserved":0,"allocated":80871424,"data_stored":80554205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":106360491},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,1,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247837,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":510388,"kb_used_data":338532,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93861452,"statfs":{"total":96636764160,"available":96114126848,"internally_reserved":0,"allocated":346656768,"data_stored":346386518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":175959720},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,1,5,6,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":347648000,"data_stored":347611870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81690624,"data_stored":81607525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":323010560,"data_stored":322974430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:27:09.974 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:27:09.974 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'5643", 'reported_seq': 5715, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:05.284328+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:27:05.284328+0000', 'last_peered': '2024-09-17T15:27:05.284328+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:27:05.284328+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 5643, 'log_dups_size': 0, 'ondisk_log_size': 5643, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23113728, 'num_objects': 5643, 'num_object_clones': 0, 'num_object_copies': 11286, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4070, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5643, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5643, 'num_write_kb': 22572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1573}, {'shards': '0,2', 'objects': 4070}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:09.974 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:27:09.974 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'5809", 'reported_seq': 5927, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:06.845856+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:27:06.845856+0000', 'last_peered': '2024-09-17T15:27:06.845856+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:27:06.845856+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 5809, 'log_dups_size': 0, 'ondisk_log_size': 5809, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23777280, 'num_objects': 5805, 'num_object_clones': 0, 'num_object_copies': 11610, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4169, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5805, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3966, 'num_write_kb': 15864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4168}, {'shards': '2', 'objects': 1637}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:27:09.974 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:27:09.974 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'5845", 'reported_seq': 5974, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:05.278811+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:27:05.278811+0000', 'last_peered': '2024-09-17T15:27:05.278811+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:27:05.278811+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 5845, 'log_dups_size': 0, 'ondisk_log_size': 5845, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23921123, 'num_objects': 5841, 'num_object_clones': 0, 'num_object_copies': 11682, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4271, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5841, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2344, 'num_write_kb': 9376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1571}, {'shards': '0,2', 'objects': 4268}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:09.975 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:27:09.975 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'5777", 'reported_seq': 5910, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:05.238201+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:27:05.238201+0000', 'last_peered': '2024-09-17T15:27:05.238201+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:27:05.238201+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 5777, 'log_dups_size': 0, 'ondisk_log_size': 5777, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23638982, 'num_objects': 5773, 'num_object_clones': 0, 'num_object_copies': 11546, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4174, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5773, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 2546, 'num_write_kb': 10184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1599}, {'shards': '0,2', 'objects': 4174}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:09.975 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-17T15:27:09.975 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "41'5695", 'reported_seq': 8318, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:27:06.841750+0000', 'last_change': '2024-09-17T15:26:22.588242+0000', 'last_active': '2024-09-17T15:27:06.841750+0000', 'last_peered': '2024-09-17T15:27:06.841750+0000', 'last_clean': '2024-09-17T15:26:20.155646+0000', 'last_became_active': '2024-09-17T15:26:22.557742+0000', 'last_became_peered': '2024-09-17T15:26:22.557742+0000', 'last_unstale': '2024-09-17T15:27:06.841750+0000', 'last_undegraded': '2024-09-17T15:27:06.841750+0000', 'last_fullsized': '2024-09-17T15:26:22.188850+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2016", 'last_scrub_stamp': '2024-09-17T15:25:53.959011+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:53.959011+0000', 'objects_scrubbed': 1994, 'log_size': 5695, 'log_dups_size': 0, 'ondisk_log_size': 5695, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 568, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23298531, 'num_objects': 5689, 'num_object_clones': 0, 'num_object_copies': 11378, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2821, 'num_objects_unfound': 0, 'num_objects_dirty': 5689, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3695, 'num_write_kb': 14780, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1250, 'num_bytes_recovered': 5120000, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2821}, {'shards': '2', 'objects': 2868}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:27:09.975 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:27:09.975 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'5586", 'reported_seq': 5711, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:05.267397+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:27:05.267397+0000', 'last_peered': '2024-09-17T15:27:05.267397+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:27:05.267397+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 5586, 'log_dups_size': 0, 'ondisk_log_size': 5586, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22847488, 'num_objects': 5578, 'num_object_clones': 0, 'num_object_copies': 11156, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4012, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5578, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2182, 'num_write_kb': 8728, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1568}, {'shards': '0,2', 'objects': 4010}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:09.975 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:27:09.975 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'5698", 'reported_seq': 5812, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:06.820685+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:27:06.820685+0000', 'last_peered': '2024-09-17T15:27:06.820685+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:27:06.820685+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 5698, 'log_dups_size': 0, 'ondisk_log_size': 5698, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23319011, 'num_objects': 5694, 'num_object_clones': 0, 'num_object_copies': 11388, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4078, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5694, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3593, 'num_write_kb': 14372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4077}, {'shards': '2', 'objects': 1617}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:27:09.975 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:27:09.975 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'5766", 'reported_seq': 5879, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:06.799508+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:27:06.799508+0000', 'last_peered': '2024-09-17T15:27:06.799508+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:27:06.799508+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 5766, 'log_dups_size': 0, 'ondisk_log_size': 5766, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23592960, 'num_objects': 5760, 'num_object_clones': 0, 'num_object_copies': 11520, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5760, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3528, 'num_write_kb': 14112, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4137}, {'shards': '2', 'objects': 1623}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:27:09.976 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:27:09.976 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'5765", 'reported_seq': 5895, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:05.167785+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:27:05.167785+0000', 'last_peered': '2024-09-17T15:27:05.167785+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:27:05.167785+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 5765, 'log_dups_size': 0, 'ondisk_log_size': 5765, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23605248, 'num_objects': 5763, 'num_object_clones': 0, 'num_object_copies': 11526, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5763, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2313, 'num_write_kb': 9252, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1625}, {'shards': '0,2', 'objects': 4138}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:09.976 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:27:09.976 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'5726", 'reported_seq': 5862, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:05.221310+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:27:05.221310+0000', 'last_peered': '2024-09-17T15:27:05.221310+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:27:05.221310+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 5726, 'log_dups_size': 0, 'ondisk_log_size': 5726, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23420928, 'num_objects': 5718, 'num_object_clones': 0, 'num_object_copies': 11436, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4160, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5718, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2013, 'num_write_kb': 8052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1561}, {'shards': '0,2', 'objects': 4157}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:09.976 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:27:09.976 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'5730", 'reported_seq': 5856, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:05.241189+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:27:05.241189+0000', 'last_peered': '2024-09-17T15:27:05.241189+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:27:05.241189+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 5730, 'log_dups_size': 0, 'ondisk_log_size': 5730, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23453696, 'num_objects': 5726, 'num_object_clones': 0, 'num_object_copies': 11452, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4065, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5726, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2517, 'num_write_kb': 10068, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1663}, {'shards': '0,2', 'objects': 4063}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:09.976 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:27:09.976 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'5709", 'reported_seq': 5845, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:05.331930+0000', 'last_change': '2024-09-17T15:26:22.654521+0000', 'last_active': '2024-09-17T15:27:05.331930+0000', 'last_peered': '2024-09-17T15:27:05.331930+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:27:05.331930+0000', 'last_undegraded': '2024-09-17T15:26:22.307795+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 5709, 'log_dups_size': 0, 'ondisk_log_size': 5709, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23335878, 'num_objects': 5699, 'num_object_clones': 0, 'num_object_copies': 11398, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4111, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5699, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2151, 'num_write_kb': 8604, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1590}, {'shards': '0,2', 'objects': 4109}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:09.977 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:27:10.301 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 91 20 94025 94005 4.03442 2.05469 0.00835211 0.0193547 2024-09-17T15:27:10.676 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:27:10.676 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45388/182982 objects degraded (24.805%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":167,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":91491,"data_bytes":374702833,"bytes_used":1194778624,"bytes_avail":288715513856,"bytes_total":289910292480,"degraded_objects":45388,"degraded_total":182982,"degraded_ratio":0.24804625591588245,"misplaced_objects":2821,"misplaced_total":182982,"misplaced_ratio":0.015416816954673139,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":95845,"recovering_keys_per_sec":0,"num_objects_recovered":281,"num_bytes_recovered":1150976,"num_keys_recovered":0,"write_bytes_sec":1961243,"read_op_per_sec":0,"write_op_per_sec":478},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-17T15:26:23.592815+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-17T15:27:10.704 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:27:10.704 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:27:11.261 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:27:11.264 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:27:11.283 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":117,"stamp":"2024-09-17T15:27:09.612841+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":374702833,"num_objects":91491,"num_object_clones":0,"num_object_copies":182982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":2821,"num_objects_unfound":0,"num_objects_dirty":91491,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":51598,"num_write_kb":206386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1251,"num_bytes_recovered":5120019,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91564,"ondisk_log_size":91564,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1166776,"kb_used_data":733000,"kb_used_omap":13,"kb_used_meta":433714,"kb_avail":281948744,"statfs":{"total":289910292480,"available":288715513856,"internally_reserved":0,"allocated":750592000,"data_stored":749738358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":444124159},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,5,1,2,11,14,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23552000,"num_objects":5750,"num_object_clones":0,"num_object_copies":11500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-281,"num_objects_unfound":0,"num_objects_dirty":5750,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5750,"num_write_kb":23000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":281,"num_bytes_recovered":1150976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008706"},"pg_stats":[{"pgid":"2.f","version":"41'5643","reported_seq":5715,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:05.284328+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:27:05.284328+0000","last_peered":"2024-09-17T15:27:05.284328+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:27:05.284328+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":5643,"log_dups_size":0,"ondisk_log_size":5643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23113728,"num_objects":5643,"num_object_clones":0,"num_object_copies":11286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5643,"num_write_kb":22572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1573},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'5704","reported_seq":5801,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:06.833189+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:27:06.833189+0000","last_peered":"2024-09-17T15:27:06.833189+0000","last_clean":"2024-09-17T15:27:06.833189+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:27:06.833189+0000","last_undegraded":"2024-09-17T15:27:06.833189+0000","last_fullsized":"2024-09-17T15:27:06.833189+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":5704,"log_dups_size":0,"ondisk_log_size":5704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23339974,"num_objects":5700,"num_object_clones":0,"num_object_copies":11400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5700,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4061,"num_write_kb":16244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'5809","reported_seq":5927,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:06.845856+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:27:06.845856+0000","last_peered":"2024-09-17T15:27:06.845856+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:27:06.845856+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":5809,"log_dups_size":0,"ondisk_log_size":5809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23777280,"num_objects":5805,"num_object_clones":0,"num_object_copies":11610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3966,"num_write_kb":15864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":1637}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'5793","reported_seq":5857,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:06.842626+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:27:06.842626+0000","last_peered":"2024-09-17T15:27:06.842626+0000","last_clean":"2024-09-17T15:27:06.842626+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:27:06.842626+0000","last_undegraded":"2024-09-17T15:27:06.842626+0000","last_fullsized":"2024-09-17T15:27:06.842626+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":5793,"log_dups_size":0,"ondisk_log_size":5793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23728128,"num_objects":5793,"num_object_clones":0,"num_object_copies":11586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5793,"num_write_kb":23172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'5595","reported_seq":5719,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:05.319199+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:27:05.319199+0000","last_peered":"2024-09-17T15:27:05.319199+0000","last_clean":"2024-09-17T15:27:05.319199+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:27:05.319199+0000","last_undegraded":"2024-09-17T15:27:05.319199+0000","last_fullsized":"2024-09-17T15:27:05.319199+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":5595,"log_dups_size":0,"ondisk_log_size":5595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22888931,"num_objects":5589,"num_object_clones":0,"num_object_copies":11178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5589,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2577,"num_write_kb":10308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'5845","reported_seq":5974,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:05.278811+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:27:05.278811+0000","last_peered":"2024-09-17T15:27:05.278811+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:27:05.278811+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":5845,"log_dups_size":0,"ondisk_log_size":5845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23921123,"num_objects":5841,"num_object_clones":0,"num_object_copies":11682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5841,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2344,"num_write_kb":9376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1571},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'5777","reported_seq":5910,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:05.238201+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:27:05.238201+0000","last_peered":"2024-09-17T15:27:05.238201+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:27:05.238201+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":5777,"log_dups_size":0,"ondisk_log_size":5777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23638982,"num_objects":5773,"num_object_clones":0,"num_object_copies":11546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5773,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2546,"num_write_kb":10184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1599},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'5720","reported_seq":5842,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:05.241282+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:27:05.241282+0000","last_peered":"2024-09-17T15:27:05.241282+0000","last_clean":"2024-09-17T15:27:05.241282+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:27:05.241282+0000","last_undegraded":"2024-09-17T15:27:05.241282+0000","last_fullsized":"2024-09-17T15:27:05.241282+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":5720,"log_dups_size":0,"ondisk_log_size":5720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23420928,"num_objects":5718,"num_object_clones":0,"num_object_copies":11436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2674,"num_write_kb":10696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'5695","reported_seq":8318,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:27:06.841750+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:27:06.841750+0000","last_peered":"2024-09-17T15:27:06.841750+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:27:06.841750+0000","last_undegraded":"2024-09-17T15:27:06.841750+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":5695,"log_dups_size":0,"ondisk_log_size":5695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23298531,"num_objects":5689,"num_object_clones":0,"num_object_copies":11378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2821,"num_objects_unfound":0,"num_objects_dirty":5689,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3695,"num_write_kb":14780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1250,"num_bytes_recovered":5120000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2821},{"shards":"2","objects":2868}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233552+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233552+0000","last_peered":"2024-09-17T15:26:22.233552+0000","last_clean":"2024-09-17T15:26:22.233552+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233552+0000","last_undegraded":"2024-09-17T15:26:22.233552+0000","last_fullsized":"2024-09-17T15:26:22.233552+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'5586","reported_seq":5711,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:05.267397+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:27:05.267397+0000","last_peered":"2024-09-17T15:27:05.267397+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:27:05.267397+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":5586,"log_dups_size":0,"ondisk_log_size":5586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22847488,"num_objects":5578,"num_object_clones":0,"num_object_copies":11156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5578,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2182,"num_write_kb":8728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1568},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":97,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412553+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412553+0000","last_peered":"2024-09-17T15:26:22.412553+0000","last_clean":"2024-09-17T15:26:22.412553+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412553+0000","last_undegraded":"2024-09-17T15:26:22.412553+0000","last_fullsized":"2024-09-17T15:26:22.412553+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'5698","reported_seq":5812,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:06.820685+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:27:06.820685+0000","last_peered":"2024-09-17T15:27:06.820685+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:27:06.820685+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":5698,"log_dups_size":0,"ondisk_log_size":5698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23319011,"num_objects":5694,"num_object_clones":0,"num_object_copies":11388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5694,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3593,"num_write_kb":14372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":1617}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'5766","reported_seq":5879,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:06.799508+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:27:06.799508+0000","last_peered":"2024-09-17T15:27:06.799508+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:27:06.799508+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":5766,"log_dups_size":0,"ondisk_log_size":5766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23592960,"num_objects":5760,"num_object_clones":0,"num_object_copies":11520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5760,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3528,"num_write_kb":14112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":1623}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'5765","reported_seq":5895,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:05.167785+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:27:05.167785+0000","last_peered":"2024-09-17T15:27:05.167785+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:27:05.167785+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":5765,"log_dups_size":0,"ondisk_log_size":5765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23605248,"num_objects":5763,"num_object_clones":0,"num_object_copies":11526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2313,"num_write_kb":9252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1625},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'5726","reported_seq":5862,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:05.221310+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:27:05.221310+0000","last_peered":"2024-09-17T15:27:05.221310+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:27:05.221310+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":5726,"log_dups_size":0,"ondisk_log_size":5726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23420928,"num_objects":5718,"num_object_clones":0,"num_object_copies":11436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2013,"num_write_kb":8052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1561},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229013+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229013+0000","last_peered":"2024-09-17T15:26:22.229013+0000","last_clean":"2024-09-17T15:26:22.229013+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229013+0000","last_undegraded":"2024-09-17T15:26:22.229013+0000","last_fullsized":"2024-09-17T15:26:22.229013+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'5730","reported_seq":5856,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:05.241189+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:27:05.241189+0000","last_peered":"2024-09-17T15:27:05.241189+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:27:05.241189+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":5730,"log_dups_size":0,"ondisk_log_size":5730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23453696,"num_objects":5726,"num_object_clones":0,"num_object_copies":11452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2517,"num_write_kb":10068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1663},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'5709","reported_seq":5845,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:05.331930+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:27:05.331930+0000","last_peered":"2024-09-17T15:27:05.331930+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:27:05.331930+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":5709,"log_dups_size":0,"ondisk_log_size":5709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23335878,"num_objects":5699,"num_object_clones":0,"num_object_copies":11398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5699,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2151,"num_write_kb":8604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1590},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":374702814,"num_objects":91489,"num_object_clones":0,"num_object_copies":182978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":2821,"num_objects_unfound":0,"num_objects_dirty":91489,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":51596,"num_write_kb":206384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1250,"num_bytes_recovered":5120000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":752349184,"data_stored":752193825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91561,"ondisk_log_size":91561,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051609,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473524,"kb_used_data":315492,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":93898316,"statfs":{"total":96636764160,"available":96151875584,"internally_reserved":0,"allocated":323063808,"data_stored":322797635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":161803948},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,4,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724554,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":182864,"kb_used_data":78976,"kb_used_omap":4,"kb_used_meta":103867,"kb_avail":94188976,"statfs":{"total":96636764160,"available":96449511424,"internally_reserved":0,"allocated":80871424,"data_stored":80554205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":106360491},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,1,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247837,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":510388,"kb_used_data":338532,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93861452,"statfs":{"total":96636764160,"available":96114126848,"internally_reserved":0,"allocated":346656768,"data_stored":346386518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":175959720},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,1,5,6,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":347648000,"data_stored":347611870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81690624,"data_stored":81607525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":323010560,"data_stored":322974430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:27:11.301 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 92 20 94550 94530 4.01286 2.05078 0.040988 0.0194612 2024-09-17T15:27:12.302 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 93 20 95132 95112 3.99415 2.27344 0.0405847 0.0195518 2024-09-17T15:27:13.302 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 94 20 95741 95721 3.97697 2.37891 0.0373333 0.0196365 2024-09-17T15:27:14.288 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:27:14.303 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 95 20 96340 96320 3.95966 2.33984 0.016519 0.0197218 2024-09-17T15:27:14.836 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:27:14.837 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:27:14.859 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2024-09-17T15:27:13.614269+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":386384625,"num_objects":94343,"num_object_clones":0,"num_object_copies":188686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":2673,"num_objects_unfound":0,"num_objects_dirty":94343,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":54450,"num_write_kb":217794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1399,"num_bytes_recovered":5726227,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94416,"ondisk_log_size":94416,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1220648,"kb_used_data":756344,"kb_used_omap":13,"kb_used_meta":464242,"kb_avail":281894872,"statfs":{"total":289910292480,"available":288660348928,"internally_reserved":0,"allocated":774496256,"data_stored":773642614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":475384831},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,9,5,11,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27709440,"num_objects":6765,"num_object_clones":0,"num_object_copies":13530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-429,"num_objects_unfound":0,"num_objects_dirty":6765,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6765,"num_write_kb":27060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":429,"num_bytes_recovered":1757184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006949"},"pg_stats":[{"pgid":"2.f","version":"41'5833","reported_seq":5905,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:10.245544+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:27:10.245544+0000","last_peered":"2024-09-17T15:27:10.245544+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:27:10.245544+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":5833,"log_dups_size":0,"ondisk_log_size":5833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23891968,"num_objects":5833,"num_object_clones":0,"num_object_copies":11666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5833,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5833,"num_write_kb":23332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1763},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'5888","reported_seq":5985,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:11.849696+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:27:11.849696+0000","last_peered":"2024-09-17T15:27:11.849696+0000","last_clean":"2024-09-17T15:27:11.849696+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:27:11.849696+0000","last_undegraded":"2024-09-17T15:27:11.849696+0000","last_fullsized":"2024-09-17T15:27:11.849696+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":5888,"log_dups_size":0,"ondisk_log_size":5888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24093638,"num_objects":5884,"num_object_clones":0,"num_object_copies":11768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5884,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4245,"num_write_kb":16980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'5977","reported_seq":6096,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:11.852883+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:27:11.852883+0000","last_peered":"2024-09-17T15:27:11.852883+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:27:11.852883+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":5977,"log_dups_size":0,"ondisk_log_size":5977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24465408,"num_objects":5973,"num_object_clones":0,"num_object_copies":11946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4134,"num_write_kb":16536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":1805}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'5954","reported_seq":6018,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:11.799666+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:27:11.799666+0000","last_peered":"2024-09-17T15:27:11.799666+0000","last_clean":"2024-09-17T15:27:11.799666+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:27:11.799666+0000","last_undegraded":"2024-09-17T15:27:11.799666+0000","last_fullsized":"2024-09-17T15:27:11.799666+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":5954,"log_dups_size":0,"ondisk_log_size":5954,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24387584,"num_objects":5954,"num_object_clones":0,"num_object_copies":11908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5954,"num_write_kb":23816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'5789","reported_seq":5913,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:10.244183+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:27:10.244183+0000","last_peered":"2024-09-17T15:27:10.244183+0000","last_clean":"2024-09-17T15:27:10.244183+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:27:10.244183+0000","last_undegraded":"2024-09-17T15:27:10.244183+0000","last_fullsized":"2024-09-17T15:27:10.244183+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":5789,"log_dups_size":0,"ondisk_log_size":5789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23683555,"num_objects":5783,"num_object_clones":0,"num_object_copies":11566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5783,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2771,"num_write_kb":11084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'6011","reported_seq":6140,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:10.176667+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:27:10.176667+0000","last_peered":"2024-09-17T15:27:10.176667+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:27:10.176667+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":6011,"log_dups_size":0,"ondisk_log_size":6011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24601059,"num_objects":6007,"num_object_clones":0,"num_object_copies":12014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6007,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2510,"num_write_kb":10040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1737},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'5959","reported_seq":6092,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:10.267421+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:27:10.267421+0000","last_peered":"2024-09-17T15:27:10.267421+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:27:10.267421+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":5959,"log_dups_size":0,"ondisk_log_size":5959,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24384454,"num_objects":5955,"num_object_clones":0,"num_object_copies":11910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5955,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2728,"num_write_kb":10912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1781},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'5886","reported_seq":6008,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:10.300276+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:27:10.300276+0000","last_peered":"2024-09-17T15:27:10.300276+0000","last_clean":"2024-09-17T15:27:10.300276+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:27:10.300276+0000","last_undegraded":"2024-09-17T15:27:10.300276+0000","last_fullsized":"2024-09-17T15:27:10.300276+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":5886,"log_dups_size":0,"ondisk_log_size":5886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24100864,"num_objects":5884,"num_object_clones":0,"num_object_copies":11768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5884,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2840,"num_write_kb":11360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'5878","reported_seq":8797,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:27:11.846924+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:27:11.846924+0000","last_peered":"2024-09-17T15:27:11.846924+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:27:11.846924+0000","last_undegraded":"2024-09-17T15:27:11.846924+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":5878,"log_dups_size":0,"ondisk_log_size":5878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24048099,"num_objects":5872,"num_object_clones":0,"num_object_copies":11744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2673,"num_objects_unfound":0,"num_objects_dirty":5872,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3878,"num_write_kb":15512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1398,"num_bytes_recovered":5726208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2673},{"shards":"2","objects":3199}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233552+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233552+0000","last_peered":"2024-09-17T15:26:22.233552+0000","last_clean":"2024-09-17T15:26:22.233552+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233552+0000","last_undegraded":"2024-09-17T15:26:22.233552+0000","last_fullsized":"2024-09-17T15:26:22.233552+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'5757","reported_seq":5882,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:10.335245+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:27:10.335245+0000","last_peered":"2024-09-17T15:27:10.335245+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:27:10.335245+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":5757,"log_dups_size":0,"ondisk_log_size":5757,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23547904,"num_objects":5749,"num_object_clones":0,"num_object_copies":11498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2353,"num_write_kb":9412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1739},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":98,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412941+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412941+0000","last_peered":"2024-09-17T15:26:22.412941+0000","last_clean":"2024-09-17T15:26:22.412941+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412941+0000","last_undegraded":"2024-09-17T15:26:22.412941+0000","last_fullsized":"2024-09-17T15:26:22.412941+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'5881","reported_seq":5996,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:11.757778+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:27:11.757778+0000","last_peered":"2024-09-17T15:27:11.757778+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:27:11.757778+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":5881,"log_dups_size":0,"ondisk_log_size":5881,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24068579,"num_objects":5877,"num_object_clones":0,"num_object_copies":11754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5877,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3776,"num_write_kb":15104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":1800}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'5929","reported_seq":6042,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:11.759722+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:27:11.759722+0000","last_peered":"2024-09-17T15:27:11.759722+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:27:11.759722+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":5929,"log_dups_size":0,"ondisk_log_size":5929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24260608,"num_objects":5923,"num_object_clones":0,"num_object_copies":11846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3691,"num_write_kb":14764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":1786}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'5974","reported_seq":6104,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:10.294139+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:27:10.294139+0000","last_peered":"2024-09-17T15:27:10.294139+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:27:10.294139+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":5974,"log_dups_size":0,"ondisk_log_size":5974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24461312,"num_objects":5972,"num_object_clones":0,"num_object_copies":11944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5972,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2522,"num_write_kb":10088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1834},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'5896","reported_seq":6032,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:10.313852+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:27:10.313852+0000","last_peered":"2024-09-17T15:27:10.313852+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:27:10.313852+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":5896,"log_dups_size":0,"ondisk_log_size":5896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24117248,"num_objects":5888,"num_object_clones":0,"num_object_copies":11776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2183,"num_write_kb":8732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1731},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229013+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229013+0000","last_peered":"2024-09-17T15:26:22.229013+0000","last_clean":"2024-09-17T15:26:22.229013+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229013+0000","last_undegraded":"2024-09-17T15:26:22.229013+0000","last_fullsized":"2024-09-17T15:26:22.229013+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'5930","reported_seq":6051,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:10.331440+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:27:10.331440+0000","last_peered":"2024-09-17T15:27:10.331440+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:27:10.331440+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":5930,"log_dups_size":0,"ondisk_log_size":5930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24272896,"num_objects":5926,"num_object_clones":0,"num_object_copies":11852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2717,"num_write_kb":10868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1863},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'5871","reported_seq":6008,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:10.220629+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:27:10.220629+0000","last_peered":"2024-09-17T15:27:10.220629+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:27:10.220629+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":5871,"log_dups_size":0,"ondisk_log_size":5871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23999430,"num_objects":5861,"num_object_clones":0,"num_object_copies":11722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2313,"num_write_kb":9252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1752},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":386384606,"num_objects":94341,"num_object_clones":0,"num_object_copies":188682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":2673,"num_objects_unfound":0,"num_objects_dirty":94341,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":54448,"num_write_kb":217792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1398,"num_bytes_recovered":5726208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":776482816,"data_stored":776327457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94413,"ondisk_log_size":94413,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051610,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501304,"kb_used_data":321064,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93870536,"statfs":{"total":96636764160,"available":96123428864,"internally_reserved":0,"allocated":328769536,"data_stored":328503363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":184544940},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1,1,2,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724555,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":196072,"kb_used_data":88024,"kb_used_omap":4,"kb_used_meta":108027,"kb_avail":94175768,"statfs":{"total":96636764160,"available":96435986432,"internally_reserved":0,"allocated":90136576,"data_stored":89819357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":110620331},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247838,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523272,"kb_used_data":347256,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93848568,"statfs":{"total":96636764160,"available":96100933632,"internally_reserved":0,"allocated":355590144,"data_stored":355319894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":180219560},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":356564992,"data_stored":356528862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":91230208,"data_stored":91147109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":328687616,"data_stored":328651486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:27:14.861 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:27:15.304 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 96 20 96852 96832 3.93925 2 0.0438123 0.019825 2024-09-17T15:27:15.435 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:27:15.438 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:27:15.468 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2024-09-17T15:27:13.614269+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":386384625,"num_objects":94343,"num_object_clones":0,"num_object_copies":188686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":2673,"num_objects_unfound":0,"num_objects_dirty":94343,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":54450,"num_write_kb":217794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1399,"num_bytes_recovered":5726227,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94416,"ondisk_log_size":94416,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1220648,"kb_used_data":756344,"kb_used_omap":13,"kb_used_meta":464242,"kb_avail":281894872,"statfs":{"total":289910292480,"available":288660348928,"internally_reserved":0,"allocated":774496256,"data_stored":773642614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":475384831},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,9,5,11,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27709440,"num_objects":6765,"num_object_clones":0,"num_object_copies":13530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-429,"num_objects_unfound":0,"num_objects_dirty":6765,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6765,"num_write_kb":27060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":429,"num_bytes_recovered":1757184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006949"},"pg_stats":[{"pgid":"2.f","version":"41'5833","reported_seq":5905,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:10.245544+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:27:10.245544+0000","last_peered":"2024-09-17T15:27:10.245544+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:27:10.245544+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":5833,"log_dups_size":0,"ondisk_log_size":5833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23891968,"num_objects":5833,"num_object_clones":0,"num_object_copies":11666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5833,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5833,"num_write_kb":23332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1763},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'5888","reported_seq":5985,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:11.849696+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:27:11.849696+0000","last_peered":"2024-09-17T15:27:11.849696+0000","last_clean":"2024-09-17T15:27:11.849696+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:27:11.849696+0000","last_undegraded":"2024-09-17T15:27:11.849696+0000","last_fullsized":"2024-09-17T15:27:11.849696+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":5888,"log_dups_size":0,"ondisk_log_size":5888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24093638,"num_objects":5884,"num_object_clones":0,"num_object_copies":11768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5884,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4245,"num_write_kb":16980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'5977","reported_seq":6096,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:11.852883+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:27:11.852883+0000","last_peered":"2024-09-17T15:27:11.852883+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:27:11.852883+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":5977,"log_dups_size":0,"ondisk_log_size":5977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24465408,"num_objects":5973,"num_object_clones":0,"num_object_copies":11946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4134,"num_write_kb":16536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":1805}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'5954","reported_seq":6018,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:11.799666+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:27:11.799666+0000","last_peered":"2024-09-17T15:27:11.799666+0000","last_clean":"2024-09-17T15:27:11.799666+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:27:11.799666+0000","last_undegraded":"2024-09-17T15:27:11.799666+0000","last_fullsized":"2024-09-17T15:27:11.799666+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":5954,"log_dups_size":0,"ondisk_log_size":5954,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24387584,"num_objects":5954,"num_object_clones":0,"num_object_copies":11908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5954,"num_write_kb":23816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'5789","reported_seq":5913,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:10.244183+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:27:10.244183+0000","last_peered":"2024-09-17T15:27:10.244183+0000","last_clean":"2024-09-17T15:27:10.244183+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:27:10.244183+0000","last_undegraded":"2024-09-17T15:27:10.244183+0000","last_fullsized":"2024-09-17T15:27:10.244183+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":5789,"log_dups_size":0,"ondisk_log_size":5789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23683555,"num_objects":5783,"num_object_clones":0,"num_object_copies":11566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5783,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2771,"num_write_kb":11084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'6011","reported_seq":6140,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:10.176667+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:27:10.176667+0000","last_peered":"2024-09-17T15:27:10.176667+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:27:10.176667+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":6011,"log_dups_size":0,"ondisk_log_size":6011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24601059,"num_objects":6007,"num_object_clones":0,"num_object_copies":12014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6007,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2510,"num_write_kb":10040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1737},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'5959","reported_seq":6092,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:10.267421+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:27:10.267421+0000","last_peered":"2024-09-17T15:27:10.267421+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:27:10.267421+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":5959,"log_dups_size":0,"ondisk_log_size":5959,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24384454,"num_objects":5955,"num_object_clones":0,"num_object_copies":11910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5955,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2728,"num_write_kb":10912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1781},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'5886","reported_seq":6008,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:10.300276+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:27:10.300276+0000","last_peered":"2024-09-17T15:27:10.300276+0000","last_clean":"2024-09-17T15:27:10.300276+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:27:10.300276+0000","last_undegraded":"2024-09-17T15:27:10.300276+0000","last_fullsized":"2024-09-17T15:27:10.300276+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":5886,"log_dups_size":0,"ondisk_log_size":5886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24100864,"num_objects":5884,"num_object_clones":0,"num_object_copies":11768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5884,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2840,"num_write_kb":11360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'5878","reported_seq":8797,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:27:11.846924+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:27:11.846924+0000","last_peered":"2024-09-17T15:27:11.846924+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:27:11.846924+0000","last_undegraded":"2024-09-17T15:27:11.846924+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":5878,"log_dups_size":0,"ondisk_log_size":5878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24048099,"num_objects":5872,"num_object_clones":0,"num_object_copies":11744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2673,"num_objects_unfound":0,"num_objects_dirty":5872,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3878,"num_write_kb":15512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1398,"num_bytes_recovered":5726208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2673},{"shards":"2","objects":3199}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233552+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233552+0000","last_peered":"2024-09-17T15:26:22.233552+0000","last_clean":"2024-09-17T15:26:22.233552+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233552+0000","last_undegraded":"2024-09-17T15:26:22.233552+0000","last_fullsized":"2024-09-17T15:26:22.233552+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'5757","reported_seq":5882,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:10.335245+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:27:10.335245+0000","last_peered":"2024-09-17T15:27:10.335245+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:27:10.335245+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":5757,"log_dups_size":0,"ondisk_log_size":5757,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23547904,"num_objects":5749,"num_object_clones":0,"num_object_copies":11498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2353,"num_write_kb":9412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1739},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":98,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412941+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412941+0000","last_peered":"2024-09-17T15:26:22.412941+0000","last_clean":"2024-09-17T15:26:22.412941+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412941+0000","last_undegraded":"2024-09-17T15:26:22.412941+0000","last_fullsized":"2024-09-17T15:26:22.412941+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'5881","reported_seq":5996,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:11.757778+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:27:11.757778+0000","last_peered":"2024-09-17T15:27:11.757778+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:27:11.757778+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":5881,"log_dups_size":0,"ondisk_log_size":5881,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24068579,"num_objects":5877,"num_object_clones":0,"num_object_copies":11754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5877,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3776,"num_write_kb":15104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":1800}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'5929","reported_seq":6042,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:11.759722+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:27:11.759722+0000","last_peered":"2024-09-17T15:27:11.759722+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:27:11.759722+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":5929,"log_dups_size":0,"ondisk_log_size":5929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24260608,"num_objects":5923,"num_object_clones":0,"num_object_copies":11846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3691,"num_write_kb":14764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":1786}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'5974","reported_seq":6104,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:10.294139+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:27:10.294139+0000","last_peered":"2024-09-17T15:27:10.294139+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:27:10.294139+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":5974,"log_dups_size":0,"ondisk_log_size":5974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24461312,"num_objects":5972,"num_object_clones":0,"num_object_copies":11944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5972,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2522,"num_write_kb":10088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1834},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'5896","reported_seq":6032,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:10.313852+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:27:10.313852+0000","last_peered":"2024-09-17T15:27:10.313852+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:27:10.313852+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":5896,"log_dups_size":0,"ondisk_log_size":5896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24117248,"num_objects":5888,"num_object_clones":0,"num_object_copies":11776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2183,"num_write_kb":8732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1731},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229013+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229013+0000","last_peered":"2024-09-17T15:26:22.229013+0000","last_clean":"2024-09-17T15:26:22.229013+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229013+0000","last_undegraded":"2024-09-17T15:26:22.229013+0000","last_fullsized":"2024-09-17T15:26:22.229013+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'5930","reported_seq":6051,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:10.331440+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:27:10.331440+0000","last_peered":"2024-09-17T15:27:10.331440+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:27:10.331440+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":5930,"log_dups_size":0,"ondisk_log_size":5930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24272896,"num_objects":5926,"num_object_clones":0,"num_object_copies":11852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2717,"num_write_kb":10868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1863},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'5871","reported_seq":6008,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:10.220629+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:27:10.220629+0000","last_peered":"2024-09-17T15:27:10.220629+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:27:10.220629+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":5871,"log_dups_size":0,"ondisk_log_size":5871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23999430,"num_objects":5861,"num_object_clones":0,"num_object_copies":11722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2313,"num_write_kb":9252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1752},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":386384606,"num_objects":94341,"num_object_clones":0,"num_object_copies":188682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":2673,"num_objects_unfound":0,"num_objects_dirty":94341,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":54448,"num_write_kb":217792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1398,"num_bytes_recovered":5726208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":776482816,"data_stored":776327457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94413,"ondisk_log_size":94413,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051610,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501304,"kb_used_data":321064,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93870536,"statfs":{"total":96636764160,"available":96123428864,"internally_reserved":0,"allocated":328769536,"data_stored":328503363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":184544940},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1,1,2,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724555,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":196072,"kb_used_data":88024,"kb_used_omap":4,"kb_used_meta":108027,"kb_avail":94175768,"statfs":{"total":96636764160,"available":96435986432,"internally_reserved":0,"allocated":90136576,"data_stored":89819357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":110620331},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247838,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523272,"kb_used_data":347256,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93848568,"statfs":{"total":96636764160,"available":96100933632,"internally_reserved":0,"allocated":355590144,"data_stored":355319894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":180219560},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":356564992,"data_stored":356528862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":91230208,"data_stored":91147109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":328687616,"data_stored":328651486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:27:15.469 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:27:15.469 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'5833", 'reported_seq': 5905, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:10.245544+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:27:10.245544+0000', 'last_peered': '2024-09-17T15:27:10.245544+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:27:10.245544+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 5833, 'log_dups_size': 0, 'ondisk_log_size': 5833, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23891968, 'num_objects': 5833, 'num_object_clones': 0, 'num_object_copies': 11666, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4070, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5833, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5833, 'num_write_kb': 23332, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1763}, {'shards': '0,2', 'objects': 4070}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:15.469 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:27:15.469 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'5977", 'reported_seq': 6096, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:11.852883+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:27:11.852883+0000', 'last_peered': '2024-09-17T15:27:11.852883+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:27:11.852883+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 5977, 'log_dups_size': 0, 'ondisk_log_size': 5977, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24465408, 'num_objects': 5973, 'num_object_clones': 0, 'num_object_copies': 11946, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4169, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5973, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4134, 'num_write_kb': 16536, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4168}, {'shards': '2', 'objects': 1805}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:27:15.469 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:27:15.469 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'6011", 'reported_seq': 6140, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:10.176667+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:27:10.176667+0000', 'last_peered': '2024-09-17T15:27:10.176667+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:27:10.176667+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 6011, 'log_dups_size': 0, 'ondisk_log_size': 6011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24601059, 'num_objects': 6007, 'num_object_clones': 0, 'num_object_copies': 12014, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4271, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6007, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2510, 'num_write_kb': 10040, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1737}, {'shards': '0,2', 'objects': 4268}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:15.470 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:27:15.470 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'5959", 'reported_seq': 6092, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:10.267421+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:27:10.267421+0000', 'last_peered': '2024-09-17T15:27:10.267421+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:27:10.267421+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 5959, 'log_dups_size': 0, 'ondisk_log_size': 5959, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24384454, 'num_objects': 5955, 'num_object_clones': 0, 'num_object_copies': 11910, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4174, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5955, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 2728, 'num_write_kb': 10912, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1781}, {'shards': '0,2', 'objects': 4174}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:15.470 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-17T15:27:15.470 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "41'5878", 'reported_seq': 8797, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:27:11.846924+0000', 'last_change': '2024-09-17T15:26:22.588242+0000', 'last_active': '2024-09-17T15:27:11.846924+0000', 'last_peered': '2024-09-17T15:27:11.846924+0000', 'last_clean': '2024-09-17T15:26:20.155646+0000', 'last_became_active': '2024-09-17T15:26:22.557742+0000', 'last_became_peered': '2024-09-17T15:26:22.557742+0000', 'last_unstale': '2024-09-17T15:27:11.846924+0000', 'last_undegraded': '2024-09-17T15:27:11.846924+0000', 'last_fullsized': '2024-09-17T15:26:22.188850+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2016", 'last_scrub_stamp': '2024-09-17T15:25:53.959011+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:53.959011+0000', 'objects_scrubbed': 1994, 'log_size': 5878, 'log_dups_size': 0, 'ondisk_log_size': 5878, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 568, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24048099, 'num_objects': 5872, 'num_object_clones': 0, 'num_object_copies': 11744, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2673, 'num_objects_unfound': 0, 'num_objects_dirty': 5872, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3878, 'num_write_kb': 15512, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1398, 'num_bytes_recovered': 5726208, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2673}, {'shards': '2', 'objects': 3199}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:27:15.470 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:27:15.470 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'5757", 'reported_seq': 5882, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:10.335245+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:27:10.335245+0000', 'last_peered': '2024-09-17T15:27:10.335245+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:27:10.335245+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 5757, 'log_dups_size': 0, 'ondisk_log_size': 5757, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23547904, 'num_objects': 5749, 'num_object_clones': 0, 'num_object_copies': 11498, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4012, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5749, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2353, 'num_write_kb': 9412, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1739}, {'shards': '0,2', 'objects': 4010}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:15.470 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:27:15.470 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'5881", 'reported_seq': 5996, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:11.757778+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:27:11.757778+0000', 'last_peered': '2024-09-17T15:27:11.757778+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:27:11.757778+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 5881, 'log_dups_size': 0, 'ondisk_log_size': 5881, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24068579, 'num_objects': 5877, 'num_object_clones': 0, 'num_object_copies': 11754, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4078, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5877, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3776, 'num_write_kb': 15104, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4077}, {'shards': '2', 'objects': 1800}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:27:15.470 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:27:15.470 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'5929", 'reported_seq': 6042, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:11.759722+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:27:11.759722+0000', 'last_peered': '2024-09-17T15:27:11.759722+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:27:11.759722+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 5929, 'log_dups_size': 0, 'ondisk_log_size': 5929, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24260608, 'num_objects': 5923, 'num_object_clones': 0, 'num_object_copies': 11846, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5923, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3691, 'num_write_kb': 14764, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4137}, {'shards': '2', 'objects': 1786}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:27:15.471 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:27:15.471 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'5974", 'reported_seq': 6104, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:10.294139+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:27:10.294139+0000', 'last_peered': '2024-09-17T15:27:10.294139+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:27:10.294139+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 5974, 'log_dups_size': 0, 'ondisk_log_size': 5974, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24461312, 'num_objects': 5972, 'num_object_clones': 0, 'num_object_copies': 11944, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5972, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2522, 'num_write_kb': 10088, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1834}, {'shards': '0,2', 'objects': 4138}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:15.471 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:27:15.471 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'5896", 'reported_seq': 6032, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:10.313852+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:27:10.313852+0000', 'last_peered': '2024-09-17T15:27:10.313852+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:27:10.313852+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 5896, 'log_dups_size': 0, 'ondisk_log_size': 5896, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24117248, 'num_objects': 5888, 'num_object_clones': 0, 'num_object_copies': 11776, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4160, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5888, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2183, 'num_write_kb': 8732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1731}, {'shards': '0,2', 'objects': 4157}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:15.471 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:27:15.471 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'5930", 'reported_seq': 6051, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:10.331440+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:27:10.331440+0000', 'last_peered': '2024-09-17T15:27:10.331440+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:27:10.331440+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 5930, 'log_dups_size': 0, 'ondisk_log_size': 5930, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24272896, 'num_objects': 5926, 'num_object_clones': 0, 'num_object_copies': 11852, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4065, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5926, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2717, 'num_write_kb': 10868, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1863}, {'shards': '0,2', 'objects': 4063}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:15.471 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:27:15.471 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'5871", 'reported_seq': 6008, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:10.220629+0000', 'last_change': '2024-09-17T15:26:22.654521+0000', 'last_active': '2024-09-17T15:27:10.220629+0000', 'last_peered': '2024-09-17T15:27:10.220629+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:27:10.220629+0000', 'last_undegraded': '2024-09-17T15:26:22.307795+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 5871, 'log_dups_size': 0, 'ondisk_log_size': 5871, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23999430, 'num_objects': 5861, 'num_object_clones': 0, 'num_object_copies': 11722, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4111, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5861, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2313, 'num_write_kb': 9252, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1752}, {'shards': '0,2', 'objects': 4109}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:15.471 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:27:16.184 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:27:16.185 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45388/192280 objects degraded (23.605%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":173,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":96140,"data_bytes":393745137,"bytes_used":1299320832,"bytes_avail":288610971648,"bytes_total":289910292480,"degraded_objects":45388,"degraded_total":192280,"degraded_ratio":0.23605159142916579,"misplaced_objects":2673,"misplaced_total":192280,"misplaced_ratio":0.013901601830663614,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":100999,"recovering_keys_per_sec":0,"num_objects_recovered":296,"num_bytes_recovered":1212416,"num_keys_recovered":0,"write_bytes_sec":2558091,"read_op_per_sec":0,"write_op_per_sec":624},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-17T15:26:23.592815+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-17T15:27:16.208 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:27:16.208 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:27:16.304 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 97 20 97374 97354 3.91966 2.03906 0.0143303 0.0199221 2024-09-17T15:27:16.754 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:27:16.755 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:27:16.777 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":120,"stamp":"2024-09-17T15:27:15.614994+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":393745137,"num_objects":96140,"num_object_clones":0,"num_object_copies":192280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":2673,"num_objects_unfound":0,"num_objects_dirty":96140,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":56247,"num_write_kb":224982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1399,"num_bytes_recovered":5726227,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96213,"ondisk_log_size":96213,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1268868,"kb_used_data":774036,"kb_used_omap":13,"kb_used_meta":494770,"kb_avail":281846652,"statfs":{"total":289910292480,"available":288610971648,"internally_reserved":0,"allocated":792612864,"data_stored":791759222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":506645503},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,4,8,8,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30707712,"num_objects":7497,"num_object_clones":0,"num_object_copies":14994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-296,"num_objects_unfound":0,"num_objects_dirty":7497,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7497,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":1212416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004150"},"pg_stats":[{"pgid":"2.f","version":"41'6010","reported_seq":6082,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:15.327601+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:27:15.327601+0000","last_peered":"2024-09-17T15:27:15.327601+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:27:15.327601+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":6010,"log_dups_size":0,"ondisk_log_size":6010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24616960,"num_objects":6010,"num_object_clones":0,"num_object_copies":12020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6010,"num_write_kb":24040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1940},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'5888","reported_seq":5985,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:11.849696+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:27:11.849696+0000","last_peered":"2024-09-17T15:27:11.849696+0000","last_clean":"2024-09-17T15:27:11.849696+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:27:11.849696+0000","last_undegraded":"2024-09-17T15:27:11.849696+0000","last_fullsized":"2024-09-17T15:27:11.849696+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":5888,"log_dups_size":0,"ondisk_log_size":5888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24093638,"num_objects":5884,"num_object_clones":0,"num_object_copies":11768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5884,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4245,"num_write_kb":16980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'5977","reported_seq":6096,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:11.852883+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:27:11.852883+0000","last_peered":"2024-09-17T15:27:11.852883+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:27:11.852883+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":5977,"log_dups_size":0,"ondisk_log_size":5977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24465408,"num_objects":5973,"num_object_clones":0,"num_object_copies":11946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4134,"num_write_kb":16536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":1805}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'5954","reported_seq":6018,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:11.799666+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:27:11.799666+0000","last_peered":"2024-09-17T15:27:11.799666+0000","last_clean":"2024-09-17T15:27:11.799666+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:27:11.799666+0000","last_undegraded":"2024-09-17T15:27:11.799666+0000","last_fullsized":"2024-09-17T15:27:11.799666+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":5954,"log_dups_size":0,"ondisk_log_size":5954,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24387584,"num_objects":5954,"num_object_clones":0,"num_object_copies":11908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5954,"num_write_kb":23816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'5984","reported_seq":6107,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:15.352928+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:27:15.352928+0000","last_peered":"2024-09-17T15:27:15.352928+0000","last_clean":"2024-09-17T15:27:15.352928+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:27:15.352928+0000","last_undegraded":"2024-09-17T15:27:15.352928+0000","last_fullsized":"2024-09-17T15:27:15.352928+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":5984,"log_dups_size":0,"ondisk_log_size":5984,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24482275,"num_objects":5978,"num_object_clones":0,"num_object_copies":11956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5978,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2966,"num_write_kb":11864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'6187","reported_seq":6316,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:15.283948+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:27:15.283948+0000","last_peered":"2024-09-17T15:27:15.283948+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:27:15.283948+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":6187,"log_dups_size":0,"ondisk_log_size":6187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25321955,"num_objects":6183,"num_object_clones":0,"num_object_copies":12366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6183,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2686,"num_write_kb":10744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1913},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'6152","reported_seq":6285,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:15.332583+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:27:15.332583+0000","last_peered":"2024-09-17T15:27:15.332583+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:27:15.332583+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":6152,"log_dups_size":0,"ondisk_log_size":6152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25174982,"num_objects":6148,"num_object_clones":0,"num_object_copies":12296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6148,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2921,"num_write_kb":11684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1974},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'6047","reported_seq":6169,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:15.353041+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:27:15.353041+0000","last_peered":"2024-09-17T15:27:15.353041+0000","last_clean":"2024-09-17T15:27:15.353041+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:27:15.353041+0000","last_undegraded":"2024-09-17T15:27:15.353041+0000","last_fullsized":"2024-09-17T15:27:15.353041+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":6047,"log_dups_size":0,"ondisk_log_size":6047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24760320,"num_objects":6045,"num_object_clones":0,"num_object_copies":12090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3001,"num_write_kb":12004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'5878","reported_seq":8797,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:27:11.846924+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:27:11.846924+0000","last_peered":"2024-09-17T15:27:11.846924+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:27:11.846924+0000","last_undegraded":"2024-09-17T15:27:11.846924+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":5878,"log_dups_size":0,"ondisk_log_size":5878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24048099,"num_objects":5872,"num_object_clones":0,"num_object_copies":11744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2673,"num_objects_unfound":0,"num_objects_dirty":5872,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3878,"num_write_kb":15512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1398,"num_bytes_recovered":5726208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2673},{"shards":"2","objects":3199}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233552+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233552+0000","last_peered":"2024-09-17T15:26:22.233552+0000","last_clean":"2024-09-17T15:26:22.233552+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233552+0000","last_undegraded":"2024-09-17T15:26:22.233552+0000","last_fullsized":"2024-09-17T15:26:22.233552+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'5920","reported_seq":6045,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:15.289982+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:27:15.289982+0000","last_peered":"2024-09-17T15:27:15.289982+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:27:15.289982+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":5920,"log_dups_size":0,"ondisk_log_size":5920,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24215552,"num_objects":5912,"num_object_clones":0,"num_object_copies":11824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2516,"num_write_kb":10064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1902},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":98,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412941+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412941+0000","last_peered":"2024-09-17T15:26:22.412941+0000","last_clean":"2024-09-17T15:26:22.412941+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412941+0000","last_undegraded":"2024-09-17T15:26:22.412941+0000","last_fullsized":"2024-09-17T15:26:22.412941+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'5881","reported_seq":5996,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:11.757778+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:27:11.757778+0000","last_peered":"2024-09-17T15:27:11.757778+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:27:11.757778+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":5881,"log_dups_size":0,"ondisk_log_size":5881,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24068579,"num_objects":5877,"num_object_clones":0,"num_object_copies":11754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5877,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3776,"num_write_kb":15104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":1800}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'5929","reported_seq":6042,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:11.759722+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:27:11.759722+0000","last_peered":"2024-09-17T15:27:11.759722+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:27:11.759722+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":5929,"log_dups_size":0,"ondisk_log_size":5929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24260608,"num_objects":5923,"num_object_clones":0,"num_object_copies":11846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3691,"num_write_kb":14764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":1786}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'6166","reported_seq":6295,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:15.364149+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:27:15.364149+0000","last_peered":"2024-09-17T15:27:15.364149+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:27:15.364149+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":6166,"log_dups_size":0,"ondisk_log_size":6166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25247744,"num_objects":6164,"num_object_clones":0,"num_object_copies":12328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2714,"num_write_kb":10856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2026},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'6080","reported_seq":6216,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:15.191493+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:27:15.191493+0000","last_peered":"2024-09-17T15:27:15.191493+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:27:15.191493+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":6080,"log_dups_size":0,"ondisk_log_size":6080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24870912,"num_objects":6072,"num_object_clones":0,"num_object_copies":12144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2367,"num_write_kb":9468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1915},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229013+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229013+0000","last_peered":"2024-09-17T15:26:22.229013+0000","last_clean":"2024-09-17T15:26:22.229013+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229013+0000","last_undegraded":"2024-09-17T15:26:22.229013+0000","last_fullsized":"2024-09-17T15:26:22.229013+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'6106","reported_seq":6232,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:15.288925+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:27:15.288925+0000","last_peered":"2024-09-17T15:27:15.288925+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:27:15.288925+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":6106,"log_dups_size":0,"ondisk_log_size":6106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24993792,"num_objects":6102,"num_object_clones":0,"num_object_copies":12204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2893,"num_write_kb":11572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2039},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'6051","reported_seq":6188,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:15.363237+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:27:15.363237+0000","last_peered":"2024-09-17T15:27:15.363237+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:27:15.363237+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":6051,"log_dups_size":0,"ondisk_log_size":6051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24736710,"num_objects":6041,"num_object_clones":0,"num_object_copies":12082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2493,"num_write_kb":9972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1932},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":393745118,"num_objects":96138,"num_object_clones":0,"num_object_copies":192276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":2673,"num_objects_unfound":0,"num_objects_dirty":96138,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":56245,"num_write_kb":224980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1398,"num_bytes_recovered":5726208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":794800128,"data_stored":794644769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96210,"ondisk_log_size":96210,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051610,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501304,"kb_used_data":321064,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93870536,"statfs":{"total":96636764160,"available":96123428864,"internally_reserved":0,"allocated":328769536,"data_stored":328503363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":184544940},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1,1,2,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724556,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":227188,"kb_used_data":96932,"kb_used_omap":4,"kb_used_meta":130235,"kb_avail":94144652,"statfs":{"total":96636764160,"available":96404123648,"internally_reserved":0,"allocated":99258368,"data_stored":98941149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":133361323},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,3,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247839,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":540376,"kb_used_data":356040,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93831464,"statfs":{"total":96636764160,"available":96083419136,"internally_reserved":0,"allocated":364584960,"data_stored":364314710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":188739240},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":365453312,"data_stored":365417182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":100659200,"data_stored":100576101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":328687616,"data_stored":328651486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:27:17.304 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 98 20 97927 97907 3.90171 2.16016 0.0182601 0.0200134 2024-09-17T15:27:18.304 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 99 20 98512 98492 3.88538 2.28516 0.0253072 0.020099 2024-09-17T15:27:19.304 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout:2024-09-17T15:27:19.304810+0000 min lat: 0.00187689 max lat: 2.64156 avg lat: 0.0201813 2024-09-17T15:27:19.304 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-17T15:27:19.304 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 100 20 99101 99081 3.86953 2.30078 0.021249 0.0201813 2024-09-17T15:27:19.779 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:27:20.306 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 101 20 99671 99651 3.8532 2.22656 0.0168277 0.0202675 2024-09-17T15:27:20.341 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:27:20.342 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:27:20.378 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":122,"stamp":"2024-09-17T15:27:19.616342+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":398074609,"num_objects":97197,"num_object_clones":0,"num_object_copies":194394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":2521,"num_objects_unfound":0,"num_objects_dirty":97197,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":57304,"num_write_kb":229210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1551,"num_bytes_recovered":6348819,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97270,"ondisk_log_size":97270,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1278680,"kb_used_data":779688,"kb_used_omap":13,"kb_used_meta":498930,"kb_avail":281836840,"statfs":{"total":289910292480,"available":288600924160,"internally_reserved":0,"allocated":798400512,"data_stored":797546870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":510905343},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,4,10,6,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23371776,"num_objects":5706,"num_object_clones":0,"num_object_copies":11412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-300,"num_objects_unfound":0,"num_objects_dirty":5706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5706,"num_write_kb":22824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":1228800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004273"},"pg_stats":[{"pgid":"2.f","version":"41'6010","reported_seq":6082,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:15.327601+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:27:15.327601+0000","last_peered":"2024-09-17T15:27:15.327601+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:27:15.327601+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":6010,"log_dups_size":0,"ondisk_log_size":6010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24616960,"num_objects":6010,"num_object_clones":0,"num_object_copies":12020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6010,"num_write_kb":24040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1940},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'6076","reported_seq":6173,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:16.887964+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:27:16.887964+0000","last_peered":"2024-09-17T15:27:16.887964+0000","last_clean":"2024-09-17T15:27:16.887964+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:27:16.887964+0000","last_undegraded":"2024-09-17T15:27:16.887964+0000","last_fullsized":"2024-09-17T15:27:16.887964+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":6076,"log_dups_size":0,"ondisk_log_size":6076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24863686,"num_objects":6072,"num_object_clones":0,"num_object_copies":12144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6072,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4433,"num_write_kb":17732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'6144","reported_seq":6261,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:16.805257+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:27:16.805257+0000","last_peered":"2024-09-17T15:27:16.805257+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:27:16.805257+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":6144,"log_dups_size":0,"ondisk_log_size":6144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25149440,"num_objects":6140,"num_object_clones":0,"num_object_copies":12280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6140,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4301,"num_write_kb":17204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":1972}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'6133","reported_seq":6197,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:16.876485+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:27:16.876485+0000","last_peered":"2024-09-17T15:27:16.876485+0000","last_clean":"2024-09-17T15:27:16.876485+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:27:16.876485+0000","last_undegraded":"2024-09-17T15:27:16.876485+0000","last_fullsized":"2024-09-17T15:27:16.876485+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":6133,"log_dups_size":0,"ondisk_log_size":6133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25120768,"num_objects":6133,"num_object_clones":0,"num_object_copies":12266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6133,"num_write_kb":24532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'5984","reported_seq":6107,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:15.352928+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:27:15.352928+0000","last_peered":"2024-09-17T15:27:15.352928+0000","last_clean":"2024-09-17T15:27:15.352928+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:27:15.352928+0000","last_undegraded":"2024-09-17T15:27:15.352928+0000","last_fullsized":"2024-09-17T15:27:15.352928+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":5984,"log_dups_size":0,"ondisk_log_size":5984,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24482275,"num_objects":5978,"num_object_clones":0,"num_object_copies":11956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5978,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2966,"num_write_kb":11864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'6187","reported_seq":6316,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:15.283948+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:27:15.283948+0000","last_peered":"2024-09-17T15:27:15.283948+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:27:15.283948+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":6187,"log_dups_size":0,"ondisk_log_size":6187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25321955,"num_objects":6183,"num_object_clones":0,"num_object_copies":12366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6183,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2686,"num_write_kb":10744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1913},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'6152","reported_seq":6285,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:15.332583+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:27:15.332583+0000","last_peered":"2024-09-17T15:27:15.332583+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:27:15.332583+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":6152,"log_dups_size":0,"ondisk_log_size":6152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25174982,"num_objects":6148,"num_object_clones":0,"num_object_copies":12296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6148,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2921,"num_write_kb":11684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1974},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'6047","reported_seq":6169,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:15.353041+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:27:15.353041+0000","last_peered":"2024-09-17T15:27:15.353041+0000","last_clean":"2024-09-17T15:27:15.353041+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:27:15.353041+0000","last_undegraded":"2024-09-17T15:27:15.353041+0000","last_fullsized":"2024-09-17T15:27:15.353041+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":6047,"log_dups_size":0,"ondisk_log_size":6047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24760320,"num_objects":6045,"num_object_clones":0,"num_object_copies":12090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3001,"num_write_kb":12004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'6067","reported_seq":9289,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:27:16.862129+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:27:16.862129+0000","last_peered":"2024-09-17T15:27:16.862129+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:27:16.862129+0000","last_undegraded":"2024-09-17T15:27:16.862129+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":6067,"log_dups_size":0,"ondisk_log_size":6067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24822243,"num_objects":6061,"num_object_clones":0,"num_object_copies":12122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2521,"num_objects_unfound":0,"num_objects_dirty":6061,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4067,"num_write_kb":16268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1550,"num_bytes_recovered":6348800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2521},{"shards":"2","objects":3540}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233552+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233552+0000","last_peered":"2024-09-17T15:26:22.233552+0000","last_clean":"2024-09-17T15:26:22.233552+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233552+0000","last_undegraded":"2024-09-17T15:26:22.233552+0000","last_fullsized":"2024-09-17T15:26:22.233552+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'5920","reported_seq":6045,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:15.289982+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:27:15.289982+0000","last_peered":"2024-09-17T15:27:15.289982+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:27:15.289982+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":5920,"log_dups_size":0,"ondisk_log_size":5920,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24215552,"num_objects":5912,"num_object_clones":0,"num_object_copies":11824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2516,"num_write_kb":10064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1902},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":98,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412941+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412941+0000","last_peered":"2024-09-17T15:26:22.412941+0000","last_clean":"2024-09-17T15:26:22.412941+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412941+0000","last_undegraded":"2024-09-17T15:26:22.412941+0000","last_fullsized":"2024-09-17T15:26:22.412941+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'6049","reported_seq":6164,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:16.836670+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:27:16.836670+0000","last_peered":"2024-09-17T15:27:16.836670+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:27:16.836670+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":6049,"log_dups_size":0,"ondisk_log_size":6049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24756707,"num_objects":6045,"num_object_clones":0,"num_object_copies":12090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6045,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3944,"num_write_kb":15776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":1968}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'6095","reported_seq":6208,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:16.865202+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:27:16.865202+0000","last_peered":"2024-09-17T15:27:16.865202+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:27:16.865202+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":6095,"log_dups_size":0,"ondisk_log_size":6095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24940544,"num_objects":6089,"num_object_clones":0,"num_object_copies":12178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3857,"num_write_kb":15428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":1952}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'6166","reported_seq":6295,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:15.364149+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:27:15.364149+0000","last_peered":"2024-09-17T15:27:15.364149+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:27:15.364149+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":6166,"log_dups_size":0,"ondisk_log_size":6166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25247744,"num_objects":6164,"num_object_clones":0,"num_object_copies":12328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2714,"num_write_kb":10856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2026},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'6080","reported_seq":6216,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:15.191493+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:27:15.191493+0000","last_peered":"2024-09-17T15:27:15.191493+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:27:15.191493+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":6080,"log_dups_size":0,"ondisk_log_size":6080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24870912,"num_objects":6072,"num_object_clones":0,"num_object_copies":12144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2367,"num_write_kb":9468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1915},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229013+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229013+0000","last_peered":"2024-09-17T15:26:22.229013+0000","last_clean":"2024-09-17T15:26:22.229013+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229013+0000","last_undegraded":"2024-09-17T15:26:22.229013+0000","last_fullsized":"2024-09-17T15:26:22.229013+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'6106","reported_seq":6232,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:15.288925+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:27:15.288925+0000","last_peered":"2024-09-17T15:27:15.288925+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:27:15.288925+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":6106,"log_dups_size":0,"ondisk_log_size":6106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24993792,"num_objects":6102,"num_object_clones":0,"num_object_copies":12204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2893,"num_write_kb":11572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2039},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'6051","reported_seq":6188,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:15.363237+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:27:15.363237+0000","last_peered":"2024-09-17T15:27:15.363237+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:27:15.363237+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":6051,"log_dups_size":0,"ondisk_log_size":6051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24736710,"num_objects":6041,"num_object_clones":0,"num_object_copies":12082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2493,"num_write_kb":9972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1932},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":398074590,"num_objects":97195,"num_object_clones":0,"num_object_copies":194390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":2521,"num_objects_unfound":0,"num_objects_dirty":97195,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":57302,"num_write_kb":229208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1550,"num_bytes_recovered":6348800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":800567296,"data_stored":800411937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97267,"ondisk_log_size":97267,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051611,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511116,"kb_used_data":326716,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93860724,"statfs":{"total":96636764160,"available":96113381376,"internally_reserved":0,"allocated":334557184,"data_stored":334291011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":188804780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,3,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724556,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":227188,"kb_used_data":96932,"kb_used_omap":4,"kb_used_meta":130235,"kb_avail":94144652,"statfs":{"total":96636764160,"available":96404123648,"internally_reserved":0,"allocated":99258368,"data_stored":98941149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":133361323},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,3,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247839,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":540376,"kb_used_data":356040,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93831464,"statfs":{"total":96636764160,"available":96083419136,"internally_reserved":0,"allocated":364584960,"data_stored":364314710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":188739240},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":365453312,"data_stored":365417182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":100659200,"data_stored":100576101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":334454784,"data_stored":334418654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:27:20.380 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:27:20.927 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:27:20.928 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:27:20.957 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":122,"stamp":"2024-09-17T15:27:19.616342+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":398074609,"num_objects":97197,"num_object_clones":0,"num_object_copies":194394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":2521,"num_objects_unfound":0,"num_objects_dirty":97197,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":57304,"num_write_kb":229210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1551,"num_bytes_recovered":6348819,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97270,"ondisk_log_size":97270,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1278680,"kb_used_data":779688,"kb_used_omap":13,"kb_used_meta":498930,"kb_avail":281836840,"statfs":{"total":289910292480,"available":288600924160,"internally_reserved":0,"allocated":798400512,"data_stored":797546870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":510905343},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,4,10,6,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23371776,"num_objects":5706,"num_object_clones":0,"num_object_copies":11412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-300,"num_objects_unfound":0,"num_objects_dirty":5706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5706,"num_write_kb":22824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":1228800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004273"},"pg_stats":[{"pgid":"2.f","version":"41'6010","reported_seq":6082,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:15.327601+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:27:15.327601+0000","last_peered":"2024-09-17T15:27:15.327601+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:27:15.327601+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":6010,"log_dups_size":0,"ondisk_log_size":6010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24616960,"num_objects":6010,"num_object_clones":0,"num_object_copies":12020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6010,"num_write_kb":24040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1940},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'6076","reported_seq":6173,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:16.887964+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:27:16.887964+0000","last_peered":"2024-09-17T15:27:16.887964+0000","last_clean":"2024-09-17T15:27:16.887964+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:27:16.887964+0000","last_undegraded":"2024-09-17T15:27:16.887964+0000","last_fullsized":"2024-09-17T15:27:16.887964+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":6076,"log_dups_size":0,"ondisk_log_size":6076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24863686,"num_objects":6072,"num_object_clones":0,"num_object_copies":12144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6072,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4433,"num_write_kb":17732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'6144","reported_seq":6261,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:16.805257+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:27:16.805257+0000","last_peered":"2024-09-17T15:27:16.805257+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:27:16.805257+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":6144,"log_dups_size":0,"ondisk_log_size":6144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25149440,"num_objects":6140,"num_object_clones":0,"num_object_copies":12280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6140,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4301,"num_write_kb":17204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":1972}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'6133","reported_seq":6197,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:16.876485+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:27:16.876485+0000","last_peered":"2024-09-17T15:27:16.876485+0000","last_clean":"2024-09-17T15:27:16.876485+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:27:16.876485+0000","last_undegraded":"2024-09-17T15:27:16.876485+0000","last_fullsized":"2024-09-17T15:27:16.876485+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":6133,"log_dups_size":0,"ondisk_log_size":6133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25120768,"num_objects":6133,"num_object_clones":0,"num_object_copies":12266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6133,"num_write_kb":24532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'5984","reported_seq":6107,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:15.352928+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:27:15.352928+0000","last_peered":"2024-09-17T15:27:15.352928+0000","last_clean":"2024-09-17T15:27:15.352928+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:27:15.352928+0000","last_undegraded":"2024-09-17T15:27:15.352928+0000","last_fullsized":"2024-09-17T15:27:15.352928+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":5984,"log_dups_size":0,"ondisk_log_size":5984,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24482275,"num_objects":5978,"num_object_clones":0,"num_object_copies":11956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5978,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2966,"num_write_kb":11864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'6187","reported_seq":6316,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:15.283948+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:27:15.283948+0000","last_peered":"2024-09-17T15:27:15.283948+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:27:15.283948+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":6187,"log_dups_size":0,"ondisk_log_size":6187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25321955,"num_objects":6183,"num_object_clones":0,"num_object_copies":12366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6183,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2686,"num_write_kb":10744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1913},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'6152","reported_seq":6285,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:15.332583+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:27:15.332583+0000","last_peered":"2024-09-17T15:27:15.332583+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:27:15.332583+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":6152,"log_dups_size":0,"ondisk_log_size":6152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25174982,"num_objects":6148,"num_object_clones":0,"num_object_copies":12296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6148,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2921,"num_write_kb":11684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1974},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'6047","reported_seq":6169,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:15.353041+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:27:15.353041+0000","last_peered":"2024-09-17T15:27:15.353041+0000","last_clean":"2024-09-17T15:27:15.353041+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:27:15.353041+0000","last_undegraded":"2024-09-17T15:27:15.353041+0000","last_fullsized":"2024-09-17T15:27:15.353041+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":6047,"log_dups_size":0,"ondisk_log_size":6047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24760320,"num_objects":6045,"num_object_clones":0,"num_object_copies":12090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3001,"num_write_kb":12004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'6067","reported_seq":9289,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:27:16.862129+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:27:16.862129+0000","last_peered":"2024-09-17T15:27:16.862129+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:27:16.862129+0000","last_undegraded":"2024-09-17T15:27:16.862129+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":6067,"log_dups_size":0,"ondisk_log_size":6067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24822243,"num_objects":6061,"num_object_clones":0,"num_object_copies":12122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2521,"num_objects_unfound":0,"num_objects_dirty":6061,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4067,"num_write_kb":16268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1550,"num_bytes_recovered":6348800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2521},{"shards":"2","objects":3540}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233552+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233552+0000","last_peered":"2024-09-17T15:26:22.233552+0000","last_clean":"2024-09-17T15:26:22.233552+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233552+0000","last_undegraded":"2024-09-17T15:26:22.233552+0000","last_fullsized":"2024-09-17T15:26:22.233552+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'5920","reported_seq":6045,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:15.289982+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:27:15.289982+0000","last_peered":"2024-09-17T15:27:15.289982+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:27:15.289982+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":5920,"log_dups_size":0,"ondisk_log_size":5920,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24215552,"num_objects":5912,"num_object_clones":0,"num_object_copies":11824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2516,"num_write_kb":10064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1902},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":98,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412941+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412941+0000","last_peered":"2024-09-17T15:26:22.412941+0000","last_clean":"2024-09-17T15:26:22.412941+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412941+0000","last_undegraded":"2024-09-17T15:26:22.412941+0000","last_fullsized":"2024-09-17T15:26:22.412941+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'6049","reported_seq":6164,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:16.836670+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:27:16.836670+0000","last_peered":"2024-09-17T15:27:16.836670+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:27:16.836670+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":6049,"log_dups_size":0,"ondisk_log_size":6049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24756707,"num_objects":6045,"num_object_clones":0,"num_object_copies":12090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6045,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3944,"num_write_kb":15776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":1968}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'6095","reported_seq":6208,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:16.865202+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:27:16.865202+0000","last_peered":"2024-09-17T15:27:16.865202+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:27:16.865202+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":6095,"log_dups_size":0,"ondisk_log_size":6095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24940544,"num_objects":6089,"num_object_clones":0,"num_object_copies":12178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3857,"num_write_kb":15428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":1952}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'6166","reported_seq":6295,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:15.364149+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:27:15.364149+0000","last_peered":"2024-09-17T15:27:15.364149+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:27:15.364149+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":6166,"log_dups_size":0,"ondisk_log_size":6166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25247744,"num_objects":6164,"num_object_clones":0,"num_object_copies":12328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2714,"num_write_kb":10856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2026},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'6080","reported_seq":6216,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:15.191493+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:27:15.191493+0000","last_peered":"2024-09-17T15:27:15.191493+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:27:15.191493+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":6080,"log_dups_size":0,"ondisk_log_size":6080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24870912,"num_objects":6072,"num_object_clones":0,"num_object_copies":12144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2367,"num_write_kb":9468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1915},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229013+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229013+0000","last_peered":"2024-09-17T15:26:22.229013+0000","last_clean":"2024-09-17T15:26:22.229013+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229013+0000","last_undegraded":"2024-09-17T15:26:22.229013+0000","last_fullsized":"2024-09-17T15:26:22.229013+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'6106","reported_seq":6232,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:15.288925+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:27:15.288925+0000","last_peered":"2024-09-17T15:27:15.288925+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:27:15.288925+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":6106,"log_dups_size":0,"ondisk_log_size":6106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24993792,"num_objects":6102,"num_object_clones":0,"num_object_copies":12204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2893,"num_write_kb":11572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2039},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'6051","reported_seq":6188,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:15.363237+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:27:15.363237+0000","last_peered":"2024-09-17T15:27:15.363237+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:27:15.363237+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":6051,"log_dups_size":0,"ondisk_log_size":6051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24736710,"num_objects":6041,"num_object_clones":0,"num_object_copies":12082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2493,"num_write_kb":9972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1932},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":398074590,"num_objects":97195,"num_object_clones":0,"num_object_copies":194390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":2521,"num_objects_unfound":0,"num_objects_dirty":97195,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":57302,"num_write_kb":229208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1550,"num_bytes_recovered":6348800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":800567296,"data_stored":800411937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97267,"ondisk_log_size":97267,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051611,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511116,"kb_used_data":326716,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93860724,"statfs":{"total":96636764160,"available":96113381376,"internally_reserved":0,"allocated":334557184,"data_stored":334291011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":188804780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,3,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724556,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":227188,"kb_used_data":96932,"kb_used_omap":4,"kb_used_meta":130235,"kb_avail":94144652,"statfs":{"total":96636764160,"available":96404123648,"internally_reserved":0,"allocated":99258368,"data_stored":98941149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":133361323},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,3,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247839,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":540376,"kb_used_data":356040,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93831464,"statfs":{"total":96636764160,"available":96083419136,"internally_reserved":0,"allocated":364584960,"data_stored":364314710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":188739240},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":365453312,"data_stored":365417182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":100659200,"data_stored":100576101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":334454784,"data_stored":334418654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:27:20.959 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:27:20.959 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'6010", 'reported_seq': 6082, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:15.327601+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:27:15.327601+0000', 'last_peered': '2024-09-17T15:27:15.327601+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:27:15.327601+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 6010, 'log_dups_size': 0, 'ondisk_log_size': 6010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24616960, 'num_objects': 6010, 'num_object_clones': 0, 'num_object_copies': 12020, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4070, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6010, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6010, 'num_write_kb': 24040, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1940}, {'shards': '0,2', 'objects': 4070}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:20.959 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:27:20.959 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'6144", 'reported_seq': 6261, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:16.805257+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:27:16.805257+0000', 'last_peered': '2024-09-17T15:27:16.805257+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:27:16.805257+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 6144, 'log_dups_size': 0, 'ondisk_log_size': 6144, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25149440, 'num_objects': 6140, 'num_object_clones': 0, 'num_object_copies': 12280, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4169, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6140, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4301, 'num_write_kb': 17204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4168}, {'shards': '2', 'objects': 1972}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:27:20.959 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:27:20.959 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'6187", 'reported_seq': 6316, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:15.283948+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:27:15.283948+0000', 'last_peered': '2024-09-17T15:27:15.283948+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:27:15.283948+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 6187, 'log_dups_size': 0, 'ondisk_log_size': 6187, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25321955, 'num_objects': 6183, 'num_object_clones': 0, 'num_object_copies': 12366, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4271, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6183, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2686, 'num_write_kb': 10744, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1913}, {'shards': '0,2', 'objects': 4268}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:20.959 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:27:20.959 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'6152", 'reported_seq': 6285, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:15.332583+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:27:15.332583+0000', 'last_peered': '2024-09-17T15:27:15.332583+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:27:15.332583+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 6152, 'log_dups_size': 0, 'ondisk_log_size': 6152, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25174982, 'num_objects': 6148, 'num_object_clones': 0, 'num_object_copies': 12296, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4174, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6148, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 2921, 'num_write_kb': 11684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1974}, {'shards': '0,2', 'objects': 4174}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:20.959 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-17T15:27:20.959 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "41'6067", 'reported_seq': 9289, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:27:16.862129+0000', 'last_change': '2024-09-17T15:26:22.588242+0000', 'last_active': '2024-09-17T15:27:16.862129+0000', 'last_peered': '2024-09-17T15:27:16.862129+0000', 'last_clean': '2024-09-17T15:26:20.155646+0000', 'last_became_active': '2024-09-17T15:26:22.557742+0000', 'last_became_peered': '2024-09-17T15:26:22.557742+0000', 'last_unstale': '2024-09-17T15:27:16.862129+0000', 'last_undegraded': '2024-09-17T15:27:16.862129+0000', 'last_fullsized': '2024-09-17T15:26:22.188850+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2016", 'last_scrub_stamp': '2024-09-17T15:25:53.959011+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:53.959011+0000', 'objects_scrubbed': 1994, 'log_size': 6067, 'log_dups_size': 0, 'ondisk_log_size': 6067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 568, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24822243, 'num_objects': 6061, 'num_object_clones': 0, 'num_object_copies': 12122, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2521, 'num_objects_unfound': 0, 'num_objects_dirty': 6061, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4067, 'num_write_kb': 16268, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1550, 'num_bytes_recovered': 6348800, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2521}, {'shards': '2', 'objects': 3540}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:27:20.960 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:27:20.960 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'5920", 'reported_seq': 6045, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:15.289982+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:27:15.289982+0000', 'last_peered': '2024-09-17T15:27:15.289982+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:27:15.289982+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 5920, 'log_dups_size': 0, 'ondisk_log_size': 5920, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24215552, 'num_objects': 5912, 'num_object_clones': 0, 'num_object_copies': 11824, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4012, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5912, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2516, 'num_write_kb': 10064, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1902}, {'shards': '0,2', 'objects': 4010}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:20.960 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:27:20.960 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'6049", 'reported_seq': 6164, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:16.836670+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:27:16.836670+0000', 'last_peered': '2024-09-17T15:27:16.836670+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:27:16.836670+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 6049, 'log_dups_size': 0, 'ondisk_log_size': 6049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24756707, 'num_objects': 6045, 'num_object_clones': 0, 'num_object_copies': 12090, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4078, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6045, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3944, 'num_write_kb': 15776, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4077}, {'shards': '2', 'objects': 1968}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:27:20.960 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:27:20.960 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'6095", 'reported_seq': 6208, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:16.865202+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:27:16.865202+0000', 'last_peered': '2024-09-17T15:27:16.865202+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:27:16.865202+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 6095, 'log_dups_size': 0, 'ondisk_log_size': 6095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24940544, 'num_objects': 6089, 'num_object_clones': 0, 'num_object_copies': 12178, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6089, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3857, 'num_write_kb': 15428, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4137}, {'shards': '2', 'objects': 1952}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:27:20.960 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:27:20.960 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'6166", 'reported_seq': 6295, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:15.364149+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:27:15.364149+0000', 'last_peered': '2024-09-17T15:27:15.364149+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:27:15.364149+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 6166, 'log_dups_size': 0, 'ondisk_log_size': 6166, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25247744, 'num_objects': 6164, 'num_object_clones': 0, 'num_object_copies': 12328, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6164, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2714, 'num_write_kb': 10856, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2026}, {'shards': '0,2', 'objects': 4138}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:20.960 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:27:20.960 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'6080", 'reported_seq': 6216, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:15.191493+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:27:15.191493+0000', 'last_peered': '2024-09-17T15:27:15.191493+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:27:15.191493+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 6080, 'log_dups_size': 0, 'ondisk_log_size': 6080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24870912, 'num_objects': 6072, 'num_object_clones': 0, 'num_object_copies': 12144, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4160, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6072, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2367, 'num_write_kb': 9468, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1915}, {'shards': '0,2', 'objects': 4157}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:20.960 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:27:20.961 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'6106", 'reported_seq': 6232, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:15.288925+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:27:15.288925+0000', 'last_peered': '2024-09-17T15:27:15.288925+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:27:15.288925+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 6106, 'log_dups_size': 0, 'ondisk_log_size': 6106, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24993792, 'num_objects': 6102, 'num_object_clones': 0, 'num_object_copies': 12204, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4065, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6102, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2893, 'num_write_kb': 11572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2039}, {'shards': '0,2', 'objects': 4063}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:20.961 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:27:20.961 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'6051", 'reported_seq': 6188, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:15.363237+0000', 'last_change': '2024-09-17T15:26:22.654521+0000', 'last_active': '2024-09-17T15:27:15.363237+0000', 'last_peered': '2024-09-17T15:27:15.363237+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:27:15.363237+0000', 'last_undegraded': '2024-09-17T15:26:22.307795+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 6051, 'log_dups_size': 0, 'ondisk_log_size': 6051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24736710, 'num_objects': 6041, 'num_object_clones': 0, 'num_object_copies': 12082, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4111, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6041, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2493, 'num_write_kb': 9972, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1932}, {'shards': '0,2', 'objects': 4109}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:20.961 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:27:21.307 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 102 20 100204 100184 3.8358 2.08203 0.0773384 0.0203593 2024-09-17T15:27:21.629 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:27:21.629 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45388/194394 objects degraded (23.348%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":178,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":97197,"data_bytes":398074609,"bytes_used":1309368320,"bytes_avail":288600924160,"bytes_total":289910292480,"degraded_objects":45388,"degraded_total":194394,"degraded_ratio":0.2334845725691122,"misplaced_objects":2521,"misplaced_total":194394,"misplaced_ratio":0.012968507258454477,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":102363,"recovering_keys_per_sec":0,"num_objects_recovered":300,"num_bytes_recovered":1228800,"num_keys_recovered":0,"write_bytes_sec":1946954,"read_op_per_sec":0,"write_op_per_sec":475},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T15:27:19.616376+0000","services":{}},"progress_events":{}} 2024-09-17T15:27:21.657 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:27:21.658 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:27:22.212 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:27:22.215 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:27:22.234 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":123,"stamp":"2024-09-17T15:27:21.616885+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":405258993,"num_objects":98951,"num_object_clones":0,"num_object_copies":197902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":2521,"num_objects_unfound":0,"num_objects_dirty":98951,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":59058,"num_write_kb":236226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1551,"num_bytes_recovered":6348819,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99024,"ondisk_log_size":99024,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1329264,"kb_used_data":796928,"kb_used_omap":13,"kb_used_meta":532274,"kb_avail":281786256,"statfs":{"total":289910292480,"available":288549126144,"internally_reserved":0,"allocated":816054272,"data_stored":815200630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":545049599},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,4,6,8,8,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30556160,"num_objects":7460,"num_object_clones":0,"num_object_copies":14920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-300,"num_objects_unfound":0,"num_objects_dirty":7460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7460,"num_write_kb":29840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":1228800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004043"},"pg_stats":[{"pgid":"2.f","version":"41'6190","reported_seq":6262,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:20.374296+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:27:20.374296+0000","last_peered":"2024-09-17T15:27:20.374296+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:27:20.374296+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":6190,"log_dups_size":0,"ondisk_log_size":6190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25354240,"num_objects":6190,"num_object_clones":0,"num_object_copies":12380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6190,"num_write_kb":24760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2120},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'6076","reported_seq":6173,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:16.887964+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:27:16.887964+0000","last_peered":"2024-09-17T15:27:16.887964+0000","last_clean":"2024-09-17T15:27:16.887964+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:27:16.887964+0000","last_undegraded":"2024-09-17T15:27:16.887964+0000","last_fullsized":"2024-09-17T15:27:16.887964+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":6076,"log_dups_size":0,"ondisk_log_size":6076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24863686,"num_objects":6072,"num_object_clones":0,"num_object_copies":12144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6072,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4433,"num_write_kb":17732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'6144","reported_seq":6261,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:16.805257+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:27:16.805257+0000","last_peered":"2024-09-17T15:27:16.805257+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:27:16.805257+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":6144,"log_dups_size":0,"ondisk_log_size":6144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25149440,"num_objects":6140,"num_object_clones":0,"num_object_copies":12280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6140,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4301,"num_write_kb":17204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":1972}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'6133","reported_seq":6197,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:16.876485+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:27:16.876485+0000","last_peered":"2024-09-17T15:27:16.876485+0000","last_clean":"2024-09-17T15:27:16.876485+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:27:16.876485+0000","last_undegraded":"2024-09-17T15:27:16.876485+0000","last_fullsized":"2024-09-17T15:27:16.876485+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":6133,"log_dups_size":0,"ondisk_log_size":6133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25120768,"num_objects":6133,"num_object_clones":0,"num_object_copies":12266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6133,"num_write_kb":24532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'6179","reported_seq":6303,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:20.365177+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:27:20.365177+0000","last_peered":"2024-09-17T15:27:20.365177+0000","last_clean":"2024-09-17T15:27:20.365177+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:27:20.365177+0000","last_undegraded":"2024-09-17T15:27:20.365177+0000","last_fullsized":"2024-09-17T15:27:20.365177+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":6179,"log_dups_size":0,"ondisk_log_size":6179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25280995,"num_objects":6173,"num_object_clones":0,"num_object_copies":12346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6173,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3161,"num_write_kb":12644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'6387","reported_seq":6516,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:20.369992+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:27:20.369992+0000","last_peered":"2024-09-17T15:27:20.369992+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:27:20.369992+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":6387,"log_dups_size":0,"ondisk_log_size":6387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26141155,"num_objects":6383,"num_object_clones":0,"num_object_copies":12766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6383,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2886,"num_write_kb":11544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2113},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'6331","reported_seq":6464,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:20.334569+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:27:20.334569+0000","last_peered":"2024-09-17T15:27:20.334569+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:27:20.334569+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":6331,"log_dups_size":0,"ondisk_log_size":6331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25908166,"num_objects":6327,"num_object_clones":0,"num_object_copies":12654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6327,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3100,"num_write_kb":12400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2153},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'6221","reported_seq":6344,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:20.394513+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:27:20.394513+0000","last_peered":"2024-09-17T15:27:20.394513+0000","last_clean":"2024-09-17T15:27:20.394513+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:27:20.394513+0000","last_undegraded":"2024-09-17T15:27:20.394513+0000","last_fullsized":"2024-09-17T15:27:20.394513+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":6221,"log_dups_size":0,"ondisk_log_size":6221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25473024,"num_objects":6219,"num_object_clones":0,"num_object_copies":12438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6219,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3175,"num_write_kb":12700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'6067","reported_seq":9289,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:27:16.862129+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:27:16.862129+0000","last_peered":"2024-09-17T15:27:16.862129+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:27:16.862129+0000","last_undegraded":"2024-09-17T15:27:16.862129+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":6067,"log_dups_size":0,"ondisk_log_size":6067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24822243,"num_objects":6061,"num_object_clones":0,"num_object_copies":12122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2521,"num_objects_unfound":0,"num_objects_dirty":6061,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4067,"num_write_kb":16268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1550,"num_bytes_recovered":6348800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2521},{"shards":"2","objects":3540}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233552+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233552+0000","last_peered":"2024-09-17T15:26:22.233552+0000","last_clean":"2024-09-17T15:26:22.233552+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233552+0000","last_undegraded":"2024-09-17T15:26:22.233552+0000","last_fullsized":"2024-09-17T15:26:22.233552+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'6093","reported_seq":6218,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:20.297808+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:27:20.297808+0000","last_peered":"2024-09-17T15:27:20.297808+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:27:20.297808+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":6093,"log_dups_size":0,"ondisk_log_size":6093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24924160,"num_objects":6085,"num_object_clones":0,"num_object_copies":12170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2689,"num_write_kb":10756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2075},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":98,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412941+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412941+0000","last_peered":"2024-09-17T15:26:22.412941+0000","last_clean":"2024-09-17T15:26:22.412941+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412941+0000","last_undegraded":"2024-09-17T15:26:22.412941+0000","last_fullsized":"2024-09-17T15:26:22.412941+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'6049","reported_seq":6164,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:16.836670+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:27:16.836670+0000","last_peered":"2024-09-17T15:27:16.836670+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:27:16.836670+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":6049,"log_dups_size":0,"ondisk_log_size":6049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24756707,"num_objects":6045,"num_object_clones":0,"num_object_copies":12090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6045,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3944,"num_write_kb":15776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":1968}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'6095","reported_seq":6208,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:16.865202+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:27:16.865202+0000","last_peered":"2024-09-17T15:27:16.865202+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:27:16.865202+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":6095,"log_dups_size":0,"ondisk_log_size":6095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24940544,"num_objects":6089,"num_object_clones":0,"num_object_copies":12178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3857,"num_write_kb":15428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":1952}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'6328","reported_seq":6458,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:20.375166+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:27:20.375166+0000","last_peered":"2024-09-17T15:27:20.375166+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:27:20.375166+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":6328,"log_dups_size":0,"ondisk_log_size":6328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25911296,"num_objects":6326,"num_object_clones":0,"num_object_copies":12652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2876,"num_write_kb":11504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2188},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'6250","reported_seq":6386,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:20.380579+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:27:20.380579+0000","last_peered":"2024-09-17T15:27:20.380579+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:27:20.380579+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":6250,"log_dups_size":0,"ondisk_log_size":6250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25567232,"num_objects":6242,"num_object_clones":0,"num_object_copies":12484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6242,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2537,"num_write_kb":10148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2085},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229013+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229013+0000","last_peered":"2024-09-17T15:26:22.229013+0000","last_clean":"2024-09-17T15:26:22.229013+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229013+0000","last_undegraded":"2024-09-17T15:26:22.229013+0000","last_fullsized":"2024-09-17T15:26:22.229013+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'6275","reported_seq":6402,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:20.407582+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:27:20.407582+0000","last_peered":"2024-09-17T15:27:20.407582+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:27:20.407582+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":6275,"log_dups_size":0,"ondisk_log_size":6275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25686016,"num_objects":6271,"num_object_clones":0,"num_object_copies":12542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3062,"num_write_kb":12248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2208},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'6203","reported_seq":6340,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:20.333934+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:27:20.333934+0000","last_peered":"2024-09-17T15:27:20.333934+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:27:20.333934+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":6203,"log_dups_size":0,"ondisk_log_size":6203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25359302,"num_objects":6193,"num_object_clones":0,"num_object_copies":12386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6193,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2645,"num_write_kb":10580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2084},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":405258974,"num_objects":98949,"num_object_clones":0,"num_object_copies":197898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":2521,"num_objects_unfound":0,"num_objects_dirty":98949,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":59056,"num_write_kb":236224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1550,"num_bytes_recovered":6348800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":818470912,"data_stored":818315553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99021,"ondisk_log_size":99021,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051611,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511116,"kb_used_data":326716,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93860724,"statfs":{"total":96636764160,"available":96113381376,"internally_reserved":0,"allocated":334557184,"data_stored":334291011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":188804780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,3,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724557,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":227772,"kb_used_data":105836,"kb_used_omap":4,"kb_used_meta":121915,"kb_avail":94144068,"statfs":{"total":96636764160,"available":96403525632,"internally_reserved":0,"allocated":108376064,"data_stored":108058845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":124841643},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,5,4,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247840,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":590376,"kb_used_data":364376,"kb_used_omap":4,"kb_used_meta":225979,"kb_avail":93781464,"statfs":{"total":96636764160,"available":96032219136,"internally_reserved":0,"allocated":373121024,"data_stored":372850774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":231403176},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,0,1,7,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":374116352,"data_stored":374080222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":109899776,"data_stored":109816677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":334454784,"data_stored":334418654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:27:22.307 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 103 20 100736 100716 3.81874 2.07812 0.0297723 0.0204518 2024-09-17T15:27:23.308 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 104 20 101314 101294 3.80371 2.25781 0.0519793 0.0205327 2024-09-17T15:27:24.308 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 105 20 101899 101879 3.78925 2.28516 0.0879416 0.0206096 2024-09-17T15:27:25.237 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:27:25.308 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 106 20 102377 102357 3.77112 1.86719 0.0216409 0.0207091 2024-09-17T15:27:25.795 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:27:25.798 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:27:25.821 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2024-09-17T15:27:25.618733+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":416629489,"num_objects":101727,"num_object_clones":0,"num_object_copies":203454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":2370,"num_objects_unfound":0,"num_objects_dirty":101727,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":61834,"num_write_kb":247330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1702,"num_bytes_recovered":6967315,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":101800,"ondisk_log_size":101800,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1352320,"kb_used_data":819984,"kb_used_omap":13,"kb_used_meta":532274,"kb_avail":281763200,"statfs":{"total":289910292480,"available":288525516800,"internally_reserved":0,"allocated":839663616,"data_stored":838809974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":545049599},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,4,0,4,12,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30244864,"num_objects":7384,"num_object_clones":0,"num_object_copies":14768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-303,"num_objects_unfound":0,"num_objects_dirty":7384,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7384,"num_write_kb":29536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":1241088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004464"},"pg_stats":[{"pgid":"2.f","version":"41'6363","reported_seq":6436,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:25.415098+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:27:25.415098+0000","last_peered":"2024-09-17T15:27:25.415098+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:27:25.415098+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":6363,"log_dups_size":0,"ondisk_log_size":6363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26062848,"num_objects":6363,"num_object_clones":0,"num_object_copies":12726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6363,"num_write_kb":25452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2293},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'6266","reported_seq":6362,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:21.867644+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:27:21.867644+0000","last_peered":"2024-09-17T15:27:21.867644+0000","last_clean":"2024-09-17T15:27:21.867644+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:27:21.867644+0000","last_undegraded":"2024-09-17T15:27:21.867644+0000","last_fullsized":"2024-09-17T15:27:21.867644+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":6266,"log_dups_size":0,"ondisk_log_size":6266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25641926,"num_objects":6262,"num_object_clones":0,"num_object_copies":12524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6262,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4623,"num_write_kb":18492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'6301","reported_seq":6421,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:21.892375+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:27:21.892375+0000","last_peered":"2024-09-17T15:27:21.892375+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:27:21.892375+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":6301,"log_dups_size":0,"ondisk_log_size":6301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25792512,"num_objects":6297,"num_object_clones":0,"num_object_copies":12594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4458,"num_write_kb":17832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":2129}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'6325","reported_seq":6387,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:21.884072+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:27:21.884072+0000","last_peered":"2024-09-17T15:27:21.884072+0000","last_clean":"2024-09-17T15:27:21.884072+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:27:21.884072+0000","last_undegraded":"2024-09-17T15:27:21.884072+0000","last_fullsized":"2024-09-17T15:27:21.884072+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":6325,"log_dups_size":0,"ondisk_log_size":6325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25907200,"num_objects":6325,"num_object_clones":0,"num_object_copies":12650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6325,"num_write_kb":25300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'6367","reported_seq":6491,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:25.422455+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:27:25.422455+0000","last_peered":"2024-09-17T15:27:25.422455+0000","last_clean":"2024-09-17T15:27:25.422455+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:27:25.422455+0000","last_undegraded":"2024-09-17T15:27:25.422455+0000","last_fullsized":"2024-09-17T15:27:25.422455+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":6367,"log_dups_size":0,"ondisk_log_size":6367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26051043,"num_objects":6361,"num_object_clones":0,"num_object_copies":12722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6361,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3349,"num_write_kb":13396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'6541","reported_seq":6670,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:25.433303+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:27:25.433303+0000","last_peered":"2024-09-17T15:27:25.433303+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:27:25.433303+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":6541,"log_dups_size":0,"ondisk_log_size":6541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26771939,"num_objects":6537,"num_object_clones":0,"num_object_copies":13074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6537,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3040,"num_write_kb":12160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2267},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'6490","reported_seq":6624,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:25.401239+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:27:25.401239+0000","last_peered":"2024-09-17T15:27:25.401239+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:27:25.401239+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":6490,"log_dups_size":0,"ondisk_log_size":6490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26559430,"num_objects":6486,"num_object_clones":0,"num_object_copies":12972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6486,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3259,"num_write_kb":13036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2312},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'6405","reported_seq":6528,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:25.397872+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:27:25.397872+0000","last_peered":"2024-09-17T15:27:25.397872+0000","last_clean":"2024-09-17T15:27:25.397872+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:27:25.397872+0000","last_undegraded":"2024-09-17T15:27:25.397872+0000","last_fullsized":"2024-09-17T15:27:25.397872+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":6405,"log_dups_size":0,"ondisk_log_size":6405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26226688,"num_objects":6403,"num_object_clones":0,"num_object_copies":12806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6403,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3359,"num_write_kb":13436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'6255","reported_seq":9780,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:27:21.882572+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:27:21.882572+0000","last_peered":"2024-09-17T15:27:21.882572+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:27:21.882572+0000","last_undegraded":"2024-09-17T15:27:21.882572+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":6255,"log_dups_size":0,"ondisk_log_size":6255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25592291,"num_objects":6249,"num_object_clones":0,"num_object_copies":12498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2370,"num_objects_unfound":0,"num_objects_dirty":6249,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4255,"num_write_kb":17020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1701,"num_bytes_recovered":6967296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2370},{"shards":"2","objects":3879}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233552+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233552+0000","last_peered":"2024-09-17T15:26:22.233552+0000","last_clean":"2024-09-17T15:26:22.233552+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233552+0000","last_undegraded":"2024-09-17T15:26:22.233552+0000","last_fullsized":"2024-09-17T15:26:22.233552+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'6275","reported_seq":6400,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:25.431189+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:27:25.431189+0000","last_peered":"2024-09-17T15:27:25.431189+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:27:25.431189+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":6275,"log_dups_size":0,"ondisk_log_size":6275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25669632,"num_objects":6267,"num_object_clones":0,"num_object_copies":12534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2871,"num_write_kb":11484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2257},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":98,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412941+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412941+0000","last_peered":"2024-09-17T15:26:22.412941+0000","last_clean":"2024-09-17T15:26:22.412941+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412941+0000","last_undegraded":"2024-09-17T15:26:22.412941+0000","last_fullsized":"2024-09-17T15:26:22.412941+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'6237","reported_seq":6352,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:21.767591+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:27:21.767591+0000","last_peered":"2024-09-17T15:27:21.767591+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:27:21.767591+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":6237,"log_dups_size":0,"ondisk_log_size":6237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25526755,"num_objects":6233,"num_object_clones":0,"num_object_copies":12466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6233,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4132,"num_write_kb":16528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":2156}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'6279","reported_seq":6392,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:21.811345+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:27:21.811345+0000","last_peered":"2024-09-17T15:27:21.811345+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:27:21.811345+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":6279,"log_dups_size":0,"ondisk_log_size":6279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25694208,"num_objects":6273,"num_object_clones":0,"num_object_copies":12546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6273,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4041,"num_write_kb":16164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":2136}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'6488","reported_seq":6618,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:25.381013+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:27:25.381013+0000","last_peered":"2024-09-17T15:27:25.381013+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:27:25.381013+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":6488,"log_dups_size":0,"ondisk_log_size":6488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26566656,"num_objects":6486,"num_object_clones":0,"num_object_copies":12972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6486,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3036,"num_write_kb":12144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2348},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'6405","reported_seq":6540,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:25.363755+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:27:25.363755+0000","last_peered":"2024-09-17T15:27:25.363755+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:27:25.363755+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":6405,"log_dups_size":0,"ondisk_log_size":6405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26202112,"num_objects":6397,"num_object_clones":0,"num_object_copies":12794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2692,"num_write_kb":10768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2240},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229013+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229013+0000","last_peered":"2024-09-17T15:26:22.229013+0000","last_clean":"2024-09-17T15:26:22.229013+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229013+0000","last_undegraded":"2024-09-17T15:26:22.229013+0000","last_fullsized":"2024-09-17T15:26:22.229013+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'6433","reported_seq":6560,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:25.292832+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:27:25.292832+0000","last_peered":"2024-09-17T15:27:25.292832+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:27:25.292832+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":6433,"log_dups_size":0,"ondisk_log_size":6433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26333184,"num_objects":6429,"num_object_clones":0,"num_object_copies":12858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3220,"num_write_kb":12880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2366},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'6367","reported_seq":6504,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:25.430192+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:27:25.430192+0000","last_peered":"2024-09-17T15:27:25.430192+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:27:25.430192+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":6367,"log_dups_size":0,"ondisk_log_size":6367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26031046,"num_objects":6357,"num_object_clones":0,"num_object_copies":12714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2809,"num_write_kb":11236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2248},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":416629470,"num_objects":101725,"num_object_clones":0,"num_object_copies":203450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":2370,"num_objects_unfound":0,"num_objects_dirty":101725,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":61832,"num_write_kb":247328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1701,"num_bytes_recovered":6967296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":841674752,"data_stored":841519393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":101797,"ondisk_log_size":101797,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051612,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508720,"kb_used_data":332640,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93863120,"statfs":{"total":96636764160,"available":96115834880,"internally_reserved":0,"allocated":340623360,"data_stored":340357187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":180285100},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,0,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724558,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":240824,"kb_used_data":114728,"kb_used_omap":4,"kb_used_meta":126075,"kb_avail":94131016,"statfs":{"total":96636764160,"available":96390160384,"internally_reserved":0,"allocated":117481472,"data_stored":117164253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":129101483},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247841,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":602776,"kb_used_data":372616,"kb_used_omap":4,"kb_used_meta":230139,"kb_avail":93769064,"statfs":{"total":96636764160,"available":96019521536,"internally_reserved":0,"allocated":381558784,"data_stored":381288534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":235663016},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,0,1,7,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":382443520,"data_stored":382407390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":118730752,"data_stored":118647653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":340500480,"data_stored":340464350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:27:25.823 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:27:26.308 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 107 20 102874 102854 3.75402 1.94141 0.0338681 0.0208025 2024-09-17T15:27:26.387 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:27:26.389 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:27:26.417 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2024-09-17T15:27:25.618733+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":416629489,"num_objects":101727,"num_object_clones":0,"num_object_copies":203454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":2370,"num_objects_unfound":0,"num_objects_dirty":101727,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":61834,"num_write_kb":247330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1702,"num_bytes_recovered":6967315,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":101800,"ondisk_log_size":101800,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1352320,"kb_used_data":819984,"kb_used_omap":13,"kb_used_meta":532274,"kb_avail":281763200,"statfs":{"total":289910292480,"available":288525516800,"internally_reserved":0,"allocated":839663616,"data_stored":838809974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":545049599},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,4,0,4,12,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30244864,"num_objects":7384,"num_object_clones":0,"num_object_copies":14768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-303,"num_objects_unfound":0,"num_objects_dirty":7384,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7384,"num_write_kb":29536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":1241088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004464"},"pg_stats":[{"pgid":"2.f","version":"41'6363","reported_seq":6436,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:25.415098+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:27:25.415098+0000","last_peered":"2024-09-17T15:27:25.415098+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:27:25.415098+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":6363,"log_dups_size":0,"ondisk_log_size":6363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26062848,"num_objects":6363,"num_object_clones":0,"num_object_copies":12726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6363,"num_write_kb":25452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2293},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'6266","reported_seq":6362,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:21.867644+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:27:21.867644+0000","last_peered":"2024-09-17T15:27:21.867644+0000","last_clean":"2024-09-17T15:27:21.867644+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:27:21.867644+0000","last_undegraded":"2024-09-17T15:27:21.867644+0000","last_fullsized":"2024-09-17T15:27:21.867644+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":6266,"log_dups_size":0,"ondisk_log_size":6266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25641926,"num_objects":6262,"num_object_clones":0,"num_object_copies":12524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6262,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4623,"num_write_kb":18492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'6301","reported_seq":6421,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:21.892375+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:27:21.892375+0000","last_peered":"2024-09-17T15:27:21.892375+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:27:21.892375+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":6301,"log_dups_size":0,"ondisk_log_size":6301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25792512,"num_objects":6297,"num_object_clones":0,"num_object_copies":12594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4458,"num_write_kb":17832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":2129}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'6325","reported_seq":6387,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:21.884072+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:27:21.884072+0000","last_peered":"2024-09-17T15:27:21.884072+0000","last_clean":"2024-09-17T15:27:21.884072+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:27:21.884072+0000","last_undegraded":"2024-09-17T15:27:21.884072+0000","last_fullsized":"2024-09-17T15:27:21.884072+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":6325,"log_dups_size":0,"ondisk_log_size":6325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25907200,"num_objects":6325,"num_object_clones":0,"num_object_copies":12650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6325,"num_write_kb":25300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'6367","reported_seq":6491,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:25.422455+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:27:25.422455+0000","last_peered":"2024-09-17T15:27:25.422455+0000","last_clean":"2024-09-17T15:27:25.422455+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:27:25.422455+0000","last_undegraded":"2024-09-17T15:27:25.422455+0000","last_fullsized":"2024-09-17T15:27:25.422455+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":6367,"log_dups_size":0,"ondisk_log_size":6367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26051043,"num_objects":6361,"num_object_clones":0,"num_object_copies":12722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6361,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3349,"num_write_kb":13396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'6541","reported_seq":6670,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:25.433303+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:27:25.433303+0000","last_peered":"2024-09-17T15:27:25.433303+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:27:25.433303+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":6541,"log_dups_size":0,"ondisk_log_size":6541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26771939,"num_objects":6537,"num_object_clones":0,"num_object_copies":13074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6537,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3040,"num_write_kb":12160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2267},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'6490","reported_seq":6624,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:25.401239+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:27:25.401239+0000","last_peered":"2024-09-17T15:27:25.401239+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:27:25.401239+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":6490,"log_dups_size":0,"ondisk_log_size":6490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26559430,"num_objects":6486,"num_object_clones":0,"num_object_copies":12972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6486,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3259,"num_write_kb":13036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2312},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'6405","reported_seq":6528,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:25.397872+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:27:25.397872+0000","last_peered":"2024-09-17T15:27:25.397872+0000","last_clean":"2024-09-17T15:27:25.397872+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:27:25.397872+0000","last_undegraded":"2024-09-17T15:27:25.397872+0000","last_fullsized":"2024-09-17T15:27:25.397872+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":6405,"log_dups_size":0,"ondisk_log_size":6405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26226688,"num_objects":6403,"num_object_clones":0,"num_object_copies":12806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6403,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3359,"num_write_kb":13436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'6255","reported_seq":9780,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:27:21.882572+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:27:21.882572+0000","last_peered":"2024-09-17T15:27:21.882572+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:27:21.882572+0000","last_undegraded":"2024-09-17T15:27:21.882572+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":6255,"log_dups_size":0,"ondisk_log_size":6255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25592291,"num_objects":6249,"num_object_clones":0,"num_object_copies":12498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2370,"num_objects_unfound":0,"num_objects_dirty":6249,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4255,"num_write_kb":17020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1701,"num_bytes_recovered":6967296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2370},{"shards":"2","objects":3879}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233552+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233552+0000","last_peered":"2024-09-17T15:26:22.233552+0000","last_clean":"2024-09-17T15:26:22.233552+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233552+0000","last_undegraded":"2024-09-17T15:26:22.233552+0000","last_fullsized":"2024-09-17T15:26:22.233552+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'6275","reported_seq":6400,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:25.431189+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:27:25.431189+0000","last_peered":"2024-09-17T15:27:25.431189+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:27:25.431189+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":6275,"log_dups_size":0,"ondisk_log_size":6275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25669632,"num_objects":6267,"num_object_clones":0,"num_object_copies":12534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2871,"num_write_kb":11484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2257},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":98,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412941+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412941+0000","last_peered":"2024-09-17T15:26:22.412941+0000","last_clean":"2024-09-17T15:26:22.412941+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412941+0000","last_undegraded":"2024-09-17T15:26:22.412941+0000","last_fullsized":"2024-09-17T15:26:22.412941+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'6237","reported_seq":6352,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:21.767591+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:27:21.767591+0000","last_peered":"2024-09-17T15:27:21.767591+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:27:21.767591+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":6237,"log_dups_size":0,"ondisk_log_size":6237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25526755,"num_objects":6233,"num_object_clones":0,"num_object_copies":12466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6233,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4132,"num_write_kb":16528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":2156}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'6279","reported_seq":6392,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:21.811345+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:27:21.811345+0000","last_peered":"2024-09-17T15:27:21.811345+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:27:21.811345+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":6279,"log_dups_size":0,"ondisk_log_size":6279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25694208,"num_objects":6273,"num_object_clones":0,"num_object_copies":12546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6273,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4041,"num_write_kb":16164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":2136}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'6488","reported_seq":6618,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:25.381013+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:27:25.381013+0000","last_peered":"2024-09-17T15:27:25.381013+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:27:25.381013+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":6488,"log_dups_size":0,"ondisk_log_size":6488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26566656,"num_objects":6486,"num_object_clones":0,"num_object_copies":12972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6486,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3036,"num_write_kb":12144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2348},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'6405","reported_seq":6540,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:25.363755+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:27:25.363755+0000","last_peered":"2024-09-17T15:27:25.363755+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:27:25.363755+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":6405,"log_dups_size":0,"ondisk_log_size":6405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26202112,"num_objects":6397,"num_object_clones":0,"num_object_copies":12794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2692,"num_write_kb":10768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2240},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229013+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229013+0000","last_peered":"2024-09-17T15:26:22.229013+0000","last_clean":"2024-09-17T15:26:22.229013+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229013+0000","last_undegraded":"2024-09-17T15:26:22.229013+0000","last_fullsized":"2024-09-17T15:26:22.229013+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'6433","reported_seq":6560,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:25.292832+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:27:25.292832+0000","last_peered":"2024-09-17T15:27:25.292832+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:27:25.292832+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":6433,"log_dups_size":0,"ondisk_log_size":6433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26333184,"num_objects":6429,"num_object_clones":0,"num_object_copies":12858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3220,"num_write_kb":12880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2366},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'6367","reported_seq":6504,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:25.430192+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:27:25.430192+0000","last_peered":"2024-09-17T15:27:25.430192+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:27:25.430192+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":6367,"log_dups_size":0,"ondisk_log_size":6367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26031046,"num_objects":6357,"num_object_clones":0,"num_object_copies":12714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2809,"num_write_kb":11236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2248},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":416629470,"num_objects":101725,"num_object_clones":0,"num_object_copies":203450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":2370,"num_objects_unfound":0,"num_objects_dirty":101725,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":61832,"num_write_kb":247328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1701,"num_bytes_recovered":6967296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":841674752,"data_stored":841519393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":101797,"ondisk_log_size":101797,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051612,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508720,"kb_used_data":332640,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93863120,"statfs":{"total":96636764160,"available":96115834880,"internally_reserved":0,"allocated":340623360,"data_stored":340357187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":180285100},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,0,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724558,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":240824,"kb_used_data":114728,"kb_used_omap":4,"kb_used_meta":126075,"kb_avail":94131016,"statfs":{"total":96636764160,"available":96390160384,"internally_reserved":0,"allocated":117481472,"data_stored":117164253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":129101483},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247841,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":602776,"kb_used_data":372616,"kb_used_omap":4,"kb_used_meta":230139,"kb_avail":93769064,"statfs":{"total":96636764160,"available":96019521536,"internally_reserved":0,"allocated":381558784,"data_stored":381288534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":235663016},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,0,1,7,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":382443520,"data_stored":382407390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":118730752,"data_stored":118647653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":340500480,"data_stored":340464350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:27:26.418 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:27:26.418 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'6363", 'reported_seq': 6436, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:25.415098+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:27:25.415098+0000', 'last_peered': '2024-09-17T15:27:25.415098+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:27:25.415098+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 6363, 'log_dups_size': 0, 'ondisk_log_size': 6363, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26062848, 'num_objects': 6363, 'num_object_clones': 0, 'num_object_copies': 12726, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4070, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6363, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6363, 'num_write_kb': 25452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2293}, {'shards': '0,2', 'objects': 4070}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:26.418 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:27:26.419 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'6301", 'reported_seq': 6421, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:21.892375+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:27:21.892375+0000', 'last_peered': '2024-09-17T15:27:21.892375+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:27:21.892375+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 6301, 'log_dups_size': 0, 'ondisk_log_size': 6301, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25792512, 'num_objects': 6297, 'num_object_clones': 0, 'num_object_copies': 12594, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4169, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6297, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4458, 'num_write_kb': 17832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4168}, {'shards': '2', 'objects': 2129}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:27:26.419 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:27:26.419 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'6541", 'reported_seq': 6670, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:25.433303+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:27:25.433303+0000', 'last_peered': '2024-09-17T15:27:25.433303+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:27:25.433303+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 6541, 'log_dups_size': 0, 'ondisk_log_size': 6541, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26771939, 'num_objects': 6537, 'num_object_clones': 0, 'num_object_copies': 13074, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4271, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6537, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3040, 'num_write_kb': 12160, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2267}, {'shards': '0,2', 'objects': 4268}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:26.419 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:27:26.419 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'6490", 'reported_seq': 6624, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:25.401239+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:27:25.401239+0000', 'last_peered': '2024-09-17T15:27:25.401239+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:27:25.401239+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 6490, 'log_dups_size': 0, 'ondisk_log_size': 6490, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26559430, 'num_objects': 6486, 'num_object_clones': 0, 'num_object_copies': 12972, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4174, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6486, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 3259, 'num_write_kb': 13036, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2312}, {'shards': '0,2', 'objects': 4174}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:26.419 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-17T15:27:26.419 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "41'6255", 'reported_seq': 9780, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:27:21.882572+0000', 'last_change': '2024-09-17T15:26:22.588242+0000', 'last_active': '2024-09-17T15:27:21.882572+0000', 'last_peered': '2024-09-17T15:27:21.882572+0000', 'last_clean': '2024-09-17T15:26:20.155646+0000', 'last_became_active': '2024-09-17T15:26:22.557742+0000', 'last_became_peered': '2024-09-17T15:26:22.557742+0000', 'last_unstale': '2024-09-17T15:27:21.882572+0000', 'last_undegraded': '2024-09-17T15:27:21.882572+0000', 'last_fullsized': '2024-09-17T15:26:22.188850+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2016", 'last_scrub_stamp': '2024-09-17T15:25:53.959011+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:53.959011+0000', 'objects_scrubbed': 1994, 'log_size': 6255, 'log_dups_size': 0, 'ondisk_log_size': 6255, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 568, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25592291, 'num_objects': 6249, 'num_object_clones': 0, 'num_object_copies': 12498, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2370, 'num_objects_unfound': 0, 'num_objects_dirty': 6249, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4255, 'num_write_kb': 17020, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1701, 'num_bytes_recovered': 6967296, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2370}, {'shards': '2', 'objects': 3879}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:27:26.419 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:27:26.419 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'6275", 'reported_seq': 6400, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:25.431189+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:27:25.431189+0000', 'last_peered': '2024-09-17T15:27:25.431189+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:27:25.431189+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 6275, 'log_dups_size': 0, 'ondisk_log_size': 6275, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25669632, 'num_objects': 6267, 'num_object_clones': 0, 'num_object_copies': 12534, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4012, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6267, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2871, 'num_write_kb': 11484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2257}, {'shards': '0,2', 'objects': 4010}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:26.419 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:27:26.420 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'6237", 'reported_seq': 6352, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:21.767591+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:27:21.767591+0000', 'last_peered': '2024-09-17T15:27:21.767591+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:27:21.767591+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 6237, 'log_dups_size': 0, 'ondisk_log_size': 6237, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25526755, 'num_objects': 6233, 'num_object_clones': 0, 'num_object_copies': 12466, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4078, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6233, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4132, 'num_write_kb': 16528, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4077}, {'shards': '2', 'objects': 2156}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:27:26.420 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:27:26.420 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'6279", 'reported_seq': 6392, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:21.811345+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:27:21.811345+0000', 'last_peered': '2024-09-17T15:27:21.811345+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:27:21.811345+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 6279, 'log_dups_size': 0, 'ondisk_log_size': 6279, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25694208, 'num_objects': 6273, 'num_object_clones': 0, 'num_object_copies': 12546, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6273, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4041, 'num_write_kb': 16164, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4137}, {'shards': '2', 'objects': 2136}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:27:26.420 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:27:26.420 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'6488", 'reported_seq': 6618, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:25.381013+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:27:25.381013+0000', 'last_peered': '2024-09-17T15:27:25.381013+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:27:25.381013+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 6488, 'log_dups_size': 0, 'ondisk_log_size': 6488, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26566656, 'num_objects': 6486, 'num_object_clones': 0, 'num_object_copies': 12972, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6486, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3036, 'num_write_kb': 12144, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2348}, {'shards': '0,2', 'objects': 4138}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:26.420 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:27:26.420 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'6405", 'reported_seq': 6540, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:25.363755+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:27:25.363755+0000', 'last_peered': '2024-09-17T15:27:25.363755+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:27:25.363755+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 6405, 'log_dups_size': 0, 'ondisk_log_size': 6405, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26202112, 'num_objects': 6397, 'num_object_clones': 0, 'num_object_copies': 12794, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4160, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6397, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2692, 'num_write_kb': 10768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2240}, {'shards': '0,2', 'objects': 4157}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:26.420 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:27:26.420 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'6433", 'reported_seq': 6560, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:25.292832+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:27:25.292832+0000', 'last_peered': '2024-09-17T15:27:25.292832+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:27:25.292832+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 6433, 'log_dups_size': 0, 'ondisk_log_size': 6433, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26333184, 'num_objects': 6429, 'num_object_clones': 0, 'num_object_copies': 12858, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4065, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6429, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3220, 'num_write_kb': 12880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2366}, {'shards': '0,2', 'objects': 4063}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:26.421 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:27:26.421 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'6367", 'reported_seq': 6504, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:25.430192+0000', 'last_change': '2024-09-17T15:26:22.654521+0000', 'last_active': '2024-09-17T15:27:25.430192+0000', 'last_peered': '2024-09-17T15:27:25.430192+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:27:25.430192+0000', 'last_undegraded': '2024-09-17T15:26:22.307795+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 6367, 'log_dups_size': 0, 'ondisk_log_size': 6367, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26031046, 'num_objects': 6357, 'num_object_clones': 0, 'num_object_copies': 12714, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4111, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6357, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2809, 'num_write_kb': 11236, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2248}, {'shards': '0,2', 'objects': 4109}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:26.421 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:27:27.118 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:27:27.118 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45388/203454 objects degraded (22.309%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":184,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":101727,"data_bytes":416629489,"bytes_used":1384775680,"bytes_avail":288525516800,"bytes_total":289910292480,"degraded_objects":45388,"degraded_total":203454,"degraded_ratio":0.22308728262899721,"misplaced_objects":2370,"misplaced_total":203454,"misplaced_ratio":0.011648824795776933,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":103385,"recovering_keys_per_sec":0,"num_objects_recovered":303,"num_bytes_recovered":1241088,"num_keys_recovered":0,"write_bytes_sec":2519467,"read_op_per_sec":0,"write_op_per_sec":615},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T15:27:19.616376+0000","services":{}},"progress_events":{}} 2024-09-17T15:27:27.147 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:27:27.147 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:27:27.308 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 108 20 103382 103362 3.73763 1.98438 0.0226875 0.0208936 2024-09-17T15:27:27.694 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:27:27.694 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:27:27.727 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":126,"stamp":"2024-09-17T15:27:27.619252+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":420782833,"num_objects":102741,"num_object_clones":0,"num_object_copies":205482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":2192,"num_objects_unfound":0,"num_objects_dirty":102741,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":62848,"num_write_kb":251386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1880,"num_bytes_recovered":7696403,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102814,"ondisk_log_size":102814,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1361836,"kb_used_data":825340,"kb_used_omap":13,"kb_used_meta":536434,"kb_avail":281753684,"statfs":{"total":289910292480,"available":288515772416,"internally_reserved":0,"allocated":845148160,"data_stored":844294518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":549309439},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,4,2,5,18,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27037696,"num_objects":6601,"num_object_clones":0,"num_object_copies":13202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-481,"num_objects_unfound":0,"num_objects_dirty":6601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6601,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":481,"num_bytes_recovered":1970176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004257"},"pg_stats":[{"pgid":"2.f","version":"41'6363","reported_seq":6436,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:25.415098+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:27:25.415098+0000","last_peered":"2024-09-17T15:27:25.415098+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:27:25.415098+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":6363,"log_dups_size":0,"ondisk_log_size":6363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26062848,"num_objects":6363,"num_object_clones":0,"num_object_copies":12726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6363,"num_write_kb":25452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2293},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'6415","reported_seq":6512,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:26.895792+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:27:26.895792+0000","last_peered":"2024-09-17T15:27:26.895792+0000","last_clean":"2024-09-17T15:27:26.895792+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:27:26.895792+0000","last_undegraded":"2024-09-17T15:27:26.895792+0000","last_fullsized":"2024-09-17T15:27:26.895792+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":6415,"log_dups_size":0,"ondisk_log_size":6415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26252230,"num_objects":6411,"num_object_clones":0,"num_object_copies":12822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6411,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4772,"num_write_kb":19088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'6478","reported_seq":6598,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:26.924269+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:27:26.924269+0000","last_peered":"2024-09-17T15:27:26.924269+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:27:26.924269+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":6478,"log_dups_size":0,"ondisk_log_size":6478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26517504,"num_objects":6474,"num_object_clones":0,"num_object_copies":12948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4635,"num_write_kb":18540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":2306}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'6494","reported_seq":6558,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:26.833290+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:27:26.833290+0000","last_peered":"2024-09-17T15:27:26.833290+0000","last_clean":"2024-09-17T15:27:26.833290+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:27:26.833290+0000","last_undegraded":"2024-09-17T15:27:26.833290+0000","last_fullsized":"2024-09-17T15:27:26.833290+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":6494,"log_dups_size":0,"ondisk_log_size":6494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26599424,"num_objects":6494,"num_object_clones":0,"num_object_copies":12988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6494,"num_write_kb":25976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'6367","reported_seq":6491,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:25.422455+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:27:25.422455+0000","last_peered":"2024-09-17T15:27:25.422455+0000","last_clean":"2024-09-17T15:27:25.422455+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:27:25.422455+0000","last_undegraded":"2024-09-17T15:27:25.422455+0000","last_fullsized":"2024-09-17T15:27:25.422455+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":6367,"log_dups_size":0,"ondisk_log_size":6367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26051043,"num_objects":6361,"num_object_clones":0,"num_object_copies":12722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6361,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3349,"num_write_kb":13396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'6541","reported_seq":6670,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:25.433303+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:27:25.433303+0000","last_peered":"2024-09-17T15:27:25.433303+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:27:25.433303+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":6541,"log_dups_size":0,"ondisk_log_size":6541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26771939,"num_objects":6537,"num_object_clones":0,"num_object_copies":13074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6537,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3040,"num_write_kb":12160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2267},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'6490","reported_seq":6624,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:25.401239+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:27:25.401239+0000","last_peered":"2024-09-17T15:27:25.401239+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:27:25.401239+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":6490,"log_dups_size":0,"ondisk_log_size":6490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26559430,"num_objects":6486,"num_object_clones":0,"num_object_copies":12972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6486,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3259,"num_write_kb":13036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2312},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'6405","reported_seq":6528,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:25.397872+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:27:25.397872+0000","last_peered":"2024-09-17T15:27:25.397872+0000","last_clean":"2024-09-17T15:27:25.397872+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:27:25.397872+0000","last_undegraded":"2024-09-17T15:27:25.397872+0000","last_fullsized":"2024-09-17T15:27:25.397872+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":6405,"log_dups_size":0,"ondisk_log_size":6405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26226688,"num_objects":6403,"num_object_clones":0,"num_object_copies":12806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6403,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3359,"num_write_kb":13436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'6430","reported_seq":10311,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:27:26.924946+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:27:26.924946+0000","last_peered":"2024-09-17T15:27:26.924946+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:27:26.924946+0000","last_undegraded":"2024-09-17T15:27:26.924946+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":6430,"log_dups_size":0,"ondisk_log_size":6430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26309091,"num_objects":6424,"num_object_clones":0,"num_object_copies":12848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2192,"num_objects_unfound":0,"num_objects_dirty":6424,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4430,"num_write_kb":17720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1879,"num_bytes_recovered":7696384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2192},{"shards":"2","objects":4232}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233600+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233600+0000","last_peered":"2024-09-17T15:26:22.233600+0000","last_clean":"2024-09-17T15:26:22.233600+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233600+0000","last_undegraded":"2024-09-17T15:26:22.233600+0000","last_fullsized":"2024-09-17T15:26:22.233600+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'6275","reported_seq":6400,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:25.431189+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:27:25.431189+0000","last_peered":"2024-09-17T15:27:25.431189+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:27:25.431189+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":6275,"log_dups_size":0,"ondisk_log_size":6275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25669632,"num_objects":6267,"num_object_clones":0,"num_object_copies":12534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2871,"num_write_kb":11484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2257},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":98,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412941+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412941+0000","last_peered":"2024-09-17T15:26:22.412941+0000","last_clean":"2024-09-17T15:26:22.412941+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412941+0000","last_undegraded":"2024-09-17T15:26:22.412941+0000","last_fullsized":"2024-09-17T15:26:22.412941+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'6412","reported_seq":6528,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:26.912692+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:27:26.912692+0000","last_peered":"2024-09-17T15:27:26.912692+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:27:26.912692+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":6412,"log_dups_size":0,"ondisk_log_size":6412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26243555,"num_objects":6408,"num_object_clones":0,"num_object_copies":12816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6408,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4307,"num_write_kb":17228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":2331}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'6448","reported_seq":6562,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:26.832869+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:27:26.832869+0000","last_peered":"2024-09-17T15:27:26.832869+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:27:26.832869+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":6448,"log_dups_size":0,"ondisk_log_size":6448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26386432,"num_objects":6442,"num_object_clones":0,"num_object_copies":12884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4210,"num_write_kb":16840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":2305}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'6488","reported_seq":6618,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:25.381013+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:27:25.381013+0000","last_peered":"2024-09-17T15:27:25.381013+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:27:25.381013+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":6488,"log_dups_size":0,"ondisk_log_size":6488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26566656,"num_objects":6486,"num_object_clones":0,"num_object_copies":12972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6486,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3036,"num_write_kb":12144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2348},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'6405","reported_seq":6540,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:25.363755+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:27:25.363755+0000","last_peered":"2024-09-17T15:27:25.363755+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:27:25.363755+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":6405,"log_dups_size":0,"ondisk_log_size":6405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26202112,"num_objects":6397,"num_object_clones":0,"num_object_copies":12794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2692,"num_write_kb":10768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2240},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229061+0000","last_peered":"2024-09-17T15:26:22.229061+0000","last_clean":"2024-09-17T15:26:22.229061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229061+0000","last_undegraded":"2024-09-17T15:26:22.229061+0000","last_fullsized":"2024-09-17T15:26:22.229061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'6433","reported_seq":6560,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:25.292832+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:27:25.292832+0000","last_peered":"2024-09-17T15:27:25.292832+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:27:25.292832+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":6433,"log_dups_size":0,"ondisk_log_size":6433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26333184,"num_objects":6429,"num_object_clones":0,"num_object_copies":12858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3220,"num_write_kb":12880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2366},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'6367","reported_seq":6504,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:25.430192+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:27:25.430192+0000","last_peered":"2024-09-17T15:27:25.430192+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:27:25.430192+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":6367,"log_dups_size":0,"ondisk_log_size":6367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26031046,"num_objects":6357,"num_object_clones":0,"num_object_copies":12714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2809,"num_write_kb":11236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2248},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":420782814,"num_objects":102739,"num_object_clones":0,"num_object_copies":205478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":2192,"num_objects_unfound":0,"num_objects_dirty":102739,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":62846,"num_write_kb":251384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1879,"num_bytes_recovered":7696384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":847286272,"data_stored":847130913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102811,"ondisk_log_size":102811,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051613,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518236,"kb_used_data":337996,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93853604,"statfs":{"total":96636764160,"available":96106090496,"internally_reserved":0,"allocated":346107904,"data_stored":345841731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":184544940},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,7,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724558,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":240824,"kb_used_data":114728,"kb_used_omap":4,"kb_used_meta":126075,"kb_avail":94131016,"statfs":{"total":96636764160,"available":96390160384,"internally_reserved":0,"allocated":117481472,"data_stored":117164253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":129101483},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247841,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":602776,"kb_used_data":372616,"kb_used_omap":4,"kb_used_meta":230139,"kb_avail":93769064,"statfs":{"total":96636764160,"available":96019521536,"internally_reserved":0,"allocated":381558784,"data_stored":381288534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":235663016},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,0,1,7,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":382443520,"data_stored":382407390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":118730752,"data_stored":118647653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":346112000,"data_stored":346075870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:27:28.308 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 109 20 103924 103904 3.72277 2.11719 0.0180771 0.0209775 2024-09-17T15:27:29.308 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 110 20 104495 104475 3.7092 2.23047 0.0165808 0.0210551 2024-09-17T15:27:30.308 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 111 20 105074 105054 3.69616 2.26172 0.0405615 0.0211291 2024-09-17T15:27:30.730 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:27:31.281 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:27:31.282 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:27:31.308 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 112 20 105597 105577 3.6814 2.04297 0.0299362 0.0212149 2024-09-17T15:27:31.311 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":127,"stamp":"2024-09-17T15:27:29.619902+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":420782833,"num_objects":102741,"num_object_clones":0,"num_object_copies":205482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":2192,"num_objects_unfound":0,"num_objects_dirty":102741,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":62848,"num_write_kb":251386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1880,"num_bytes_recovered":7696403,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102814,"ondisk_log_size":102814,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1361836,"kb_used_data":825340,"kb_used_omap":13,"kb_used_meta":536434,"kb_avail":281753684,"statfs":{"total":289910292480,"available":288515772416,"internally_reserved":0,"allocated":845148160,"data_stored":844294518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":549309439},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,4,2,5,18,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22708224,"num_objects":5544,"num_object_clones":0,"num_object_copies":11088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-329,"num_objects_unfound":0,"num_objects_dirty":5544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5544,"num_write_kb":22176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":329,"num_bytes_recovered":1347584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004129"},"pg_stats":[{"pgid":"2.f","version":"41'6363","reported_seq":6436,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:25.415098+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:27:25.415098+0000","last_peered":"2024-09-17T15:27:25.415098+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:27:25.415098+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":6363,"log_dups_size":0,"ondisk_log_size":6363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26062848,"num_objects":6363,"num_object_clones":0,"num_object_copies":12726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6363,"num_write_kb":25452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2293},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'6415","reported_seq":6512,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:26.895792+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:27:26.895792+0000","last_peered":"2024-09-17T15:27:26.895792+0000","last_clean":"2024-09-17T15:27:26.895792+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:27:26.895792+0000","last_undegraded":"2024-09-17T15:27:26.895792+0000","last_fullsized":"2024-09-17T15:27:26.895792+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":6415,"log_dups_size":0,"ondisk_log_size":6415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26252230,"num_objects":6411,"num_object_clones":0,"num_object_copies":12822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6411,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4772,"num_write_kb":19088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'6478","reported_seq":6598,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:26.924269+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:27:26.924269+0000","last_peered":"2024-09-17T15:27:26.924269+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:27:26.924269+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":6478,"log_dups_size":0,"ondisk_log_size":6478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26517504,"num_objects":6474,"num_object_clones":0,"num_object_copies":12948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4635,"num_write_kb":18540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":2306}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'6494","reported_seq":6558,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:26.833290+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:27:26.833290+0000","last_peered":"2024-09-17T15:27:26.833290+0000","last_clean":"2024-09-17T15:27:26.833290+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:27:26.833290+0000","last_undegraded":"2024-09-17T15:27:26.833290+0000","last_fullsized":"2024-09-17T15:27:26.833290+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":6494,"log_dups_size":0,"ondisk_log_size":6494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26599424,"num_objects":6494,"num_object_clones":0,"num_object_copies":12988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6494,"num_write_kb":25976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'6367","reported_seq":6491,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:25.422455+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:27:25.422455+0000","last_peered":"2024-09-17T15:27:25.422455+0000","last_clean":"2024-09-17T15:27:25.422455+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:27:25.422455+0000","last_undegraded":"2024-09-17T15:27:25.422455+0000","last_fullsized":"2024-09-17T15:27:25.422455+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":6367,"log_dups_size":0,"ondisk_log_size":6367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26051043,"num_objects":6361,"num_object_clones":0,"num_object_copies":12722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6361,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3349,"num_write_kb":13396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'6541","reported_seq":6670,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:25.433303+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:27:25.433303+0000","last_peered":"2024-09-17T15:27:25.433303+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:27:25.433303+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":6541,"log_dups_size":0,"ondisk_log_size":6541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26771939,"num_objects":6537,"num_object_clones":0,"num_object_copies":13074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6537,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3040,"num_write_kb":12160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2267},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'6490","reported_seq":6624,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:25.401239+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:27:25.401239+0000","last_peered":"2024-09-17T15:27:25.401239+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:27:25.401239+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":6490,"log_dups_size":0,"ondisk_log_size":6490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26559430,"num_objects":6486,"num_object_clones":0,"num_object_copies":12972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6486,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3259,"num_write_kb":13036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2312},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'6405","reported_seq":6528,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:25.397872+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:27:25.397872+0000","last_peered":"2024-09-17T15:27:25.397872+0000","last_clean":"2024-09-17T15:27:25.397872+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:27:25.397872+0000","last_undegraded":"2024-09-17T15:27:25.397872+0000","last_fullsized":"2024-09-17T15:27:25.397872+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":6405,"log_dups_size":0,"ondisk_log_size":6405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26226688,"num_objects":6403,"num_object_clones":0,"num_object_copies":12806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6403,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3359,"num_write_kb":13436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'6430","reported_seq":10311,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:27:26.924946+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:27:26.924946+0000","last_peered":"2024-09-17T15:27:26.924946+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:27:26.924946+0000","last_undegraded":"2024-09-17T15:27:26.924946+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":6430,"log_dups_size":0,"ondisk_log_size":6430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26309091,"num_objects":6424,"num_object_clones":0,"num_object_copies":12848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2192,"num_objects_unfound":0,"num_objects_dirty":6424,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4430,"num_write_kb":17720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1879,"num_bytes_recovered":7696384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2192},{"shards":"2","objects":4232}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233600+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233600+0000","last_peered":"2024-09-17T15:26:22.233600+0000","last_clean":"2024-09-17T15:26:22.233600+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233600+0000","last_undegraded":"2024-09-17T15:26:22.233600+0000","last_fullsized":"2024-09-17T15:26:22.233600+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'6275","reported_seq":6400,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:25.431189+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:27:25.431189+0000","last_peered":"2024-09-17T15:27:25.431189+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:27:25.431189+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":6275,"log_dups_size":0,"ondisk_log_size":6275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25669632,"num_objects":6267,"num_object_clones":0,"num_object_copies":12534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2871,"num_write_kb":11484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2257},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":98,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412941+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412941+0000","last_peered":"2024-09-17T15:26:22.412941+0000","last_clean":"2024-09-17T15:26:22.412941+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412941+0000","last_undegraded":"2024-09-17T15:26:22.412941+0000","last_fullsized":"2024-09-17T15:26:22.412941+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'6412","reported_seq":6528,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:26.912692+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:27:26.912692+0000","last_peered":"2024-09-17T15:27:26.912692+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:27:26.912692+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":6412,"log_dups_size":0,"ondisk_log_size":6412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26243555,"num_objects":6408,"num_object_clones":0,"num_object_copies":12816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6408,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4307,"num_write_kb":17228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":2331}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'6448","reported_seq":6562,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:26.832869+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:27:26.832869+0000","last_peered":"2024-09-17T15:27:26.832869+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:27:26.832869+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":6448,"log_dups_size":0,"ondisk_log_size":6448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26386432,"num_objects":6442,"num_object_clones":0,"num_object_copies":12884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4210,"num_write_kb":16840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":2305}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'6488","reported_seq":6618,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:25.381013+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:27:25.381013+0000","last_peered":"2024-09-17T15:27:25.381013+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:27:25.381013+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":6488,"log_dups_size":0,"ondisk_log_size":6488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26566656,"num_objects":6486,"num_object_clones":0,"num_object_copies":12972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6486,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3036,"num_write_kb":12144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2348},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'6405","reported_seq":6540,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:25.363755+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:27:25.363755+0000","last_peered":"2024-09-17T15:27:25.363755+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:27:25.363755+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":6405,"log_dups_size":0,"ondisk_log_size":6405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26202112,"num_objects":6397,"num_object_clones":0,"num_object_copies":12794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2692,"num_write_kb":10768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2240},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229061+0000","last_peered":"2024-09-17T15:26:22.229061+0000","last_clean":"2024-09-17T15:26:22.229061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229061+0000","last_undegraded":"2024-09-17T15:26:22.229061+0000","last_fullsized":"2024-09-17T15:26:22.229061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'6433","reported_seq":6560,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:25.292832+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:27:25.292832+0000","last_peered":"2024-09-17T15:27:25.292832+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:27:25.292832+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":6433,"log_dups_size":0,"ondisk_log_size":6433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26333184,"num_objects":6429,"num_object_clones":0,"num_object_copies":12858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3220,"num_write_kb":12880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2366},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'6367","reported_seq":6504,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:25.430192+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:27:25.430192+0000","last_peered":"2024-09-17T15:27:25.430192+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:27:25.430192+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":6367,"log_dups_size":0,"ondisk_log_size":6367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26031046,"num_objects":6357,"num_object_clones":0,"num_object_copies":12714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2809,"num_write_kb":11236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2248},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":420782814,"num_objects":102739,"num_object_clones":0,"num_object_copies":205478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":2192,"num_objects_unfound":0,"num_objects_dirty":102739,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":62846,"num_write_kb":251384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1879,"num_bytes_recovered":7696384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":847286272,"data_stored":847130913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102811,"ondisk_log_size":102811,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051613,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518236,"kb_used_data":337996,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93853604,"statfs":{"total":96636764160,"available":96106090496,"internally_reserved":0,"allocated":346107904,"data_stored":345841731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":184544940},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,7,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724558,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":240824,"kb_used_data":114728,"kb_used_omap":4,"kb_used_meta":126075,"kb_avail":94131016,"statfs":{"total":96636764160,"available":96390160384,"internally_reserved":0,"allocated":117481472,"data_stored":117164253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":129101483},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247841,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":602776,"kb_used_data":372616,"kb_used_omap":4,"kb_used_meta":230139,"kb_avail":93769064,"statfs":{"total":96636764160,"available":96019521536,"internally_reserved":0,"allocated":381558784,"data_stored":381288534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":235663016},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,0,1,7,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":382443520,"data_stored":382407390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":118730752,"data_stored":118647653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":346112000,"data_stored":346075870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:27:31.313 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:27:31.707 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:27:31.708 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:27:31.727 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":128,"stamp":"2024-09-17T15:27:31.620487+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":427602673,"num_objects":104406,"num_object_clones":0,"num_object_copies":208812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":2192,"num_objects_unfound":0,"num_objects_dirty":104406,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":64513,"num_write_kb":258046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1880,"num_bytes_recovered":7696403,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104479,"ondisk_log_size":104479,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1375492,"kb_used_data":843156,"kb_used_omap":13,"kb_used_meta":532274,"kb_avail":281740028,"statfs":{"total":289910292480,"available":288501788672,"internally_reserved":0,"allocated":863391744,"data_stored":862541715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":545049599},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,7,9,15,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29528064,"num_objects":7209,"num_object_clones":0,"num_object_copies":14418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-329,"num_objects_unfound":0,"num_objects_dirty":7209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7209,"num_write_kb":28836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":329,"num_bytes_recovered":1347584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004145"},"pg_stats":[{"pgid":"2.f","version":"41'6528","reported_seq":6600,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:30.436919+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:27:30.436919+0000","last_peered":"2024-09-17T15:27:30.436919+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:27:30.436919+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":6528,"log_dups_size":0,"ondisk_log_size":6528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26738688,"num_objects":6528,"num_object_clones":0,"num_object_copies":13056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6528,"num_write_kb":26112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2458},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'6415","reported_seq":6512,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:26.895792+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:27:26.895792+0000","last_peered":"2024-09-17T15:27:26.895792+0000","last_clean":"2024-09-17T15:27:26.895792+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:27:26.895792+0000","last_undegraded":"2024-09-17T15:27:26.895792+0000","last_fullsized":"2024-09-17T15:27:26.895792+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":6415,"log_dups_size":0,"ondisk_log_size":6415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26252230,"num_objects":6411,"num_object_clones":0,"num_object_copies":12822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6411,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4772,"num_write_kb":19088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'6478","reported_seq":6598,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:26.924269+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:27:26.924269+0000","last_peered":"2024-09-17T15:27:26.924269+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:27:26.924269+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":6478,"log_dups_size":0,"ondisk_log_size":6478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26517504,"num_objects":6474,"num_object_clones":0,"num_object_copies":12948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4635,"num_write_kb":18540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":2306}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'6494","reported_seq":6558,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:26.833290+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:27:26.833290+0000","last_peered":"2024-09-17T15:27:26.833290+0000","last_clean":"2024-09-17T15:27:26.833290+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:27:26.833290+0000","last_undegraded":"2024-09-17T15:27:26.833290+0000","last_fullsized":"2024-09-17T15:27:26.833290+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":6494,"log_dups_size":0,"ondisk_log_size":6494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26599424,"num_objects":6494,"num_object_clones":0,"num_object_copies":12988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6494,"num_write_kb":25976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'6531","reported_seq":6655,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:30.448346+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:27:30.448346+0000","last_peered":"2024-09-17T15:27:30.448346+0000","last_clean":"2024-09-17T15:27:30.448346+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:27:30.448346+0000","last_undegraded":"2024-09-17T15:27:30.448346+0000","last_fullsized":"2024-09-17T15:27:30.448346+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":6531,"log_dups_size":0,"ondisk_log_size":6531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26722787,"num_objects":6525,"num_object_clones":0,"num_object_copies":13050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6525,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3513,"num_write_kb":14052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'6710","reported_seq":6839,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:30.496638+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:27:30.496638+0000","last_peered":"2024-09-17T15:27:30.496638+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:27:30.496638+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":6710,"log_dups_size":0,"ondisk_log_size":6710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27464163,"num_objects":6706,"num_object_clones":0,"num_object_copies":13412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6706,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3209,"num_write_kb":12836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2436},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'6644","reported_seq":6778,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:30.452411+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:27:30.452411+0000","last_peered":"2024-09-17T15:27:30.452411+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:27:30.452411+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":6644,"log_dups_size":0,"ondisk_log_size":6644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27190214,"num_objects":6640,"num_object_clones":0,"num_object_copies":13280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6640,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3413,"num_write_kb":13652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2466},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'6568","reported_seq":6691,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:30.370158+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:27:30.370158+0000","last_peered":"2024-09-17T15:27:30.370158+0000","last_clean":"2024-09-17T15:27:30.370158+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:27:30.370158+0000","last_undegraded":"2024-09-17T15:27:30.370158+0000","last_fullsized":"2024-09-17T15:27:30.370158+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":6568,"log_dups_size":0,"ondisk_log_size":6568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26894336,"num_objects":6566,"num_object_clones":0,"num_object_copies":13132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3522,"num_write_kb":14088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'6430","reported_seq":10311,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:27:26.924946+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:27:26.924946+0000","last_peered":"2024-09-17T15:27:26.924946+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:27:26.924946+0000","last_undegraded":"2024-09-17T15:27:26.924946+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":6430,"log_dups_size":0,"ondisk_log_size":6430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26309091,"num_objects":6424,"num_object_clones":0,"num_object_copies":12848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2192,"num_objects_unfound":0,"num_objects_dirty":6424,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4430,"num_write_kb":17720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1879,"num_bytes_recovered":7696384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2192},{"shards":"2","objects":4232}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233600+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233600+0000","last_peered":"2024-09-17T15:26:22.233600+0000","last_clean":"2024-09-17T15:26:22.233600+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233600+0000","last_undegraded":"2024-09-17T15:26:22.233600+0000","last_fullsized":"2024-09-17T15:26:22.233600+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'6437","reported_seq":6562,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:30.464035+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:27:30.464035+0000","last_peered":"2024-09-17T15:27:30.464035+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:27:30.464035+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":6437,"log_dups_size":0,"ondisk_log_size":6437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26333184,"num_objects":6429,"num_object_clones":0,"num_object_copies":12858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3033,"num_write_kb":12132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2419},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":98,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412941+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412941+0000","last_peered":"2024-09-17T15:26:22.412941+0000","last_clean":"2024-09-17T15:26:22.412941+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412941+0000","last_undegraded":"2024-09-17T15:26:22.412941+0000","last_fullsized":"2024-09-17T15:26:22.412941+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'6412","reported_seq":6528,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:26.912692+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:27:26.912692+0000","last_peered":"2024-09-17T15:27:26.912692+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:27:26.912692+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":6412,"log_dups_size":0,"ondisk_log_size":6412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26243555,"num_objects":6408,"num_object_clones":0,"num_object_copies":12816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6408,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4307,"num_write_kb":17228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":2331}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'6448","reported_seq":6562,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:26.832869+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:27:26.832869+0000","last_peered":"2024-09-17T15:27:26.832869+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:27:26.832869+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":6448,"log_dups_size":0,"ondisk_log_size":6448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26386432,"num_objects":6442,"num_object_clones":0,"num_object_copies":12884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4210,"num_write_kb":16840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":2305}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'6652","reported_seq":6782,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:30.466786+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:27:30.466786+0000","last_peered":"2024-09-17T15:27:30.466786+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:27:30.466786+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":6652,"log_dups_size":0,"ondisk_log_size":6652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27238400,"num_objects":6650,"num_object_clones":0,"num_object_copies":13300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3200,"num_write_kb":12800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2512},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'6590","reported_seq":6725,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:30.508177+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:27:30.508177+0000","last_peered":"2024-09-17T15:27:30.508177+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:27:30.508177+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":6590,"log_dups_size":0,"ondisk_log_size":6590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26959872,"num_objects":6582,"num_object_clones":0,"num_object_copies":13164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2877,"num_write_kb":11508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2425},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229061+0000","last_peered":"2024-09-17T15:26:22.229061+0000","last_clean":"2024-09-17T15:26:22.229061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229061+0000","last_undegraded":"2024-09-17T15:26:22.229061+0000","last_fullsized":"2024-09-17T15:26:22.229061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'6606","reported_seq":6733,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:30.411431+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:27:30.411431+0000","last_peered":"2024-09-17T15:27:30.411431+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:27:30.411431+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":6606,"log_dups_size":0,"ondisk_log_size":6606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27041792,"num_objects":6602,"num_object_clones":0,"num_object_copies":13204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3393,"num_write_kb":13572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2539},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'6533","reported_seq":6671,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:30.475671+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:27:30.475671+0000","last_peered":"2024-09-17T15:27:30.475671+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:27:30.475671+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":6533,"log_dups_size":0,"ondisk_log_size":6533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26710982,"num_objects":6523,"num_object_clones":0,"num_object_copies":13046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6523,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2975,"num_write_kb":11900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2414},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":427602654,"num_objects":104404,"num_object_clones":0,"num_object_copies":208808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":2192,"num_objects_unfound":0,"num_objects_dirty":104404,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":64511,"num_write_kb":258044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1879,"num_bytes_recovered":7696384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":864423936,"data_stored":864272190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104476,"ondisk_log_size":104476,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051613,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518236,"kb_used_data":337996,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93853604,"statfs":{"total":96636764160,"available":96106090496,"internally_reserved":0,"allocated":346107904,"data_stored":345841731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":184544940},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,7,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724559,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":242592,"kb_used_data":124816,"kb_used_omap":4,"kb_used_meta":117755,"kb_avail":94129248,"statfs":{"total":96636764160,"available":96388349952,"internally_reserved":0,"allocated":127811584,"data_stored":127497978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":120581803},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247842,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":614664,"kb_used_data":380344,"kb_used_omap":4,"kb_used_meta":234299,"kb_avail":93757176,"statfs":{"total":96636764160,"available":96007348224,"internally_reserved":0,"allocated":389472256,"data_stored":389202006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":239922856},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,3,6,7,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":390815744,"data_stored":390779614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":127496192,"data_stored":127416706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":346112000,"data_stored":346075870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:27:31.729 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:27:31.729 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'6528", 'reported_seq': 6600, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:30.436919+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:27:30.436919+0000', 'last_peered': '2024-09-17T15:27:30.436919+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:27:30.436919+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 6528, 'log_dups_size': 0, 'ondisk_log_size': 6528, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26738688, 'num_objects': 6528, 'num_object_clones': 0, 'num_object_copies': 13056, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4070, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6528, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6528, 'num_write_kb': 26112, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2458}, {'shards': '0,2', 'objects': 4070}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:31.729 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:27:31.729 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'6478", 'reported_seq': 6598, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:26.924269+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:27:26.924269+0000', 'last_peered': '2024-09-17T15:27:26.924269+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:27:26.924269+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 6478, 'log_dups_size': 0, 'ondisk_log_size': 6478, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26517504, 'num_objects': 6474, 'num_object_clones': 0, 'num_object_copies': 12948, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4169, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6474, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4635, 'num_write_kb': 18540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4168}, {'shards': '2', 'objects': 2306}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:27:31.729 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:27:31.729 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'6710", 'reported_seq': 6839, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:30.496638+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:27:30.496638+0000', 'last_peered': '2024-09-17T15:27:30.496638+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:27:30.496638+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 6710, 'log_dups_size': 0, 'ondisk_log_size': 6710, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27464163, 'num_objects': 6706, 'num_object_clones': 0, 'num_object_copies': 13412, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4271, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6706, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3209, 'num_write_kb': 12836, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2436}, {'shards': '0,2', 'objects': 4268}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:31.729 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:27:31.730 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'6644", 'reported_seq': 6778, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:30.452411+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:27:30.452411+0000', 'last_peered': '2024-09-17T15:27:30.452411+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:27:30.452411+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 6644, 'log_dups_size': 0, 'ondisk_log_size': 6644, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27190214, 'num_objects': 6640, 'num_object_clones': 0, 'num_object_copies': 13280, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4174, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6640, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 3413, 'num_write_kb': 13652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2466}, {'shards': '0,2', 'objects': 4174}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:31.730 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-17T15:27:31.730 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "41'6430", 'reported_seq': 10311, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:27:26.924946+0000', 'last_change': '2024-09-17T15:26:22.588242+0000', 'last_active': '2024-09-17T15:27:26.924946+0000', 'last_peered': '2024-09-17T15:27:26.924946+0000', 'last_clean': '2024-09-17T15:26:20.155646+0000', 'last_became_active': '2024-09-17T15:26:22.557742+0000', 'last_became_peered': '2024-09-17T15:26:22.557742+0000', 'last_unstale': '2024-09-17T15:27:26.924946+0000', 'last_undegraded': '2024-09-17T15:27:26.924946+0000', 'last_fullsized': '2024-09-17T15:26:22.188850+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2016", 'last_scrub_stamp': '2024-09-17T15:25:53.959011+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:53.959011+0000', 'objects_scrubbed': 1994, 'log_size': 6430, 'log_dups_size': 0, 'ondisk_log_size': 6430, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 568, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26309091, 'num_objects': 6424, 'num_object_clones': 0, 'num_object_copies': 12848, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2192, 'num_objects_unfound': 0, 'num_objects_dirty': 6424, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4430, 'num_write_kb': 17720, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1879, 'num_bytes_recovered': 7696384, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2192}, {'shards': '2', 'objects': 4232}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:27:31.730 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:27:31.730 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'6437", 'reported_seq': 6562, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:30.464035+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:27:30.464035+0000', 'last_peered': '2024-09-17T15:27:30.464035+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:27:30.464035+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 6437, 'log_dups_size': 0, 'ondisk_log_size': 6437, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26333184, 'num_objects': 6429, 'num_object_clones': 0, 'num_object_copies': 12858, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4012, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6429, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3033, 'num_write_kb': 12132, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2419}, {'shards': '0,2', 'objects': 4010}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:31.730 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:27:31.730 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'6412", 'reported_seq': 6528, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:26.912692+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:27:26.912692+0000', 'last_peered': '2024-09-17T15:27:26.912692+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:27:26.912692+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 6412, 'log_dups_size': 0, 'ondisk_log_size': 6412, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26243555, 'num_objects': 6408, 'num_object_clones': 0, 'num_object_copies': 12816, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4078, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6408, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4307, 'num_write_kb': 17228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4077}, {'shards': '2', 'objects': 2331}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:27:31.730 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:27:31.730 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'6448", 'reported_seq': 6562, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:26.832869+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:27:26.832869+0000', 'last_peered': '2024-09-17T15:27:26.832869+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:27:26.832869+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 6448, 'log_dups_size': 0, 'ondisk_log_size': 6448, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26386432, 'num_objects': 6442, 'num_object_clones': 0, 'num_object_copies': 12884, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6442, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4210, 'num_write_kb': 16840, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4137}, {'shards': '2', 'objects': 2305}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:27:31.730 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:27:31.730 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'6652", 'reported_seq': 6782, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:30.466786+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:27:30.466786+0000', 'last_peered': '2024-09-17T15:27:30.466786+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:27:30.466786+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 6652, 'log_dups_size': 0, 'ondisk_log_size': 6652, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27238400, 'num_objects': 6650, 'num_object_clones': 0, 'num_object_copies': 13300, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6650, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3200, 'num_write_kb': 12800, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2512}, {'shards': '0,2', 'objects': 4138}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:31.731 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:27:31.731 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'6590", 'reported_seq': 6725, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:30.508177+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:27:30.508177+0000', 'last_peered': '2024-09-17T15:27:30.508177+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:27:30.508177+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 6590, 'log_dups_size': 0, 'ondisk_log_size': 6590, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26959872, 'num_objects': 6582, 'num_object_clones': 0, 'num_object_copies': 13164, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4160, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6582, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2877, 'num_write_kb': 11508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2425}, {'shards': '0,2', 'objects': 4157}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:31.731 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:27:31.731 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'6606", 'reported_seq': 6733, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:30.411431+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:27:30.411431+0000', 'last_peered': '2024-09-17T15:27:30.411431+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:27:30.411431+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 6606, 'log_dups_size': 0, 'ondisk_log_size': 6606, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27041792, 'num_objects': 6602, 'num_object_clones': 0, 'num_object_copies': 13204, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4065, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6602, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3393, 'num_write_kb': 13572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2539}, {'shards': '0,2', 'objects': 4063}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:31.731 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:27:31.731 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'6533", 'reported_seq': 6671, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:30.475671+0000', 'last_change': '2024-09-17T15:26:22.654521+0000', 'last_active': '2024-09-17T15:27:30.475671+0000', 'last_peered': '2024-09-17T15:27:30.475671+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:27:30.475671+0000', 'last_undegraded': '2024-09-17T15:26:22.307795+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 6533, 'log_dups_size': 0, 'ondisk_log_size': 6533, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26710982, 'num_objects': 6523, 'num_object_clones': 0, 'num_object_copies': 13046, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4111, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6523, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2975, 'num_write_kb': 11900, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2414}, {'shards': '0,2', 'objects': 4109}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:31.731 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:27:32.308 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 113 20 105993 105973 3.66251 1.54688 0.0381111 0.0213202 2024-09-17T15:27:32.430 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:27:32.430 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45388/208812 objects degraded (21.736%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":189,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":104406,"data_bytes":427602673,"bytes_used":1408503808,"bytes_avail":288501788672,"bytes_total":289910292480,"degraded_objects":45388,"degraded_total":208812,"degraded_ratio":0.21736298680152483,"misplaced_objects":2192,"misplaced_total":208812,"misplaced_ratio":0.0104974809876827,"recovering_objects_per_sec":27,"recovering_bytes_per_sec":112259,"recovering_keys_per_sec":0,"num_objects_recovered":329,"num_bytes_recovered":1347584,"num_keys_recovered":0,"write_bytes_sec":2459822,"read_op_per_sec":0,"write_op_per_sec":600},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T15:27:19.616376+0000","services":{}},"progress_events":{}} 2024-09-17T15:27:32.450 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:27:32.450 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:27:33.003 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:27:33.006 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:27:33.026 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":128,"stamp":"2024-09-17T15:27:31.620487+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":427602673,"num_objects":104406,"num_object_clones":0,"num_object_copies":208812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":2192,"num_objects_unfound":0,"num_objects_dirty":104406,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":64513,"num_write_kb":258046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1880,"num_bytes_recovered":7696403,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104479,"ondisk_log_size":104479,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1375492,"kb_used_data":843156,"kb_used_omap":13,"kb_used_meta":532274,"kb_avail":281740028,"statfs":{"total":289910292480,"available":288501788672,"internally_reserved":0,"allocated":863391744,"data_stored":862541715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":545049599},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,7,9,15,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29528064,"num_objects":7209,"num_object_clones":0,"num_object_copies":14418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-329,"num_objects_unfound":0,"num_objects_dirty":7209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7209,"num_write_kb":28836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":329,"num_bytes_recovered":1347584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004145"},"pg_stats":[{"pgid":"2.f","version":"41'6528","reported_seq":6600,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:30.436919+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:27:30.436919+0000","last_peered":"2024-09-17T15:27:30.436919+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:27:30.436919+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":6528,"log_dups_size":0,"ondisk_log_size":6528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26738688,"num_objects":6528,"num_object_clones":0,"num_object_copies":13056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6528,"num_write_kb":26112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2458},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'6415","reported_seq":6512,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:26.895792+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:27:26.895792+0000","last_peered":"2024-09-17T15:27:26.895792+0000","last_clean":"2024-09-17T15:27:26.895792+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:27:26.895792+0000","last_undegraded":"2024-09-17T15:27:26.895792+0000","last_fullsized":"2024-09-17T15:27:26.895792+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":6415,"log_dups_size":0,"ondisk_log_size":6415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26252230,"num_objects":6411,"num_object_clones":0,"num_object_copies":12822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6411,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4772,"num_write_kb":19088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'6478","reported_seq":6598,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:26.924269+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:27:26.924269+0000","last_peered":"2024-09-17T15:27:26.924269+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:27:26.924269+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":6478,"log_dups_size":0,"ondisk_log_size":6478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26517504,"num_objects":6474,"num_object_clones":0,"num_object_copies":12948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4635,"num_write_kb":18540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":2306}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'6494","reported_seq":6558,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:26.833290+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:27:26.833290+0000","last_peered":"2024-09-17T15:27:26.833290+0000","last_clean":"2024-09-17T15:27:26.833290+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:27:26.833290+0000","last_undegraded":"2024-09-17T15:27:26.833290+0000","last_fullsized":"2024-09-17T15:27:26.833290+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":6494,"log_dups_size":0,"ondisk_log_size":6494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26599424,"num_objects":6494,"num_object_clones":0,"num_object_copies":12988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6494,"num_write_kb":25976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'6531","reported_seq":6655,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:30.448346+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:27:30.448346+0000","last_peered":"2024-09-17T15:27:30.448346+0000","last_clean":"2024-09-17T15:27:30.448346+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:27:30.448346+0000","last_undegraded":"2024-09-17T15:27:30.448346+0000","last_fullsized":"2024-09-17T15:27:30.448346+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":6531,"log_dups_size":0,"ondisk_log_size":6531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26722787,"num_objects":6525,"num_object_clones":0,"num_object_copies":13050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6525,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3513,"num_write_kb":14052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'6710","reported_seq":6839,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:30.496638+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:27:30.496638+0000","last_peered":"2024-09-17T15:27:30.496638+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:27:30.496638+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":6710,"log_dups_size":0,"ondisk_log_size":6710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27464163,"num_objects":6706,"num_object_clones":0,"num_object_copies":13412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6706,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3209,"num_write_kb":12836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2436},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'6644","reported_seq":6778,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:30.452411+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:27:30.452411+0000","last_peered":"2024-09-17T15:27:30.452411+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:27:30.452411+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":6644,"log_dups_size":0,"ondisk_log_size":6644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27190214,"num_objects":6640,"num_object_clones":0,"num_object_copies":13280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6640,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3413,"num_write_kb":13652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2466},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'6568","reported_seq":6691,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:30.370158+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:27:30.370158+0000","last_peered":"2024-09-17T15:27:30.370158+0000","last_clean":"2024-09-17T15:27:30.370158+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:27:30.370158+0000","last_undegraded":"2024-09-17T15:27:30.370158+0000","last_fullsized":"2024-09-17T15:27:30.370158+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":6568,"log_dups_size":0,"ondisk_log_size":6568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26894336,"num_objects":6566,"num_object_clones":0,"num_object_copies":13132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3522,"num_write_kb":14088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'6430","reported_seq":10311,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:27:26.924946+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:27:26.924946+0000","last_peered":"2024-09-17T15:27:26.924946+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:27:26.924946+0000","last_undegraded":"2024-09-17T15:27:26.924946+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":6430,"log_dups_size":0,"ondisk_log_size":6430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26309091,"num_objects":6424,"num_object_clones":0,"num_object_copies":12848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2192,"num_objects_unfound":0,"num_objects_dirty":6424,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4430,"num_write_kb":17720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1879,"num_bytes_recovered":7696384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2192},{"shards":"2","objects":4232}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233600+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233600+0000","last_peered":"2024-09-17T15:26:22.233600+0000","last_clean":"2024-09-17T15:26:22.233600+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233600+0000","last_undegraded":"2024-09-17T15:26:22.233600+0000","last_fullsized":"2024-09-17T15:26:22.233600+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'6437","reported_seq":6562,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:30.464035+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:27:30.464035+0000","last_peered":"2024-09-17T15:27:30.464035+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:27:30.464035+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":6437,"log_dups_size":0,"ondisk_log_size":6437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26333184,"num_objects":6429,"num_object_clones":0,"num_object_copies":12858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3033,"num_write_kb":12132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2419},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":98,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412941+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412941+0000","last_peered":"2024-09-17T15:26:22.412941+0000","last_clean":"2024-09-17T15:26:22.412941+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412941+0000","last_undegraded":"2024-09-17T15:26:22.412941+0000","last_fullsized":"2024-09-17T15:26:22.412941+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'6412","reported_seq":6528,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:26.912692+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:27:26.912692+0000","last_peered":"2024-09-17T15:27:26.912692+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:27:26.912692+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":6412,"log_dups_size":0,"ondisk_log_size":6412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26243555,"num_objects":6408,"num_object_clones":0,"num_object_copies":12816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6408,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4307,"num_write_kb":17228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":2331}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'6448","reported_seq":6562,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:26.832869+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:27:26.832869+0000","last_peered":"2024-09-17T15:27:26.832869+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:27:26.832869+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":6448,"log_dups_size":0,"ondisk_log_size":6448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26386432,"num_objects":6442,"num_object_clones":0,"num_object_copies":12884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4210,"num_write_kb":16840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":2305}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'6652","reported_seq":6782,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:30.466786+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:27:30.466786+0000","last_peered":"2024-09-17T15:27:30.466786+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:27:30.466786+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":6652,"log_dups_size":0,"ondisk_log_size":6652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27238400,"num_objects":6650,"num_object_clones":0,"num_object_copies":13300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3200,"num_write_kb":12800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2512},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'6590","reported_seq":6725,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:30.508177+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:27:30.508177+0000","last_peered":"2024-09-17T15:27:30.508177+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:27:30.508177+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":6590,"log_dups_size":0,"ondisk_log_size":6590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26959872,"num_objects":6582,"num_object_clones":0,"num_object_copies":13164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2877,"num_write_kb":11508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2425},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229061+0000","last_peered":"2024-09-17T15:26:22.229061+0000","last_clean":"2024-09-17T15:26:22.229061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229061+0000","last_undegraded":"2024-09-17T15:26:22.229061+0000","last_fullsized":"2024-09-17T15:26:22.229061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'6606","reported_seq":6733,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:30.411431+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:27:30.411431+0000","last_peered":"2024-09-17T15:27:30.411431+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:27:30.411431+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":6606,"log_dups_size":0,"ondisk_log_size":6606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27041792,"num_objects":6602,"num_object_clones":0,"num_object_copies":13204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3393,"num_write_kb":13572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2539},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'6533","reported_seq":6671,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:30.475671+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:27:30.475671+0000","last_peered":"2024-09-17T15:27:30.475671+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:27:30.475671+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":6533,"log_dups_size":0,"ondisk_log_size":6533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26710982,"num_objects":6523,"num_object_clones":0,"num_object_copies":13046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6523,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2975,"num_write_kb":11900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2414},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":427602654,"num_objects":104404,"num_object_clones":0,"num_object_copies":208808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":2192,"num_objects_unfound":0,"num_objects_dirty":104404,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":64511,"num_write_kb":258044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1879,"num_bytes_recovered":7696384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":864423936,"data_stored":864272190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104476,"ondisk_log_size":104476,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051613,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518236,"kb_used_data":337996,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93853604,"statfs":{"total":96636764160,"available":96106090496,"internally_reserved":0,"allocated":346107904,"data_stored":345841731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":184544940},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,7,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724559,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":242592,"kb_used_data":124816,"kb_used_omap":4,"kb_used_meta":117755,"kb_avail":94129248,"statfs":{"total":96636764160,"available":96388349952,"internally_reserved":0,"allocated":127811584,"data_stored":127497978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":120581803},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247842,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":614664,"kb_used_data":380344,"kb_used_omap":4,"kb_used_meta":234299,"kb_avail":93757176,"statfs":{"total":96636764160,"available":96007348224,"internally_reserved":0,"allocated":389472256,"data_stored":389202006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":239922856},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,3,6,7,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":390815744,"data_stored":390779614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":127496192,"data_stored":127416706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":346112000,"data_stored":346075870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:27:33.308 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 114 20 106506 106486 3.64797 2.00391 0.0375612 0.0214094 2024-09-17T15:27:34.309 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 115 20 107078 107058 3.63567 2.23438 0.0223997 0.0214794 2024-09-17T15:27:35.310 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 116 20 107636 107616 3.62308 2.17969 0.0521071 0.0215554 2024-09-17T15:27:36.028 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:27:36.310 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 117 20 108179 108159 3.61024 2.12109 0.0331982 0.0216295 2024-09-17T15:27:36.590 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:27:36.593 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:27:36.616 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":130,"stamp":"2024-09-17T15:27:35.621512+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":438522609,"num_objects":107072,"num_object_clones":0,"num_object_copies":214144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":1998,"num_objects_unfound":0,"num_objects_dirty":107072,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":67179,"num_write_kb":268710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2075,"num_bytes_recovered":8487414,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107145,"ondisk_log_size":107145,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1397240,"kb_used_data":864904,"kb_used_omap":13,"kb_used_meta":532274,"kb_avail":281718280,"statfs":{"total":289910292480,"available":288479518720,"internally_reserved":0,"allocated":885661696,"data_stored":884811667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":545049599},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,4,10,7,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28762112,"num_objects":7022,"num_object_clones":0,"num_object_copies":14044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-372,"num_objects_unfound":0,"num_objects_dirty":7022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7022,"num_write_kb":28088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":373,"num_bytes_recovered":1520099,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003378"},"pg_stats":[{"pgid":"2.f","version":"41'6690","reported_seq":6763,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:35.526511+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:27:35.526511+0000","last_peered":"2024-09-17T15:27:35.526511+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:27:35.526511+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":6690,"log_dups_size":0,"ondisk_log_size":6690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27402240,"num_objects":6690,"num_object_clones":0,"num_object_copies":13380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6690,"num_write_kb":26760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2620},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'6590","reported_seq":6687,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:31.882800+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:27:31.882800+0000","last_peered":"2024-09-17T15:27:31.882800+0000","last_clean":"2024-09-17T15:27:31.882800+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:27:31.882800+0000","last_undegraded":"2024-09-17T15:27:31.882800+0000","last_fullsized":"2024-09-17T15:27:31.882800+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":6590,"log_dups_size":0,"ondisk_log_size":6590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26969030,"num_objects":6586,"num_object_clones":0,"num_object_copies":13172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6586,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4947,"num_write_kb":19788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'6637","reported_seq":6757,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:31.918670+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:27:31.918670+0000","last_peered":"2024-09-17T15:27:31.918670+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:27:31.918670+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":6637,"log_dups_size":0,"ondisk_log_size":6637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27168768,"num_objects":6633,"num_object_clones":0,"num_object_copies":13266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4794,"num_write_kb":19176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":2465}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'6675","reported_seq":6739,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:31.908348+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:27:31.908348+0000","last_peered":"2024-09-17T15:27:31.908348+0000","last_clean":"2024-09-17T15:27:31.908348+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:27:31.908348+0000","last_undegraded":"2024-09-17T15:27:31.908348+0000","last_fullsized":"2024-09-17T15:27:31.908348+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":6675,"log_dups_size":0,"ondisk_log_size":6675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27340800,"num_objects":6675,"num_object_clones":0,"num_object_copies":13350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6675,"num_write_kb":26700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'6690","reported_seq":6810,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:35.514960+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:27:35.514960+0000","last_peered":"2024-09-17T15:27:35.514960+0000","last_clean":"2024-09-17T15:27:35.514960+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:27:35.514960+0000","last_undegraded":"2024-09-17T15:27:35.514960+0000","last_fullsized":"2024-09-17T15:27:35.514960+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":6690,"log_dups_size":0,"ondisk_log_size":6690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27374051,"num_objects":6684,"num_object_clones":0,"num_object_copies":13368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6684,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3672,"num_write_kb":14688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'6887","reported_seq":7017,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:35.526783+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:27:35.526783+0000","last_peered":"2024-09-17T15:27:35.526783+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:27:35.526783+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":6887,"log_dups_size":0,"ondisk_log_size":6887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28189155,"num_objects":6883,"num_object_clones":0,"num_object_copies":13766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6883,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3386,"num_write_kb":13544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2613},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'6803","reported_seq":6937,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:35.478378+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:27:35.478378+0000","last_peered":"2024-09-17T15:27:35.478378+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:27:35.478378+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":6803,"log_dups_size":0,"ondisk_log_size":6803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27841478,"num_objects":6799,"num_object_clones":0,"num_object_copies":13598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6799,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3572,"num_write_kb":14288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2625},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'6733","reported_seq":6856,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:35.485666+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:27:35.485666+0000","last_peered":"2024-09-17T15:27:35.485666+0000","last_clean":"2024-09-17T15:27:35.485666+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:27:35.485666+0000","last_undegraded":"2024-09-17T15:27:35.485666+0000","last_fullsized":"2024-09-17T15:27:35.485666+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":6733,"log_dups_size":0,"ondisk_log_size":6733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27570176,"num_objects":6731,"num_object_clones":0,"num_object_copies":13462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3687,"num_write_kb":14748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'6585","reported_seq":10856,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:27:31.908643+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:27:31.908643+0000","last_peered":"2024-09-17T15:27:31.908643+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:27:31.908643+0000","last_undegraded":"2024-09-17T15:27:31.908643+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":6585,"log_dups_size":0,"ondisk_log_size":6585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26943971,"num_objects":6579,"num_object_clones":0,"num_object_copies":13158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1998,"num_objects_unfound":0,"num_objects_dirty":6579,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4585,"num_write_kb":18340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2074,"num_bytes_recovered":8487395,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1998},{"shards":"2","objects":4581}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233600+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233600+0000","last_peered":"2024-09-17T15:26:22.233600+0000","last_clean":"2024-09-17T15:26:22.233600+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233600+0000","last_undegraded":"2024-09-17T15:26:22.233600+0000","last_fullsized":"2024-09-17T15:26:22.233600+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'6606","reported_seq":6730,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:35.567069+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:27:35.567069+0000","last_peered":"2024-09-17T15:27:35.567069+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:27:35.567069+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":6606,"log_dups_size":0,"ondisk_log_size":6606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27025408,"num_objects":6598,"num_object_clones":0,"num_object_copies":13196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3202,"num_write_kb":12808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2588},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":98,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412941+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412941+0000","last_peered":"2024-09-17T15:26:22.412941+0000","last_clean":"2024-09-17T15:26:22.412941+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412941+0000","last_undegraded":"2024-09-17T15:26:22.412941+0000","last_fullsized":"2024-09-17T15:26:22.412941+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'6561","reported_seq":6675,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:31.934180+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:27:31.934180+0000","last_peered":"2024-09-17T15:27:31.934180+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:27:31.934180+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":6561,"log_dups_size":0,"ondisk_log_size":6561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26853859,"num_objects":6557,"num_object_clones":0,"num_object_copies":13114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6557,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4456,"num_write_kb":17824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":2480}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'6620","reported_seq":6734,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:31.877913+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:27:31.877913+0000","last_peered":"2024-09-17T15:27:31.877913+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:27:31.877913+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":6620,"log_dups_size":0,"ondisk_log_size":6620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27090944,"num_objects":6614,"num_object_clones":0,"num_object_copies":13228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4382,"num_write_kb":17528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":2477}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'6826","reported_seq":6956,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:35.561103+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:27:35.561103+0000","last_peered":"2024-09-17T15:27:35.561103+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:27:35.561103+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":6826,"log_dups_size":0,"ondisk_log_size":6826,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27951104,"num_objects":6824,"num_object_clones":0,"num_object_copies":13648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3374,"num_write_kb":13496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2686},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'6742","reported_seq":6879,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:35.544433+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:27:35.544433+0000","last_peered":"2024-09-17T15:27:35.544433+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:27:35.544433+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27582464,"num_objects":6734,"num_object_clones":0,"num_object_copies":13468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6734,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3029,"num_write_kb":12116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2577},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229061+0000","last_peered":"2024-09-17T15:26:22.229061+0000","last_clean":"2024-09-17T15:26:22.229061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229061+0000","last_undegraded":"2024-09-17T15:26:22.229061+0000","last_fullsized":"2024-09-17T15:26:22.229061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'6787","reported_seq":6911,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:35.523187+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:27:35.523187+0000","last_peered":"2024-09-17T15:27:35.523187+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:27:35.523187+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":6787,"log_dups_size":0,"ondisk_log_size":6787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27783168,"num_objects":6783,"num_object_clones":0,"num_object_copies":13566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6783,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3574,"num_write_kb":14296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2720},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'6710","reported_seq":6848,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:35.542171+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:27:35.542171+0000","last_peered":"2024-09-17T15:27:35.542171+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:27:35.542171+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":6710,"log_dups_size":0,"ondisk_log_size":6710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27435974,"num_objects":6700,"num_object_clones":0,"num_object_copies":13400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3152,"num_write_kb":12608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2591},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":438522590,"num_objects":107070,"num_object_clones":0,"num_object_copies":214140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":1998,"num_objects_unfound":0,"num_objects_dirty":107070,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":67177,"num_write_kb":268708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2074,"num_bytes_recovered":8487395,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":886759424,"data_stored":886607678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107142,"ondisk_log_size":107142,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051614,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527672,"kb_used_data":343272,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93844168,"statfs":{"total":96636764160,"available":96096428032,"internally_reserved":0,"allocated":351510528,"data_stored":351244355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":188804780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724560,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":253756,"kb_used_data":131820,"kb_used_omap":4,"kb_used_meta":121915,"kb_avail":94118084,"statfs":{"total":96636764160,"available":96376918016,"internally_reserved":0,"allocated":134983680,"data_stored":134670074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":124841643},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,1,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247843,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":615812,"kb_used_data":389812,"kb_used_omap":4,"kb_used_meta":225979,"kb_avail":93756028,"statfs":{"total":96636764160,"available":96006172672,"internally_reserved":0,"allocated":399167488,"data_stored":398897238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":231403176},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,0,8,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":398974976,"data_stored":398938846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":136306688,"data_stored":136227202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":351477760,"data_stored":351441630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:27:36.618 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:27:37.187 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:27:37.190 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:27:37.217 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":130,"stamp":"2024-09-17T15:27:35.621512+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":438522609,"num_objects":107072,"num_object_clones":0,"num_object_copies":214144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":1998,"num_objects_unfound":0,"num_objects_dirty":107072,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":67179,"num_write_kb":268710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2075,"num_bytes_recovered":8487414,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107145,"ondisk_log_size":107145,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1397240,"kb_used_data":864904,"kb_used_omap":13,"kb_used_meta":532274,"kb_avail":281718280,"statfs":{"total":289910292480,"available":288479518720,"internally_reserved":0,"allocated":885661696,"data_stored":884811667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":545049599},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,4,10,7,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28762112,"num_objects":7022,"num_object_clones":0,"num_object_copies":14044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-372,"num_objects_unfound":0,"num_objects_dirty":7022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7022,"num_write_kb":28088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":373,"num_bytes_recovered":1520099,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003378"},"pg_stats":[{"pgid":"2.f","version":"41'6690","reported_seq":6763,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:35.526511+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:27:35.526511+0000","last_peered":"2024-09-17T15:27:35.526511+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:27:35.526511+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":6690,"log_dups_size":0,"ondisk_log_size":6690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27402240,"num_objects":6690,"num_object_clones":0,"num_object_copies":13380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6690,"num_write_kb":26760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2620},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'6590","reported_seq":6687,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:31.882800+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:27:31.882800+0000","last_peered":"2024-09-17T15:27:31.882800+0000","last_clean":"2024-09-17T15:27:31.882800+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:27:31.882800+0000","last_undegraded":"2024-09-17T15:27:31.882800+0000","last_fullsized":"2024-09-17T15:27:31.882800+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":6590,"log_dups_size":0,"ondisk_log_size":6590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26969030,"num_objects":6586,"num_object_clones":0,"num_object_copies":13172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6586,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4947,"num_write_kb":19788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'6637","reported_seq":6757,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:31.918670+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:27:31.918670+0000","last_peered":"2024-09-17T15:27:31.918670+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:27:31.918670+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":6637,"log_dups_size":0,"ondisk_log_size":6637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27168768,"num_objects":6633,"num_object_clones":0,"num_object_copies":13266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4794,"num_write_kb":19176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":2465}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'6675","reported_seq":6739,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:31.908348+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:27:31.908348+0000","last_peered":"2024-09-17T15:27:31.908348+0000","last_clean":"2024-09-17T15:27:31.908348+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:27:31.908348+0000","last_undegraded":"2024-09-17T15:27:31.908348+0000","last_fullsized":"2024-09-17T15:27:31.908348+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":6675,"log_dups_size":0,"ondisk_log_size":6675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27340800,"num_objects":6675,"num_object_clones":0,"num_object_copies":13350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6675,"num_write_kb":26700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'6690","reported_seq":6810,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:35.514960+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:27:35.514960+0000","last_peered":"2024-09-17T15:27:35.514960+0000","last_clean":"2024-09-17T15:27:35.514960+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:27:35.514960+0000","last_undegraded":"2024-09-17T15:27:35.514960+0000","last_fullsized":"2024-09-17T15:27:35.514960+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":6690,"log_dups_size":0,"ondisk_log_size":6690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27374051,"num_objects":6684,"num_object_clones":0,"num_object_copies":13368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6684,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3672,"num_write_kb":14688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'6887","reported_seq":7017,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:35.526783+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:27:35.526783+0000","last_peered":"2024-09-17T15:27:35.526783+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:27:35.526783+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":6887,"log_dups_size":0,"ondisk_log_size":6887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28189155,"num_objects":6883,"num_object_clones":0,"num_object_copies":13766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6883,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3386,"num_write_kb":13544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2613},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'6803","reported_seq":6937,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:35.478378+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:27:35.478378+0000","last_peered":"2024-09-17T15:27:35.478378+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:27:35.478378+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":6803,"log_dups_size":0,"ondisk_log_size":6803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27841478,"num_objects":6799,"num_object_clones":0,"num_object_copies":13598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6799,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3572,"num_write_kb":14288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2625},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'6733","reported_seq":6856,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:35.485666+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:27:35.485666+0000","last_peered":"2024-09-17T15:27:35.485666+0000","last_clean":"2024-09-17T15:27:35.485666+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:27:35.485666+0000","last_undegraded":"2024-09-17T15:27:35.485666+0000","last_fullsized":"2024-09-17T15:27:35.485666+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":6733,"log_dups_size":0,"ondisk_log_size":6733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27570176,"num_objects":6731,"num_object_clones":0,"num_object_copies":13462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3687,"num_write_kb":14748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'6585","reported_seq":10856,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:27:31.908643+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:27:31.908643+0000","last_peered":"2024-09-17T15:27:31.908643+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:27:31.908643+0000","last_undegraded":"2024-09-17T15:27:31.908643+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":6585,"log_dups_size":0,"ondisk_log_size":6585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26943971,"num_objects":6579,"num_object_clones":0,"num_object_copies":13158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1998,"num_objects_unfound":0,"num_objects_dirty":6579,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4585,"num_write_kb":18340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2074,"num_bytes_recovered":8487395,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1998},{"shards":"2","objects":4581}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233600+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233600+0000","last_peered":"2024-09-17T15:26:22.233600+0000","last_clean":"2024-09-17T15:26:22.233600+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233600+0000","last_undegraded":"2024-09-17T15:26:22.233600+0000","last_fullsized":"2024-09-17T15:26:22.233600+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'6606","reported_seq":6730,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:35.567069+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:27:35.567069+0000","last_peered":"2024-09-17T15:27:35.567069+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:27:35.567069+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":6606,"log_dups_size":0,"ondisk_log_size":6606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27025408,"num_objects":6598,"num_object_clones":0,"num_object_copies":13196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3202,"num_write_kb":12808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2588},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":98,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412941+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412941+0000","last_peered":"2024-09-17T15:26:22.412941+0000","last_clean":"2024-09-17T15:26:22.412941+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412941+0000","last_undegraded":"2024-09-17T15:26:22.412941+0000","last_fullsized":"2024-09-17T15:26:22.412941+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'6561","reported_seq":6675,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:31.934180+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:27:31.934180+0000","last_peered":"2024-09-17T15:27:31.934180+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:27:31.934180+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":6561,"log_dups_size":0,"ondisk_log_size":6561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26853859,"num_objects":6557,"num_object_clones":0,"num_object_copies":13114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6557,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4456,"num_write_kb":17824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":2480}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'6620","reported_seq":6734,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:31.877913+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:27:31.877913+0000","last_peered":"2024-09-17T15:27:31.877913+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:27:31.877913+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":6620,"log_dups_size":0,"ondisk_log_size":6620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27090944,"num_objects":6614,"num_object_clones":0,"num_object_copies":13228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4382,"num_write_kb":17528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":2477}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'6826","reported_seq":6956,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:35.561103+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:27:35.561103+0000","last_peered":"2024-09-17T15:27:35.561103+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:27:35.561103+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":6826,"log_dups_size":0,"ondisk_log_size":6826,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27951104,"num_objects":6824,"num_object_clones":0,"num_object_copies":13648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3374,"num_write_kb":13496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2686},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'6742","reported_seq":6879,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:35.544433+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:27:35.544433+0000","last_peered":"2024-09-17T15:27:35.544433+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:27:35.544433+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27582464,"num_objects":6734,"num_object_clones":0,"num_object_copies":13468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6734,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3029,"num_write_kb":12116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2577},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229061+0000","last_peered":"2024-09-17T15:26:22.229061+0000","last_clean":"2024-09-17T15:26:22.229061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229061+0000","last_undegraded":"2024-09-17T15:26:22.229061+0000","last_fullsized":"2024-09-17T15:26:22.229061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'6787","reported_seq":6911,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:35.523187+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:27:35.523187+0000","last_peered":"2024-09-17T15:27:35.523187+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:27:35.523187+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":6787,"log_dups_size":0,"ondisk_log_size":6787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27783168,"num_objects":6783,"num_object_clones":0,"num_object_copies":13566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6783,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3574,"num_write_kb":14296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2720},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'6710","reported_seq":6848,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:35.542171+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:27:35.542171+0000","last_peered":"2024-09-17T15:27:35.542171+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:27:35.542171+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":6710,"log_dups_size":0,"ondisk_log_size":6710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27435974,"num_objects":6700,"num_object_clones":0,"num_object_copies":13400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3152,"num_write_kb":12608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2591},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":438522590,"num_objects":107070,"num_object_clones":0,"num_object_copies":214140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":1998,"num_objects_unfound":0,"num_objects_dirty":107070,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":67177,"num_write_kb":268708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2074,"num_bytes_recovered":8487395,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":886759424,"data_stored":886607678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107142,"ondisk_log_size":107142,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051614,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527672,"kb_used_data":343272,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93844168,"statfs":{"total":96636764160,"available":96096428032,"internally_reserved":0,"allocated":351510528,"data_stored":351244355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":188804780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724560,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":253756,"kb_used_data":131820,"kb_used_omap":4,"kb_used_meta":121915,"kb_avail":94118084,"statfs":{"total":96636764160,"available":96376918016,"internally_reserved":0,"allocated":134983680,"data_stored":134670074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":124841643},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,1,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247843,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":615812,"kb_used_data":389812,"kb_used_omap":4,"kb_used_meta":225979,"kb_avail":93756028,"statfs":{"total":96636764160,"available":96006172672,"internally_reserved":0,"allocated":399167488,"data_stored":398897238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":231403176},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,0,8,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":398974976,"data_stored":398938846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":136306688,"data_stored":136227202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":351477760,"data_stored":351441630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:27:37.219 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:27:37.219 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'6690", 'reported_seq': 6763, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:35.526511+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:27:35.526511+0000', 'last_peered': '2024-09-17T15:27:35.526511+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:27:35.526511+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 6690, 'log_dups_size': 0, 'ondisk_log_size': 6690, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27402240, 'num_objects': 6690, 'num_object_clones': 0, 'num_object_copies': 13380, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4070, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6690, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6690, 'num_write_kb': 26760, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2620}, {'shards': '0,2', 'objects': 4070}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:37.219 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:27:37.219 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'6637", 'reported_seq': 6757, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:31.918670+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:27:31.918670+0000', 'last_peered': '2024-09-17T15:27:31.918670+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:27:31.918670+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 6637, 'log_dups_size': 0, 'ondisk_log_size': 6637, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27168768, 'num_objects': 6633, 'num_object_clones': 0, 'num_object_copies': 13266, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4169, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6633, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4794, 'num_write_kb': 19176, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4168}, {'shards': '2', 'objects': 2465}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:27:37.219 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:27:37.219 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'6887", 'reported_seq': 7017, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:35.526783+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:27:35.526783+0000', 'last_peered': '2024-09-17T15:27:35.526783+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:27:35.526783+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 6887, 'log_dups_size': 0, 'ondisk_log_size': 6887, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28189155, 'num_objects': 6883, 'num_object_clones': 0, 'num_object_copies': 13766, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4271, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6883, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3386, 'num_write_kb': 13544, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2613}, {'shards': '0,2', 'objects': 4268}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:37.219 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:27:37.219 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'6803", 'reported_seq': 6937, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:35.478378+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:27:35.478378+0000', 'last_peered': '2024-09-17T15:27:35.478378+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:27:35.478378+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 6803, 'log_dups_size': 0, 'ondisk_log_size': 6803, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27841478, 'num_objects': 6799, 'num_object_clones': 0, 'num_object_copies': 13598, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4174, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6799, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 3572, 'num_write_kb': 14288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2625}, {'shards': '0,2', 'objects': 4174}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:37.220 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-17T15:27:37.220 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "41'6585", 'reported_seq': 10856, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:27:31.908643+0000', 'last_change': '2024-09-17T15:26:22.588242+0000', 'last_active': '2024-09-17T15:27:31.908643+0000', 'last_peered': '2024-09-17T15:27:31.908643+0000', 'last_clean': '2024-09-17T15:26:20.155646+0000', 'last_became_active': '2024-09-17T15:26:22.557742+0000', 'last_became_peered': '2024-09-17T15:26:22.557742+0000', 'last_unstale': '2024-09-17T15:27:31.908643+0000', 'last_undegraded': '2024-09-17T15:27:31.908643+0000', 'last_fullsized': '2024-09-17T15:26:22.188850+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2016", 'last_scrub_stamp': '2024-09-17T15:25:53.959011+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:53.959011+0000', 'objects_scrubbed': 1994, 'log_size': 6585, 'log_dups_size': 0, 'ondisk_log_size': 6585, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 568, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26943971, 'num_objects': 6579, 'num_object_clones': 0, 'num_object_copies': 13158, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1998, 'num_objects_unfound': 0, 'num_objects_dirty': 6579, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4585, 'num_write_kb': 18340, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2074, 'num_bytes_recovered': 8487395, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1998}, {'shards': '2', 'objects': 4581}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:27:37.220 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:27:37.220 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'6606", 'reported_seq': 6730, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:35.567069+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:27:35.567069+0000', 'last_peered': '2024-09-17T15:27:35.567069+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:27:35.567069+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 6606, 'log_dups_size': 0, 'ondisk_log_size': 6606, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27025408, 'num_objects': 6598, 'num_object_clones': 0, 'num_object_copies': 13196, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4012, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6598, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3202, 'num_write_kb': 12808, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2588}, {'shards': '0,2', 'objects': 4010}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:37.220 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:27:37.220 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'6561", 'reported_seq': 6675, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:31.934180+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:27:31.934180+0000', 'last_peered': '2024-09-17T15:27:31.934180+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:27:31.934180+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 6561, 'log_dups_size': 0, 'ondisk_log_size': 6561, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26853859, 'num_objects': 6557, 'num_object_clones': 0, 'num_object_copies': 13114, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4078, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6557, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4456, 'num_write_kb': 17824, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4077}, {'shards': '2', 'objects': 2480}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:27:37.220 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:27:37.220 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'6620", 'reported_seq': 6734, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:31.877913+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:27:31.877913+0000', 'last_peered': '2024-09-17T15:27:31.877913+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:27:31.877913+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 6620, 'log_dups_size': 0, 'ondisk_log_size': 6620, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27090944, 'num_objects': 6614, 'num_object_clones': 0, 'num_object_copies': 13228, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6614, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4382, 'num_write_kb': 17528, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4137}, {'shards': '2', 'objects': 2477}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:27:37.220 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:27:37.220 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'6826", 'reported_seq': 6956, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:35.561103+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:27:35.561103+0000', 'last_peered': '2024-09-17T15:27:35.561103+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:27:35.561103+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 6826, 'log_dups_size': 0, 'ondisk_log_size': 6826, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27951104, 'num_objects': 6824, 'num_object_clones': 0, 'num_object_copies': 13648, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6824, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3374, 'num_write_kb': 13496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2686}, {'shards': '0,2', 'objects': 4138}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:37.221 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:27:37.221 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'6742", 'reported_seq': 6879, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:35.544433+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:27:35.544433+0000', 'last_peered': '2024-09-17T15:27:35.544433+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:27:35.544433+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 6742, 'log_dups_size': 0, 'ondisk_log_size': 6742, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27582464, 'num_objects': 6734, 'num_object_clones': 0, 'num_object_copies': 13468, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4160, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6734, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3029, 'num_write_kb': 12116, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2577}, {'shards': '0,2', 'objects': 4157}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:37.221 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:27:37.221 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'6787", 'reported_seq': 6911, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:35.523187+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:27:35.523187+0000', 'last_peered': '2024-09-17T15:27:35.523187+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:27:35.523187+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 6787, 'log_dups_size': 0, 'ondisk_log_size': 6787, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27783168, 'num_objects': 6783, 'num_object_clones': 0, 'num_object_copies': 13566, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4065, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6783, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3574, 'num_write_kb': 14296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2720}, {'shards': '0,2', 'objects': 4063}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:37.221 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:27:37.221 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'6710", 'reported_seq': 6848, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:35.542171+0000', 'last_change': '2024-09-17T15:26:22.654521+0000', 'last_active': '2024-09-17T15:27:35.542171+0000', 'last_peered': '2024-09-17T15:27:35.542171+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:27:35.542171+0000', 'last_undegraded': '2024-09-17T15:26:22.307795+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 6710, 'log_dups_size': 0, 'ondisk_log_size': 6710, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27435974, 'num_objects': 6700, 'num_object_clones': 0, 'num_object_copies': 13400, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4111, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6700, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3152, 'num_write_kb': 12608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2591}, {'shards': '0,2', 'objects': 4109}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:37.221 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:27:37.310 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 118 20 108660 108640 3.59557 1.87891 0.0414994 0.0217207 2024-09-17T15:27:37.889 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:27:37.889 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45388/214144 objects degraded (21.195%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":194,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":107072,"data_bytes":438522609,"bytes_used":1430773760,"bytes_avail":288479518720,"bytes_total":289910292480,"degraded_objects":45388,"degraded_total":214144,"degraded_ratio":0.21195083682008367,"misplaced_objects":1998,"misplaced_total":214144,"misplaced_ratio":0.0093301703526598931,"recovering_objects_per_sec":31,"recovering_bytes_per_sec":126639,"recovering_keys_per_sec":0,"num_objects_recovered":373,"num_bytes_recovered":1520099,"num_keys_recovered":0,"write_bytes_sec":2396168,"read_op_per_sec":0,"write_op_per_sec":585},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T15:27:19.616376+0000","services":{}},"progress_events":{}} 2024-09-17T15:27:37.908 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:27:37.908 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:27:38.310 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 119 20 109149 109129 3.58141 1.91016 0.0215756 0.0218042 2024-09-17T15:27:38.468 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:27:38.469 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:27:38.496 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":131,"stamp":"2024-09-17T15:27:37.622016+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":442479345,"num_objects":108038,"num_object_clones":0,"num_object_copies":216076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":1821,"num_objects_unfound":0,"num_objects_dirty":108038,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":68145,"num_write_kb":272574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2252,"num_bytes_recovered":9212406,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":108111,"ondisk_log_size":108111,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1390012,"kb_used_data":870156,"kb_used_omap":13,"kb_used_meta":519794,"kb_avail":281725508,"statfs":{"total":289910292480,"available":288486920192,"internally_reserved":0,"allocated":891039744,"data_stored":890189715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":532270079},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,3,9,9,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25849856,"num_objects":6311,"num_object_clones":0,"num_object_copies":12622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-549,"num_objects_unfound":0,"num_objects_dirty":6311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6311,"num_write_kb":25244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":550,"num_bytes_recovered":2245091,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003283"},"pg_stats":[{"pgid":"2.f","version":"41'6690","reported_seq":6763,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:35.526511+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:27:35.526511+0000","last_peered":"2024-09-17T15:27:35.526511+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:27:35.526511+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":6690,"log_dups_size":0,"ondisk_log_size":6690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27402240,"num_objects":6690,"num_object_clones":0,"num_object_copies":13380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6690,"num_write_kb":26760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2620},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'6752","reported_seq":6849,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:36.929567+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:27:36.929567+0000","last_peered":"2024-09-17T15:27:36.929567+0000","last_clean":"2024-09-17T15:27:36.929567+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:27:36.929567+0000","last_undegraded":"2024-09-17T15:27:36.929567+0000","last_fullsized":"2024-09-17T15:27:36.929567+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":6752,"log_dups_size":0,"ondisk_log_size":6752,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27632582,"num_objects":6748,"num_object_clones":0,"num_object_copies":13496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6748,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5109,"num_write_kb":20436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'6784","reported_seq":6902,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:36.878301+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:27:36.878301+0000","last_peered":"2024-09-17T15:27:36.878301+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:27:36.878301+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":6784,"log_dups_size":0,"ondisk_log_size":6784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27770880,"num_objects":6780,"num_object_clones":0,"num_object_copies":13560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4941,"num_write_kb":19764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":2612}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'6845","reported_seq":6909,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:36.954485+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:27:36.954485+0000","last_peered":"2024-09-17T15:27:36.954485+0000","last_clean":"2024-09-17T15:27:36.954485+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:27:36.954485+0000","last_undegraded":"2024-09-17T15:27:36.954485+0000","last_fullsized":"2024-09-17T15:27:36.954485+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":6845,"log_dups_size":0,"ondisk_log_size":6845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28037120,"num_objects":6845,"num_object_clones":0,"num_object_copies":13690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6845,"num_write_kb":27380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'6690","reported_seq":6810,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:35.514960+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:27:35.514960+0000","last_peered":"2024-09-17T15:27:35.514960+0000","last_clean":"2024-09-17T15:27:35.514960+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:27:35.514960+0000","last_undegraded":"2024-09-17T15:27:35.514960+0000","last_fullsized":"2024-09-17T15:27:35.514960+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":6690,"log_dups_size":0,"ondisk_log_size":6690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27374051,"num_objects":6684,"num_object_clones":0,"num_object_copies":13368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6684,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3672,"num_write_kb":14688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'6887","reported_seq":7017,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:35.526783+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:27:35.526783+0000","last_peered":"2024-09-17T15:27:35.526783+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:27:35.526783+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":6887,"log_dups_size":0,"ondisk_log_size":6887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28189155,"num_objects":6883,"num_object_clones":0,"num_object_copies":13766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6883,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3386,"num_write_kb":13544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2613},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'6803","reported_seq":6937,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:35.478378+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:27:35.478378+0000","last_peered":"2024-09-17T15:27:35.478378+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:27:35.478378+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":6803,"log_dups_size":0,"ondisk_log_size":6803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27841478,"num_objects":6799,"num_object_clones":0,"num_object_copies":13598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6799,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3572,"num_write_kb":14288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2625},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'6733","reported_seq":6856,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:35.485666+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:27:35.485666+0000","last_peered":"2024-09-17T15:27:35.485666+0000","last_clean":"2024-09-17T15:27:35.485666+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:27:35.485666+0000","last_undegraded":"2024-09-17T15:27:35.485666+0000","last_fullsized":"2024-09-17T15:27:35.485666+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":6733,"log_dups_size":0,"ondisk_log_size":6733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27570176,"num_objects":6731,"num_object_clones":0,"num_object_copies":13462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3687,"num_write_kb":14748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'6754","reported_seq":11380,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:27:36.946290+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:27:36.946290+0000","last_peered":"2024-09-17T15:27:36.946290+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:27:36.946290+0000","last_undegraded":"2024-09-17T15:27:36.946290+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":6754,"log_dups_size":0,"ondisk_log_size":6754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27636195,"num_objects":6748,"num_object_clones":0,"num_object_copies":13496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1821,"num_objects_unfound":0,"num_objects_dirty":6748,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4754,"num_write_kb":19016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2251,"num_bytes_recovered":9212387,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1821},{"shards":"2","objects":4927}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233600+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233600+0000","last_peered":"2024-09-17T15:26:22.233600+0000","last_clean":"2024-09-17T15:26:22.233600+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233600+0000","last_undegraded":"2024-09-17T15:26:22.233600+0000","last_fullsized":"2024-09-17T15:26:22.233600+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'6606","reported_seq":6730,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:35.567069+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:27:35.567069+0000","last_peered":"2024-09-17T15:27:35.567069+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:27:35.567069+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":6606,"log_dups_size":0,"ondisk_log_size":6606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27025408,"num_objects":6598,"num_object_clones":0,"num_object_copies":13196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3202,"num_write_kb":12808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2588},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":98,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412941+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412941+0000","last_peered":"2024-09-17T15:26:22.412941+0000","last_clean":"2024-09-17T15:26:22.412941+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412941+0000","last_undegraded":"2024-09-17T15:26:22.412941+0000","last_fullsized":"2024-09-17T15:26:22.412941+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'6732","reported_seq":6848,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:36.899161+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:27:36.899161+0000","last_peered":"2024-09-17T15:27:36.899161+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:27:36.899161+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":6732,"log_dups_size":0,"ondisk_log_size":6732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27554275,"num_objects":6728,"num_object_clones":0,"num_object_copies":13456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6728,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4627,"num_write_kb":18508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":2651}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'6767","reported_seq":6881,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:36.865869+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:27:36.865869+0000","last_peered":"2024-09-17T15:27:36.865869+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:27:36.865869+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":6767,"log_dups_size":0,"ondisk_log_size":6767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27693056,"num_objects":6761,"num_object_clones":0,"num_object_copies":13522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4529,"num_write_kb":18116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":2624}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'6826","reported_seq":6956,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:35.561103+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:27:35.561103+0000","last_peered":"2024-09-17T15:27:35.561103+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:27:35.561103+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":6826,"log_dups_size":0,"ondisk_log_size":6826,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27951104,"num_objects":6824,"num_object_clones":0,"num_object_copies":13648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3374,"num_write_kb":13496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2686},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'6742","reported_seq":6879,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:35.544433+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:27:35.544433+0000","last_peered":"2024-09-17T15:27:35.544433+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:27:35.544433+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27582464,"num_objects":6734,"num_object_clones":0,"num_object_copies":13468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6734,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3029,"num_write_kb":12116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2577},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229061+0000","last_peered":"2024-09-17T15:26:22.229061+0000","last_clean":"2024-09-17T15:26:22.229061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229061+0000","last_undegraded":"2024-09-17T15:26:22.229061+0000","last_fullsized":"2024-09-17T15:26:22.229061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'6787","reported_seq":6911,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:35.523187+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:27:35.523187+0000","last_peered":"2024-09-17T15:27:35.523187+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:27:35.523187+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":6787,"log_dups_size":0,"ondisk_log_size":6787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27783168,"num_objects":6783,"num_object_clones":0,"num_object_copies":13566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6783,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3574,"num_write_kb":14296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2720},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'6710","reported_seq":6848,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:35.542171+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:27:35.542171+0000","last_peered":"2024-09-17T15:27:35.542171+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:27:35.542171+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":6710,"log_dups_size":0,"ondisk_log_size":6710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27435974,"num_objects":6700,"num_object_clones":0,"num_object_copies":13400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3152,"num_write_kb":12608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2591},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":442479326,"num_objects":108036,"num_object_clones":0,"num_object_copies":216072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":1821,"num_objects_unfound":0,"num_objects_dirty":108036,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":68143,"num_write_kb":272572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2251,"num_bytes_recovered":9212387,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":892084224,"data_stored":891932478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":108108,"ondisk_log_size":108108,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051615,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":520444,"kb_used_data":348524,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93851396,"statfs":{"total":96636764160,"available":96103829504,"internally_reserved":0,"allocated":356888576,"data_stored":356622403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":176025260},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,0,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724560,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":253756,"kb_used_data":131820,"kb_used_omap":4,"kb_used_meta":121915,"kb_avail":94118084,"statfs":{"total":96636764160,"available":96376918016,"internally_reserved":0,"allocated":134983680,"data_stored":134670074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":124841643},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,1,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247843,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":615812,"kb_used_data":389812,"kb_used_omap":4,"kb_used_meta":225979,"kb_avail":93756028,"statfs":{"total":96636764160,"available":96006172672,"internally_reserved":0,"allocated":399167488,"data_stored":398897238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":231403176},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,0,8,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":398974976,"data_stored":398938846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":136306688,"data_stored":136227202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":356802560,"data_stored":356766430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:27:39.310 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout:2024-09-17T15:27:39.310927+0000 min lat: 0.00187689 max lat: 2.64156 avg lat: 0.0218804 2024-09-17T15:27:39.310 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-17T15:27:39.310 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 120 20 109693 109673 3.56927 2.125 0.0513523 0.0218804 2024-09-17T15:27:40.310 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 121 20 110247 110227 3.55766 2.16406 0.0219653 0.021952 2024-09-17T15:27:41.311 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 122 20 110794 110774 3.54601 2.13672 0.0347295 0.0220243 2024-09-17T15:27:41.498 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:27:42.071 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:27:42.072 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:27:42.093 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":133,"stamp":"2024-09-17T15:27:41.622915+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":449364721,"num_objects":109719,"num_object_clones":0,"num_object_copies":219438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":1821,"num_objects_unfound":0,"num_objects_dirty":109719,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":69826,"num_write_kb":279298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2252,"num_bytes_recovered":9212406,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109792,"ondisk_log_size":109792,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1432916,"kb_used_data":886688,"kb_used_omap":13,"kb_used_meta":546162,"kb_avail":281682604,"statfs":{"total":289910292480,"available":288442986496,"internally_reserved":0,"allocated":907968512,"data_stored":907118483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":559270911},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,9,3,9,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28581888,"num_objects":6978,"num_object_clones":0,"num_object_copies":13956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-371,"num_objects_unfound":0,"num_objects_dirty":6978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6978,"num_write_kb":27912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":372,"num_bytes_recovered":1516003,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003013"},"pg_stats":[{"pgid":"2.f","version":"41'6854","reported_seq":6927,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:40.588045+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:27:40.588045+0000","last_peered":"2024-09-17T15:27:40.588045+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:27:40.588045+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":6854,"log_dups_size":0,"ondisk_log_size":6854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28073984,"num_objects":6854,"num_object_clones":0,"num_object_copies":13708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6854,"num_write_kb":27416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2784},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'6752","reported_seq":6849,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:36.929567+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:27:36.929567+0000","last_peered":"2024-09-17T15:27:36.929567+0000","last_clean":"2024-09-17T15:27:36.929567+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:27:36.929567+0000","last_undegraded":"2024-09-17T15:27:36.929567+0000","last_fullsized":"2024-09-17T15:27:36.929567+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":6752,"log_dups_size":0,"ondisk_log_size":6752,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27632582,"num_objects":6748,"num_object_clones":0,"num_object_copies":13496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6748,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5109,"num_write_kb":20436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'6784","reported_seq":6902,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:36.878301+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:27:36.878301+0000","last_peered":"2024-09-17T15:27:36.878301+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:27:36.878301+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":6784,"log_dups_size":0,"ondisk_log_size":6784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27770880,"num_objects":6780,"num_object_clones":0,"num_object_copies":13560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4941,"num_write_kb":19764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":2612}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'6845","reported_seq":6909,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:36.954485+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:27:36.954485+0000","last_peered":"2024-09-17T15:27:36.954485+0000","last_clean":"2024-09-17T15:27:36.954485+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:27:36.954485+0000","last_undegraded":"2024-09-17T15:27:36.954485+0000","last_fullsized":"2024-09-17T15:27:36.954485+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":6845,"log_dups_size":0,"ondisk_log_size":6845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28037120,"num_objects":6845,"num_object_clones":0,"num_object_copies":13690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6845,"num_write_kb":27380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'6851","reported_seq":6975,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:40.598974+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:27:40.598974+0000","last_peered":"2024-09-17T15:27:40.598974+0000","last_clean":"2024-09-17T15:27:40.598974+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:27:40.598974+0000","last_undegraded":"2024-09-17T15:27:40.598974+0000","last_fullsized":"2024-09-17T15:27:40.598974+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":6851,"log_dups_size":0,"ondisk_log_size":6851,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28033507,"num_objects":6845,"num_object_clones":0,"num_object_copies":13690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6845,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3833,"num_write_kb":15332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'7065","reported_seq":7195,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:40.590179+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:27:40.590179+0000","last_peered":"2024-09-17T15:27:40.590179+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:27:40.590179+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":7065,"log_dups_size":0,"ondisk_log_size":7065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28918243,"num_objects":7061,"num_object_clones":0,"num_object_copies":14122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7061,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3564,"num_write_kb":14256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2791},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'6971","reported_seq":7102,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:40.570648+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:27:40.570648+0000","last_peered":"2024-09-17T15:27:40.570648+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:27:40.570648+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":6971,"log_dups_size":0,"ondisk_log_size":6971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28529606,"num_objects":6967,"num_object_clones":0,"num_object_copies":13934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6967,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3740,"num_write_kb":14960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2793},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'6899","reported_seq":7022,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:40.598349+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:27:40.598349+0000","last_peered":"2024-09-17T15:27:40.598349+0000","last_clean":"2024-09-17T15:27:40.598349+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:27:40.598349+0000","last_undegraded":"2024-09-17T15:27:40.598349+0000","last_fullsized":"2024-09-17T15:27:40.598349+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":6899,"log_dups_size":0,"ondisk_log_size":6899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28250112,"num_objects":6897,"num_object_clones":0,"num_object_copies":13794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3853,"num_write_kb":15412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'6754","reported_seq":11380,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:27:36.946290+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:27:36.946290+0000","last_peered":"2024-09-17T15:27:36.946290+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:27:36.946290+0000","last_undegraded":"2024-09-17T15:27:36.946290+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":6754,"log_dups_size":0,"ondisk_log_size":6754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27636195,"num_objects":6748,"num_object_clones":0,"num_object_copies":13496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1821,"num_objects_unfound":0,"num_objects_dirty":6748,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4754,"num_write_kb":19016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2251,"num_bytes_recovered":9212387,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1821},{"shards":"2","objects":4927}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233600+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233600+0000","last_peered":"2024-09-17T15:26:22.233600+0000","last_clean":"2024-09-17T15:26:22.233600+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233600+0000","last_undegraded":"2024-09-17T15:26:22.233600+0000","last_fullsized":"2024-09-17T15:26:22.233600+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'6789","reported_seq":6914,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:40.556773+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:27:40.556773+0000","last_peered":"2024-09-17T15:27:40.556773+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:27:40.556773+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":6789,"log_dups_size":0,"ondisk_log_size":6789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27774976,"num_objects":6781,"num_object_clones":0,"num_object_copies":13562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3385,"num_write_kb":13540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2771},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":98,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412941+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412941+0000","last_peered":"2024-09-17T15:26:22.412941+0000","last_clean":"2024-09-17T15:26:22.412941+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412941+0000","last_undegraded":"2024-09-17T15:26:22.412941+0000","last_fullsized":"2024-09-17T15:26:22.412941+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'6732","reported_seq":6848,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:36.899161+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:27:36.899161+0000","last_peered":"2024-09-17T15:27:36.899161+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:27:36.899161+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":6732,"log_dups_size":0,"ondisk_log_size":6732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27554275,"num_objects":6728,"num_object_clones":0,"num_object_copies":13456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6728,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4627,"num_write_kb":18508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":2651}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'6767","reported_seq":6881,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:36.865869+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:27:36.865869+0000","last_peered":"2024-09-17T15:27:36.865869+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:27:36.865869+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":6767,"log_dups_size":0,"ondisk_log_size":6767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27693056,"num_objects":6761,"num_object_clones":0,"num_object_copies":13522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4529,"num_write_kb":18116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":2624}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'7008","reported_seq":7138,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:40.614714+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:27:40.614714+0000","last_peered":"2024-09-17T15:27:40.614714+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:27:40.614714+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":7008,"log_dups_size":0,"ondisk_log_size":7008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28696576,"num_objects":7006,"num_object_clones":0,"num_object_copies":14012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3556,"num_write_kb":14224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2868},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'6898","reported_seq":7034,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:40.581303+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:27:40.581303+0000","last_peered":"2024-09-17T15:27:40.581303+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:27:40.581303+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":6898,"log_dups_size":0,"ondisk_log_size":6898,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28221440,"num_objects":6890,"num_object_clones":0,"num_object_copies":13780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3185,"num_write_kb":12740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2733},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229061+0000","last_peered":"2024-09-17T15:26:22.229061+0000","last_clean":"2024-09-17T15:26:22.229061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229061+0000","last_undegraded":"2024-09-17T15:26:22.229061+0000","last_fullsized":"2024-09-17T15:26:22.229061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'6959","reported_seq":7086,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:40.529098+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:27:40.529098+0000","last_peered":"2024-09-17T15:27:40.529098+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:27:40.529098+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":6959,"log_dups_size":0,"ondisk_log_size":6959,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28487680,"num_objects":6955,"num_object_clones":0,"num_object_copies":13910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6955,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3746,"num_write_kb":14984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2892},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'6861","reported_seq":6998,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:40.620763+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:27:40.620763+0000","last_peered":"2024-09-17T15:27:40.620763+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:27:40.620763+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":6861,"log_dups_size":0,"ondisk_log_size":6861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28054470,"num_objects":6851,"num_object_clones":0,"num_object_copies":13702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3303,"num_write_kb":13212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2742},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":449364702,"num_objects":109717,"num_object_clones":0,"num_object_copies":219434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":1821,"num_objects_unfound":0,"num_objects_dirty":109717,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":69824,"num_write_kb":279296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2251,"num_bytes_recovered":9212387,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":909131776,"data_stored":908980030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109789,"ondisk_log_size":109789,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051615,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":520444,"kb_used_data":348524,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93851396,"statfs":{"total":96636764160,"available":96103829504,"internally_reserved":0,"allocated":356888576,"data_stored":356622403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":176025260},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,0,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724561,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":266460,"kb_used_data":140360,"kb_used_omap":4,"kb_used_meta":126075,"kb_avail":94105380,"statfs":{"total":96636764160,"available":96363909120,"internally_reserved":0,"allocated":143728640,"data_stored":143415034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":129101483},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,0,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247844,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":646012,"kb_used_data":397804,"kb_used_omap":4,"kb_used_meta":248187,"kb_avail":93725828,"statfs":{"total":96636764160,"available":95975247872,"internally_reserved":0,"allocated":407351296,"data_stored":407081046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":254144168},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,5,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":407220224,"data_stored":407184094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":145108992,"data_stored":145029506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":356802560,"data_stored":356766430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:27:42.094 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:27:42.310 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 123 20 111306 111286 3.53344 2 0.0235936 0.0221031 2024-09-17T15:27:42.644 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:27:42.644 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:27:42.670 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":133,"stamp":"2024-09-17T15:27:41.622915+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":449364721,"num_objects":109719,"num_object_clones":0,"num_object_copies":219438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":1821,"num_objects_unfound":0,"num_objects_dirty":109719,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":69826,"num_write_kb":279298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2252,"num_bytes_recovered":9212406,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109792,"ondisk_log_size":109792,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1432916,"kb_used_data":886688,"kb_used_omap":13,"kb_used_meta":546162,"kb_avail":281682604,"statfs":{"total":289910292480,"available":288442986496,"internally_reserved":0,"allocated":907968512,"data_stored":907118483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":559270911},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,9,3,9,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28581888,"num_objects":6978,"num_object_clones":0,"num_object_copies":13956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-371,"num_objects_unfound":0,"num_objects_dirty":6978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6978,"num_write_kb":27912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":372,"num_bytes_recovered":1516003,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003013"},"pg_stats":[{"pgid":"2.f","version":"41'6854","reported_seq":6927,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:40.588045+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:27:40.588045+0000","last_peered":"2024-09-17T15:27:40.588045+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:27:40.588045+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":6854,"log_dups_size":0,"ondisk_log_size":6854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28073984,"num_objects":6854,"num_object_clones":0,"num_object_copies":13708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6854,"num_write_kb":27416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2784},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'6752","reported_seq":6849,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:36.929567+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:27:36.929567+0000","last_peered":"2024-09-17T15:27:36.929567+0000","last_clean":"2024-09-17T15:27:36.929567+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:27:36.929567+0000","last_undegraded":"2024-09-17T15:27:36.929567+0000","last_fullsized":"2024-09-17T15:27:36.929567+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":6752,"log_dups_size":0,"ondisk_log_size":6752,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27632582,"num_objects":6748,"num_object_clones":0,"num_object_copies":13496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6748,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5109,"num_write_kb":20436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'6784","reported_seq":6902,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:36.878301+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:27:36.878301+0000","last_peered":"2024-09-17T15:27:36.878301+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:27:36.878301+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":6784,"log_dups_size":0,"ondisk_log_size":6784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27770880,"num_objects":6780,"num_object_clones":0,"num_object_copies":13560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4941,"num_write_kb":19764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":2612}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'6845","reported_seq":6909,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:36.954485+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:27:36.954485+0000","last_peered":"2024-09-17T15:27:36.954485+0000","last_clean":"2024-09-17T15:27:36.954485+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:27:36.954485+0000","last_undegraded":"2024-09-17T15:27:36.954485+0000","last_fullsized":"2024-09-17T15:27:36.954485+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":6845,"log_dups_size":0,"ondisk_log_size":6845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28037120,"num_objects":6845,"num_object_clones":0,"num_object_copies":13690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6845,"num_write_kb":27380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'6851","reported_seq":6975,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:40.598974+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:27:40.598974+0000","last_peered":"2024-09-17T15:27:40.598974+0000","last_clean":"2024-09-17T15:27:40.598974+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:27:40.598974+0000","last_undegraded":"2024-09-17T15:27:40.598974+0000","last_fullsized":"2024-09-17T15:27:40.598974+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":6851,"log_dups_size":0,"ondisk_log_size":6851,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28033507,"num_objects":6845,"num_object_clones":0,"num_object_copies":13690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6845,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3833,"num_write_kb":15332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'7065","reported_seq":7195,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:40.590179+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:27:40.590179+0000","last_peered":"2024-09-17T15:27:40.590179+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:27:40.590179+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":7065,"log_dups_size":0,"ondisk_log_size":7065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28918243,"num_objects":7061,"num_object_clones":0,"num_object_copies":14122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7061,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3564,"num_write_kb":14256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2791},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'6971","reported_seq":7102,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:40.570648+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:27:40.570648+0000","last_peered":"2024-09-17T15:27:40.570648+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:27:40.570648+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":6971,"log_dups_size":0,"ondisk_log_size":6971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28529606,"num_objects":6967,"num_object_clones":0,"num_object_copies":13934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6967,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3740,"num_write_kb":14960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2793},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'6899","reported_seq":7022,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:40.598349+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:27:40.598349+0000","last_peered":"2024-09-17T15:27:40.598349+0000","last_clean":"2024-09-17T15:27:40.598349+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:27:40.598349+0000","last_undegraded":"2024-09-17T15:27:40.598349+0000","last_fullsized":"2024-09-17T15:27:40.598349+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":6899,"log_dups_size":0,"ondisk_log_size":6899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28250112,"num_objects":6897,"num_object_clones":0,"num_object_copies":13794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3853,"num_write_kb":15412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'6754","reported_seq":11380,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:27:36.946290+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:27:36.946290+0000","last_peered":"2024-09-17T15:27:36.946290+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:27:36.946290+0000","last_undegraded":"2024-09-17T15:27:36.946290+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":6754,"log_dups_size":0,"ondisk_log_size":6754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27636195,"num_objects":6748,"num_object_clones":0,"num_object_copies":13496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1821,"num_objects_unfound":0,"num_objects_dirty":6748,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4754,"num_write_kb":19016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2251,"num_bytes_recovered":9212387,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1821},{"shards":"2","objects":4927}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233600+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233600+0000","last_peered":"2024-09-17T15:26:22.233600+0000","last_clean":"2024-09-17T15:26:22.233600+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233600+0000","last_undegraded":"2024-09-17T15:26:22.233600+0000","last_fullsized":"2024-09-17T15:26:22.233600+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'6789","reported_seq":6914,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:40.556773+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:27:40.556773+0000","last_peered":"2024-09-17T15:27:40.556773+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:27:40.556773+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":6789,"log_dups_size":0,"ondisk_log_size":6789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27774976,"num_objects":6781,"num_object_clones":0,"num_object_copies":13562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3385,"num_write_kb":13540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2771},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":98,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412941+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412941+0000","last_peered":"2024-09-17T15:26:22.412941+0000","last_clean":"2024-09-17T15:26:22.412941+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412941+0000","last_undegraded":"2024-09-17T15:26:22.412941+0000","last_fullsized":"2024-09-17T15:26:22.412941+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'6732","reported_seq":6848,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:36.899161+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:27:36.899161+0000","last_peered":"2024-09-17T15:27:36.899161+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:27:36.899161+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":6732,"log_dups_size":0,"ondisk_log_size":6732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27554275,"num_objects":6728,"num_object_clones":0,"num_object_copies":13456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6728,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4627,"num_write_kb":18508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":2651}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'6767","reported_seq":6881,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:36.865869+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:27:36.865869+0000","last_peered":"2024-09-17T15:27:36.865869+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:27:36.865869+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":6767,"log_dups_size":0,"ondisk_log_size":6767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27693056,"num_objects":6761,"num_object_clones":0,"num_object_copies":13522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4529,"num_write_kb":18116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":2624}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'7008","reported_seq":7138,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:40.614714+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:27:40.614714+0000","last_peered":"2024-09-17T15:27:40.614714+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:27:40.614714+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":7008,"log_dups_size":0,"ondisk_log_size":7008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28696576,"num_objects":7006,"num_object_clones":0,"num_object_copies":14012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3556,"num_write_kb":14224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2868},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'6898","reported_seq":7034,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:40.581303+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:27:40.581303+0000","last_peered":"2024-09-17T15:27:40.581303+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:27:40.581303+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":6898,"log_dups_size":0,"ondisk_log_size":6898,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28221440,"num_objects":6890,"num_object_clones":0,"num_object_copies":13780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3185,"num_write_kb":12740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2733},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229061+0000","last_peered":"2024-09-17T15:26:22.229061+0000","last_clean":"2024-09-17T15:26:22.229061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229061+0000","last_undegraded":"2024-09-17T15:26:22.229061+0000","last_fullsized":"2024-09-17T15:26:22.229061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'6959","reported_seq":7086,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:40.529098+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:27:40.529098+0000","last_peered":"2024-09-17T15:27:40.529098+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:27:40.529098+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":6959,"log_dups_size":0,"ondisk_log_size":6959,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28487680,"num_objects":6955,"num_object_clones":0,"num_object_copies":13910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6955,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3746,"num_write_kb":14984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2892},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'6861","reported_seq":6998,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:40.620763+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:27:40.620763+0000","last_peered":"2024-09-17T15:27:40.620763+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:27:40.620763+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":6861,"log_dups_size":0,"ondisk_log_size":6861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28054470,"num_objects":6851,"num_object_clones":0,"num_object_copies":13702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3303,"num_write_kb":13212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2742},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":449364702,"num_objects":109717,"num_object_clones":0,"num_object_copies":219434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":1821,"num_objects_unfound":0,"num_objects_dirty":109717,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":69824,"num_write_kb":279296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2251,"num_bytes_recovered":9212387,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":909131776,"data_stored":908980030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109789,"ondisk_log_size":109789,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051615,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":520444,"kb_used_data":348524,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93851396,"statfs":{"total":96636764160,"available":96103829504,"internally_reserved":0,"allocated":356888576,"data_stored":356622403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":176025260},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,0,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724561,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":266460,"kb_used_data":140360,"kb_used_omap":4,"kb_used_meta":126075,"kb_avail":94105380,"statfs":{"total":96636764160,"available":96363909120,"internally_reserved":0,"allocated":143728640,"data_stored":143415034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":129101483},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,0,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247844,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":646012,"kb_used_data":397804,"kb_used_omap":4,"kb_used_meta":248187,"kb_avail":93725828,"statfs":{"total":96636764160,"available":95975247872,"internally_reserved":0,"allocated":407351296,"data_stored":407081046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":254144168},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,5,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":407220224,"data_stored":407184094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":145108992,"data_stored":145029506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":356802560,"data_stored":356766430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:27:42.671 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:27:42.671 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'6854", 'reported_seq': 6927, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:40.588045+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:27:40.588045+0000', 'last_peered': '2024-09-17T15:27:40.588045+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:27:40.588045+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 6854, 'log_dups_size': 0, 'ondisk_log_size': 6854, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28073984, 'num_objects': 6854, 'num_object_clones': 0, 'num_object_copies': 13708, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4070, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6854, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6854, 'num_write_kb': 27416, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2784}, {'shards': '0,2', 'objects': 4070}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:42.672 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:27:42.672 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'6784", 'reported_seq': 6902, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:36.878301+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:27:36.878301+0000', 'last_peered': '2024-09-17T15:27:36.878301+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:27:36.878301+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 6784, 'log_dups_size': 0, 'ondisk_log_size': 6784, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27770880, 'num_objects': 6780, 'num_object_clones': 0, 'num_object_copies': 13560, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4169, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6780, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4941, 'num_write_kb': 19764, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4168}, {'shards': '2', 'objects': 2612}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:27:42.672 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:27:42.672 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'7065", 'reported_seq': 7195, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:40.590179+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:27:40.590179+0000', 'last_peered': '2024-09-17T15:27:40.590179+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:27:40.590179+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 7065, 'log_dups_size': 0, 'ondisk_log_size': 7065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28918243, 'num_objects': 7061, 'num_object_clones': 0, 'num_object_copies': 14122, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4271, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7061, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3564, 'num_write_kb': 14256, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2791}, {'shards': '0,2', 'objects': 4268}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:42.672 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:27:42.672 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'6971", 'reported_seq': 7102, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:40.570648+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:27:40.570648+0000', 'last_peered': '2024-09-17T15:27:40.570648+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:27:40.570648+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 6971, 'log_dups_size': 0, 'ondisk_log_size': 6971, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28529606, 'num_objects': 6967, 'num_object_clones': 0, 'num_object_copies': 13934, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4174, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6967, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 3740, 'num_write_kb': 14960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2793}, {'shards': '0,2', 'objects': 4174}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:42.672 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-17T15:27:42.672 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "41'6754", 'reported_seq': 11380, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:27:36.946290+0000', 'last_change': '2024-09-17T15:26:22.588242+0000', 'last_active': '2024-09-17T15:27:36.946290+0000', 'last_peered': '2024-09-17T15:27:36.946290+0000', 'last_clean': '2024-09-17T15:26:20.155646+0000', 'last_became_active': '2024-09-17T15:26:22.557742+0000', 'last_became_peered': '2024-09-17T15:26:22.557742+0000', 'last_unstale': '2024-09-17T15:27:36.946290+0000', 'last_undegraded': '2024-09-17T15:27:36.946290+0000', 'last_fullsized': '2024-09-17T15:26:22.188850+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2016", 'last_scrub_stamp': '2024-09-17T15:25:53.959011+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:53.959011+0000', 'objects_scrubbed': 1994, 'log_size': 6754, 'log_dups_size': 0, 'ondisk_log_size': 6754, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 568, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27636195, 'num_objects': 6748, 'num_object_clones': 0, 'num_object_copies': 13496, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1821, 'num_objects_unfound': 0, 'num_objects_dirty': 6748, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4754, 'num_write_kb': 19016, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2251, 'num_bytes_recovered': 9212387, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1821}, {'shards': '2', 'objects': 4927}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:27:42.672 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:27:42.673 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'6789", 'reported_seq': 6914, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:40.556773+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:27:40.556773+0000', 'last_peered': '2024-09-17T15:27:40.556773+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:27:40.556773+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 6789, 'log_dups_size': 0, 'ondisk_log_size': 6789, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27774976, 'num_objects': 6781, 'num_object_clones': 0, 'num_object_copies': 13562, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4012, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6781, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3385, 'num_write_kb': 13540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2771}, {'shards': '0,2', 'objects': 4010}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:42.673 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:27:42.673 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'6732", 'reported_seq': 6848, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:36.899161+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:27:36.899161+0000', 'last_peered': '2024-09-17T15:27:36.899161+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:27:36.899161+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 6732, 'log_dups_size': 0, 'ondisk_log_size': 6732, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27554275, 'num_objects': 6728, 'num_object_clones': 0, 'num_object_copies': 13456, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4078, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6728, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4627, 'num_write_kb': 18508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4077}, {'shards': '2', 'objects': 2651}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:27:42.673 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:27:42.673 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'6767", 'reported_seq': 6881, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:36.865869+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:27:36.865869+0000', 'last_peered': '2024-09-17T15:27:36.865869+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:27:36.865869+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 6767, 'log_dups_size': 0, 'ondisk_log_size': 6767, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27693056, 'num_objects': 6761, 'num_object_clones': 0, 'num_object_copies': 13522, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6761, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4529, 'num_write_kb': 18116, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4137}, {'shards': '2', 'objects': 2624}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:27:42.673 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:27:42.673 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'7008", 'reported_seq': 7138, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:40.614714+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:27:40.614714+0000', 'last_peered': '2024-09-17T15:27:40.614714+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:27:40.614714+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 7008, 'log_dups_size': 0, 'ondisk_log_size': 7008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28696576, 'num_objects': 7006, 'num_object_clones': 0, 'num_object_copies': 14012, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7006, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3556, 'num_write_kb': 14224, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2868}, {'shards': '0,2', 'objects': 4138}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:42.673 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:27:42.673 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'6898", 'reported_seq': 7034, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:40.581303+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:27:40.581303+0000', 'last_peered': '2024-09-17T15:27:40.581303+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:27:40.581303+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 6898, 'log_dups_size': 0, 'ondisk_log_size': 6898, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28221440, 'num_objects': 6890, 'num_object_clones': 0, 'num_object_copies': 13780, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4160, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6890, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3185, 'num_write_kb': 12740, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2733}, {'shards': '0,2', 'objects': 4157}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:42.673 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:27:42.674 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'6959", 'reported_seq': 7086, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:40.529098+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:27:40.529098+0000', 'last_peered': '2024-09-17T15:27:40.529098+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:27:40.529098+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 6959, 'log_dups_size': 0, 'ondisk_log_size': 6959, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28487680, 'num_objects': 6955, 'num_object_clones': 0, 'num_object_copies': 13910, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4065, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6955, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3746, 'num_write_kb': 14984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2892}, {'shards': '0,2', 'objects': 4063}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:42.674 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:27:42.674 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'6861", 'reported_seq': 6998, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:40.620763+0000', 'last_change': '2024-09-17T15:26:22.654521+0000', 'last_active': '2024-09-17T15:27:40.620763+0000', 'last_peered': '2024-09-17T15:27:40.620763+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:27:40.620763+0000', 'last_undegraded': '2024-09-17T15:26:22.307795+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 6861, 'log_dups_size': 0, 'ondisk_log_size': 6861, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28054470, 'num_objects': 6851, 'num_object_clones': 0, 'num_object_copies': 13702, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4111, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6851, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3303, 'num_write_kb': 13212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2742}, {'shards': '0,2', 'objects': 4109}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:42.674 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:27:43.313 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 124 20 111794 111774 3.52032 1.90625 0.0308647 0.0221863 2024-09-17T15:27:43.358 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:27:43.359 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45388/219438 objects degraded (20.684%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":200,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":109719,"data_bytes":449364721,"bytes_used":1467305984,"bytes_avail":288442986496,"bytes_total":289910292480,"degraded_objects":45388,"degraded_total":219438,"degraded_ratio":0.20683746661927288,"misplaced_objects":1821,"misplaced_total":219438,"misplaced_ratio":0.0082984715500505832,"recovering_objects_per_sec":30,"recovering_bytes_per_sec":126301,"recovering_keys_per_sec":0,"num_objects_recovered":372,"num_bytes_recovered":1516003,"num_keys_recovered":0,"write_bytes_sec":2381225,"read_op_per_sec":0,"write_op_per_sec":581},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T15:27:19.616376+0000","services":{}},"progress_events":{}} 2024-09-17T15:27:43.380 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:27:43.380 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:27:43.926 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:27:43.928 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:27:43.953 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":134,"stamp":"2024-09-17T15:27:43.623442+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":453333745,"num_objects":110688,"num_object_clones":0,"num_object_copies":221376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":1639,"num_objects_unfound":0,"num_objects_dirty":110688,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":70795,"num_write_kb":283174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2434,"num_bytes_recovered":9957878,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110761,"ondisk_log_size":110761,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1442236,"kb_used_data":891848,"kb_used_omap":13,"kb_used_meta":550322,"kb_avail":281673284,"statfs":{"total":289910292480,"available":288433442816,"internally_reserved":0,"allocated":913252352,"data_stored":912402323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":563530751},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,8,4,6,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25731072,"num_objects":6282,"num_object_clones":0,"num_object_copies":12564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-553,"num_objects_unfound":0,"num_objects_dirty":6282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6282,"num_write_kb":25128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":554,"num_bytes_recovered":2261475,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002954"},"pg_stats":[{"pgid":"2.f","version":"41'6854","reported_seq":6927,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:40.588045+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:27:40.588045+0000","last_peered":"2024-09-17T15:27:40.588045+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:27:40.588045+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":6854,"log_dups_size":0,"ondisk_log_size":6854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28073984,"num_objects":6854,"num_object_clones":0,"num_object_copies":13708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6854,"num_write_kb":27416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2784},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'6926","reported_seq":7024,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:41.842792+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:27:41.842792+0000","last_peered":"2024-09-17T15:27:41.842792+0000","last_clean":"2024-09-17T15:27:41.842792+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:27:41.842792+0000","last_undegraded":"2024-09-17T15:27:41.842792+0000","last_fullsized":"2024-09-17T15:27:41.842792+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":6926,"log_dups_size":0,"ondisk_log_size":6926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28345286,"num_objects":6922,"num_object_clones":0,"num_object_copies":13844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6922,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5283,"num_write_kb":21132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'6936","reported_seq":7056,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:41.825438+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:27:41.825438+0000","last_peered":"2024-09-17T15:27:41.825438+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:27:41.825438+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":6936,"log_dups_size":0,"ondisk_log_size":6936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28393472,"num_objects":6932,"num_object_clones":0,"num_object_copies":13864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6932,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5093,"num_write_kb":20372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":2764}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'7008","reported_seq":7073,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:41.971819+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:27:41.971819+0000","last_peered":"2024-09-17T15:27:41.971819+0000","last_clean":"2024-09-17T15:27:41.971819+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:27:41.971819+0000","last_undegraded":"2024-09-17T15:27:41.971819+0000","last_fullsized":"2024-09-17T15:27:41.971819+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":7008,"log_dups_size":0,"ondisk_log_size":7008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28704768,"num_objects":7008,"num_object_clones":0,"num_object_copies":14016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7008,"num_write_kb":28032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'6851","reported_seq":6975,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:40.598974+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:27:40.598974+0000","last_peered":"2024-09-17T15:27:40.598974+0000","last_clean":"2024-09-17T15:27:40.598974+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:27:40.598974+0000","last_undegraded":"2024-09-17T15:27:40.598974+0000","last_fullsized":"2024-09-17T15:27:40.598974+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":6851,"log_dups_size":0,"ondisk_log_size":6851,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28033507,"num_objects":6845,"num_object_clones":0,"num_object_copies":13690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6845,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3833,"num_write_kb":15332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'7065","reported_seq":7195,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:40.590179+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:27:40.590179+0000","last_peered":"2024-09-17T15:27:40.590179+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:27:40.590179+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":7065,"log_dups_size":0,"ondisk_log_size":7065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28918243,"num_objects":7061,"num_object_clones":0,"num_object_copies":14122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7061,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3564,"num_write_kb":14256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2791},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'6971","reported_seq":7102,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:40.570648+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:27:40.570648+0000","last_peered":"2024-09-17T15:27:40.570648+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:27:40.570648+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":6971,"log_dups_size":0,"ondisk_log_size":6971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28529606,"num_objects":6967,"num_object_clones":0,"num_object_copies":13934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6967,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3740,"num_write_kb":14960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2793},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'6899","reported_seq":7022,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:40.598349+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:27:40.598349+0000","last_peered":"2024-09-17T15:27:40.598349+0000","last_clean":"2024-09-17T15:27:40.598349+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:27:40.598349+0000","last_undegraded":"2024-09-17T15:27:40.598349+0000","last_fullsized":"2024-09-17T15:27:40.598349+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":6899,"log_dups_size":0,"ondisk_log_size":6899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28250112,"num_objects":6897,"num_object_clones":0,"num_object_copies":13794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3853,"num_write_kb":15412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'6899","reported_seq":11889,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:27:41.952447+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:27:41.952447+0000","last_peered":"2024-09-17T15:27:41.952447+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:27:41.952447+0000","last_undegraded":"2024-09-17T15:27:41.952447+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":6899,"log_dups_size":0,"ondisk_log_size":6899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28230115,"num_objects":6893,"num_object_clones":0,"num_object_copies":13786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1639,"num_objects_unfound":0,"num_objects_dirty":6893,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4899,"num_write_kb":19596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2433,"num_bytes_recovered":9957859,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1639},{"shards":"2","objects":5254}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233600+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233600+0000","last_peered":"2024-09-17T15:26:22.233600+0000","last_clean":"2024-09-17T15:26:22.233600+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233600+0000","last_undegraded":"2024-09-17T15:26:22.233600+0000","last_fullsized":"2024-09-17T15:26:22.233600+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'6789","reported_seq":6914,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:40.556773+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:27:40.556773+0000","last_peered":"2024-09-17T15:27:40.556773+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:27:40.556773+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":6789,"log_dups_size":0,"ondisk_log_size":6789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27774976,"num_objects":6781,"num_object_clones":0,"num_object_copies":13562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3385,"num_write_kb":13540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2771},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":98,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412941+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412941+0000","last_peered":"2024-09-17T15:26:22.412941+0000","last_clean":"2024-09-17T15:26:22.412941+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412941+0000","last_undegraded":"2024-09-17T15:26:22.412941+0000","last_fullsized":"2024-09-17T15:26:22.412941+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'6899","reported_seq":7015,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:41.918270+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:27:41.918270+0000","last_peered":"2024-09-17T15:27:41.918270+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:27:41.918270+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":6899,"log_dups_size":0,"ondisk_log_size":6899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28238307,"num_objects":6895,"num_object_clones":0,"num_object_copies":13790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6895,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4794,"num_write_kb":19176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":2818}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'6935","reported_seq":7049,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:41.906293+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:27:41.906293+0000","last_peered":"2024-09-17T15:27:41.906293+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:27:41.906293+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":6935,"log_dups_size":0,"ondisk_log_size":6935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28381184,"num_objects":6929,"num_object_clones":0,"num_object_copies":13858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4697,"num_write_kb":18788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":2792}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'7008","reported_seq":7138,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:40.614714+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:27:40.614714+0000","last_peered":"2024-09-17T15:27:40.614714+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:27:40.614714+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":7008,"log_dups_size":0,"ondisk_log_size":7008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28696576,"num_objects":7006,"num_object_clones":0,"num_object_copies":14012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3556,"num_write_kb":14224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2868},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'6898","reported_seq":7034,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:40.581303+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:27:40.581303+0000","last_peered":"2024-09-17T15:27:40.581303+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:27:40.581303+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":6898,"log_dups_size":0,"ondisk_log_size":6898,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28221440,"num_objects":6890,"num_object_clones":0,"num_object_copies":13780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3185,"num_write_kb":12740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2733},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229061+0000","last_peered":"2024-09-17T15:26:22.229061+0000","last_clean":"2024-09-17T15:26:22.229061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229061+0000","last_undegraded":"2024-09-17T15:26:22.229061+0000","last_fullsized":"2024-09-17T15:26:22.229061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'6959","reported_seq":7086,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:40.529098+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:27:40.529098+0000","last_peered":"2024-09-17T15:27:40.529098+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:27:40.529098+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":6959,"log_dups_size":0,"ondisk_log_size":6959,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28487680,"num_objects":6955,"num_object_clones":0,"num_object_copies":13910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6955,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3746,"num_write_kb":14984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2892},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'6861","reported_seq":6998,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:40.620763+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:27:40.620763+0000","last_peered":"2024-09-17T15:27:40.620763+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:27:40.620763+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":6861,"log_dups_size":0,"ondisk_log_size":6861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28054470,"num_objects":6851,"num_object_clones":0,"num_object_copies":13702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3303,"num_write_kb":13212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2742},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":453333726,"num_objects":110686,"num_object_clones":0,"num_object_copies":221372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":1639,"num_objects_unfound":0,"num_objects_dirty":110686,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":70793,"num_write_kb":283172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2433,"num_bytes_recovered":9957859,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":914419712,"data_stored":914267966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110758,"ondisk_log_size":110758,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051616,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":529764,"kb_used_data":353684,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93842076,"statfs":{"total":96636764160,"available":96094285824,"internally_reserved":0,"allocated":362172416,"data_stored":361906243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":180285100},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724561,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":266460,"kb_used_data":140360,"kb_used_omap":4,"kb_used_meta":126075,"kb_avail":94105380,"statfs":{"total":96636764160,"available":96363909120,"internally_reserved":0,"allocated":143728640,"data_stored":143415034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":129101483},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,0,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247844,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":646012,"kb_used_data":397804,"kb_used_omap":4,"kb_used_meta":248187,"kb_avail":93725828,"statfs":{"total":96636764160,"available":95975247872,"internally_reserved":0,"allocated":407351296,"data_stored":407081046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":254144168},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,5,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":407220224,"data_stored":407184094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":145108992,"data_stored":145029506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":362090496,"data_stored":362054366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:27:44.311 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 125 20 112305 112285 3.50813 1.99609 0.0400432 0.0222623 2024-09-17T15:27:45.311 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 126 20 112853 112833 3.49727 2.14062 0.0408994 0.0223314 2024-09-17T15:27:46.311 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 127 20 113415 113395 3.48702 2.19531 0.0232376 0.022397 2024-09-17T15:27:46.955 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:27:47.311 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 128 20 113945 113925 3.47595 2.07031 0.0198507 0.0224675 2024-09-17T15:27:47.510 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:27:47.513 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:27:47.533 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":135,"stamp":"2024-09-17T15:27:45.623966+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":453333745,"num_objects":110688,"num_object_clones":0,"num_object_copies":221376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":1639,"num_objects_unfound":0,"num_objects_dirty":110688,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":70795,"num_write_kb":283174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2434,"num_bytes_recovered":9957878,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110761,"ondisk_log_size":110761,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1462208,"kb_used_data":902096,"kb_used_omap":13,"kb_used_meta":560050,"kb_avail":281653312,"statfs":{"total":289910292480,"available":288412991488,"internally_reserved":0,"allocated":923746304,"data_stored":922896275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":573492223},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,0,4,5,7,18,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21671936,"num_objects":5291,"num_object_clones":0,"num_object_copies":10582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-359,"num_objects_unfound":0,"num_objects_dirty":5291,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5291,"num_write_kb":21164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":359,"num_bytes_recovered":1470464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002980"},"pg_stats":[{"pgid":"2.f","version":"41'6854","reported_seq":6927,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:40.588045+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:27:40.588045+0000","last_peered":"2024-09-17T15:27:40.588045+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:27:40.588045+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":6854,"log_dups_size":0,"ondisk_log_size":6854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28073984,"num_objects":6854,"num_object_clones":0,"num_object_copies":13708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6854,"num_write_kb":27416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2784},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'6926","reported_seq":7024,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:41.842792+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:27:41.842792+0000","last_peered":"2024-09-17T15:27:41.842792+0000","last_clean":"2024-09-17T15:27:41.842792+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:27:41.842792+0000","last_undegraded":"2024-09-17T15:27:41.842792+0000","last_fullsized":"2024-09-17T15:27:41.842792+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":6926,"log_dups_size":0,"ondisk_log_size":6926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28345286,"num_objects":6922,"num_object_clones":0,"num_object_copies":13844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6922,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5283,"num_write_kb":21132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'6936","reported_seq":7056,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:41.825438+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:27:41.825438+0000","last_peered":"2024-09-17T15:27:41.825438+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:27:41.825438+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":6936,"log_dups_size":0,"ondisk_log_size":6936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28393472,"num_objects":6932,"num_object_clones":0,"num_object_copies":13864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6932,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5093,"num_write_kb":20372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":2764}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'7008","reported_seq":7073,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:41.971819+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:27:41.971819+0000","last_peered":"2024-09-17T15:27:41.971819+0000","last_clean":"2024-09-17T15:27:41.971819+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:27:41.971819+0000","last_undegraded":"2024-09-17T15:27:41.971819+0000","last_fullsized":"2024-09-17T15:27:41.971819+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":7008,"log_dups_size":0,"ondisk_log_size":7008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28704768,"num_objects":7008,"num_object_clones":0,"num_object_copies":14016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7008,"num_write_kb":28032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'6851","reported_seq":6975,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:40.598974+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:27:40.598974+0000","last_peered":"2024-09-17T15:27:40.598974+0000","last_clean":"2024-09-17T15:27:40.598974+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:27:40.598974+0000","last_undegraded":"2024-09-17T15:27:40.598974+0000","last_fullsized":"2024-09-17T15:27:40.598974+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":6851,"log_dups_size":0,"ondisk_log_size":6851,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28033507,"num_objects":6845,"num_object_clones":0,"num_object_copies":13690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6845,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3833,"num_write_kb":15332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'7065","reported_seq":7195,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:40.590179+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:27:40.590179+0000","last_peered":"2024-09-17T15:27:40.590179+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:27:40.590179+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":7065,"log_dups_size":0,"ondisk_log_size":7065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28918243,"num_objects":7061,"num_object_clones":0,"num_object_copies":14122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7061,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3564,"num_write_kb":14256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2791},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'6971","reported_seq":7102,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:40.570648+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:27:40.570648+0000","last_peered":"2024-09-17T15:27:40.570648+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:27:40.570648+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":6971,"log_dups_size":0,"ondisk_log_size":6971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28529606,"num_objects":6967,"num_object_clones":0,"num_object_copies":13934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6967,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3740,"num_write_kb":14960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2793},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'6899","reported_seq":7022,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:40.598349+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:27:40.598349+0000","last_peered":"2024-09-17T15:27:40.598349+0000","last_clean":"2024-09-17T15:27:40.598349+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:27:40.598349+0000","last_undegraded":"2024-09-17T15:27:40.598349+0000","last_fullsized":"2024-09-17T15:27:40.598349+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":6899,"log_dups_size":0,"ondisk_log_size":6899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28250112,"num_objects":6897,"num_object_clones":0,"num_object_copies":13794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3853,"num_write_kb":15412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'6899","reported_seq":11889,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:27:41.952447+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:27:41.952447+0000","last_peered":"2024-09-17T15:27:41.952447+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:27:41.952447+0000","last_undegraded":"2024-09-17T15:27:41.952447+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":6899,"log_dups_size":0,"ondisk_log_size":6899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28230115,"num_objects":6893,"num_object_clones":0,"num_object_copies":13786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1639,"num_objects_unfound":0,"num_objects_dirty":6893,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4899,"num_write_kb":19596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2433,"num_bytes_recovered":9957859,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1639},{"shards":"2","objects":5254}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233600+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233600+0000","last_peered":"2024-09-17T15:26:22.233600+0000","last_clean":"2024-09-17T15:26:22.233600+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233600+0000","last_undegraded":"2024-09-17T15:26:22.233600+0000","last_fullsized":"2024-09-17T15:26:22.233600+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'6789","reported_seq":6914,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:40.556773+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:27:40.556773+0000","last_peered":"2024-09-17T15:27:40.556773+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:27:40.556773+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":6789,"log_dups_size":0,"ondisk_log_size":6789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27774976,"num_objects":6781,"num_object_clones":0,"num_object_copies":13562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3385,"num_write_kb":13540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2771},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":98,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412941+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412941+0000","last_peered":"2024-09-17T15:26:22.412941+0000","last_clean":"2024-09-17T15:26:22.412941+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412941+0000","last_undegraded":"2024-09-17T15:26:22.412941+0000","last_fullsized":"2024-09-17T15:26:22.412941+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'6899","reported_seq":7015,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:41.918270+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:27:41.918270+0000","last_peered":"2024-09-17T15:27:41.918270+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:27:41.918270+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":6899,"log_dups_size":0,"ondisk_log_size":6899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28238307,"num_objects":6895,"num_object_clones":0,"num_object_copies":13790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6895,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4794,"num_write_kb":19176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":2818}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'6935","reported_seq":7049,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:41.906293+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:27:41.906293+0000","last_peered":"2024-09-17T15:27:41.906293+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:27:41.906293+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":6935,"log_dups_size":0,"ondisk_log_size":6935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28381184,"num_objects":6929,"num_object_clones":0,"num_object_copies":13858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4697,"num_write_kb":18788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":2792}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'7008","reported_seq":7138,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:40.614714+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:27:40.614714+0000","last_peered":"2024-09-17T15:27:40.614714+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:27:40.614714+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":7008,"log_dups_size":0,"ondisk_log_size":7008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28696576,"num_objects":7006,"num_object_clones":0,"num_object_copies":14012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3556,"num_write_kb":14224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2868},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'6898","reported_seq":7034,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:40.581303+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:27:40.581303+0000","last_peered":"2024-09-17T15:27:40.581303+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:27:40.581303+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":6898,"log_dups_size":0,"ondisk_log_size":6898,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28221440,"num_objects":6890,"num_object_clones":0,"num_object_copies":13780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3185,"num_write_kb":12740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2733},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229061+0000","last_peered":"2024-09-17T15:26:22.229061+0000","last_clean":"2024-09-17T15:26:22.229061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229061+0000","last_undegraded":"2024-09-17T15:26:22.229061+0000","last_fullsized":"2024-09-17T15:26:22.229061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'6959","reported_seq":7086,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:40.529098+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:27:40.529098+0000","last_peered":"2024-09-17T15:27:40.529098+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:27:40.529098+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":6959,"log_dups_size":0,"ondisk_log_size":6959,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28487680,"num_objects":6955,"num_object_clones":0,"num_object_copies":13910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6955,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3746,"num_write_kb":14984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2892},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'6861","reported_seq":6998,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:40.620763+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:27:40.620763+0000","last_peered":"2024-09-17T15:27:40.620763+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:27:40.620763+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":6861,"log_dups_size":0,"ondisk_log_size":6861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28054470,"num_objects":6851,"num_object_clones":0,"num_object_copies":13702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3303,"num_write_kb":13212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2742},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":453333726,"num_objects":110686,"num_object_clones":0,"num_object_copies":221372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":1639,"num_objects_unfound":0,"num_objects_dirty":110686,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":70793,"num_write_kb":283172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2433,"num_bytes_recovered":9957859,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":923295744,"data_stored":923143998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110758,"ondisk_log_size":110758,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051616,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":529764,"kb_used_data":353684,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93842076,"statfs":{"total":96636764160,"available":96094285824,"internally_reserved":0,"allocated":362172416,"data_stored":361906243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":180285100},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724562,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":286432,"kb_used_data":150608,"kb_used_omap":4,"kb_used_meta":135803,"kb_avail":94085408,"statfs":{"total":96636764160,"available":96343457792,"internally_reserved":0,"allocated":154222592,"data_stored":153908986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":139062955},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,0,0,1,2,13],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247844,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":646012,"kb_used_data":397804,"kb_used_omap":4,"kb_used_meta":248187,"kb_avail":93725828,"statfs":{"total":96636764160,"available":95975247872,"internally_reserved":0,"allocated":407351296,"data_stored":407081046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":254144168},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,5,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":407220224,"data_stored":407184094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":153985024,"data_stored":153905538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":362090496,"data_stored":362054366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:27:47.535 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:27:48.130 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:27:48.130 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:27:48.163 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":136,"stamp":"2024-09-17T15:27:47.624432+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":464360177,"num_objects":113380,"num_object_clones":0,"num_object_copies":226760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":1470,"num_objects_unfound":0,"num_objects_dirty":113380,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":73487,"num_write_kb":293942,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2603,"num_bytes_recovered":10650102,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113453,"ondisk_log_size":113453,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1483668,"kb_used_data":915236,"kb_used_omap":13,"kb_used_meta":568370,"kb_avail":281631852,"statfs":{"total":289910292480,"available":288391016448,"internally_reserved":0,"allocated":937201664,"data_stored":936351635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":582011903},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,4,3,7,6,15],"upper_bound":128},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25837568,"num_objects":6308,"num_object_clones":0,"num_object_copies":12616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-528,"num_objects_unfound":0,"num_objects_dirty":6308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6308,"num_write_kb":25232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":528,"num_bytes_recovered":2162688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002919"},"pg_stats":[{"pgid":"2.f","version":"41'7041","reported_seq":7113,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:45.651291+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:27:45.651291+0000","last_peered":"2024-09-17T15:27:45.651291+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:27:45.651291+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":7041,"log_dups_size":0,"ondisk_log_size":7041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28839936,"num_objects":7041,"num_object_clones":0,"num_object_copies":14082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7041,"num_write_kb":28164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2971},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'7079","reported_seq":7177,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:46.937757+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:27:46.937757+0000","last_peered":"2024-09-17T15:27:46.937757+0000","last_clean":"2024-09-17T15:27:46.937757+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:27:46.937757+0000","last_undegraded":"2024-09-17T15:27:46.937757+0000","last_fullsized":"2024-09-17T15:27:46.937757+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":7079,"log_dups_size":0,"ondisk_log_size":7079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28971974,"num_objects":7075,"num_object_clones":0,"num_object_copies":14150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7075,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5436,"num_write_kb":21744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'7108","reported_seq":7227,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:47.049514+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:27:47.049514+0000","last_peered":"2024-09-17T15:27:47.049514+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:27:47.049514+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":7108,"log_dups_size":0,"ondisk_log_size":7108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29097984,"num_objects":7104,"num_object_clones":0,"num_object_copies":14208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7104,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5265,"num_write_kb":21060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":2936}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'7194","reported_seq":7259,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:47.034556+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:27:47.034556+0000","last_peered":"2024-09-17T15:27:47.034556+0000","last_clean":"2024-09-17T15:27:47.034556+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:27:47.034556+0000","last_undegraded":"2024-09-17T15:27:47.034556+0000","last_fullsized":"2024-09-17T15:27:47.034556+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":7194,"log_dups_size":0,"ondisk_log_size":7194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29466624,"num_objects":7194,"num_object_clones":0,"num_object_copies":14388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7194,"num_write_kb":28776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'7003","reported_seq":7125,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:45.682334+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:27:45.682334+0000","last_peered":"2024-09-17T15:27:45.682334+0000","last_clean":"2024-09-17T15:27:45.682334+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:27:45.682334+0000","last_undegraded":"2024-09-17T15:27:45.682334+0000","last_fullsized":"2024-09-17T15:27:45.682334+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":7003,"log_dups_size":0,"ondisk_log_size":7003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28656099,"num_objects":6997,"num_object_clones":0,"num_object_copies":13994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6997,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3985,"num_write_kb":15940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'7242","reported_seq":7372,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:45.610149+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:27:45.610149+0000","last_peered":"2024-09-17T15:27:45.610149+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:27:45.610149+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":7242,"log_dups_size":0,"ondisk_log_size":7242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29643235,"num_objects":7238,"num_object_clones":0,"num_object_copies":14476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7238,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3741,"num_write_kb":14964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2968},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'7145","reported_seq":7279,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:45.661795+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:27:45.661795+0000","last_peered":"2024-09-17T15:27:45.661795+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:27:45.661795+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":7145,"log_dups_size":0,"ondisk_log_size":7145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29242310,"num_objects":7141,"num_object_clones":0,"num_object_copies":14282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7141,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3914,"num_write_kb":15656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2967},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'7084","reported_seq":7207,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:45.672739+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:27:45.672739+0000","last_peered":"2024-09-17T15:27:45.672739+0000","last_clean":"2024-09-17T15:27:45.672739+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:27:45.672739+0000","last_undegraded":"2024-09-17T15:27:45.672739+0000","last_fullsized":"2024-09-17T15:27:45.672739+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":7084,"log_dups_size":0,"ondisk_log_size":7084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29007872,"num_objects":7082,"num_object_clones":0,"num_object_copies":14164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4038,"num_write_kb":16152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'7074","reported_seq":12402,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:27:47.037120+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:27:47.037120+0000","last_peered":"2024-09-17T15:27:47.037120+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:27:47.037120+0000","last_undegraded":"2024-09-17T15:27:47.037120+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":7074,"log_dups_size":0,"ondisk_log_size":7074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28946915,"num_objects":7068,"num_object_clones":0,"num_object_copies":14136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1470,"num_objects_unfound":0,"num_objects_dirty":7068,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5074,"num_write_kb":20296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2602,"num_bytes_recovered":10650083,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1470},{"shards":"2","objects":5598}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233600+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233600+0000","last_peered":"2024-09-17T15:26:22.233600+0000","last_clean":"2024-09-17T15:26:22.233600+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233600+0000","last_undegraded":"2024-09-17T15:26:22.233600+0000","last_fullsized":"2024-09-17T15:26:22.233600+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'6964","reported_seq":7089,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:45.653170+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:27:45.653170+0000","last_peered":"2024-09-17T15:27:45.653170+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:27:45.653170+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":6964,"log_dups_size":0,"ondisk_log_size":6964,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28491776,"num_objects":6956,"num_object_clones":0,"num_object_copies":13912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6956,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3560,"num_write_kb":14240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2946},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":98,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412941+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412941+0000","last_peered":"2024-09-17T15:26:22.412941+0000","last_clean":"2024-09-17T15:26:22.412941+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412941+0000","last_undegraded":"2024-09-17T15:26:22.412941+0000","last_fullsized":"2024-09-17T15:26:22.412941+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'7053","reported_seq":7169,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:46.971967+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:27:46.971967+0000","last_peered":"2024-09-17T15:27:46.971967+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:27:46.971967+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":7053,"log_dups_size":0,"ondisk_log_size":7053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28869091,"num_objects":7049,"num_object_clones":0,"num_object_copies":14098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7049,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4948,"num_write_kb":19792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":2972}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'7123","reported_seq":7237,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:47.004767+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:27:47.004767+0000","last_peered":"2024-09-17T15:27:47.004767+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:27:47.004767+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":7123,"log_dups_size":0,"ondisk_log_size":7123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29151232,"num_objects":7117,"num_object_clones":0,"num_object_copies":14234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4885,"num_write_kb":19540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":2980}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'7146","reported_seq":7276,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:45.606842+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:27:45.606842+0000","last_peered":"2024-09-17T15:27:45.606842+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:27:45.606842+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":7146,"log_dups_size":0,"ondisk_log_size":7146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29261824,"num_objects":7144,"num_object_clones":0,"num_object_copies":14288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3694,"num_write_kb":14776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3006},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'7059","reported_seq":7196,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:45.635424+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:27:45.635424+0000","last_peered":"2024-09-17T15:27:45.635424+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:27:45.635424+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":7059,"log_dups_size":0,"ondisk_log_size":7059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28880896,"num_objects":7051,"num_object_clones":0,"num_object_copies":14102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3346,"num_write_kb":13384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2894},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229061+0000","last_peered":"2024-09-17T15:26:22.229061+0000","last_clean":"2024-09-17T15:26:22.229061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229061+0000","last_undegraded":"2024-09-17T15:26:22.229061+0000","last_fullsized":"2024-09-17T15:26:22.229061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'7109","reported_seq":7235,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:45.662231+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:27:45.662231+0000","last_peered":"2024-09-17T15:27:45.662231+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:27:45.662231+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":7109,"log_dups_size":0,"ondisk_log_size":7109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29102080,"num_objects":7105,"num_object_clones":0,"num_object_copies":14210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3896,"num_write_kb":15584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3042},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'7026","reported_seq":7164,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:45.663776+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:27:45.663776+0000","last_peered":"2024-09-17T15:27:45.663776+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:27:45.663776+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":7026,"log_dups_size":0,"ondisk_log_size":7026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28730310,"num_objects":7016,"num_object_clones":0,"num_object_copies":14032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3468,"num_write_kb":13872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2907},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":464360158,"num_objects":113378,"num_object_clones":0,"num_object_copies":226756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":1470,"num_objects_unfound":0,"num_objects_dirty":113378,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":73485,"num_write_kb":293940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2602,"num_bytes_recovered":10650083,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":936935424,"data_stored":936783678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113450,"ondisk_log_size":113450,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051617,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539208,"kb_used_data":358968,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93832632,"statfs":{"total":96636764160,"available":96084615168,"internally_reserved":0,"allocated":367583232,"data_stored":367317059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":184544940},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724562,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":286432,"kb_used_data":150608,"kb_used_omap":4,"kb_used_meta":135803,"kb_avail":94085408,"statfs":{"total":96636764160,"available":96343457792,"internally_reserved":0,"allocated":154222592,"data_stored":153908986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":139062955},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,0,0,1,2,13],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247845,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":658028,"kb_used_data":405660,"kb_used_omap":4,"kb_used_meta":252347,"kb_avail":93713812,"statfs":{"total":96636764160,"available":95962943488,"internally_reserved":0,"allocated":415395840,"data_stored":415125590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":258404008},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,4,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":415272960,"data_stored":415236830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":153985024,"data_stored":153905538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":367677440,"data_stored":367641310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:27:48.164 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:27:48.165 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'7041", 'reported_seq': 7113, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:45.651291+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:27:45.651291+0000', 'last_peered': '2024-09-17T15:27:45.651291+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:27:45.651291+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 7041, 'log_dups_size': 0, 'ondisk_log_size': 7041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28839936, 'num_objects': 7041, 'num_object_clones': 0, 'num_object_copies': 14082, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4070, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7041, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7041, 'num_write_kb': 28164, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2971}, {'shards': '0,2', 'objects': 4070}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:48.165 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:27:48.165 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'7108", 'reported_seq': 7227, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:47.049514+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:27:47.049514+0000', 'last_peered': '2024-09-17T15:27:47.049514+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:27:47.049514+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 7108, 'log_dups_size': 0, 'ondisk_log_size': 7108, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29097984, 'num_objects': 7104, 'num_object_clones': 0, 'num_object_copies': 14208, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4169, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7104, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5265, 'num_write_kb': 21060, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4168}, {'shards': '2', 'objects': 2936}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:27:48.165 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:27:48.165 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'7242", 'reported_seq': 7372, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:45.610149+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:27:45.610149+0000', 'last_peered': '2024-09-17T15:27:45.610149+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:27:45.610149+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 7242, 'log_dups_size': 0, 'ondisk_log_size': 7242, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29643235, 'num_objects': 7238, 'num_object_clones': 0, 'num_object_copies': 14476, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4271, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7238, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3741, 'num_write_kb': 14964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2968}, {'shards': '0,2', 'objects': 4268}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:48.165 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:27:48.165 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'7145", 'reported_seq': 7279, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:45.661795+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:27:45.661795+0000', 'last_peered': '2024-09-17T15:27:45.661795+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:27:45.661795+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 7145, 'log_dups_size': 0, 'ondisk_log_size': 7145, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29242310, 'num_objects': 7141, 'num_object_clones': 0, 'num_object_copies': 14282, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4174, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7141, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 3914, 'num_write_kb': 15656, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2967}, {'shards': '0,2', 'objects': 4174}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:48.165 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-17T15:27:48.165 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "41'7074", 'reported_seq': 12402, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:27:47.037120+0000', 'last_change': '2024-09-17T15:26:22.588242+0000', 'last_active': '2024-09-17T15:27:47.037120+0000', 'last_peered': '2024-09-17T15:27:47.037120+0000', 'last_clean': '2024-09-17T15:26:20.155646+0000', 'last_became_active': '2024-09-17T15:26:22.557742+0000', 'last_became_peered': '2024-09-17T15:26:22.557742+0000', 'last_unstale': '2024-09-17T15:27:47.037120+0000', 'last_undegraded': '2024-09-17T15:27:47.037120+0000', 'last_fullsized': '2024-09-17T15:26:22.188850+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2016", 'last_scrub_stamp': '2024-09-17T15:25:53.959011+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:53.959011+0000', 'objects_scrubbed': 1994, 'log_size': 7074, 'log_dups_size': 0, 'ondisk_log_size': 7074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 568, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28946915, 'num_objects': 7068, 'num_object_clones': 0, 'num_object_copies': 14136, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1470, 'num_objects_unfound': 0, 'num_objects_dirty': 7068, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5074, 'num_write_kb': 20296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2602, 'num_bytes_recovered': 10650083, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1470}, {'shards': '2', 'objects': 5598}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:27:48.165 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:27:48.166 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'6964", 'reported_seq': 7089, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:45.653170+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:27:45.653170+0000', 'last_peered': '2024-09-17T15:27:45.653170+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:27:45.653170+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 6964, 'log_dups_size': 0, 'ondisk_log_size': 6964, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28491776, 'num_objects': 6956, 'num_object_clones': 0, 'num_object_copies': 13912, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4012, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6956, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3560, 'num_write_kb': 14240, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2946}, {'shards': '0,2', 'objects': 4010}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:48.166 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:27:48.166 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'7053", 'reported_seq': 7169, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:46.971967+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:27:46.971967+0000', 'last_peered': '2024-09-17T15:27:46.971967+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:27:46.971967+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 7053, 'log_dups_size': 0, 'ondisk_log_size': 7053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28869091, 'num_objects': 7049, 'num_object_clones': 0, 'num_object_copies': 14098, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4078, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7049, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4948, 'num_write_kb': 19792, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4077}, {'shards': '2', 'objects': 2972}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:27:48.166 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:27:48.166 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'7123", 'reported_seq': 7237, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:47.004767+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:27:47.004767+0000', 'last_peered': '2024-09-17T15:27:47.004767+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:27:47.004767+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 7123, 'log_dups_size': 0, 'ondisk_log_size': 7123, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29151232, 'num_objects': 7117, 'num_object_clones': 0, 'num_object_copies': 14234, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7117, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4885, 'num_write_kb': 19540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4137}, {'shards': '2', 'objects': 2980}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:27:48.166 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:27:48.166 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'7146", 'reported_seq': 7276, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:45.606842+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:27:45.606842+0000', 'last_peered': '2024-09-17T15:27:45.606842+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:27:45.606842+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 7146, 'log_dups_size': 0, 'ondisk_log_size': 7146, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29261824, 'num_objects': 7144, 'num_object_clones': 0, 'num_object_copies': 14288, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7144, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3694, 'num_write_kb': 14776, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3006}, {'shards': '0,2', 'objects': 4138}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:48.166 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:27:48.166 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'7059", 'reported_seq': 7196, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:45.635424+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:27:45.635424+0000', 'last_peered': '2024-09-17T15:27:45.635424+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:27:45.635424+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 7059, 'log_dups_size': 0, 'ondisk_log_size': 7059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28880896, 'num_objects': 7051, 'num_object_clones': 0, 'num_object_copies': 14102, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4160, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7051, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3346, 'num_write_kb': 13384, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2894}, {'shards': '0,2', 'objects': 4157}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:48.166 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:27:48.167 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'7109", 'reported_seq': 7235, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:45.662231+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:27:45.662231+0000', 'last_peered': '2024-09-17T15:27:45.662231+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:27:45.662231+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 7109, 'log_dups_size': 0, 'ondisk_log_size': 7109, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29102080, 'num_objects': 7105, 'num_object_clones': 0, 'num_object_copies': 14210, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4065, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7105, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3896, 'num_write_kb': 15584, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3042}, {'shards': '0,2', 'objects': 4063}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:48.167 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:27:48.167 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'7026", 'reported_seq': 7164, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:45.663776+0000', 'last_change': '2024-09-17T15:26:22.654521+0000', 'last_active': '2024-09-17T15:27:45.663776+0000', 'last_peered': '2024-09-17T15:27:45.663776+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:27:45.663776+0000', 'last_undegraded': '2024-09-17T15:26:22.307795+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 7026, 'log_dups_size': 0, 'ondisk_log_size': 7026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28730310, 'num_objects': 7016, 'num_object_clones': 0, 'num_object_copies': 14032, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4111, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7016, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3468, 'num_write_kb': 13872, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2907}, {'shards': '0,2', 'objects': 4109}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:48.167 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:27:48.311 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 129 20 114428 114408 3.46364 1.88672 0.0340688 0.0225466 2024-09-17T15:27:48.850 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:27:48.850 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45388/226760 objects degraded (20.016%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":205,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":113380,"data_bytes":464360177,"bytes_used":1519276032,"bytes_avail":288391016448,"bytes_total":289910292480,"degraded_objects":45388,"degraded_total":226760,"degraded_ratio":0.20015875815840536,"misplaced_objects":1470,"misplaced_total":226760,"misplaced_ratio":0.0064826248015523022,"recovering_objects_per_sec":43,"recovering_bytes_per_sec":180180,"recovering_keys_per_sec":0,"num_objects_recovered":528,"num_bytes_recovered":2162688,"num_keys_recovered":0,"write_bytes_sec":2152606,"read_op_per_sec":0,"write_op_per_sec":525},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T15:27:19.616376+0000","services":{}},"progress_events":{}} 2024-09-17T15:27:48.875 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:27:48.876 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:27:49.314 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 130 20 114904 114884 3.45122 1.85938 0.0231439 0.0226275 2024-09-17T15:27:49.408 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:27:49.409 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:27:49.431 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":136,"stamp":"2024-09-17T15:27:47.624432+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":464360177,"num_objects":113380,"num_object_clones":0,"num_object_copies":226760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":1470,"num_objects_unfound":0,"num_objects_dirty":113380,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":73487,"num_write_kb":293942,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2603,"num_bytes_recovered":10650102,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113453,"ondisk_log_size":113453,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1483668,"kb_used_data":915236,"kb_used_omap":13,"kb_used_meta":568370,"kb_avail":281631852,"statfs":{"total":289910292480,"available":288391016448,"internally_reserved":0,"allocated":937201664,"data_stored":936351635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":582011903},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,4,3,7,6,15],"upper_bound":128},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25837568,"num_objects":6308,"num_object_clones":0,"num_object_copies":12616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-528,"num_objects_unfound":0,"num_objects_dirty":6308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6308,"num_write_kb":25232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":528,"num_bytes_recovered":2162688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002919"},"pg_stats":[{"pgid":"2.f","version":"41'7041","reported_seq":7113,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:45.651291+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:27:45.651291+0000","last_peered":"2024-09-17T15:27:45.651291+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:27:45.651291+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":7041,"log_dups_size":0,"ondisk_log_size":7041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28839936,"num_objects":7041,"num_object_clones":0,"num_object_copies":14082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7041,"num_write_kb":28164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2971},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'7079","reported_seq":7177,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:46.937757+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:27:46.937757+0000","last_peered":"2024-09-17T15:27:46.937757+0000","last_clean":"2024-09-17T15:27:46.937757+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:27:46.937757+0000","last_undegraded":"2024-09-17T15:27:46.937757+0000","last_fullsized":"2024-09-17T15:27:46.937757+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":7079,"log_dups_size":0,"ondisk_log_size":7079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28971974,"num_objects":7075,"num_object_clones":0,"num_object_copies":14150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7075,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5436,"num_write_kb":21744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'7108","reported_seq":7227,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:47.049514+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:27:47.049514+0000","last_peered":"2024-09-17T15:27:47.049514+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:27:47.049514+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":7108,"log_dups_size":0,"ondisk_log_size":7108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29097984,"num_objects":7104,"num_object_clones":0,"num_object_copies":14208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7104,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5265,"num_write_kb":21060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":2936}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'7194","reported_seq":7259,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:47.034556+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:27:47.034556+0000","last_peered":"2024-09-17T15:27:47.034556+0000","last_clean":"2024-09-17T15:27:47.034556+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:27:47.034556+0000","last_undegraded":"2024-09-17T15:27:47.034556+0000","last_fullsized":"2024-09-17T15:27:47.034556+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":7194,"log_dups_size":0,"ondisk_log_size":7194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29466624,"num_objects":7194,"num_object_clones":0,"num_object_copies":14388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7194,"num_write_kb":28776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'7003","reported_seq":7125,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:45.682334+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:27:45.682334+0000","last_peered":"2024-09-17T15:27:45.682334+0000","last_clean":"2024-09-17T15:27:45.682334+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:27:45.682334+0000","last_undegraded":"2024-09-17T15:27:45.682334+0000","last_fullsized":"2024-09-17T15:27:45.682334+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":7003,"log_dups_size":0,"ondisk_log_size":7003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28656099,"num_objects":6997,"num_object_clones":0,"num_object_copies":13994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6997,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3985,"num_write_kb":15940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'7242","reported_seq":7372,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:45.610149+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:27:45.610149+0000","last_peered":"2024-09-17T15:27:45.610149+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:27:45.610149+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":7242,"log_dups_size":0,"ondisk_log_size":7242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29643235,"num_objects":7238,"num_object_clones":0,"num_object_copies":14476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7238,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3741,"num_write_kb":14964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2968},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'7145","reported_seq":7279,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:45.661795+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:27:45.661795+0000","last_peered":"2024-09-17T15:27:45.661795+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:27:45.661795+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":7145,"log_dups_size":0,"ondisk_log_size":7145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29242310,"num_objects":7141,"num_object_clones":0,"num_object_copies":14282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7141,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3914,"num_write_kb":15656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2967},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'7084","reported_seq":7207,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:45.672739+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:27:45.672739+0000","last_peered":"2024-09-17T15:27:45.672739+0000","last_clean":"2024-09-17T15:27:45.672739+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:27:45.672739+0000","last_undegraded":"2024-09-17T15:27:45.672739+0000","last_fullsized":"2024-09-17T15:27:45.672739+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":7084,"log_dups_size":0,"ondisk_log_size":7084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29007872,"num_objects":7082,"num_object_clones":0,"num_object_copies":14164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4038,"num_write_kb":16152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'7074","reported_seq":12402,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:27:47.037120+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:27:47.037120+0000","last_peered":"2024-09-17T15:27:47.037120+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:27:47.037120+0000","last_undegraded":"2024-09-17T15:27:47.037120+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":7074,"log_dups_size":0,"ondisk_log_size":7074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28946915,"num_objects":7068,"num_object_clones":0,"num_object_copies":14136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1470,"num_objects_unfound":0,"num_objects_dirty":7068,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5074,"num_write_kb":20296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2602,"num_bytes_recovered":10650083,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1470},{"shards":"2","objects":5598}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233600+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233600+0000","last_peered":"2024-09-17T15:26:22.233600+0000","last_clean":"2024-09-17T15:26:22.233600+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233600+0000","last_undegraded":"2024-09-17T15:26:22.233600+0000","last_fullsized":"2024-09-17T15:26:22.233600+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'6964","reported_seq":7089,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:45.653170+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:27:45.653170+0000","last_peered":"2024-09-17T15:27:45.653170+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:27:45.653170+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":6964,"log_dups_size":0,"ondisk_log_size":6964,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28491776,"num_objects":6956,"num_object_clones":0,"num_object_copies":13912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6956,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3560,"num_write_kb":14240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2946},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":98,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412941+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412941+0000","last_peered":"2024-09-17T15:26:22.412941+0000","last_clean":"2024-09-17T15:26:22.412941+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412941+0000","last_undegraded":"2024-09-17T15:26:22.412941+0000","last_fullsized":"2024-09-17T15:26:22.412941+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'7053","reported_seq":7169,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:46.971967+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:27:46.971967+0000","last_peered":"2024-09-17T15:27:46.971967+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:27:46.971967+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":7053,"log_dups_size":0,"ondisk_log_size":7053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28869091,"num_objects":7049,"num_object_clones":0,"num_object_copies":14098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7049,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4948,"num_write_kb":19792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":2972}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'7123","reported_seq":7237,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:47.004767+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:27:47.004767+0000","last_peered":"2024-09-17T15:27:47.004767+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:27:47.004767+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":7123,"log_dups_size":0,"ondisk_log_size":7123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29151232,"num_objects":7117,"num_object_clones":0,"num_object_copies":14234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4885,"num_write_kb":19540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":2980}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'7146","reported_seq":7276,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:45.606842+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:27:45.606842+0000","last_peered":"2024-09-17T15:27:45.606842+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:27:45.606842+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":7146,"log_dups_size":0,"ondisk_log_size":7146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29261824,"num_objects":7144,"num_object_clones":0,"num_object_copies":14288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3694,"num_write_kb":14776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3006},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'7059","reported_seq":7196,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:45.635424+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:27:45.635424+0000","last_peered":"2024-09-17T15:27:45.635424+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:27:45.635424+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":7059,"log_dups_size":0,"ondisk_log_size":7059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28880896,"num_objects":7051,"num_object_clones":0,"num_object_copies":14102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3346,"num_write_kb":13384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2894},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229061+0000","last_peered":"2024-09-17T15:26:22.229061+0000","last_clean":"2024-09-17T15:26:22.229061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229061+0000","last_undegraded":"2024-09-17T15:26:22.229061+0000","last_fullsized":"2024-09-17T15:26:22.229061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'7109","reported_seq":7235,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:45.662231+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:27:45.662231+0000","last_peered":"2024-09-17T15:27:45.662231+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:27:45.662231+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":7109,"log_dups_size":0,"ondisk_log_size":7109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29102080,"num_objects":7105,"num_object_clones":0,"num_object_copies":14210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3896,"num_write_kb":15584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3042},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'7026","reported_seq":7164,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:45.663776+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:27:45.663776+0000","last_peered":"2024-09-17T15:27:45.663776+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:27:45.663776+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":7026,"log_dups_size":0,"ondisk_log_size":7026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28730310,"num_objects":7016,"num_object_clones":0,"num_object_copies":14032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3468,"num_write_kb":13872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2907},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":464360158,"num_objects":113378,"num_object_clones":0,"num_object_copies":226756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":1470,"num_objects_unfound":0,"num_objects_dirty":113378,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":73485,"num_write_kb":293940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2602,"num_bytes_recovered":10650083,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":936935424,"data_stored":936783678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113450,"ondisk_log_size":113450,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051617,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539208,"kb_used_data":358968,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93832632,"statfs":{"total":96636764160,"available":96084615168,"internally_reserved":0,"allocated":367583232,"data_stored":367317059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":184544940},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724562,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":286432,"kb_used_data":150608,"kb_used_omap":4,"kb_used_meta":135803,"kb_avail":94085408,"statfs":{"total":96636764160,"available":96343457792,"internally_reserved":0,"allocated":154222592,"data_stored":153908986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":139062955},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,0,0,1,2,13],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247845,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":658028,"kb_used_data":405660,"kb_used_omap":4,"kb_used_meta":252347,"kb_avail":93713812,"statfs":{"total":96636764160,"available":95962943488,"internally_reserved":0,"allocated":415395840,"data_stored":415125590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":258404008},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,4,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":415272960,"data_stored":415236830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":153985024,"data_stored":153905538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":367677440,"data_stored":367641310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:27:50.314 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 131 20 115448 115428 3.4411 2.125 0.0568116 0.0226978 2024-09-17T15:27:51.314 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 132 20 115987 115967 3.43098 2.10547 0.0253205 0.0227636 2024-09-17T15:27:52.314 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 133 20 116537 116517 3.42134 2.14844 0.0259804 0.0228271 2024-09-17T15:27:52.433 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:27:52.976 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:27:52.979 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:27:53.003 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":138,"stamp":"2024-09-17T15:27:51.625407+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":470897393,"num_objects":114976,"num_object_clones":0,"num_object_copies":229952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":1470,"num_objects_unfound":0,"num_objects_dirty":114976,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":75083,"num_write_kb":300326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2603,"num_bytes_recovered":10650102,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115049,"ondisk_log_size":115049,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1530332,"kb_used_data":931368,"kb_used_omap":13,"kb_used_meta":598898,"kb_avail":281585188,"statfs":{"total":289910292480,"available":288343232512,"internally_reserved":0,"allocated":953720832,"data_stored":952870803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":613272575},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,3,4,4,10,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28418048,"num_objects":6938,"num_object_clones":0,"num_object_copies":13876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-351,"num_objects_unfound":0,"num_objects_dirty":6938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6938,"num_write_kb":27752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":351,"num_bytes_recovered":1437696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002932"},"pg_stats":[{"pgid":"2.f","version":"41'7189","reported_seq":7260,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:50.688333+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:27:50.688333+0000","last_peered":"2024-09-17T15:27:50.688333+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:27:50.688333+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":7189,"log_dups_size":0,"ondisk_log_size":7189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29446144,"num_objects":7189,"num_object_clones":0,"num_object_copies":14378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7189,"num_write_kb":28756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3119},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'7079","reported_seq":7177,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:46.937757+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:27:46.937757+0000","last_peered":"2024-09-17T15:27:46.937757+0000","last_clean":"2024-09-17T15:27:46.937757+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:27:46.937757+0000","last_undegraded":"2024-09-17T15:27:46.937757+0000","last_fullsized":"2024-09-17T15:27:46.937757+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":7079,"log_dups_size":0,"ondisk_log_size":7079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28971974,"num_objects":7075,"num_object_clones":0,"num_object_copies":14150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7075,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5436,"num_write_kb":21744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'7108","reported_seq":7227,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:47.049514+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:27:47.049514+0000","last_peered":"2024-09-17T15:27:47.049514+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:27:47.049514+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":7108,"log_dups_size":0,"ondisk_log_size":7108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29097984,"num_objects":7104,"num_object_clones":0,"num_object_copies":14208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7104,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5265,"num_write_kb":21060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":2936}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'7194","reported_seq":7259,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:47.034556+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:27:47.034556+0000","last_peered":"2024-09-17T15:27:47.034556+0000","last_clean":"2024-09-17T15:27:47.034556+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:27:47.034556+0000","last_undegraded":"2024-09-17T15:27:47.034556+0000","last_fullsized":"2024-09-17T15:27:47.034556+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":7194,"log_dups_size":0,"ondisk_log_size":7194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29466624,"num_objects":7194,"num_object_clones":0,"num_object_copies":14388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7194,"num_write_kb":28776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'7158","reported_seq":7282,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:50.636781+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:27:50.636781+0000","last_peered":"2024-09-17T15:27:50.636781+0000","last_clean":"2024-09-17T15:27:50.636781+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:27:50.636781+0000","last_undegraded":"2024-09-17T15:27:50.636781+0000","last_fullsized":"2024-09-17T15:27:50.636781+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":7158,"log_dups_size":0,"ondisk_log_size":7158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29290979,"num_objects":7152,"num_object_clones":0,"num_object_copies":14304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7152,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4140,"num_write_kb":16560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'7440","reported_seq":7570,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:50.713781+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:27:50.713781+0000","last_peered":"2024-09-17T15:27:50.713781+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:27:50.713781+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":7440,"log_dups_size":0,"ondisk_log_size":7440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30454243,"num_objects":7436,"num_object_clones":0,"num_object_copies":14872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7436,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3939,"num_write_kb":15756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3166},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'7295","reported_seq":7429,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:50.650905+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:27:50.650905+0000","last_peered":"2024-09-17T15:27:50.650905+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:27:50.650905+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":7295,"log_dups_size":0,"ondisk_log_size":7295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29856710,"num_objects":7291,"num_object_clones":0,"num_object_copies":14582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7291,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4064,"num_write_kb":16256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3117},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'7239","reported_seq":7362,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:50.656580+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:27:50.656580+0000","last_peered":"2024-09-17T15:27:50.656580+0000","last_clean":"2024-09-17T15:27:50.656580+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:27:50.656580+0000","last_undegraded":"2024-09-17T15:27:50.656580+0000","last_fullsized":"2024-09-17T15:27:50.656580+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":7239,"log_dups_size":0,"ondisk_log_size":7239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29642752,"num_objects":7237,"num_object_clones":0,"num_object_copies":14474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4193,"num_write_kb":16772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'7074","reported_seq":12402,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:27:47.037120+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:27:47.037120+0000","last_peered":"2024-09-17T15:27:47.037120+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:27:47.037120+0000","last_undegraded":"2024-09-17T15:27:47.037120+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":7074,"log_dups_size":0,"ondisk_log_size":7074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28946915,"num_objects":7068,"num_object_clones":0,"num_object_copies":14136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1470,"num_objects_unfound":0,"num_objects_dirty":7068,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5074,"num_write_kb":20296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2602,"num_bytes_recovered":10650083,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1470},{"shards":"2","objects":5598}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233600+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233600+0000","last_peered":"2024-09-17T15:26:22.233600+0000","last_clean":"2024-09-17T15:26:22.233600+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233600+0000","last_undegraded":"2024-09-17T15:26:22.233600+0000","last_fullsized":"2024-09-17T15:26:22.233600+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'7141","reported_seq":7265,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:50.715252+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:27:50.715252+0000","last_peered":"2024-09-17T15:27:50.715252+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:27:50.715252+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":7141,"log_dups_size":0,"ondisk_log_size":7141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29216768,"num_objects":7133,"num_object_clones":0,"num_object_copies":14266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3737,"num_write_kb":14948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3123},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":98,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412941+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412941+0000","last_peered":"2024-09-17T15:26:22.412941+0000","last_clean":"2024-09-17T15:26:22.412941+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412941+0000","last_undegraded":"2024-09-17T15:26:22.412941+0000","last_fullsized":"2024-09-17T15:26:22.412941+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'7053","reported_seq":7169,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:46.971967+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:27:46.971967+0000","last_peered":"2024-09-17T15:27:46.971967+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:27:46.971967+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":7053,"log_dups_size":0,"ondisk_log_size":7053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28869091,"num_objects":7049,"num_object_clones":0,"num_object_copies":14098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7049,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4948,"num_write_kb":19792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":2972}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'7123","reported_seq":7237,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:47.004767+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:27:47.004767+0000","last_peered":"2024-09-17T15:27:47.004767+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:27:47.004767+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":7123,"log_dups_size":0,"ondisk_log_size":7123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29151232,"num_objects":7117,"num_object_clones":0,"num_object_copies":14234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4885,"num_write_kb":19540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":2980}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'7284","reported_seq":7414,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:50.587945+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:27:50.587945+0000","last_peered":"2024-09-17T15:27:50.587945+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:27:50.587945+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":7284,"log_dups_size":0,"ondisk_log_size":7284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29827072,"num_objects":7282,"num_object_clones":0,"num_object_copies":14564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3832,"num_write_kb":15328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3144},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'7218","reported_seq":7355,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:50.694902+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:27:50.694902+0000","last_peered":"2024-09-17T15:27:50.694902+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:27:50.694902+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":7218,"log_dups_size":0,"ondisk_log_size":7218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29532160,"num_objects":7210,"num_object_clones":0,"num_object_copies":14420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3505,"num_write_kb":14020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3053},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229061+0000","last_peered":"2024-09-17T15:26:22.229061+0000","last_clean":"2024-09-17T15:26:22.229061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229061+0000","last_undegraded":"2024-09-17T15:26:22.229061+0000","last_fullsized":"2024-09-17T15:26:22.229061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'7251","reported_seq":7378,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:50.674310+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:27:50.674310+0000","last_peered":"2024-09-17T15:27:50.674310+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:27:50.674310+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":7251,"log_dups_size":0,"ondisk_log_size":7251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29683712,"num_objects":7247,"num_object_clones":0,"num_object_copies":14494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4038,"num_write_kb":16152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3184},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'7200","reported_seq":7338,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:50.639218+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:27:50.639218+0000","last_peered":"2024-09-17T15:27:50.639218+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:27:50.639218+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":7200,"log_dups_size":0,"ondisk_log_size":7200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29443014,"num_objects":7190,"num_object_clones":0,"num_object_copies":14380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3642,"num_write_kb":14568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3081},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":470897374,"num_objects":114974,"num_object_clones":0,"num_object_copies":229948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":1470,"num_objects_unfound":0,"num_objects_dirty":114974,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":75081,"num_write_kb":300324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2602,"num_bytes_recovered":10650083,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":953688064,"data_stored":953536318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115046,"ondisk_log_size":115046,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051617,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539208,"kb_used_data":358968,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93832632,"statfs":{"total":96636764160,"available":96084615168,"internally_reserved":0,"allocated":367583232,"data_stored":367317059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":184544940},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724563,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":317196,"kb_used_data":159164,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":94054644,"statfs":{"total":96636764160,"available":96311955456,"internally_reserved":0,"allocated":162983936,"data_stored":162670330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":161803947},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,0,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247846,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":673928,"kb_used_data":413236,"kb_used_omap":4,"kb_used_meta":260667,"kb_avail":93697912,"statfs":{"total":96636764160,"available":95946661888,"internally_reserved":0,"allocated":423153664,"data_stored":422883414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":266923688},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,0,3,2,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":423235584,"data_stored":423199454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":162775040,"data_stored":162695554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":367677440,"data_stored":367641310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:27:53.005 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:27:53.314 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 134 20 116990 116970 3.40901 1.76953 0.131069 0.0229082 2024-09-17T15:27:53.508 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:27:53.509 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:27:53.529 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":138,"stamp":"2024-09-17T15:27:51.625407+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":470897393,"num_objects":114976,"num_object_clones":0,"num_object_copies":229952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":1470,"num_objects_unfound":0,"num_objects_dirty":114976,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":75083,"num_write_kb":300326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2603,"num_bytes_recovered":10650102,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115049,"ondisk_log_size":115049,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1530332,"kb_used_data":931368,"kb_used_omap":13,"kb_used_meta":598898,"kb_avail":281585188,"statfs":{"total":289910292480,"available":288343232512,"internally_reserved":0,"allocated":953720832,"data_stored":952870803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":613272575},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,3,4,4,10,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28418048,"num_objects":6938,"num_object_clones":0,"num_object_copies":13876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-351,"num_objects_unfound":0,"num_objects_dirty":6938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6938,"num_write_kb":27752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":351,"num_bytes_recovered":1437696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002932"},"pg_stats":[{"pgid":"2.f","version":"41'7189","reported_seq":7260,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:50.688333+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:27:50.688333+0000","last_peered":"2024-09-17T15:27:50.688333+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:27:50.688333+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":7189,"log_dups_size":0,"ondisk_log_size":7189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29446144,"num_objects":7189,"num_object_clones":0,"num_object_copies":14378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7189,"num_write_kb":28756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3119},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'7079","reported_seq":7177,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:46.937757+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:27:46.937757+0000","last_peered":"2024-09-17T15:27:46.937757+0000","last_clean":"2024-09-17T15:27:46.937757+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:27:46.937757+0000","last_undegraded":"2024-09-17T15:27:46.937757+0000","last_fullsized":"2024-09-17T15:27:46.937757+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":7079,"log_dups_size":0,"ondisk_log_size":7079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28971974,"num_objects":7075,"num_object_clones":0,"num_object_copies":14150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7075,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5436,"num_write_kb":21744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'7108","reported_seq":7227,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:47.049514+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:27:47.049514+0000","last_peered":"2024-09-17T15:27:47.049514+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:27:47.049514+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":7108,"log_dups_size":0,"ondisk_log_size":7108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29097984,"num_objects":7104,"num_object_clones":0,"num_object_copies":14208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7104,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5265,"num_write_kb":21060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":2936}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'7194","reported_seq":7259,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:47.034556+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:27:47.034556+0000","last_peered":"2024-09-17T15:27:47.034556+0000","last_clean":"2024-09-17T15:27:47.034556+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:27:47.034556+0000","last_undegraded":"2024-09-17T15:27:47.034556+0000","last_fullsized":"2024-09-17T15:27:47.034556+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":7194,"log_dups_size":0,"ondisk_log_size":7194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29466624,"num_objects":7194,"num_object_clones":0,"num_object_copies":14388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7194,"num_write_kb":28776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'7158","reported_seq":7282,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:50.636781+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:27:50.636781+0000","last_peered":"2024-09-17T15:27:50.636781+0000","last_clean":"2024-09-17T15:27:50.636781+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:27:50.636781+0000","last_undegraded":"2024-09-17T15:27:50.636781+0000","last_fullsized":"2024-09-17T15:27:50.636781+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":7158,"log_dups_size":0,"ondisk_log_size":7158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29290979,"num_objects":7152,"num_object_clones":0,"num_object_copies":14304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7152,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4140,"num_write_kb":16560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'7440","reported_seq":7570,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:50.713781+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:27:50.713781+0000","last_peered":"2024-09-17T15:27:50.713781+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:27:50.713781+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":7440,"log_dups_size":0,"ondisk_log_size":7440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30454243,"num_objects":7436,"num_object_clones":0,"num_object_copies":14872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7436,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3939,"num_write_kb":15756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3166},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'7295","reported_seq":7429,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:50.650905+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:27:50.650905+0000","last_peered":"2024-09-17T15:27:50.650905+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:27:50.650905+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":7295,"log_dups_size":0,"ondisk_log_size":7295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29856710,"num_objects":7291,"num_object_clones":0,"num_object_copies":14582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7291,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4064,"num_write_kb":16256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3117},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'7239","reported_seq":7362,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:50.656580+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:27:50.656580+0000","last_peered":"2024-09-17T15:27:50.656580+0000","last_clean":"2024-09-17T15:27:50.656580+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:27:50.656580+0000","last_undegraded":"2024-09-17T15:27:50.656580+0000","last_fullsized":"2024-09-17T15:27:50.656580+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":7239,"log_dups_size":0,"ondisk_log_size":7239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29642752,"num_objects":7237,"num_object_clones":0,"num_object_copies":14474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4193,"num_write_kb":16772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'7074","reported_seq":12402,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:27:47.037120+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:27:47.037120+0000","last_peered":"2024-09-17T15:27:47.037120+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:27:47.037120+0000","last_undegraded":"2024-09-17T15:27:47.037120+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":7074,"log_dups_size":0,"ondisk_log_size":7074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28946915,"num_objects":7068,"num_object_clones":0,"num_object_copies":14136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1470,"num_objects_unfound":0,"num_objects_dirty":7068,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5074,"num_write_kb":20296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2602,"num_bytes_recovered":10650083,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1470},{"shards":"2","objects":5598}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233600+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233600+0000","last_peered":"2024-09-17T15:26:22.233600+0000","last_clean":"2024-09-17T15:26:22.233600+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233600+0000","last_undegraded":"2024-09-17T15:26:22.233600+0000","last_fullsized":"2024-09-17T15:26:22.233600+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'7141","reported_seq":7265,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:50.715252+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:27:50.715252+0000","last_peered":"2024-09-17T15:27:50.715252+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:27:50.715252+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":7141,"log_dups_size":0,"ondisk_log_size":7141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29216768,"num_objects":7133,"num_object_clones":0,"num_object_copies":14266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3737,"num_write_kb":14948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3123},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":98,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412941+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412941+0000","last_peered":"2024-09-17T15:26:22.412941+0000","last_clean":"2024-09-17T15:26:22.412941+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412941+0000","last_undegraded":"2024-09-17T15:26:22.412941+0000","last_fullsized":"2024-09-17T15:26:22.412941+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'7053","reported_seq":7169,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:46.971967+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:27:46.971967+0000","last_peered":"2024-09-17T15:27:46.971967+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:27:46.971967+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":7053,"log_dups_size":0,"ondisk_log_size":7053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28869091,"num_objects":7049,"num_object_clones":0,"num_object_copies":14098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7049,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4948,"num_write_kb":19792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":2972}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'7123","reported_seq":7237,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:47.004767+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:27:47.004767+0000","last_peered":"2024-09-17T15:27:47.004767+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:27:47.004767+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":7123,"log_dups_size":0,"ondisk_log_size":7123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29151232,"num_objects":7117,"num_object_clones":0,"num_object_copies":14234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4885,"num_write_kb":19540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":2980}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'7284","reported_seq":7414,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:50.587945+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:27:50.587945+0000","last_peered":"2024-09-17T15:27:50.587945+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:27:50.587945+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":7284,"log_dups_size":0,"ondisk_log_size":7284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29827072,"num_objects":7282,"num_object_clones":0,"num_object_copies":14564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3832,"num_write_kb":15328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3144},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'7218","reported_seq":7355,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:50.694902+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:27:50.694902+0000","last_peered":"2024-09-17T15:27:50.694902+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:27:50.694902+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":7218,"log_dups_size":0,"ondisk_log_size":7218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29532160,"num_objects":7210,"num_object_clones":0,"num_object_copies":14420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3505,"num_write_kb":14020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3053},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229061+0000","last_peered":"2024-09-17T15:26:22.229061+0000","last_clean":"2024-09-17T15:26:22.229061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229061+0000","last_undegraded":"2024-09-17T15:26:22.229061+0000","last_fullsized":"2024-09-17T15:26:22.229061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'7251","reported_seq":7378,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:50.674310+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:27:50.674310+0000","last_peered":"2024-09-17T15:27:50.674310+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:27:50.674310+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":7251,"log_dups_size":0,"ondisk_log_size":7251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29683712,"num_objects":7247,"num_object_clones":0,"num_object_copies":14494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4038,"num_write_kb":16152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3184},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'7200","reported_seq":7338,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:50.639218+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:27:50.639218+0000","last_peered":"2024-09-17T15:27:50.639218+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:27:50.639218+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":7200,"log_dups_size":0,"ondisk_log_size":7200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29443014,"num_objects":7190,"num_object_clones":0,"num_object_copies":14380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3642,"num_write_kb":14568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3081},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":470897374,"num_objects":114974,"num_object_clones":0,"num_object_copies":229948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":1470,"num_objects_unfound":0,"num_objects_dirty":114974,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":75081,"num_write_kb":300324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2602,"num_bytes_recovered":10650083,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":953688064,"data_stored":953536318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115046,"ondisk_log_size":115046,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051617,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539208,"kb_used_data":358968,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93832632,"statfs":{"total":96636764160,"available":96084615168,"internally_reserved":0,"allocated":367583232,"data_stored":367317059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":184544940},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724563,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":317196,"kb_used_data":159164,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":94054644,"statfs":{"total":96636764160,"available":96311955456,"internally_reserved":0,"allocated":162983936,"data_stored":162670330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":161803947},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,0,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247846,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":673928,"kb_used_data":413236,"kb_used_omap":4,"kb_used_meta":260667,"kb_avail":93697912,"statfs":{"total":96636764160,"available":95946661888,"internally_reserved":0,"allocated":423153664,"data_stored":422883414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":266923688},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,0,3,2,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":423235584,"data_stored":423199454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":162775040,"data_stored":162695554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":367677440,"data_stored":367641310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:27:53.531 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:27:53.531 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'7189", 'reported_seq': 7260, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:50.688333+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:27:50.688333+0000', 'last_peered': '2024-09-17T15:27:50.688333+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:27:50.688333+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 7189, 'log_dups_size': 0, 'ondisk_log_size': 7189, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29446144, 'num_objects': 7189, 'num_object_clones': 0, 'num_object_copies': 14378, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4070, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7189, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7189, 'num_write_kb': 28756, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3119}, {'shards': '0,2', 'objects': 4070}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:53.531 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:27:53.531 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'7108", 'reported_seq': 7227, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:47.049514+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:27:47.049514+0000', 'last_peered': '2024-09-17T15:27:47.049514+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:27:47.049514+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 7108, 'log_dups_size': 0, 'ondisk_log_size': 7108, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29097984, 'num_objects': 7104, 'num_object_clones': 0, 'num_object_copies': 14208, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4169, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7104, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5265, 'num_write_kb': 21060, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4168}, {'shards': '2', 'objects': 2936}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:27:53.531 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:27:53.531 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'7440", 'reported_seq': 7570, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:50.713781+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:27:50.713781+0000', 'last_peered': '2024-09-17T15:27:50.713781+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:27:50.713781+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 7440, 'log_dups_size': 0, 'ondisk_log_size': 7440, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30454243, 'num_objects': 7436, 'num_object_clones': 0, 'num_object_copies': 14872, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4271, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7436, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3939, 'num_write_kb': 15756, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3166}, {'shards': '0,2', 'objects': 4268}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:53.531 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:27:53.531 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'7295", 'reported_seq': 7429, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:50.650905+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:27:50.650905+0000', 'last_peered': '2024-09-17T15:27:50.650905+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:27:50.650905+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 7295, 'log_dups_size': 0, 'ondisk_log_size': 7295, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29856710, 'num_objects': 7291, 'num_object_clones': 0, 'num_object_copies': 14582, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4174, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7291, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 4064, 'num_write_kb': 16256, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3117}, {'shards': '0,2', 'objects': 4174}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:53.532 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-17T15:27:53.532 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "41'7074", 'reported_seq': 12402, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:27:47.037120+0000', 'last_change': '2024-09-17T15:26:22.588242+0000', 'last_active': '2024-09-17T15:27:47.037120+0000', 'last_peered': '2024-09-17T15:27:47.037120+0000', 'last_clean': '2024-09-17T15:26:20.155646+0000', 'last_became_active': '2024-09-17T15:26:22.557742+0000', 'last_became_peered': '2024-09-17T15:26:22.557742+0000', 'last_unstale': '2024-09-17T15:27:47.037120+0000', 'last_undegraded': '2024-09-17T15:27:47.037120+0000', 'last_fullsized': '2024-09-17T15:26:22.188850+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2016", 'last_scrub_stamp': '2024-09-17T15:25:53.959011+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:53.959011+0000', 'objects_scrubbed': 1994, 'log_size': 7074, 'log_dups_size': 0, 'ondisk_log_size': 7074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 568, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28946915, 'num_objects': 7068, 'num_object_clones': 0, 'num_object_copies': 14136, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1470, 'num_objects_unfound': 0, 'num_objects_dirty': 7068, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5074, 'num_write_kb': 20296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2602, 'num_bytes_recovered': 10650083, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1470}, {'shards': '2', 'objects': 5598}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:27:53.532 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:27:53.532 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'7141", 'reported_seq': 7265, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:50.715252+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:27:50.715252+0000', 'last_peered': '2024-09-17T15:27:50.715252+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:27:50.715252+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 7141, 'log_dups_size': 0, 'ondisk_log_size': 7141, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29216768, 'num_objects': 7133, 'num_object_clones': 0, 'num_object_copies': 14266, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4012, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7133, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3737, 'num_write_kb': 14948, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3123}, {'shards': '0,2', 'objects': 4010}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:53.532 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:27:53.532 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'7053", 'reported_seq': 7169, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:46.971967+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:27:46.971967+0000', 'last_peered': '2024-09-17T15:27:46.971967+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:27:46.971967+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 7053, 'log_dups_size': 0, 'ondisk_log_size': 7053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28869091, 'num_objects': 7049, 'num_object_clones': 0, 'num_object_copies': 14098, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4078, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7049, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4948, 'num_write_kb': 19792, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4077}, {'shards': '2', 'objects': 2972}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:27:53.532 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:27:53.532 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'7123", 'reported_seq': 7237, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:47.004767+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:27:47.004767+0000', 'last_peered': '2024-09-17T15:27:47.004767+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:27:47.004767+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 7123, 'log_dups_size': 0, 'ondisk_log_size': 7123, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29151232, 'num_objects': 7117, 'num_object_clones': 0, 'num_object_copies': 14234, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7117, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4885, 'num_write_kb': 19540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4137}, {'shards': '2', 'objects': 2980}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:27:53.532 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:27:53.533 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'7284", 'reported_seq': 7414, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:50.587945+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:27:50.587945+0000', 'last_peered': '2024-09-17T15:27:50.587945+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:27:50.587945+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 7284, 'log_dups_size': 0, 'ondisk_log_size': 7284, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29827072, 'num_objects': 7282, 'num_object_clones': 0, 'num_object_copies': 14564, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7282, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3832, 'num_write_kb': 15328, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3144}, {'shards': '0,2', 'objects': 4138}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:53.533 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:27:53.533 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'7218", 'reported_seq': 7355, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:50.694902+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:27:50.694902+0000', 'last_peered': '2024-09-17T15:27:50.694902+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:27:50.694902+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 7218, 'log_dups_size': 0, 'ondisk_log_size': 7218, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29532160, 'num_objects': 7210, 'num_object_clones': 0, 'num_object_copies': 14420, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4160, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7210, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3505, 'num_write_kb': 14020, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3053}, {'shards': '0,2', 'objects': 4157}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:53.533 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:27:53.533 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'7251", 'reported_seq': 7378, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:50.674310+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:27:50.674310+0000', 'last_peered': '2024-09-17T15:27:50.674310+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:27:50.674310+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 7251, 'log_dups_size': 0, 'ondisk_log_size': 7251, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29683712, 'num_objects': 7247, 'num_object_clones': 0, 'num_object_copies': 14494, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4065, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7247, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4038, 'num_write_kb': 16152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3184}, {'shards': '0,2', 'objects': 4063}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:53.533 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:27:53.533 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'7200", 'reported_seq': 7338, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:50.639218+0000', 'last_change': '2024-09-17T15:26:22.654521+0000', 'last_active': '2024-09-17T15:27:50.639218+0000', 'last_peered': '2024-09-17T15:27:50.639218+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:27:50.639218+0000', 'last_undegraded': '2024-09-17T15:26:22.307795+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 7200, 'log_dups_size': 0, 'ondisk_log_size': 7200, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29443014, 'num_objects': 7190, 'num_object_clones': 0, 'num_object_copies': 14380, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4111, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7190, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3642, 'num_write_kb': 14568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3081}, {'shards': '0,2', 'objects': 4109}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:53.533 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:27:54.233 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:27:54.233 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45388/231846 objects degraded (19.577%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":211,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":115923,"data_bytes":474776305,"bytes_used":1576542208,"bytes_avail":288333750272,"bytes_total":289910292480,"degraded_objects":45388,"degraded_total":231846,"degraded_ratio":0.1957678804033712,"misplaced_objects":1283,"misplaced_total":231846,"misplaced_ratio":0.0055338457424324757,"recovering_objects_per_sec":44,"recovering_bytes_per_sec":183591,"recovering_keys_per_sec":0,"num_objects_recovered":538,"num_bytes_recovered":2203648,"num_keys_recovered":0,"write_bytes_sec":2117099,"read_op_per_sec":0,"write_op_per_sec":516},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T15:27:19.616376+0000","services":{}},"progress_events":{}} 2024-09-17T15:27:54.262 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:27:54.262 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:27:54.314 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 135 20 117470 117450 3.39765 1.875 0.0557899 0.0229865 2024-09-17T15:27:54.833 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:27:54.834 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:27:54.860 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":139,"stamp":"2024-09-17T15:27:53.625933+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":474776305,"num_objects":115923,"num_object_clones":0,"num_object_copies":231846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":1283,"num_objects_unfound":0,"num_objects_dirty":115923,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":76030,"num_write_kb":304114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2790,"num_bytes_recovered":11416054,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115996,"ondisk_log_size":115996,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1539592,"kb_used_data":936468,"kb_used_omap":13,"kb_used_meta":603058,"kb_avail":281575928,"statfs":{"total":289910292480,"available":288333750272,"internally_reserved":0,"allocated":958943232,"data_stored":958093203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":617532415},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,3,0,5,4,15,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25411584,"num_objects":6204,"num_object_clones":0,"num_object_copies":12408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-538,"num_objects_unfound":0,"num_objects_dirty":6204,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6204,"num_write_kb":24816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":538,"num_bytes_recovered":2203648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003017"},"pg_stats":[{"pgid":"2.f","version":"41'7189","reported_seq":7260,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:50.688333+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:27:50.688333+0000","last_peered":"2024-09-17T15:27:50.688333+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:27:50.688333+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":7189,"log_dups_size":0,"ondisk_log_size":7189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29446144,"num_objects":7189,"num_object_clones":0,"num_object_copies":14378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7189,"num_write_kb":28756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3119},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'7249","reported_seq":7347,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:52.047408+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:27:52.047408+0000","last_peered":"2024-09-17T15:27:52.047408+0000","last_clean":"2024-09-17T15:27:52.047408+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:27:52.047408+0000","last_undegraded":"2024-09-17T15:27:52.047408+0000","last_fullsized":"2024-09-17T15:27:52.047408+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":7249,"log_dups_size":0,"ondisk_log_size":7249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29668294,"num_objects":7245,"num_object_clones":0,"num_object_copies":14490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7245,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5606,"num_write_kb":22424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'7269","reported_seq":7389,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:51.971821+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:27:51.971821+0000","last_peered":"2024-09-17T15:27:51.971821+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:27:51.971821+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":7269,"log_dups_size":0,"ondisk_log_size":7269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29757440,"num_objects":7265,"num_object_clones":0,"num_object_copies":14530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7265,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5426,"num_write_kb":21704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":3097}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'7340","reported_seq":7405,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:52.013448+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:27:52.013448+0000","last_peered":"2024-09-17T15:27:52.013448+0000","last_clean":"2024-09-17T15:27:52.013448+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:27:52.013448+0000","last_undegraded":"2024-09-17T15:27:52.013448+0000","last_fullsized":"2024-09-17T15:27:52.013448+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":7340,"log_dups_size":0,"ondisk_log_size":7340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30064640,"num_objects":7340,"num_object_clones":0,"num_object_copies":14680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7340,"num_write_kb":29360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'7158","reported_seq":7282,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:50.636781+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:27:50.636781+0000","last_peered":"2024-09-17T15:27:50.636781+0000","last_clean":"2024-09-17T15:27:50.636781+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:27:50.636781+0000","last_undegraded":"2024-09-17T15:27:50.636781+0000","last_fullsized":"2024-09-17T15:27:50.636781+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":7158,"log_dups_size":0,"ondisk_log_size":7158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29290979,"num_objects":7152,"num_object_clones":0,"num_object_copies":14304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7152,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4140,"num_write_kb":16560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'7440","reported_seq":7570,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:50.713781+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:27:50.713781+0000","last_peered":"2024-09-17T15:27:50.713781+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:27:50.713781+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":7440,"log_dups_size":0,"ondisk_log_size":7440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30454243,"num_objects":7436,"num_object_clones":0,"num_object_copies":14872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7436,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3939,"num_write_kb":15756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3166},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'7295","reported_seq":7429,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:50.650905+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:27:50.650905+0000","last_peered":"2024-09-17T15:27:50.650905+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:27:50.650905+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":7295,"log_dups_size":0,"ondisk_log_size":7295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29856710,"num_objects":7291,"num_object_clones":0,"num_object_copies":14582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7291,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4064,"num_write_kb":16256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3117},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'7239","reported_seq":7362,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:50.656580+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:27:50.656580+0000","last_peered":"2024-09-17T15:27:50.656580+0000","last_clean":"2024-09-17T15:27:50.656580+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:27:50.656580+0000","last_undegraded":"2024-09-17T15:27:50.656580+0000","last_fullsized":"2024-09-17T15:27:50.656580+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":7239,"log_dups_size":0,"ondisk_log_size":7239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29642752,"num_objects":7237,"num_object_clones":0,"num_object_copies":14474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4193,"num_write_kb":16772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'7234","reported_seq":12936,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:27:52.060136+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:27:52.060136+0000","last_peered":"2024-09-17T15:27:52.060136+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:27:52.060136+0000","last_undegraded":"2024-09-17T15:27:52.060136+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":7234,"log_dups_size":0,"ondisk_log_size":7234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29602275,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1283,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5234,"num_write_kb":20936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2789,"num_bytes_recovered":11416035,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1283},{"shards":"2","objects":5945}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233600+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233600+0000","last_peered":"2024-09-17T15:26:22.233600+0000","last_clean":"2024-09-17T15:26:22.233600+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233600+0000","last_undegraded":"2024-09-17T15:26:22.233600+0000","last_fullsized":"2024-09-17T15:26:22.233600+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'7141","reported_seq":7265,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:50.715252+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:27:50.715252+0000","last_peered":"2024-09-17T15:27:50.715252+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:27:50.715252+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":7141,"log_dups_size":0,"ondisk_log_size":7141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29216768,"num_objects":7133,"num_object_clones":0,"num_object_copies":14266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3737,"num_write_kb":14948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3123},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":98,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412941+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412941+0000","last_peered":"2024-09-17T15:26:22.412941+0000","last_clean":"2024-09-17T15:26:22.412941+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412941+0000","last_undegraded":"2024-09-17T15:26:22.412941+0000","last_fullsized":"2024-09-17T15:26:22.412941+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'7202","reported_seq":7318,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:51.880751+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:27:51.880751+0000","last_peered":"2024-09-17T15:27:51.880751+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:27:51.880751+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":7202,"log_dups_size":0,"ondisk_log_size":7202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29479395,"num_objects":7198,"num_object_clones":0,"num_object_copies":14396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7198,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5097,"num_write_kb":20388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":3121}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'7284","reported_seq":7398,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:52.046808+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:27:52.046808+0000","last_peered":"2024-09-17T15:27:52.046808+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:27:52.046808+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":7284,"log_dups_size":0,"ondisk_log_size":7284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29810688,"num_objects":7278,"num_object_clones":0,"num_object_copies":14556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7278,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5046,"num_write_kb":20184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":3141}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'7284","reported_seq":7414,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:50.587945+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:27:50.587945+0000","last_peered":"2024-09-17T15:27:50.587945+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:27:50.587945+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":7284,"log_dups_size":0,"ondisk_log_size":7284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29827072,"num_objects":7282,"num_object_clones":0,"num_object_copies":14564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3832,"num_write_kb":15328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3144},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'7218","reported_seq":7355,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:50.694902+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:27:50.694902+0000","last_peered":"2024-09-17T15:27:50.694902+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:27:50.694902+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":7218,"log_dups_size":0,"ondisk_log_size":7218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29532160,"num_objects":7210,"num_object_clones":0,"num_object_copies":14420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3505,"num_write_kb":14020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3053},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229061+0000","last_peered":"2024-09-17T15:26:22.229061+0000","last_clean":"2024-09-17T15:26:22.229061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229061+0000","last_undegraded":"2024-09-17T15:26:22.229061+0000","last_fullsized":"2024-09-17T15:26:22.229061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'7251","reported_seq":7378,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:50.674310+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:27:50.674310+0000","last_peered":"2024-09-17T15:27:50.674310+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:27:50.674310+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":7251,"log_dups_size":0,"ondisk_log_size":7251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29683712,"num_objects":7247,"num_object_clones":0,"num_object_copies":14494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4038,"num_write_kb":16152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3184},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'7200","reported_seq":7338,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:50.639218+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:27:50.639218+0000","last_peered":"2024-09-17T15:27:50.639218+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:27:50.639218+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":7200,"log_dups_size":0,"ondisk_log_size":7200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29443014,"num_objects":7190,"num_object_clones":0,"num_object_copies":14380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3642,"num_write_kb":14568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3081},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":474776286,"num_objects":115921,"num_object_clones":0,"num_object_copies":231842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":1283,"num_objects_unfound":0,"num_objects_dirty":115921,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":76028,"num_write_kb":304112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2789,"num_bytes_recovered":11416035,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":958803968,"data_stored":958652222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115993,"ondisk_log_size":115993,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051618,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548468,"kb_used_data":364068,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93823372,"statfs":{"total":96636764160,"available":96075132928,"internally_reserved":0,"allocated":372805632,"data_stored":372539459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":188804780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724563,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":317196,"kb_used_data":159164,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":94054644,"statfs":{"total":96636764160,"available":96311955456,"internally_reserved":0,"allocated":162983936,"data_stored":162670330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":161803947},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,0,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247846,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":673928,"kb_used_data":413236,"kb_used_omap":4,"kb_used_meta":260667,"kb_avail":93697912,"statfs":{"total":96636764160,"available":95946661888,"internally_reserved":0,"allocated":423153664,"data_stored":422883414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":266923688},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,0,3,2,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":423235584,"data_stored":423199454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":162775040,"data_stored":162695554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":372793344,"data_stored":372757214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:27:55.314 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 136 20 117978 117958 3.38726 1.98438 0.0169228 0.0230578 2024-09-17T15:27:56.314 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 137 20 118526 118506 3.37816 2.14062 0.0560574 0.0231191 2024-09-17T15:27:57.318 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 138 20 119080 119060 3.36936 2.16406 0.0391 0.0231795 2024-09-17T15:27:57.863 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:27:58.315 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 139 20 119595 119575 3.35959 2.01172 0.0136195 0.0232438 2024-09-17T15:27:58.411 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:27:58.411 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:27:58.447 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2024-09-17T15:27:57.626915+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":485405425,"num_objects":118518,"num_object_clones":0,"num_object_copies":237036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":1093,"num_objects_unfound":0,"num_objects_dirty":118518,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":78625,"num_write_kb":314494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2980,"num_bytes_recovered":12194294,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118591,"ondisk_log_size":118591,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1544352,"kb_used_data":957872,"kb_used_omap":13,"kb_used_meta":586418,"kb_avail":281571168,"statfs":{"total":289910292480,"available":288328876032,"internally_reserved":0,"allocated":980860928,"data_stored":980010899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":600493055},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,5,5,11,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32071680,"num_objects":7830,"num_object_clones":0,"num_object_copies":15660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-546,"num_objects_unfound":0,"num_objects_dirty":7830,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7830,"num_write_kb":31320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":546,"num_bytes_recovered":2236416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002949"},"pg_stats":[{"pgid":"2.f","version":"41'7359","reported_seq":7432,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:55.748973+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:27:55.748973+0000","last_peered":"2024-09-17T15:27:55.748973+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:27:55.748973+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":7359,"log_dups_size":0,"ondisk_log_size":7359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30142464,"num_objects":7359,"num_object_clones":0,"num_object_copies":14718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7359,"num_write_kb":29436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3289},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'7404","reported_seq":7502,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:57.018889+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:27:57.018889+0000","last_peered":"2024-09-17T15:27:57.018889+0000","last_clean":"2024-09-17T15:27:57.018889+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:27:57.018889+0000","last_undegraded":"2024-09-17T15:27:57.018889+0000","last_fullsized":"2024-09-17T15:27:57.018889+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":7404,"log_dups_size":0,"ondisk_log_size":7404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30303174,"num_objects":7400,"num_object_clones":0,"num_object_copies":14800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7400,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5761,"num_write_kb":23044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'7418","reported_seq":7538,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:57.015343+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:27:57.015343+0000","last_peered":"2024-09-17T15:27:57.015343+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:27:57.015343+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":7418,"log_dups_size":0,"ondisk_log_size":7418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30367744,"num_objects":7414,"num_object_clones":0,"num_object_copies":14828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5575,"num_write_kb":22300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":3246}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'7518","reported_seq":7583,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:57.045103+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:27:57.045103+0000","last_peered":"2024-09-17T15:27:57.045103+0000","last_clean":"2024-09-17T15:27:57.045103+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:27:57.045103+0000","last_undegraded":"2024-09-17T15:27:57.045103+0000","last_fullsized":"2024-09-17T15:27:57.045103+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":7518,"log_dups_size":0,"ondisk_log_size":7518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30793728,"num_objects":7518,"num_object_clones":0,"num_object_copies":15036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7518,"num_write_kb":30072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'7308","reported_seq":7432,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:55.715301+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:27:55.715301+0000","last_peered":"2024-09-17T15:27:55.715301+0000","last_clean":"2024-09-17T15:27:55.715301+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:27:55.715301+0000","last_undegraded":"2024-09-17T15:27:55.715301+0000","last_fullsized":"2024-09-17T15:27:55.715301+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":7308,"log_dups_size":0,"ondisk_log_size":7308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29905379,"num_objects":7302,"num_object_clones":0,"num_object_copies":14604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7302,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4290,"num_write_kb":17160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'7629","reported_seq":7759,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:55.720139+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:27:55.720139+0000","last_peered":"2024-09-17T15:27:55.720139+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:27:55.720139+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":7629,"log_dups_size":0,"ondisk_log_size":7629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31228387,"num_objects":7625,"num_object_clones":0,"num_object_copies":15250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7625,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4128,"num_write_kb":16512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3355},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'7462","reported_seq":7596,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:55.668431+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:27:55.668431+0000","last_peered":"2024-09-17T15:27:55.668431+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:27:55.668431+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":7462,"log_dups_size":0,"ondisk_log_size":7462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30540742,"num_objects":7458,"num_object_clones":0,"num_object_copies":14916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7458,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4231,"num_write_kb":16924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3284},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'7397","reported_seq":7520,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:55.717283+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:27:55.717283+0000","last_peered":"2024-09-17T15:27:55.717283+0000","last_clean":"2024-09-17T15:27:55.717283+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:27:55.717283+0000","last_undegraded":"2024-09-17T15:27:55.717283+0000","last_fullsized":"2024-09-17T15:27:55.717283+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":7397,"log_dups_size":0,"ondisk_log_size":7397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30289920,"num_objects":7395,"num_object_clones":0,"num_object_copies":14790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4351,"num_write_kb":17404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'7401","reported_seq":13483,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:27:57.063758+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:27:57.063758+0000","last_peered":"2024-09-17T15:27:57.063758+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:27:57.063758+0000","last_undegraded":"2024-09-17T15:27:57.063758+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":7401,"log_dups_size":0,"ondisk_log_size":7401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30286307,"num_objects":7395,"num_object_clones":0,"num_object_copies":14790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1093,"num_objects_unfound":0,"num_objects_dirty":7395,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5401,"num_write_kb":21604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2979,"num_bytes_recovered":12194275,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1093},{"shards":"2","objects":6302}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233600+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233600+0000","last_peered":"2024-09-17T15:26:22.233600+0000","last_clean":"2024-09-17T15:26:22.233600+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233600+0000","last_undegraded":"2024-09-17T15:26:22.233600+0000","last_fullsized":"2024-09-17T15:26:22.233600+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'7299","reported_seq":7424,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:55.720469+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:27:55.720469+0000","last_peered":"2024-09-17T15:27:55.720469+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:27:55.720469+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":7299,"log_dups_size":0,"ondisk_log_size":7299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29863936,"num_objects":7291,"num_object_clones":0,"num_object_copies":14582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7291,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3895,"num_write_kb":15580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3281},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":98,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412941+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412941+0000","last_peered":"2024-09-17T15:26:22.412941+0000","last_clean":"2024-09-17T15:26:22.412941+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412941+0000","last_undegraded":"2024-09-17T15:26:22.412941+0000","last_fullsized":"2024-09-17T15:26:22.412941+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'7366","reported_seq":7482,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:57.042398+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:27:57.042398+0000","last_peered":"2024-09-17T15:27:57.042398+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:27:57.042398+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":7366,"log_dups_size":0,"ondisk_log_size":7366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30151139,"num_objects":7362,"num_object_clones":0,"num_object_copies":14724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7362,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5261,"num_write_kb":21044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":3285}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'7431","reported_seq":7545,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:57.041413+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:27:57.041413+0000","last_peered":"2024-09-17T15:27:57.041413+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:27:57.041413+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":7431,"log_dups_size":0,"ondisk_log_size":7431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30412800,"num_objects":7425,"num_object_clones":0,"num_object_copies":14850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5193,"num_write_kb":20772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":3288}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'7452","reported_seq":7581,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:55.676815+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:27:55.676815+0000","last_peered":"2024-09-17T15:27:55.676815+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:27:55.676815+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":7452,"log_dups_size":0,"ondisk_log_size":7452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30515200,"num_objects":7450,"num_object_clones":0,"num_object_copies":14900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4000,"num_write_kb":16000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3312},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'7379","reported_seq":7516,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:55.725079+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:27:55.725079+0000","last_peered":"2024-09-17T15:27:55.725079+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:27:55.725079+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":7379,"log_dups_size":0,"ondisk_log_size":7379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30191616,"num_objects":7371,"num_object_clones":0,"num_object_copies":14742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3666,"num_write_kb":14664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3214},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229061+0000","last_peered":"2024-09-17T15:26:22.229061+0000","last_clean":"2024-09-17T15:26:22.229061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229061+0000","last_undegraded":"2024-09-17T15:26:22.229061+0000","last_fullsized":"2024-09-17T15:26:22.229061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'7395","reported_seq":7522,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:55.751950+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:27:55.751950+0000","last_peered":"2024-09-17T15:27:55.751950+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:27:55.751950+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":7395,"log_dups_size":0,"ondisk_log_size":7395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30273536,"num_objects":7391,"num_object_clones":0,"num_object_copies":14782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4182,"num_write_kb":16728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3328},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'7370","reported_seq":7508,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:55.761314+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:27:55.761314+0000","last_peered":"2024-09-17T15:27:55.761314+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:27:55.761314+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":7370,"log_dups_size":0,"ondisk_log_size":7370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30139334,"num_objects":7360,"num_object_clones":0,"num_object_copies":14720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7360,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3812,"num_write_kb":15248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3251},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":485405406,"num_objects":118516,"num_object_clones":0,"num_object_copies":237032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":1093,"num_objects_unfound":0,"num_objects_dirty":118516,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":78623,"num_write_kb":314492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2979,"num_bytes_recovered":12194275,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":980774912,"data_stored":980623166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118588,"ondisk_log_size":118588,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051619,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":541268,"kb_used_data":369348,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93830572,"statfs":{"total":96636764160,"available":96082505728,"internally_reserved":0,"allocated":378212352,"data_stored":377946179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":176025260},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,0,1,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724564,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":333852,"kb_used_data":167500,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":94037988,"statfs":{"total":96636764160,"available":96294899712,"internally_reserved":0,"allocated":171520000,"data_stored":171206394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":170323627},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247847,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":669232,"kb_used_data":421024,"kb_used_omap":4,"kb_used_meta":248187,"kb_avail":93702608,"statfs":{"total":96636764160,"available":95951470592,"internally_reserved":0,"allocated":431128576,"data_stored":430858326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":254144168},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,5,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":431271936,"data_stored":431235806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":171454464,"data_stored":171374978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":378048512,"data_stored":378012382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:27:58.449 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:27:59.020 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:27:59.023 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:27:59.053 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2024-09-17T15:27:57.626915+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":485405425,"num_objects":118518,"num_object_clones":0,"num_object_copies":237036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":1093,"num_objects_unfound":0,"num_objects_dirty":118518,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":78625,"num_write_kb":314494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2980,"num_bytes_recovered":12194294,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118591,"ondisk_log_size":118591,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1544352,"kb_used_data":957872,"kb_used_omap":13,"kb_used_meta":586418,"kb_avail":281571168,"statfs":{"total":289910292480,"available":288328876032,"internally_reserved":0,"allocated":980860928,"data_stored":980010899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":600493055},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,5,5,11,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32071680,"num_objects":7830,"num_object_clones":0,"num_object_copies":15660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-546,"num_objects_unfound":0,"num_objects_dirty":7830,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7830,"num_write_kb":31320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":546,"num_bytes_recovered":2236416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002949"},"pg_stats":[{"pgid":"2.f","version":"41'7359","reported_seq":7432,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:55.748973+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:27:55.748973+0000","last_peered":"2024-09-17T15:27:55.748973+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:27:55.748973+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":7359,"log_dups_size":0,"ondisk_log_size":7359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30142464,"num_objects":7359,"num_object_clones":0,"num_object_copies":14718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7359,"num_write_kb":29436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3289},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'7404","reported_seq":7502,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:57.018889+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:27:57.018889+0000","last_peered":"2024-09-17T15:27:57.018889+0000","last_clean":"2024-09-17T15:27:57.018889+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:27:57.018889+0000","last_undegraded":"2024-09-17T15:27:57.018889+0000","last_fullsized":"2024-09-17T15:27:57.018889+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":7404,"log_dups_size":0,"ondisk_log_size":7404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30303174,"num_objects":7400,"num_object_clones":0,"num_object_copies":14800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7400,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5761,"num_write_kb":23044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'7418","reported_seq":7538,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:57.015343+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:27:57.015343+0000","last_peered":"2024-09-17T15:27:57.015343+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:27:57.015343+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":7418,"log_dups_size":0,"ondisk_log_size":7418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30367744,"num_objects":7414,"num_object_clones":0,"num_object_copies":14828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5575,"num_write_kb":22300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":3246}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'7518","reported_seq":7583,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:57.045103+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:27:57.045103+0000","last_peered":"2024-09-17T15:27:57.045103+0000","last_clean":"2024-09-17T15:27:57.045103+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:27:57.045103+0000","last_undegraded":"2024-09-17T15:27:57.045103+0000","last_fullsized":"2024-09-17T15:27:57.045103+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":7518,"log_dups_size":0,"ondisk_log_size":7518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30793728,"num_objects":7518,"num_object_clones":0,"num_object_copies":15036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7518,"num_write_kb":30072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'7308","reported_seq":7432,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:55.715301+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:27:55.715301+0000","last_peered":"2024-09-17T15:27:55.715301+0000","last_clean":"2024-09-17T15:27:55.715301+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:27:55.715301+0000","last_undegraded":"2024-09-17T15:27:55.715301+0000","last_fullsized":"2024-09-17T15:27:55.715301+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":7308,"log_dups_size":0,"ondisk_log_size":7308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29905379,"num_objects":7302,"num_object_clones":0,"num_object_copies":14604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7302,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4290,"num_write_kb":17160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'7629","reported_seq":7759,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:55.720139+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:27:55.720139+0000","last_peered":"2024-09-17T15:27:55.720139+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:27:55.720139+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":7629,"log_dups_size":0,"ondisk_log_size":7629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31228387,"num_objects":7625,"num_object_clones":0,"num_object_copies":15250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7625,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4128,"num_write_kb":16512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3355},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'7462","reported_seq":7596,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:55.668431+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:27:55.668431+0000","last_peered":"2024-09-17T15:27:55.668431+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:27:55.668431+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":7462,"log_dups_size":0,"ondisk_log_size":7462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30540742,"num_objects":7458,"num_object_clones":0,"num_object_copies":14916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7458,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4231,"num_write_kb":16924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3284},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'7397","reported_seq":7520,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:55.717283+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:27:55.717283+0000","last_peered":"2024-09-17T15:27:55.717283+0000","last_clean":"2024-09-17T15:27:55.717283+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:27:55.717283+0000","last_undegraded":"2024-09-17T15:27:55.717283+0000","last_fullsized":"2024-09-17T15:27:55.717283+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":7397,"log_dups_size":0,"ondisk_log_size":7397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30289920,"num_objects":7395,"num_object_clones":0,"num_object_copies":14790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4351,"num_write_kb":17404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'7401","reported_seq":13483,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:27:57.063758+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:27:57.063758+0000","last_peered":"2024-09-17T15:27:57.063758+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:27:57.063758+0000","last_undegraded":"2024-09-17T15:27:57.063758+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":7401,"log_dups_size":0,"ondisk_log_size":7401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30286307,"num_objects":7395,"num_object_clones":0,"num_object_copies":14790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1093,"num_objects_unfound":0,"num_objects_dirty":7395,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5401,"num_write_kb":21604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2979,"num_bytes_recovered":12194275,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1093},{"shards":"2","objects":6302}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233600+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233600+0000","last_peered":"2024-09-17T15:26:22.233600+0000","last_clean":"2024-09-17T15:26:22.233600+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233600+0000","last_undegraded":"2024-09-17T15:26:22.233600+0000","last_fullsized":"2024-09-17T15:26:22.233600+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'7299","reported_seq":7424,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:55.720469+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:27:55.720469+0000","last_peered":"2024-09-17T15:27:55.720469+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:27:55.720469+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":7299,"log_dups_size":0,"ondisk_log_size":7299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29863936,"num_objects":7291,"num_object_clones":0,"num_object_copies":14582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7291,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3895,"num_write_kb":15580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3281},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":98,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412941+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412941+0000","last_peered":"2024-09-17T15:26:22.412941+0000","last_clean":"2024-09-17T15:26:22.412941+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412941+0000","last_undegraded":"2024-09-17T15:26:22.412941+0000","last_fullsized":"2024-09-17T15:26:22.412941+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'7366","reported_seq":7482,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:57.042398+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:27:57.042398+0000","last_peered":"2024-09-17T15:27:57.042398+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:27:57.042398+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":7366,"log_dups_size":0,"ondisk_log_size":7366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30151139,"num_objects":7362,"num_object_clones":0,"num_object_copies":14724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7362,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5261,"num_write_kb":21044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":3285}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'7431","reported_seq":7545,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:57.041413+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:27:57.041413+0000","last_peered":"2024-09-17T15:27:57.041413+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:27:57.041413+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":7431,"log_dups_size":0,"ondisk_log_size":7431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30412800,"num_objects":7425,"num_object_clones":0,"num_object_copies":14850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5193,"num_write_kb":20772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":3288}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'7452","reported_seq":7581,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:55.676815+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:27:55.676815+0000","last_peered":"2024-09-17T15:27:55.676815+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:27:55.676815+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":7452,"log_dups_size":0,"ondisk_log_size":7452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30515200,"num_objects":7450,"num_object_clones":0,"num_object_copies":14900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4000,"num_write_kb":16000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3312},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'7379","reported_seq":7516,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:55.725079+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:27:55.725079+0000","last_peered":"2024-09-17T15:27:55.725079+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:27:55.725079+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":7379,"log_dups_size":0,"ondisk_log_size":7379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30191616,"num_objects":7371,"num_object_clones":0,"num_object_copies":14742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3666,"num_write_kb":14664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3214},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229061+0000","last_peered":"2024-09-17T15:26:22.229061+0000","last_clean":"2024-09-17T15:26:22.229061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229061+0000","last_undegraded":"2024-09-17T15:26:22.229061+0000","last_fullsized":"2024-09-17T15:26:22.229061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'7395","reported_seq":7522,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:55.751950+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:27:55.751950+0000","last_peered":"2024-09-17T15:27:55.751950+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:27:55.751950+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":7395,"log_dups_size":0,"ondisk_log_size":7395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30273536,"num_objects":7391,"num_object_clones":0,"num_object_copies":14782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4182,"num_write_kb":16728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3328},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'7370","reported_seq":7508,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:55.761314+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:27:55.761314+0000","last_peered":"2024-09-17T15:27:55.761314+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:27:55.761314+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":7370,"log_dups_size":0,"ondisk_log_size":7370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30139334,"num_objects":7360,"num_object_clones":0,"num_object_copies":14720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7360,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3812,"num_write_kb":15248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3251},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":485405406,"num_objects":118516,"num_object_clones":0,"num_object_copies":237032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":1093,"num_objects_unfound":0,"num_objects_dirty":118516,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":78623,"num_write_kb":314492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2979,"num_bytes_recovered":12194275,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":980774912,"data_stored":980623166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118588,"ondisk_log_size":118588,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051619,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":541268,"kb_used_data":369348,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93830572,"statfs":{"total":96636764160,"available":96082505728,"internally_reserved":0,"allocated":378212352,"data_stored":377946179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":176025260},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,0,1,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724564,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":333852,"kb_used_data":167500,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":94037988,"statfs":{"total":96636764160,"available":96294899712,"internally_reserved":0,"allocated":171520000,"data_stored":171206394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":170323627},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247847,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":669232,"kb_used_data":421024,"kb_used_omap":4,"kb_used_meta":248187,"kb_avail":93702608,"statfs":{"total":96636764160,"available":95951470592,"internally_reserved":0,"allocated":431128576,"data_stored":430858326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":254144168},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,5,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":431271936,"data_stored":431235806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":171454464,"data_stored":171374978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":378048512,"data_stored":378012382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:27:59.055 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:27:59.055 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'7359", 'reported_seq': 7432, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:55.748973+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:27:55.748973+0000', 'last_peered': '2024-09-17T15:27:55.748973+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:27:55.748973+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 7359, 'log_dups_size': 0, 'ondisk_log_size': 7359, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30142464, 'num_objects': 7359, 'num_object_clones': 0, 'num_object_copies': 14718, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4070, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7359, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7359, 'num_write_kb': 29436, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3289}, {'shards': '0,2', 'objects': 4070}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:59.055 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:27:59.055 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'7418", 'reported_seq': 7538, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:57.015343+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:27:57.015343+0000', 'last_peered': '2024-09-17T15:27:57.015343+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:27:57.015343+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 7418, 'log_dups_size': 0, 'ondisk_log_size': 7418, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30367744, 'num_objects': 7414, 'num_object_clones': 0, 'num_object_copies': 14828, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4169, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7414, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5575, 'num_write_kb': 22300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4168}, {'shards': '2', 'objects': 3246}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:27:59.055 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:27:59.055 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'7629", 'reported_seq': 7759, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:55.720139+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:27:55.720139+0000', 'last_peered': '2024-09-17T15:27:55.720139+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:27:55.720139+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 7629, 'log_dups_size': 0, 'ondisk_log_size': 7629, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31228387, 'num_objects': 7625, 'num_object_clones': 0, 'num_object_copies': 15250, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4271, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7625, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4128, 'num_write_kb': 16512, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3355}, {'shards': '0,2', 'objects': 4268}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:59.055 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:27:59.055 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'7462", 'reported_seq': 7596, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:55.668431+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:27:55.668431+0000', 'last_peered': '2024-09-17T15:27:55.668431+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:27:55.668431+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 7462, 'log_dups_size': 0, 'ondisk_log_size': 7462, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30540742, 'num_objects': 7458, 'num_object_clones': 0, 'num_object_copies': 14916, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4174, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7458, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 4231, 'num_write_kb': 16924, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3284}, {'shards': '0,2', 'objects': 4174}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:59.055 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-17T15:27:59.056 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "41'7401", 'reported_seq': 13483, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:27:57.063758+0000', 'last_change': '2024-09-17T15:26:22.588242+0000', 'last_active': '2024-09-17T15:27:57.063758+0000', 'last_peered': '2024-09-17T15:27:57.063758+0000', 'last_clean': '2024-09-17T15:26:20.155646+0000', 'last_became_active': '2024-09-17T15:26:22.557742+0000', 'last_became_peered': '2024-09-17T15:26:22.557742+0000', 'last_unstale': '2024-09-17T15:27:57.063758+0000', 'last_undegraded': '2024-09-17T15:27:57.063758+0000', 'last_fullsized': '2024-09-17T15:26:22.188850+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2016", 'last_scrub_stamp': '2024-09-17T15:25:53.959011+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:53.959011+0000', 'objects_scrubbed': 1994, 'log_size': 7401, 'log_dups_size': 0, 'ondisk_log_size': 7401, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 568, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30286307, 'num_objects': 7395, 'num_object_clones': 0, 'num_object_copies': 14790, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1093, 'num_objects_unfound': 0, 'num_objects_dirty': 7395, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5401, 'num_write_kb': 21604, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2979, 'num_bytes_recovered': 12194275, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1093}, {'shards': '2', 'objects': 6302}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:27:59.056 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:27:59.056 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'7299", 'reported_seq': 7424, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:55.720469+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:27:55.720469+0000', 'last_peered': '2024-09-17T15:27:55.720469+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:27:55.720469+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 7299, 'log_dups_size': 0, 'ondisk_log_size': 7299, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29863936, 'num_objects': 7291, 'num_object_clones': 0, 'num_object_copies': 14582, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4012, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7291, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3895, 'num_write_kb': 15580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3281}, {'shards': '0,2', 'objects': 4010}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:59.056 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:27:59.056 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'7366", 'reported_seq': 7482, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:57.042398+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:27:57.042398+0000', 'last_peered': '2024-09-17T15:27:57.042398+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:27:57.042398+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 7366, 'log_dups_size': 0, 'ondisk_log_size': 7366, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30151139, 'num_objects': 7362, 'num_object_clones': 0, 'num_object_copies': 14724, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4078, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7362, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5261, 'num_write_kb': 21044, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4077}, {'shards': '2', 'objects': 3285}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:27:59.056 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:27:59.056 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'7431", 'reported_seq': 7545, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:57.041413+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:27:57.041413+0000', 'last_peered': '2024-09-17T15:27:57.041413+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:27:57.041413+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 7431, 'log_dups_size': 0, 'ondisk_log_size': 7431, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30412800, 'num_objects': 7425, 'num_object_clones': 0, 'num_object_copies': 14850, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7425, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5193, 'num_write_kb': 20772, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4137}, {'shards': '2', 'objects': 3288}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:27:59.056 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:27:59.056 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'7452", 'reported_seq': 7581, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:55.676815+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:27:55.676815+0000', 'last_peered': '2024-09-17T15:27:55.676815+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:27:55.676815+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 7452, 'log_dups_size': 0, 'ondisk_log_size': 7452, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30515200, 'num_objects': 7450, 'num_object_clones': 0, 'num_object_copies': 14900, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7450, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4000, 'num_write_kb': 16000, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3312}, {'shards': '0,2', 'objects': 4138}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:59.056 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:27:59.056 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'7379", 'reported_seq': 7516, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:55.725079+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:27:55.725079+0000', 'last_peered': '2024-09-17T15:27:55.725079+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:27:55.725079+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 7379, 'log_dups_size': 0, 'ondisk_log_size': 7379, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30191616, 'num_objects': 7371, 'num_object_clones': 0, 'num_object_copies': 14742, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4160, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7371, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3666, 'num_write_kb': 14664, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3214}, {'shards': '0,2', 'objects': 4157}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:59.057 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:27:59.057 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'7395", 'reported_seq': 7522, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:55.751950+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:27:55.751950+0000', 'last_peered': '2024-09-17T15:27:55.751950+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:27:55.751950+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 7395, 'log_dups_size': 0, 'ondisk_log_size': 7395, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30273536, 'num_objects': 7391, 'num_object_clones': 0, 'num_object_copies': 14782, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4065, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7391, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4182, 'num_write_kb': 16728, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3328}, {'shards': '0,2', 'objects': 4063}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:59.057 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:27:59.057 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'7370", 'reported_seq': 7508, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:27:55.761314+0000', 'last_change': '2024-09-17T15:26:22.654521+0000', 'last_active': '2024-09-17T15:27:55.761314+0000', 'last_peered': '2024-09-17T15:27:55.761314+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:27:55.761314+0000', 'last_undegraded': '2024-09-17T15:26:22.307795+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 7370, 'log_dups_size': 0, 'ondisk_log_size': 7370, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30139334, 'num_objects': 7360, 'num_object_clones': 0, 'num_object_copies': 14720, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4111, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7360, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3812, 'num_write_kb': 15248, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3251}, {'shards': '0,2', 'objects': 4109}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:27:59.057 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:27:59.315 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout:2024-09-17T15:27:59.315316+0000 min lat: 0.00187689 max lat: 2.64156 avg lat: 0.0233206 2024-09-17T15:27:59.315 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-17T15:27:59.315 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 140 20 120071 120051 3.34888 1.85938 0.0222646 0.0233206 2024-09-17T15:27:59.750 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:27:59.750 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45388/237036 objects degraded (19.148%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":216,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":118518,"data_bytes":485405425,"bytes_used":1581416448,"bytes_avail":288328876032,"bytes_total":289910292480,"degraded_objects":45388,"degraded_total":237036,"degraded_ratio":0.19148146273139946,"misplaced_objects":1093,"misplaced_total":237036,"misplaced_ratio":0.0046111139236234156,"recovering_objects_per_sec":45,"recovering_bytes_per_sec":186322,"recovering_keys_per_sec":0,"num_objects_recovered":546,"num_bytes_recovered":2236416,"num_keys_recovered":0,"write_bytes_sec":2671983,"read_op_per_sec":0,"write_op_per_sec":652},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T15:27:19.616376+0000","services":{}},"progress_events":{}} 2024-09-17T15:27:59.777 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:27:59.778 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:28:00.315 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 141 20 120548 120528 3.33834 1.86328 0.0222335 0.0233938 2024-09-17T15:28:00.340 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:28:00.343 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:28:00.367 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2024-09-17T15:27:59.627417+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":485405425,"num_objects":118518,"num_object_clones":0,"num_object_copies":237036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":1093,"num_objects_unfound":0,"num_objects_dirty":118518,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":78625,"num_write_kb":314494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2980,"num_bytes_recovered":12194294,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118591,"ondisk_log_size":118591,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1544352,"kb_used_data":957872,"kb_used_omap":13,"kb_used_meta":586418,"kb_avail":281571168,"statfs":{"total":289910292480,"available":288328876032,"internally_reserved":0,"allocated":980860928,"data_stored":980010899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":600493055},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,5,5,11,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21045248,"num_objects":5138,"num_object_clones":0,"num_object_copies":10276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-377,"num_objects_unfound":0,"num_objects_dirty":5138,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5138,"num_write_kb":20552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":377,"num_bytes_recovered":1544192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002984"},"pg_stats":[{"pgid":"2.f","version":"41'7359","reported_seq":7432,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:55.748973+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:27:55.748973+0000","last_peered":"2024-09-17T15:27:55.748973+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:27:55.748973+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":7359,"log_dups_size":0,"ondisk_log_size":7359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30142464,"num_objects":7359,"num_object_clones":0,"num_object_copies":14718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7359,"num_write_kb":29436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3289},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'7404","reported_seq":7502,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:57.018889+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:27:57.018889+0000","last_peered":"2024-09-17T15:27:57.018889+0000","last_clean":"2024-09-17T15:27:57.018889+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:27:57.018889+0000","last_undegraded":"2024-09-17T15:27:57.018889+0000","last_fullsized":"2024-09-17T15:27:57.018889+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":7404,"log_dups_size":0,"ondisk_log_size":7404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30303174,"num_objects":7400,"num_object_clones":0,"num_object_copies":14800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7400,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5761,"num_write_kb":23044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'7418","reported_seq":7538,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:57.015343+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:27:57.015343+0000","last_peered":"2024-09-17T15:27:57.015343+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:27:57.015343+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":7418,"log_dups_size":0,"ondisk_log_size":7418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30367744,"num_objects":7414,"num_object_clones":0,"num_object_copies":14828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5575,"num_write_kb":22300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":3246}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'7518","reported_seq":7583,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:57.045103+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:27:57.045103+0000","last_peered":"2024-09-17T15:27:57.045103+0000","last_clean":"2024-09-17T15:27:57.045103+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:27:57.045103+0000","last_undegraded":"2024-09-17T15:27:57.045103+0000","last_fullsized":"2024-09-17T15:27:57.045103+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":7518,"log_dups_size":0,"ondisk_log_size":7518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30793728,"num_objects":7518,"num_object_clones":0,"num_object_copies":15036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7518,"num_write_kb":30072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'7308","reported_seq":7432,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:55.715301+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:27:55.715301+0000","last_peered":"2024-09-17T15:27:55.715301+0000","last_clean":"2024-09-17T15:27:55.715301+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:27:55.715301+0000","last_undegraded":"2024-09-17T15:27:55.715301+0000","last_fullsized":"2024-09-17T15:27:55.715301+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":7308,"log_dups_size":0,"ondisk_log_size":7308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29905379,"num_objects":7302,"num_object_clones":0,"num_object_copies":14604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7302,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4290,"num_write_kb":17160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'7629","reported_seq":7759,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:55.720139+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:27:55.720139+0000","last_peered":"2024-09-17T15:27:55.720139+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:27:55.720139+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":7629,"log_dups_size":0,"ondisk_log_size":7629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31228387,"num_objects":7625,"num_object_clones":0,"num_object_copies":15250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7625,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4128,"num_write_kb":16512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3355},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'7462","reported_seq":7596,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:55.668431+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:27:55.668431+0000","last_peered":"2024-09-17T15:27:55.668431+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:27:55.668431+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":7462,"log_dups_size":0,"ondisk_log_size":7462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30540742,"num_objects":7458,"num_object_clones":0,"num_object_copies":14916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7458,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4231,"num_write_kb":16924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3284},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'7397","reported_seq":7520,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:27:55.717283+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:27:55.717283+0000","last_peered":"2024-09-17T15:27:55.717283+0000","last_clean":"2024-09-17T15:27:55.717283+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:27:55.717283+0000","last_undegraded":"2024-09-17T15:27:55.717283+0000","last_fullsized":"2024-09-17T15:27:55.717283+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":7397,"log_dups_size":0,"ondisk_log_size":7397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30289920,"num_objects":7395,"num_object_clones":0,"num_object_copies":14790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4351,"num_write_kb":17404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'7401","reported_seq":13483,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:27:57.063758+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:27:57.063758+0000","last_peered":"2024-09-17T15:27:57.063758+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:27:57.063758+0000","last_undegraded":"2024-09-17T15:27:57.063758+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":7401,"log_dups_size":0,"ondisk_log_size":7401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30286307,"num_objects":7395,"num_object_clones":0,"num_object_copies":14790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1093,"num_objects_unfound":0,"num_objects_dirty":7395,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5401,"num_write_kb":21604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2979,"num_bytes_recovered":12194275,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1093},{"shards":"2","objects":6302}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233600+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233600+0000","last_peered":"2024-09-17T15:26:22.233600+0000","last_clean":"2024-09-17T15:26:22.233600+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233600+0000","last_undegraded":"2024-09-17T15:26:22.233600+0000","last_fullsized":"2024-09-17T15:26:22.233600+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'7299","reported_seq":7424,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:55.720469+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:27:55.720469+0000","last_peered":"2024-09-17T15:27:55.720469+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:27:55.720469+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":7299,"log_dups_size":0,"ondisk_log_size":7299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29863936,"num_objects":7291,"num_object_clones":0,"num_object_copies":14582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7291,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3895,"num_write_kb":15580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3281},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":98,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412941+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412941+0000","last_peered":"2024-09-17T15:26:22.412941+0000","last_clean":"2024-09-17T15:26:22.412941+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412941+0000","last_undegraded":"2024-09-17T15:26:22.412941+0000","last_fullsized":"2024-09-17T15:26:22.412941+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'7366","reported_seq":7482,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:57.042398+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:27:57.042398+0000","last_peered":"2024-09-17T15:27:57.042398+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:27:57.042398+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":7366,"log_dups_size":0,"ondisk_log_size":7366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30151139,"num_objects":7362,"num_object_clones":0,"num_object_copies":14724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7362,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5261,"num_write_kb":21044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":3285}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'7431","reported_seq":7545,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:57.041413+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:27:57.041413+0000","last_peered":"2024-09-17T15:27:57.041413+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:27:57.041413+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":7431,"log_dups_size":0,"ondisk_log_size":7431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30412800,"num_objects":7425,"num_object_clones":0,"num_object_copies":14850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5193,"num_write_kb":20772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":3288}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'7452","reported_seq":7581,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:55.676815+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:27:55.676815+0000","last_peered":"2024-09-17T15:27:55.676815+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:27:55.676815+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":7452,"log_dups_size":0,"ondisk_log_size":7452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30515200,"num_objects":7450,"num_object_clones":0,"num_object_copies":14900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4000,"num_write_kb":16000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3312},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'7379","reported_seq":7516,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:55.725079+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:27:55.725079+0000","last_peered":"2024-09-17T15:27:55.725079+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:27:55.725079+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":7379,"log_dups_size":0,"ondisk_log_size":7379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30191616,"num_objects":7371,"num_object_clones":0,"num_object_copies":14742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3666,"num_write_kb":14664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3214},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229061+0000","last_peered":"2024-09-17T15:26:22.229061+0000","last_clean":"2024-09-17T15:26:22.229061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229061+0000","last_undegraded":"2024-09-17T15:26:22.229061+0000","last_fullsized":"2024-09-17T15:26:22.229061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'7395","reported_seq":7522,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:55.751950+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:27:55.751950+0000","last_peered":"2024-09-17T15:27:55.751950+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:27:55.751950+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":7395,"log_dups_size":0,"ondisk_log_size":7395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30273536,"num_objects":7391,"num_object_clones":0,"num_object_copies":14782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4182,"num_write_kb":16728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3328},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'7370","reported_seq":7508,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:27:55.761314+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:27:55.761314+0000","last_peered":"2024-09-17T15:27:55.761314+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:27:55.761314+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":7370,"log_dups_size":0,"ondisk_log_size":7370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30139334,"num_objects":7360,"num_object_clones":0,"num_object_copies":14720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7360,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3812,"num_write_kb":15248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3251},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":485405406,"num_objects":118516,"num_object_clones":0,"num_object_copies":237032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":1093,"num_objects_unfound":0,"num_objects_dirty":118516,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":78623,"num_write_kb":314492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2979,"num_bytes_recovered":12194275,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":980774912,"data_stored":980623166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118588,"ondisk_log_size":118588,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051619,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":541268,"kb_used_data":369348,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93830572,"statfs":{"total":96636764160,"available":96082505728,"internally_reserved":0,"allocated":378212352,"data_stored":377946179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":176025260},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,0,1,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724564,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":333852,"kb_used_data":167500,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":94037988,"statfs":{"total":96636764160,"available":96294899712,"internally_reserved":0,"allocated":171520000,"data_stored":171206394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":170323627},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247847,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":669232,"kb_used_data":421024,"kb_used_omap":4,"kb_used_meta":248187,"kb_avail":93702608,"statfs":{"total":96636764160,"available":95951470592,"internally_reserved":0,"allocated":431128576,"data_stored":430858326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":254144168},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,5,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":431271936,"data_stored":431235806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":171454464,"data_stored":171374978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":378048512,"data_stored":378012382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:28:01.315 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 142 20 121094 121074 3.32985 2.13281 0.0428863 0.0234548 2024-09-17T15:28:02.315 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 143 20 121639 121619 3.32145 2.12891 0.03558 0.023514 2024-09-17T15:28:03.315 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 144 20 122187 122167 3.31325 2.14062 0.0499113 0.0235723 2024-09-17T15:28:03.369 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:28:03.943 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:28:03.945 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:28:03.974 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":144,"stamp":"2024-09-17T15:28:03.628385+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":495821553,"num_objects":121061,"num_object_clones":0,"num_object_copies":242122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":898,"num_objects_unfound":0,"num_objects_dirty":121061,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":81168,"num_write_kb":324666,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3175,"num_bytes_recovered":12993014,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121134,"ondisk_log_size":121134,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1561584,"kb_used_data":979264,"kb_used_omap":13,"kb_used_meta":582258,"kb_avail":281553936,"statfs":{"total":289910292480,"available":288311230464,"internally_reserved":0,"allocated":1002766336,"data_stored":1001916307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":596233215},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,2,6,18,10,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24924160,"num_objects":6085,"num_object_clones":0,"num_object_copies":12170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-572,"num_objects_unfound":0,"num_objects_dirty":6085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6085,"num_write_kb":24340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":572,"num_bytes_recovered":2342912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002977"},"pg_stats":[{"pgid":"2.f","version":"41'7515","reported_seq":7588,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:00.731154+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:28:00.731154+0000","last_peered":"2024-09-17T15:28:00.731154+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:28:00.731154+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":7515,"log_dups_size":0,"ondisk_log_size":7515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30781440,"num_objects":7515,"num_object_clones":0,"num_object_copies":15030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7515,"num_write_kb":30060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3445},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'7581","reported_seq":7679,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:01.982309+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:28:01.982309+0000","last_peered":"2024-09-17T15:28:01.982309+0000","last_clean":"2024-09-17T15:28:01.982309+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:28:01.982309+0000","last_undegraded":"2024-09-17T15:28:01.982309+0000","last_fullsized":"2024-09-17T15:28:01.982309+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":7581,"log_dups_size":0,"ondisk_log_size":7581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31028166,"num_objects":7577,"num_object_clones":0,"num_object_copies":15154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7577,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5938,"num_write_kb":23752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'7571","reported_seq":7690,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:01.999782+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:28:01.999782+0000","last_peered":"2024-09-17T15:28:01.999782+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:28:01.999782+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":7571,"log_dups_size":0,"ondisk_log_size":7571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30994432,"num_objects":7567,"num_object_clones":0,"num_object_copies":15134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5728,"num_write_kb":22912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'7670","reported_seq":7735,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:02.054863+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:28:02.054863+0000","last_peered":"2024-09-17T15:28:02.054863+0000","last_clean":"2024-09-17T15:28:02.054863+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:28:02.054863+0000","last_undegraded":"2024-09-17T15:28:02.054863+0000","last_fullsized":"2024-09-17T15:28:02.054863+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":7670,"log_dups_size":0,"ondisk_log_size":7670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31416320,"num_objects":7670,"num_object_clones":0,"num_object_copies":15340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7670,"num_write_kb":30680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'7465","reported_seq":7588,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:00.759463+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:28:00.759463+0000","last_peered":"2024-09-17T15:28:00.759463+0000","last_clean":"2024-09-17T15:28:00.759463+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:28:00.759463+0000","last_undegraded":"2024-09-17T15:28:00.759463+0000","last_fullsized":"2024-09-17T15:28:00.759463+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":7465,"log_dups_size":0,"ondisk_log_size":7465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30548451,"num_objects":7459,"num_object_clones":0,"num_object_copies":14918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7459,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4447,"num_write_kb":17788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'7773","reported_seq":7903,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:00.743095+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:28:00.743095+0000","last_peered":"2024-09-17T15:28:00.743095+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:28:00.743095+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":7773,"log_dups_size":0,"ondisk_log_size":7773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31818211,"num_objects":7769,"num_object_clones":0,"num_object_copies":15538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7769,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4272,"num_write_kb":17088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3499},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'7618","reported_seq":7752,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:00.704115+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:28:00.704115+0000","last_peered":"2024-09-17T15:28:00.704115+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:28:00.704115+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":7618,"log_dups_size":0,"ondisk_log_size":7618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31179718,"num_objects":7614,"num_object_clones":0,"num_object_copies":15228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7614,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4387,"num_write_kb":17548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3440},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'7589","reported_seq":7712,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:00.765780+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:28:00.765780+0000","last_peered":"2024-09-17T15:28:00.765780+0000","last_clean":"2024-09-17T15:28:00.765780+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:28:00.765780+0000","last_undegraded":"2024-09-17T15:28:00.765780+0000","last_fullsized":"2024-09-17T15:28:00.765780+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":7589,"log_dups_size":0,"ondisk_log_size":7589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31076352,"num_objects":7587,"num_object_clones":0,"num_object_copies":15174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4543,"num_write_kb":18172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'7562","reported_seq":14034,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:28:02.063012+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:28:02.063012+0000","last_peered":"2024-09-17T15:28:02.063012+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:28:02.063012+0000","last_undegraded":"2024-09-17T15:28:02.063012+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":7562,"log_dups_size":0,"ondisk_log_size":7562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30945763,"num_objects":7556,"num_object_clones":0,"num_object_copies":15112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":898,"num_objects_unfound":0,"num_objects_dirty":7556,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5562,"num_write_kb":22248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3174,"num_bytes_recovered":12992995,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":898},{"shards":"2","objects":6658}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233600+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233600+0000","last_peered":"2024-09-17T15:26:22.233600+0000","last_clean":"2024-09-17T15:26:22.233600+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233600+0000","last_undegraded":"2024-09-17T15:26:22.233600+0000","last_fullsized":"2024-09-17T15:26:22.233600+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'7445","reported_seq":7570,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:00.751311+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:28:00.751311+0000","last_peered":"2024-09-17T15:28:00.751311+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:28:00.751311+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":7445,"log_dups_size":0,"ondisk_log_size":7445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30461952,"num_objects":7437,"num_object_clones":0,"num_object_copies":14874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4041,"num_write_kb":16164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3427},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":98,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412941+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412941+0000","last_peered":"2024-09-17T15:26:22.412941+0000","last_clean":"2024-09-17T15:26:22.412941+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412941+0000","last_undegraded":"2024-09-17T15:26:22.412941+0000","last_fullsized":"2024-09-17T15:26:22.412941+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'7509","reported_seq":7625,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:02.017154+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:28:02.017154+0000","last_peered":"2024-09-17T15:28:02.017154+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:28:02.017154+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":7509,"log_dups_size":0,"ondisk_log_size":7509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30736867,"num_objects":7505,"num_object_clones":0,"num_object_copies":15010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7505,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5404,"num_write_kb":21616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":3428}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'7589","reported_seq":7703,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:02.042729+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:28:02.042729+0000","last_peered":"2024-09-17T15:28:02.042729+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:28:02.042729+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":7589,"log_dups_size":0,"ondisk_log_size":7589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31059968,"num_objects":7583,"num_object_clones":0,"num_object_copies":15166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5351,"num_write_kb":21404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":3446}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'7607","reported_seq":7737,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:00.690597+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:28:00.690597+0000","last_peered":"2024-09-17T15:28:00.690597+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:28:00.690597+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":7607,"log_dups_size":0,"ondisk_log_size":7607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31150080,"num_objects":7605,"num_object_clones":0,"num_object_copies":15210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7605,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4155,"num_write_kb":16620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3467},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'7540","reported_seq":7677,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:00.736740+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:28:00.736740+0000","last_peered":"2024-09-17T15:28:00.736740+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:28:00.736740+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":7540,"log_dups_size":0,"ondisk_log_size":7540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30851072,"num_objects":7532,"num_object_clones":0,"num_object_copies":15064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7532,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3827,"num_write_kb":15308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3375},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229061+0000","last_peered":"2024-09-17T15:26:22.229061+0000","last_clean":"2024-09-17T15:26:22.229061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229061+0000","last_undegraded":"2024-09-17T15:26:22.229061+0000","last_fullsized":"2024-09-17T15:26:22.229061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'7563","reported_seq":7688,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:00.716673+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:28:00.716673+0000","last_peered":"2024-09-17T15:28:00.716673+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:28:00.716673+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":7563,"log_dups_size":0,"ondisk_log_size":7563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30961664,"num_objects":7559,"num_object_clones":0,"num_object_copies":15118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7559,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4350,"num_write_kb":17400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3496},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'7534","reported_seq":7672,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:00.617417+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:28:00.617417+0000","last_peered":"2024-09-17T15:28:00.617417+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:28:00.617417+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":7534,"log_dups_size":0,"ondisk_log_size":7534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30811078,"num_objects":7524,"num_object_clones":0,"num_object_copies":15048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3976,"num_write_kb":15904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3415},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":495821534,"num_objects":121059,"num_object_clones":0,"num_object_copies":242118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":898,"num_objects_unfound":0,"num_objects_dirty":121059,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":81166,"num_write_kb":324664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3174,"num_bytes_recovered":12992995,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1002635264,"data_stored":1002483518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121131,"ondisk_log_size":121131,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051620,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":550628,"kb_used_data":374548,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93821212,"statfs":{"total":96636764160,"available":96072921088,"internally_reserved":0,"allocated":383537152,"data_stored":383270979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":180285100},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,1,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724565,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":329776,"kb_used_data":175904,"kb_used_omap":4,"kb_used_meta":153851,"kb_avail":94042064,"statfs":{"total":96636764160,"available":96299073536,"internally_reserved":0,"allocated":180125696,"data_stored":179812090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":157544107},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,8,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247848,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":681180,"kb_used_data":428812,"kb_used_omap":4,"kb_used_meta":252347,"kb_avail":93690660,"statfs":{"total":96636764160,"available":95939235840,"internally_reserved":0,"allocated":439103488,"data_stored":438833238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":258404008},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,3,5,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":439156736,"data_stored":439120606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":180150272,"data_stored":180070786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":383328256,"data_stored":383292126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:28:03.975 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:28:04.315 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 145 20 122668 122648 3.30336 1.87891 0.0812808 0.023643 2024-09-17T15:28:04.551 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:28:04.551 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:28:04.573 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":144,"stamp":"2024-09-17T15:28:03.628385+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":495821553,"num_objects":121061,"num_object_clones":0,"num_object_copies":242122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":898,"num_objects_unfound":0,"num_objects_dirty":121061,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":81168,"num_write_kb":324666,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3175,"num_bytes_recovered":12993014,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121134,"ondisk_log_size":121134,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1561584,"kb_used_data":979264,"kb_used_omap":13,"kb_used_meta":582258,"kb_avail":281553936,"statfs":{"total":289910292480,"available":288311230464,"internally_reserved":0,"allocated":1002766336,"data_stored":1001916307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":596233215},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,2,6,18,10,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24924160,"num_objects":6085,"num_object_clones":0,"num_object_copies":12170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-572,"num_objects_unfound":0,"num_objects_dirty":6085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6085,"num_write_kb":24340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":572,"num_bytes_recovered":2342912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002977"},"pg_stats":[{"pgid":"2.f","version":"41'7515","reported_seq":7588,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:00.731154+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:28:00.731154+0000","last_peered":"2024-09-17T15:28:00.731154+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:28:00.731154+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":7515,"log_dups_size":0,"ondisk_log_size":7515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30781440,"num_objects":7515,"num_object_clones":0,"num_object_copies":15030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7515,"num_write_kb":30060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3445},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'7581","reported_seq":7679,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:01.982309+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:28:01.982309+0000","last_peered":"2024-09-17T15:28:01.982309+0000","last_clean":"2024-09-17T15:28:01.982309+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:28:01.982309+0000","last_undegraded":"2024-09-17T15:28:01.982309+0000","last_fullsized":"2024-09-17T15:28:01.982309+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":7581,"log_dups_size":0,"ondisk_log_size":7581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31028166,"num_objects":7577,"num_object_clones":0,"num_object_copies":15154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7577,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5938,"num_write_kb":23752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'7571","reported_seq":7690,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:01.999782+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:28:01.999782+0000","last_peered":"2024-09-17T15:28:01.999782+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:28:01.999782+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":7571,"log_dups_size":0,"ondisk_log_size":7571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30994432,"num_objects":7567,"num_object_clones":0,"num_object_copies":15134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5728,"num_write_kb":22912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'7670","reported_seq":7735,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:02.054863+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:28:02.054863+0000","last_peered":"2024-09-17T15:28:02.054863+0000","last_clean":"2024-09-17T15:28:02.054863+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:28:02.054863+0000","last_undegraded":"2024-09-17T15:28:02.054863+0000","last_fullsized":"2024-09-17T15:28:02.054863+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":7670,"log_dups_size":0,"ondisk_log_size":7670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31416320,"num_objects":7670,"num_object_clones":0,"num_object_copies":15340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7670,"num_write_kb":30680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'7465","reported_seq":7588,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:00.759463+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:28:00.759463+0000","last_peered":"2024-09-17T15:28:00.759463+0000","last_clean":"2024-09-17T15:28:00.759463+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:28:00.759463+0000","last_undegraded":"2024-09-17T15:28:00.759463+0000","last_fullsized":"2024-09-17T15:28:00.759463+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":7465,"log_dups_size":0,"ondisk_log_size":7465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30548451,"num_objects":7459,"num_object_clones":0,"num_object_copies":14918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7459,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4447,"num_write_kb":17788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'7773","reported_seq":7903,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:00.743095+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:28:00.743095+0000","last_peered":"2024-09-17T15:28:00.743095+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:28:00.743095+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":7773,"log_dups_size":0,"ondisk_log_size":7773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31818211,"num_objects":7769,"num_object_clones":0,"num_object_copies":15538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7769,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4272,"num_write_kb":17088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3499},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'7618","reported_seq":7752,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:00.704115+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:28:00.704115+0000","last_peered":"2024-09-17T15:28:00.704115+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:28:00.704115+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":7618,"log_dups_size":0,"ondisk_log_size":7618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31179718,"num_objects":7614,"num_object_clones":0,"num_object_copies":15228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7614,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4387,"num_write_kb":17548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3440},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'7589","reported_seq":7712,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:00.765780+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:28:00.765780+0000","last_peered":"2024-09-17T15:28:00.765780+0000","last_clean":"2024-09-17T15:28:00.765780+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:28:00.765780+0000","last_undegraded":"2024-09-17T15:28:00.765780+0000","last_fullsized":"2024-09-17T15:28:00.765780+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":7589,"log_dups_size":0,"ondisk_log_size":7589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31076352,"num_objects":7587,"num_object_clones":0,"num_object_copies":15174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4543,"num_write_kb":18172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'7562","reported_seq":14034,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:28:02.063012+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:28:02.063012+0000","last_peered":"2024-09-17T15:28:02.063012+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:28:02.063012+0000","last_undegraded":"2024-09-17T15:28:02.063012+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":7562,"log_dups_size":0,"ondisk_log_size":7562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30945763,"num_objects":7556,"num_object_clones":0,"num_object_copies":15112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":898,"num_objects_unfound":0,"num_objects_dirty":7556,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5562,"num_write_kb":22248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3174,"num_bytes_recovered":12992995,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":898},{"shards":"2","objects":6658}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233600+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233600+0000","last_peered":"2024-09-17T15:26:22.233600+0000","last_clean":"2024-09-17T15:26:22.233600+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233600+0000","last_undegraded":"2024-09-17T15:26:22.233600+0000","last_fullsized":"2024-09-17T15:26:22.233600+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'7445","reported_seq":7570,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:00.751311+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:28:00.751311+0000","last_peered":"2024-09-17T15:28:00.751311+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:28:00.751311+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":7445,"log_dups_size":0,"ondisk_log_size":7445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30461952,"num_objects":7437,"num_object_clones":0,"num_object_copies":14874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4041,"num_write_kb":16164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3427},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":98,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412941+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412941+0000","last_peered":"2024-09-17T15:26:22.412941+0000","last_clean":"2024-09-17T15:26:22.412941+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412941+0000","last_undegraded":"2024-09-17T15:26:22.412941+0000","last_fullsized":"2024-09-17T15:26:22.412941+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'7509","reported_seq":7625,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:02.017154+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:28:02.017154+0000","last_peered":"2024-09-17T15:28:02.017154+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:28:02.017154+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":7509,"log_dups_size":0,"ondisk_log_size":7509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30736867,"num_objects":7505,"num_object_clones":0,"num_object_copies":15010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7505,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5404,"num_write_kb":21616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":3428}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'7589","reported_seq":7703,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:02.042729+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:28:02.042729+0000","last_peered":"2024-09-17T15:28:02.042729+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:28:02.042729+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":7589,"log_dups_size":0,"ondisk_log_size":7589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31059968,"num_objects":7583,"num_object_clones":0,"num_object_copies":15166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5351,"num_write_kb":21404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":3446}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'7607","reported_seq":7737,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:00.690597+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:28:00.690597+0000","last_peered":"2024-09-17T15:28:00.690597+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:28:00.690597+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":7607,"log_dups_size":0,"ondisk_log_size":7607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31150080,"num_objects":7605,"num_object_clones":0,"num_object_copies":15210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7605,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4155,"num_write_kb":16620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3467},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'7540","reported_seq":7677,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:00.736740+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:28:00.736740+0000","last_peered":"2024-09-17T15:28:00.736740+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:28:00.736740+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":7540,"log_dups_size":0,"ondisk_log_size":7540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30851072,"num_objects":7532,"num_object_clones":0,"num_object_copies":15064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7532,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3827,"num_write_kb":15308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3375},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229061+0000","last_peered":"2024-09-17T15:26:22.229061+0000","last_clean":"2024-09-17T15:26:22.229061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229061+0000","last_undegraded":"2024-09-17T15:26:22.229061+0000","last_fullsized":"2024-09-17T15:26:22.229061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'7563","reported_seq":7688,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:00.716673+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:28:00.716673+0000","last_peered":"2024-09-17T15:28:00.716673+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:28:00.716673+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":7563,"log_dups_size":0,"ondisk_log_size":7563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30961664,"num_objects":7559,"num_object_clones":0,"num_object_copies":15118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7559,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4350,"num_write_kb":17400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3496},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'7534","reported_seq":7672,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:00.617417+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:28:00.617417+0000","last_peered":"2024-09-17T15:28:00.617417+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:28:00.617417+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":7534,"log_dups_size":0,"ondisk_log_size":7534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30811078,"num_objects":7524,"num_object_clones":0,"num_object_copies":15048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3976,"num_write_kb":15904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3415},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":495821534,"num_objects":121059,"num_object_clones":0,"num_object_copies":242118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":898,"num_objects_unfound":0,"num_objects_dirty":121059,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":81166,"num_write_kb":324664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3174,"num_bytes_recovered":12992995,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1002635264,"data_stored":1002483518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121131,"ondisk_log_size":121131,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051620,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":550628,"kb_used_data":374548,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93821212,"statfs":{"total":96636764160,"available":96072921088,"internally_reserved":0,"allocated":383537152,"data_stored":383270979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":180285100},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,1,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724565,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":329776,"kb_used_data":175904,"kb_used_omap":4,"kb_used_meta":153851,"kb_avail":94042064,"statfs":{"total":96636764160,"available":96299073536,"internally_reserved":0,"allocated":180125696,"data_stored":179812090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":157544107},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,8,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247848,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":681180,"kb_used_data":428812,"kb_used_omap":4,"kb_used_meta":252347,"kb_avail":93690660,"statfs":{"total":96636764160,"available":95939235840,"internally_reserved":0,"allocated":439103488,"data_stored":438833238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":258404008},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,3,5,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":439156736,"data_stored":439120606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":180150272,"data_stored":180070786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":383328256,"data_stored":383292126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:28:04.574 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:28:04.574 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'7515", 'reported_seq': 7588, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:00.731154+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:28:00.731154+0000', 'last_peered': '2024-09-17T15:28:00.731154+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:28:00.731154+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 7515, 'log_dups_size': 0, 'ondisk_log_size': 7515, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30781440, 'num_objects': 7515, 'num_object_clones': 0, 'num_object_copies': 15030, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4070, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7515, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7515, 'num_write_kb': 30060, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3445}, {'shards': '0,2', 'objects': 4070}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:04.574 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:28:04.574 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'7571", 'reported_seq': 7690, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:01.999782+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:28:01.999782+0000', 'last_peered': '2024-09-17T15:28:01.999782+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:28:01.999782+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 7571, 'log_dups_size': 0, 'ondisk_log_size': 7571, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30994432, 'num_objects': 7567, 'num_object_clones': 0, 'num_object_copies': 15134, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4169, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7567, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5728, 'num_write_kb': 22912, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4168}, {'shards': '2', 'objects': 3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:28:04.575 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:28:04.575 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'7773", 'reported_seq': 7903, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:00.743095+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:28:00.743095+0000', 'last_peered': '2024-09-17T15:28:00.743095+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:28:00.743095+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 7773, 'log_dups_size': 0, 'ondisk_log_size': 7773, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31818211, 'num_objects': 7769, 'num_object_clones': 0, 'num_object_copies': 15538, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4271, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7769, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4272, 'num_write_kb': 17088, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3499}, {'shards': '0,2', 'objects': 4268}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:04.575 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:28:04.575 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'7618", 'reported_seq': 7752, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:00.704115+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:28:00.704115+0000', 'last_peered': '2024-09-17T15:28:00.704115+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:28:00.704115+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 7618, 'log_dups_size': 0, 'ondisk_log_size': 7618, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31179718, 'num_objects': 7614, 'num_object_clones': 0, 'num_object_copies': 15228, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4174, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7614, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 4387, 'num_write_kb': 17548, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3440}, {'shards': '0,2', 'objects': 4174}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:04.575 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-17T15:28:04.575 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "41'7562", 'reported_seq': 14034, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:28:02.063012+0000', 'last_change': '2024-09-17T15:26:22.588242+0000', 'last_active': '2024-09-17T15:28:02.063012+0000', 'last_peered': '2024-09-17T15:28:02.063012+0000', 'last_clean': '2024-09-17T15:26:20.155646+0000', 'last_became_active': '2024-09-17T15:26:22.557742+0000', 'last_became_peered': '2024-09-17T15:26:22.557742+0000', 'last_unstale': '2024-09-17T15:28:02.063012+0000', 'last_undegraded': '2024-09-17T15:28:02.063012+0000', 'last_fullsized': '2024-09-17T15:26:22.188850+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2016", 'last_scrub_stamp': '2024-09-17T15:25:53.959011+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:53.959011+0000', 'objects_scrubbed': 1994, 'log_size': 7562, 'log_dups_size': 0, 'ondisk_log_size': 7562, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 568, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30945763, 'num_objects': 7556, 'num_object_clones': 0, 'num_object_copies': 15112, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 898, 'num_objects_unfound': 0, 'num_objects_dirty': 7556, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5562, 'num_write_kb': 22248, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3174, 'num_bytes_recovered': 12992995, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 898}, {'shards': '2', 'objects': 6658}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:28:04.575 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:28:04.575 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'7445", 'reported_seq': 7570, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:00.751311+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:28:00.751311+0000', 'last_peered': '2024-09-17T15:28:00.751311+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:28:00.751311+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 7445, 'log_dups_size': 0, 'ondisk_log_size': 7445, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30461952, 'num_objects': 7437, 'num_object_clones': 0, 'num_object_copies': 14874, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4012, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7437, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4041, 'num_write_kb': 16164, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3427}, {'shards': '0,2', 'objects': 4010}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:04.575 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:28:04.575 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'7509", 'reported_seq': 7625, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:02.017154+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:28:02.017154+0000', 'last_peered': '2024-09-17T15:28:02.017154+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:28:02.017154+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 7509, 'log_dups_size': 0, 'ondisk_log_size': 7509, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30736867, 'num_objects': 7505, 'num_object_clones': 0, 'num_object_copies': 15010, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4078, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7505, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5404, 'num_write_kb': 21616, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4077}, {'shards': '2', 'objects': 3428}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:28:04.576 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:28:04.576 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'7589", 'reported_seq': 7703, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:02.042729+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:28:02.042729+0000', 'last_peered': '2024-09-17T15:28:02.042729+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:28:02.042729+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 7589, 'log_dups_size': 0, 'ondisk_log_size': 7589, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31059968, 'num_objects': 7583, 'num_object_clones': 0, 'num_object_copies': 15166, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7583, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5351, 'num_write_kb': 21404, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4137}, {'shards': '2', 'objects': 3446}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:28:04.576 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:28:04.576 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'7607", 'reported_seq': 7737, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:00.690597+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:28:00.690597+0000', 'last_peered': '2024-09-17T15:28:00.690597+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:28:00.690597+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 7607, 'log_dups_size': 0, 'ondisk_log_size': 7607, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31150080, 'num_objects': 7605, 'num_object_clones': 0, 'num_object_copies': 15210, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7605, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4155, 'num_write_kb': 16620, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3467}, {'shards': '0,2', 'objects': 4138}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:04.576 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:28:04.576 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'7540", 'reported_seq': 7677, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:00.736740+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:28:00.736740+0000', 'last_peered': '2024-09-17T15:28:00.736740+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:28:00.736740+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 7540, 'log_dups_size': 0, 'ondisk_log_size': 7540, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30851072, 'num_objects': 7532, 'num_object_clones': 0, 'num_object_copies': 15064, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4160, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7532, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3827, 'num_write_kb': 15308, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3375}, {'shards': '0,2', 'objects': 4157}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:04.576 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:28:04.576 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'7563", 'reported_seq': 7688, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:00.716673+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:28:00.716673+0000', 'last_peered': '2024-09-17T15:28:00.716673+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:28:00.716673+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 7563, 'log_dups_size': 0, 'ondisk_log_size': 7563, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30961664, 'num_objects': 7559, 'num_object_clones': 0, 'num_object_copies': 15118, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4065, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7559, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4350, 'num_write_kb': 17400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3496}, {'shards': '0,2', 'objects': 4063}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:04.576 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:28:04.577 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'7534", 'reported_seq': 7672, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:00.617417+0000', 'last_change': '2024-09-17T15:26:22.654521+0000', 'last_active': '2024-09-17T15:28:00.617417+0000', 'last_peered': '2024-09-17T15:28:00.617417+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:28:00.617417+0000', 'last_undegraded': '2024-09-17T15:26:22.307795+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 7534, 'log_dups_size': 0, 'ondisk_log_size': 7534, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30811078, 'num_objects': 7524, 'num_object_clones': 0, 'num_object_copies': 15048, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4111, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7524, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3976, 'num_write_kb': 15904, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3415}, {'shards': '0,2', 'objects': 4109}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:04.577 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:28:05.276 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:28:05.276 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45388/242122 objects degraded (18.746%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":222,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":121061,"data_bytes":495821553,"bytes_used":1599062016,"bytes_avail":288311230464,"bytes_total":289910292480,"degraded_objects":45388,"degraded_total":242122,"degraded_ratio":0.18745921477602201,"misplaced_objects":898,"misplaced_total":242122,"misplaced_ratio":0.0037088740387077588,"recovering_objects_per_sec":47,"recovering_bytes_per_sec":195194,"recovering_keys_per_sec":0,"num_objects_recovered":572,"num_bytes_recovered":2342912,"num_keys_recovered":0,"write_bytes_sec":2076498,"read_op_per_sec":0,"write_op_per_sec":506},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T15:27:19.616376+0000","services":{}},"progress_events":{}} 2024-09-17T15:28:05.301 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:28:05.301 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:28:05.315 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 146 20 123130 123110 3.2931 1.80469 0.0910077 0.0237143 2024-09-17T15:28:05.857 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:28:05.857 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:28:05.885 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2024-09-17T15:28:05.628927+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":495821553,"num_objects":121061,"num_object_clones":0,"num_object_copies":242122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":898,"num_objects_unfound":0,"num_objects_dirty":121061,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":81168,"num_write_kb":324666,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3175,"num_bytes_recovered":12993014,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121134,"ondisk_log_size":121134,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1574360,"kb_used_data":987880,"kb_used_omap":13,"kb_used_meta":586418,"kb_avail":281541160,"statfs":{"total":289910292480,"available":288298147840,"internally_reserved":0,"allocated":1011589120,"data_stored":1010739091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":600493055},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,5,8,16,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21045248,"num_objects":5138,"num_object_clones":0,"num_object_copies":10276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-385,"num_objects_unfound":0,"num_objects_dirty":5138,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5138,"num_write_kb":20552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":385,"num_bytes_recovered":1576960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002994"},"pg_stats":[{"pgid":"2.f","version":"41'7515","reported_seq":7588,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:00.731154+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:28:00.731154+0000","last_peered":"2024-09-17T15:28:00.731154+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:28:00.731154+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":7515,"log_dups_size":0,"ondisk_log_size":7515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30781440,"num_objects":7515,"num_object_clones":0,"num_object_copies":15030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7515,"num_write_kb":30060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3445},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'7581","reported_seq":7679,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:01.982309+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:28:01.982309+0000","last_peered":"2024-09-17T15:28:01.982309+0000","last_clean":"2024-09-17T15:28:01.982309+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:28:01.982309+0000","last_undegraded":"2024-09-17T15:28:01.982309+0000","last_fullsized":"2024-09-17T15:28:01.982309+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":7581,"log_dups_size":0,"ondisk_log_size":7581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31028166,"num_objects":7577,"num_object_clones":0,"num_object_copies":15154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7577,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5938,"num_write_kb":23752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'7571","reported_seq":7690,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:01.999782+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:28:01.999782+0000","last_peered":"2024-09-17T15:28:01.999782+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:28:01.999782+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":7571,"log_dups_size":0,"ondisk_log_size":7571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30994432,"num_objects":7567,"num_object_clones":0,"num_object_copies":15134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5728,"num_write_kb":22912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'7670","reported_seq":7735,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:02.054863+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:28:02.054863+0000","last_peered":"2024-09-17T15:28:02.054863+0000","last_clean":"2024-09-17T15:28:02.054863+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:28:02.054863+0000","last_undegraded":"2024-09-17T15:28:02.054863+0000","last_fullsized":"2024-09-17T15:28:02.054863+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":7670,"log_dups_size":0,"ondisk_log_size":7670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31416320,"num_objects":7670,"num_object_clones":0,"num_object_copies":15340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7670,"num_write_kb":30680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'7465","reported_seq":7588,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:00.759463+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:28:00.759463+0000","last_peered":"2024-09-17T15:28:00.759463+0000","last_clean":"2024-09-17T15:28:00.759463+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:28:00.759463+0000","last_undegraded":"2024-09-17T15:28:00.759463+0000","last_fullsized":"2024-09-17T15:28:00.759463+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":7465,"log_dups_size":0,"ondisk_log_size":7465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30548451,"num_objects":7459,"num_object_clones":0,"num_object_copies":14918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7459,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4447,"num_write_kb":17788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'7773","reported_seq":7903,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:00.743095+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:28:00.743095+0000","last_peered":"2024-09-17T15:28:00.743095+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:28:00.743095+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":7773,"log_dups_size":0,"ondisk_log_size":7773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31818211,"num_objects":7769,"num_object_clones":0,"num_object_copies":15538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7769,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4272,"num_write_kb":17088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3499},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'7618","reported_seq":7752,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:00.704115+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:28:00.704115+0000","last_peered":"2024-09-17T15:28:00.704115+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:28:00.704115+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":7618,"log_dups_size":0,"ondisk_log_size":7618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31179718,"num_objects":7614,"num_object_clones":0,"num_object_copies":15228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7614,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4387,"num_write_kb":17548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3440},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'7589","reported_seq":7712,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:00.765780+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:28:00.765780+0000","last_peered":"2024-09-17T15:28:00.765780+0000","last_clean":"2024-09-17T15:28:00.765780+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:28:00.765780+0000","last_undegraded":"2024-09-17T15:28:00.765780+0000","last_fullsized":"2024-09-17T15:28:00.765780+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":7589,"log_dups_size":0,"ondisk_log_size":7589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31076352,"num_objects":7587,"num_object_clones":0,"num_object_copies":15174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4543,"num_write_kb":18172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'7562","reported_seq":14034,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:28:02.063012+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:28:02.063012+0000","last_peered":"2024-09-17T15:28:02.063012+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:28:02.063012+0000","last_undegraded":"2024-09-17T15:28:02.063012+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":7562,"log_dups_size":0,"ondisk_log_size":7562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30945763,"num_objects":7556,"num_object_clones":0,"num_object_copies":15112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":898,"num_objects_unfound":0,"num_objects_dirty":7556,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5562,"num_write_kb":22248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3174,"num_bytes_recovered":12992995,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":898},{"shards":"2","objects":6658}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233600+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233600+0000","last_peered":"2024-09-17T15:26:22.233600+0000","last_clean":"2024-09-17T15:26:22.233600+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233600+0000","last_undegraded":"2024-09-17T15:26:22.233600+0000","last_fullsized":"2024-09-17T15:26:22.233600+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'7445","reported_seq":7570,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:00.751311+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:28:00.751311+0000","last_peered":"2024-09-17T15:28:00.751311+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:28:00.751311+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":7445,"log_dups_size":0,"ondisk_log_size":7445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30461952,"num_objects":7437,"num_object_clones":0,"num_object_copies":14874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4041,"num_write_kb":16164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3427},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":98,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412941+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412941+0000","last_peered":"2024-09-17T15:26:22.412941+0000","last_clean":"2024-09-17T15:26:22.412941+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412941+0000","last_undegraded":"2024-09-17T15:26:22.412941+0000","last_fullsized":"2024-09-17T15:26:22.412941+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'7509","reported_seq":7625,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:02.017154+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:28:02.017154+0000","last_peered":"2024-09-17T15:28:02.017154+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:28:02.017154+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":7509,"log_dups_size":0,"ondisk_log_size":7509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30736867,"num_objects":7505,"num_object_clones":0,"num_object_copies":15010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7505,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5404,"num_write_kb":21616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":3428}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'7589","reported_seq":7703,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:02.042729+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:28:02.042729+0000","last_peered":"2024-09-17T15:28:02.042729+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:28:02.042729+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":7589,"log_dups_size":0,"ondisk_log_size":7589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31059968,"num_objects":7583,"num_object_clones":0,"num_object_copies":15166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5351,"num_write_kb":21404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":3446}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'7607","reported_seq":7737,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:00.690597+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:28:00.690597+0000","last_peered":"2024-09-17T15:28:00.690597+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:28:00.690597+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":7607,"log_dups_size":0,"ondisk_log_size":7607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31150080,"num_objects":7605,"num_object_clones":0,"num_object_copies":15210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7605,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4155,"num_write_kb":16620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3467},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'7540","reported_seq":7677,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:00.736740+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:28:00.736740+0000","last_peered":"2024-09-17T15:28:00.736740+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:28:00.736740+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":7540,"log_dups_size":0,"ondisk_log_size":7540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30851072,"num_objects":7532,"num_object_clones":0,"num_object_copies":15064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7532,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3827,"num_write_kb":15308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3375},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229061+0000","last_peered":"2024-09-17T15:26:22.229061+0000","last_clean":"2024-09-17T15:26:22.229061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229061+0000","last_undegraded":"2024-09-17T15:26:22.229061+0000","last_fullsized":"2024-09-17T15:26:22.229061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'7563","reported_seq":7688,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:00.716673+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:28:00.716673+0000","last_peered":"2024-09-17T15:28:00.716673+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:28:00.716673+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":7563,"log_dups_size":0,"ondisk_log_size":7563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30961664,"num_objects":7559,"num_object_clones":0,"num_object_copies":15118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7559,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4350,"num_write_kb":17400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3496},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'7534","reported_seq":7672,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:00.617417+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:28:00.617417+0000","last_peered":"2024-09-17T15:28:00.617417+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:28:00.617417+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":7534,"log_dups_size":0,"ondisk_log_size":7534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30811078,"num_objects":7524,"num_object_clones":0,"num_object_copies":15048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3976,"num_write_kb":15904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3415},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":495821534,"num_objects":121059,"num_object_clones":0,"num_object_copies":242118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":898,"num_objects_unfound":0,"num_objects_dirty":121059,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":81166,"num_write_kb":324664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3174,"num_bytes_recovered":12992995,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1011363840,"data_stored":1011212094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121131,"ondisk_log_size":121131,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051620,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":550628,"kb_used_data":374548,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93821212,"statfs":{"total":96636764160,"available":96072921088,"internally_reserved":0,"allocated":383537152,"data_stored":383270979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":180285100},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,1,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724566,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":342552,"kb_used_data":184520,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":94029288,"statfs":{"total":96636764160,"available":96285990912,"internally_reserved":0,"allocated":188948480,"data_stored":188634874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":161803947},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,4,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247848,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":681180,"kb_used_data":428812,"kb_used_omap":4,"kb_used_meta":252347,"kb_avail":93690660,"statfs":{"total":96636764160,"available":95939235840,"internally_reserved":0,"allocated":439103488,"data_stored":438833238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":258404008},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,3,5,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":439156736,"data_stored":439120606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":188878848,"data_stored":188799362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":383328256,"data_stored":383292126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:28:06.315 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 147 20 123624 123604 3.28382 1.92969 0.0423754 0.0237841 2024-09-17T15:28:07.316 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 148 20 124147 124127 3.27544 2.04297 0.0344948 0.0238446 2024-09-17T15:28:08.317 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 149 20 124673 124653 3.26724 2.05469 0.0267528 0.0239033 2024-09-17T15:28:08.887 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:28:09.316 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 150 20 125182 125162 3.25872 1.98828 0.0358096 0.0239665 2024-09-17T15:28:09.463 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:28:09.464 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:28:09.493 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":146,"stamp":"2024-09-17T15:28:07.629404+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":506286833,"num_objects":123616,"num_object_clones":0,"num_object_copies":247232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":706,"num_objects_unfound":0,"num_objects_dirty":123616,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":83723,"num_write_kb":334886,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":13779446,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":123689,"ondisk_log_size":123689,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1613776,"kb_used_data":1000928,"kb_used_omap":13,"kb_used_meta":612786,"kb_avail":281501744,"statfs":{"total":289910292480,"available":288257785856,"internally_reserved":0,"allocated":1024950272,"data_stored":1024100243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":627493887},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,12,13,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":31510528,"num_objects":7693,"num_object_clones":0,"num_object_copies":15386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-577,"num_objects_unfound":0,"num_objects_dirty":7693,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7693,"num_write_kb":30772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":577,"num_bytes_recovered":2363392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002958"},"pg_stats":[{"pgid":"2.f","version":"41'7672","reported_seq":7745,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:05.766623+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:28:05.766623+0000","last_peered":"2024-09-17T15:28:05.766623+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:28:05.766623+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":7672,"log_dups_size":0,"ondisk_log_size":7672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31424512,"num_objects":7672,"num_object_clones":0,"num_object_copies":15344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7672,"num_write_kb":30688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3602},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'7759","reported_seq":7857,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:07.043441+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:28:07.043441+0000","last_peered":"2024-09-17T15:28:07.043441+0000","last_clean":"2024-09-17T15:28:07.043441+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:28:07.043441+0000","last_undegraded":"2024-09-17T15:28:07.043441+0000","last_fullsized":"2024-09-17T15:28:07.043441+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":7759,"log_dups_size":0,"ondisk_log_size":7759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31757254,"num_objects":7755,"num_object_clones":0,"num_object_copies":15510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7755,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6116,"num_write_kb":24464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'7732","reported_seq":7852,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:07.074791+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:28:07.074791+0000","last_peered":"2024-09-17T15:28:07.074791+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:28:07.074791+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":7732,"log_dups_size":0,"ondisk_log_size":7732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31653888,"num_objects":7728,"num_object_clones":0,"num_object_copies":15456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7728,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5889,"num_write_kb":23556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":3560}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'7811","reported_seq":7876,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:07.001506+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:28:07.001506+0000","last_peered":"2024-09-17T15:28:07.001506+0000","last_clean":"2024-09-17T15:28:07.001506+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:28:07.001506+0000","last_undegraded":"2024-09-17T15:28:07.001506+0000","last_fullsized":"2024-09-17T15:28:07.001506+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":7811,"log_dups_size":0,"ondisk_log_size":7811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31993856,"num_objects":7811,"num_object_clones":0,"num_object_copies":15622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7811,"num_write_kb":31244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'7609","reported_seq":7733,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:05.694586+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:28:05.694586+0000","last_peered":"2024-09-17T15:28:05.694586+0000","last_clean":"2024-09-17T15:28:05.694586+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:28:05.694586+0000","last_undegraded":"2024-09-17T15:28:05.694586+0000","last_fullsized":"2024-09-17T15:28:05.694586+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":7609,"log_dups_size":0,"ondisk_log_size":7609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31138275,"num_objects":7603,"num_object_clones":0,"num_object_copies":15206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7603,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4591,"num_write_kb":18364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'7923","reported_seq":8053,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:05.790641+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:28:05.790641+0000","last_peered":"2024-09-17T15:28:05.790641+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:28:05.790641+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":7923,"log_dups_size":0,"ondisk_log_size":7923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32432611,"num_objects":7919,"num_object_clones":0,"num_object_copies":15838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7919,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4422,"num_write_kb":17688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3649},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'7761","reported_seq":7895,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:05.784883+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:28:05.784883+0000","last_peered":"2024-09-17T15:28:05.784883+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:28:05.784883+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":7761,"log_dups_size":0,"ondisk_log_size":7761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31765446,"num_objects":7757,"num_object_clones":0,"num_object_copies":15514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7757,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4530,"num_write_kb":18120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3583},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'7759","reported_seq":7882,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:05.684883+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:28:05.684883+0000","last_peered":"2024-09-17T15:28:05.684883+0000","last_clean":"2024-09-17T15:28:05.684883+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:28:05.684883+0000","last_undegraded":"2024-09-17T15:28:05.684883+0000","last_fullsized":"2024-09-17T15:28:05.684883+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":7759,"log_dups_size":0,"ondisk_log_size":7759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31772672,"num_objects":7757,"num_object_clones":0,"num_object_copies":15514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4713,"num_write_kb":18852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'7720","reported_seq":14576,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:28:07.066535+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:28:07.066535+0000","last_peered":"2024-09-17T15:28:07.066535+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:28:07.066535+0000","last_undegraded":"2024-09-17T15:28:07.066535+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":7720,"log_dups_size":0,"ondisk_log_size":7720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31592931,"num_objects":7714,"num_object_clones":0,"num_object_copies":15428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":706,"num_objects_unfound":0,"num_objects_dirty":7714,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5720,"num_write_kb":22880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3366,"num_bytes_recovered":13779427,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":706},{"shards":"2","objects":7008}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233600+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233600+0000","last_peered":"2024-09-17T15:26:22.233600+0000","last_clean":"2024-09-17T15:26:22.233600+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233600+0000","last_undegraded":"2024-09-17T15:26:22.233600+0000","last_fullsized":"2024-09-17T15:26:22.233600+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'7612","reported_seq":7737,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:05.797806+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:28:05.797806+0000","last_peered":"2024-09-17T15:28:05.797806+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:28:05.797806+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":7612,"log_dups_size":0,"ondisk_log_size":7612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31145984,"num_objects":7604,"num_object_clones":0,"num_object_copies":15208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4208,"num_write_kb":16832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3594},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":98,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412941+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412941+0000","last_peered":"2024-09-17T15:26:22.412941+0000","last_clean":"2024-09-17T15:26:22.412941+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412941+0000","last_undegraded":"2024-09-17T15:26:22.412941+0000","last_fullsized":"2024-09-17T15:26:22.412941+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'7661","reported_seq":7771,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:07.070499+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:28:07.070499+0000","last_peered":"2024-09-17T15:28:07.070499+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:28:07.070499+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":7661,"log_dups_size":0,"ondisk_log_size":7661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31359459,"num_objects":7657,"num_object_clones":0,"num_object_copies":15314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7657,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5556,"num_write_kb":22224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":3580}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'7763","reported_seq":7877,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:07.083662+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:28:07.083662+0000","last_peered":"2024-09-17T15:28:07.083662+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:28:07.083662+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":7763,"log_dups_size":0,"ondisk_log_size":7763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31772672,"num_objects":7757,"num_object_clones":0,"num_object_copies":15514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5525,"num_write_kb":22100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":3620}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'7784","reported_seq":7914,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:05.779973+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:28:05.779973+0000","last_peered":"2024-09-17T15:28:05.779973+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:28:05.779973+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":7784,"log_dups_size":0,"ondisk_log_size":7784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31875072,"num_objects":7782,"num_object_clones":0,"num_object_copies":15564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4332,"num_write_kb":17328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3644},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'7690","reported_seq":7826,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:05.837007+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:28:05.837007+0000","last_peered":"2024-09-17T15:28:05.837007+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:28:05.837007+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":7690,"log_dups_size":0,"ondisk_log_size":7690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31465472,"num_objects":7682,"num_object_clones":0,"num_object_copies":15364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3977,"num_write_kb":15908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3525},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229061+0000","last_peered":"2024-09-17T15:26:22.229061+0000","last_clean":"2024-09-17T15:26:22.229061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229061+0000","last_undegraded":"2024-09-17T15:26:22.229061+0000","last_fullsized":"2024-09-17T15:26:22.229061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'7735","reported_seq":7862,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:05.820387+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:28:05.820387+0000","last_peered":"2024-09-17T15:28:05.820387+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:28:05.820387+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":7735,"log_dups_size":0,"ondisk_log_size":7735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31666176,"num_objects":7731,"num_object_clones":0,"num_object_copies":15462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4522,"num_write_kb":18088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3668},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'7695","reported_seq":7833,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:05.732000+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:28:05.732000+0000","last_peered":"2024-09-17T15:28:05.732000+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:28:05.732000+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":7695,"log_dups_size":0,"ondisk_log_size":7695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31470534,"num_objects":7685,"num_object_clones":0,"num_object_copies":15370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7685,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4137,"num_write_kb":16548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3576},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":506286814,"num_objects":123614,"num_object_clones":0,"num_object_copies":247228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":706,"num_objects_unfound":0,"num_objects_dirty":123614,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":83721,"num_write_kb":334884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3366,"num_bytes_recovered":13779427,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1024446464,"data_stored":1024294718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":123686,"ondisk_log_size":123686,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051621,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":577948,"kb_used_data":379660,"kb_used_omap":4,"kb_used_meta":198267,"kb_avail":93793892,"statfs":{"total":96636764160,"available":96044945408,"internally_reserved":0,"allocated":388771840,"data_stored":388505667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":203026092},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724566,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":342552,"kb_used_data":184520,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":94029288,"statfs":{"total":96636764160,"available":96285990912,"internally_reserved":0,"allocated":188948480,"data_stored":188634874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":161803947},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,4,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247849,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":693276,"kb_used_data":436748,"kb_used_omap":4,"kb_used_meta":256507,"kb_avail":93678564,"statfs":{"total":96636764160,"available":95926849536,"internally_reserved":0,"allocated":447229952,"data_stored":446959702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":262663848},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,4,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":447057920,"data_stored":447021790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":188878848,"data_stored":188799362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":388509696,"data_stored":388473566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:28:09.495 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:28:10.039 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:28:10.041 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:28:10.064 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2024-09-17T15:28:09.629917+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":506286833,"num_objects":123616,"num_object_clones":0,"num_object_copies":247232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":706,"num_objects_unfound":0,"num_objects_dirty":123616,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":83723,"num_write_kb":334886,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":13779446,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":123689,"ondisk_log_size":123689,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1613776,"kb_used_data":1000928,"kb_used_omap":13,"kb_used_meta":612786,"kb_avail":281501744,"statfs":{"total":289910292480,"available":288257785856,"internally_reserved":0,"allocated":1024950272,"data_stored":1024100243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":627493887},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,12,13,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20881408,"num_objects":5098,"num_object_clones":0,"num_object_copies":10196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-387,"num_objects_unfound":0,"num_objects_dirty":5098,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5098,"num_write_kb":20392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":387,"num_bytes_recovered":1585152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003001"},"pg_stats":[{"pgid":"2.f","version":"41'7672","reported_seq":7745,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:05.766623+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:28:05.766623+0000","last_peered":"2024-09-17T15:28:05.766623+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:28:05.766623+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":7672,"log_dups_size":0,"ondisk_log_size":7672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31424512,"num_objects":7672,"num_object_clones":0,"num_object_copies":15344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7672,"num_write_kb":30688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3602},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'7759","reported_seq":7857,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:07.043441+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:28:07.043441+0000","last_peered":"2024-09-17T15:28:07.043441+0000","last_clean":"2024-09-17T15:28:07.043441+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:28:07.043441+0000","last_undegraded":"2024-09-17T15:28:07.043441+0000","last_fullsized":"2024-09-17T15:28:07.043441+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":7759,"log_dups_size":0,"ondisk_log_size":7759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31757254,"num_objects":7755,"num_object_clones":0,"num_object_copies":15510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7755,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6116,"num_write_kb":24464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'7732","reported_seq":7852,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:07.074791+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:28:07.074791+0000","last_peered":"2024-09-17T15:28:07.074791+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:28:07.074791+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":7732,"log_dups_size":0,"ondisk_log_size":7732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31653888,"num_objects":7728,"num_object_clones":0,"num_object_copies":15456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7728,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5889,"num_write_kb":23556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":3560}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'7811","reported_seq":7876,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:07.001506+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:28:07.001506+0000","last_peered":"2024-09-17T15:28:07.001506+0000","last_clean":"2024-09-17T15:28:07.001506+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:28:07.001506+0000","last_undegraded":"2024-09-17T15:28:07.001506+0000","last_fullsized":"2024-09-17T15:28:07.001506+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":7811,"log_dups_size":0,"ondisk_log_size":7811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31993856,"num_objects":7811,"num_object_clones":0,"num_object_copies":15622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7811,"num_write_kb":31244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'7609","reported_seq":7733,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:05.694586+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:28:05.694586+0000","last_peered":"2024-09-17T15:28:05.694586+0000","last_clean":"2024-09-17T15:28:05.694586+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:28:05.694586+0000","last_undegraded":"2024-09-17T15:28:05.694586+0000","last_fullsized":"2024-09-17T15:28:05.694586+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":7609,"log_dups_size":0,"ondisk_log_size":7609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31138275,"num_objects":7603,"num_object_clones":0,"num_object_copies":15206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7603,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4591,"num_write_kb":18364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'7923","reported_seq":8053,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:05.790641+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:28:05.790641+0000","last_peered":"2024-09-17T15:28:05.790641+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:28:05.790641+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":7923,"log_dups_size":0,"ondisk_log_size":7923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32432611,"num_objects":7919,"num_object_clones":0,"num_object_copies":15838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7919,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4422,"num_write_kb":17688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3649},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'7761","reported_seq":7895,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:05.784883+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:28:05.784883+0000","last_peered":"2024-09-17T15:28:05.784883+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:28:05.784883+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":7761,"log_dups_size":0,"ondisk_log_size":7761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31765446,"num_objects":7757,"num_object_clones":0,"num_object_copies":15514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7757,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4530,"num_write_kb":18120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3583},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'7759","reported_seq":7882,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:05.684883+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:28:05.684883+0000","last_peered":"2024-09-17T15:28:05.684883+0000","last_clean":"2024-09-17T15:28:05.684883+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:28:05.684883+0000","last_undegraded":"2024-09-17T15:28:05.684883+0000","last_fullsized":"2024-09-17T15:28:05.684883+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":7759,"log_dups_size":0,"ondisk_log_size":7759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31772672,"num_objects":7757,"num_object_clones":0,"num_object_copies":15514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4713,"num_write_kb":18852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'7720","reported_seq":14576,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:28:07.066535+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:28:07.066535+0000","last_peered":"2024-09-17T15:28:07.066535+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:28:07.066535+0000","last_undegraded":"2024-09-17T15:28:07.066535+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":7720,"log_dups_size":0,"ondisk_log_size":7720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31592931,"num_objects":7714,"num_object_clones":0,"num_object_copies":15428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":706,"num_objects_unfound":0,"num_objects_dirty":7714,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5720,"num_write_kb":22880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3366,"num_bytes_recovered":13779427,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":706},{"shards":"2","objects":7008}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233600+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233600+0000","last_peered":"2024-09-17T15:26:22.233600+0000","last_clean":"2024-09-17T15:26:22.233600+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233600+0000","last_undegraded":"2024-09-17T15:26:22.233600+0000","last_fullsized":"2024-09-17T15:26:22.233600+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'7612","reported_seq":7737,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:05.797806+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:28:05.797806+0000","last_peered":"2024-09-17T15:28:05.797806+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:28:05.797806+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":7612,"log_dups_size":0,"ondisk_log_size":7612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31145984,"num_objects":7604,"num_object_clones":0,"num_object_copies":15208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4208,"num_write_kb":16832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3594},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":98,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412941+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412941+0000","last_peered":"2024-09-17T15:26:22.412941+0000","last_clean":"2024-09-17T15:26:22.412941+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412941+0000","last_undegraded":"2024-09-17T15:26:22.412941+0000","last_fullsized":"2024-09-17T15:26:22.412941+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'7661","reported_seq":7771,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:07.070499+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:28:07.070499+0000","last_peered":"2024-09-17T15:28:07.070499+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:28:07.070499+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":7661,"log_dups_size":0,"ondisk_log_size":7661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31359459,"num_objects":7657,"num_object_clones":0,"num_object_copies":15314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7657,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5556,"num_write_kb":22224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":3580}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'7763","reported_seq":7877,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:07.083662+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:28:07.083662+0000","last_peered":"2024-09-17T15:28:07.083662+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:28:07.083662+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":7763,"log_dups_size":0,"ondisk_log_size":7763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31772672,"num_objects":7757,"num_object_clones":0,"num_object_copies":15514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5525,"num_write_kb":22100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":3620}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'7784","reported_seq":7914,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:05.779973+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:28:05.779973+0000","last_peered":"2024-09-17T15:28:05.779973+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:28:05.779973+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":7784,"log_dups_size":0,"ondisk_log_size":7784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31875072,"num_objects":7782,"num_object_clones":0,"num_object_copies":15564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4332,"num_write_kb":17328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3644},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'7690","reported_seq":7826,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:05.837007+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:28:05.837007+0000","last_peered":"2024-09-17T15:28:05.837007+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:28:05.837007+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":7690,"log_dups_size":0,"ondisk_log_size":7690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31465472,"num_objects":7682,"num_object_clones":0,"num_object_copies":15364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3977,"num_write_kb":15908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3525},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229061+0000","last_peered":"2024-09-17T15:26:22.229061+0000","last_clean":"2024-09-17T15:26:22.229061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229061+0000","last_undegraded":"2024-09-17T15:26:22.229061+0000","last_fullsized":"2024-09-17T15:26:22.229061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'7735","reported_seq":7862,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:05.820387+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:28:05.820387+0000","last_peered":"2024-09-17T15:28:05.820387+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:28:05.820387+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":7735,"log_dups_size":0,"ondisk_log_size":7735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31666176,"num_objects":7731,"num_object_clones":0,"num_object_copies":15462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4522,"num_write_kb":18088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3668},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'7695","reported_seq":7833,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:05.732000+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:28:05.732000+0000","last_peered":"2024-09-17T15:28:05.732000+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:28:05.732000+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":7695,"log_dups_size":0,"ondisk_log_size":7695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31470534,"num_objects":7685,"num_object_clones":0,"num_object_copies":15370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7685,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4137,"num_write_kb":16548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3576},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":506286814,"num_objects":123614,"num_object_clones":0,"num_object_copies":247228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":706,"num_objects_unfound":0,"num_objects_dirty":123614,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":83721,"num_write_kb":334884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3366,"num_bytes_recovered":13779427,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1024446464,"data_stored":1024294718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":123686,"ondisk_log_size":123686,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051621,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":577948,"kb_used_data":379660,"kb_used_omap":4,"kb_used_meta":198267,"kb_avail":93793892,"statfs":{"total":96636764160,"available":96044945408,"internally_reserved":0,"allocated":388771840,"data_stored":388505667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":203026092},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724566,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":342552,"kb_used_data":184520,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":94029288,"statfs":{"total":96636764160,"available":96285990912,"internally_reserved":0,"allocated":188948480,"data_stored":188634874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":161803947},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,4,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247849,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":693276,"kb_used_data":436748,"kb_used_omap":4,"kb_used_meta":256507,"kb_avail":93678564,"statfs":{"total":96636764160,"available":95926849536,"internally_reserved":0,"allocated":447229952,"data_stored":446959702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":262663848},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,4,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":447057920,"data_stored":447021790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":188878848,"data_stored":188799362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":388509696,"data_stored":388473566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:28:10.065 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:28:10.065 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'7672", 'reported_seq': 7745, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:05.766623+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:28:05.766623+0000', 'last_peered': '2024-09-17T15:28:05.766623+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:28:05.766623+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 7672, 'log_dups_size': 0, 'ondisk_log_size': 7672, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31424512, 'num_objects': 7672, 'num_object_clones': 0, 'num_object_copies': 15344, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4070, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7672, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7672, 'num_write_kb': 30688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3602}, {'shards': '0,2', 'objects': 4070}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:10.065 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:28:10.065 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'7732", 'reported_seq': 7852, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:07.074791+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:28:07.074791+0000', 'last_peered': '2024-09-17T15:28:07.074791+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:28:07.074791+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 7732, 'log_dups_size': 0, 'ondisk_log_size': 7732, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31653888, 'num_objects': 7728, 'num_object_clones': 0, 'num_object_copies': 15456, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4169, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7728, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5889, 'num_write_kb': 23556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4168}, {'shards': '2', 'objects': 3560}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:28:10.066 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:28:10.066 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'7923", 'reported_seq': 8053, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:05.790641+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:28:05.790641+0000', 'last_peered': '2024-09-17T15:28:05.790641+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:28:05.790641+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 7923, 'log_dups_size': 0, 'ondisk_log_size': 7923, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32432611, 'num_objects': 7919, 'num_object_clones': 0, 'num_object_copies': 15838, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4271, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7919, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4422, 'num_write_kb': 17688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3649}, {'shards': '0,2', 'objects': 4268}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:10.066 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:28:10.066 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'7761", 'reported_seq': 7895, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:05.784883+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:28:05.784883+0000', 'last_peered': '2024-09-17T15:28:05.784883+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:28:05.784883+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 7761, 'log_dups_size': 0, 'ondisk_log_size': 7761, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31765446, 'num_objects': 7757, 'num_object_clones': 0, 'num_object_copies': 15514, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4174, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7757, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 4530, 'num_write_kb': 18120, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3583}, {'shards': '0,2', 'objects': 4174}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:10.066 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-17T15:28:10.066 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "41'7720", 'reported_seq': 14576, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:28:07.066535+0000', 'last_change': '2024-09-17T15:26:22.588242+0000', 'last_active': '2024-09-17T15:28:07.066535+0000', 'last_peered': '2024-09-17T15:28:07.066535+0000', 'last_clean': '2024-09-17T15:26:20.155646+0000', 'last_became_active': '2024-09-17T15:26:22.557742+0000', 'last_became_peered': '2024-09-17T15:26:22.557742+0000', 'last_unstale': '2024-09-17T15:28:07.066535+0000', 'last_undegraded': '2024-09-17T15:28:07.066535+0000', 'last_fullsized': '2024-09-17T15:26:22.188850+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2016", 'last_scrub_stamp': '2024-09-17T15:25:53.959011+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:53.959011+0000', 'objects_scrubbed': 1994, 'log_size': 7720, 'log_dups_size': 0, 'ondisk_log_size': 7720, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 568, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31592931, 'num_objects': 7714, 'num_object_clones': 0, 'num_object_copies': 15428, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 706, 'num_objects_unfound': 0, 'num_objects_dirty': 7714, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5720, 'num_write_kb': 22880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3366, 'num_bytes_recovered': 13779427, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 706}, {'shards': '2', 'objects': 7008}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:28:10.066 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:28:10.066 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'7612", 'reported_seq': 7737, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:05.797806+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:28:05.797806+0000', 'last_peered': '2024-09-17T15:28:05.797806+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:28:05.797806+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 7612, 'log_dups_size': 0, 'ondisk_log_size': 7612, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31145984, 'num_objects': 7604, 'num_object_clones': 0, 'num_object_copies': 15208, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4012, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7604, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4208, 'num_write_kb': 16832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3594}, {'shards': '0,2', 'objects': 4010}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:10.067 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:28:10.067 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'7661", 'reported_seq': 7771, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:07.070499+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:28:07.070499+0000', 'last_peered': '2024-09-17T15:28:07.070499+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:28:07.070499+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 7661, 'log_dups_size': 0, 'ondisk_log_size': 7661, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31359459, 'num_objects': 7657, 'num_object_clones': 0, 'num_object_copies': 15314, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4078, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7657, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5556, 'num_write_kb': 22224, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4077}, {'shards': '2', 'objects': 3580}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:28:10.067 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:28:10.067 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'7763", 'reported_seq': 7877, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:07.083662+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:28:07.083662+0000', 'last_peered': '2024-09-17T15:28:07.083662+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:28:07.083662+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 7763, 'log_dups_size': 0, 'ondisk_log_size': 7763, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31772672, 'num_objects': 7757, 'num_object_clones': 0, 'num_object_copies': 15514, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7757, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5525, 'num_write_kb': 22100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4137}, {'shards': '2', 'objects': 3620}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:28:10.067 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:28:10.067 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'7784", 'reported_seq': 7914, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:05.779973+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:28:05.779973+0000', 'last_peered': '2024-09-17T15:28:05.779973+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:28:05.779973+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 7784, 'log_dups_size': 0, 'ondisk_log_size': 7784, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31875072, 'num_objects': 7782, 'num_object_clones': 0, 'num_object_copies': 15564, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7782, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4332, 'num_write_kb': 17328, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3644}, {'shards': '0,2', 'objects': 4138}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:10.067 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:28:10.067 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'7690", 'reported_seq': 7826, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:05.837007+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:28:05.837007+0000', 'last_peered': '2024-09-17T15:28:05.837007+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:28:05.837007+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 7690, 'log_dups_size': 0, 'ondisk_log_size': 7690, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31465472, 'num_objects': 7682, 'num_object_clones': 0, 'num_object_copies': 15364, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4160, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7682, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3977, 'num_write_kb': 15908, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3525}, {'shards': '0,2', 'objects': 4157}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:10.067 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:28:10.068 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'7735", 'reported_seq': 7862, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:05.820387+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:28:05.820387+0000', 'last_peered': '2024-09-17T15:28:05.820387+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:28:05.820387+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 7735, 'log_dups_size': 0, 'ondisk_log_size': 7735, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31666176, 'num_objects': 7731, 'num_object_clones': 0, 'num_object_copies': 15462, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4065, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7731, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4522, 'num_write_kb': 18088, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3668}, {'shards': '0,2', 'objects': 4063}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:10.068 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:28:10.068 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'7695", 'reported_seq': 7833, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:05.732000+0000', 'last_change': '2024-09-17T15:26:22.654521+0000', 'last_active': '2024-09-17T15:28:05.732000+0000', 'last_peered': '2024-09-17T15:28:05.732000+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:28:05.732000+0000', 'last_undegraded': '2024-09-17T15:26:22.307795+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 7695, 'log_dups_size': 0, 'ondisk_log_size': 7695, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31470534, 'num_objects': 7685, 'num_object_clones': 0, 'num_object_copies': 15370, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4111, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7685, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4137, 'num_write_kb': 16548, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3576}, {'shards': '0,2', 'objects': 4109}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:10.068 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:28:10.316 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 151 20 125648 125628 3.24919 1.82031 0.0333655 0.0240354 2024-09-17T15:28:10.751 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:28:10.751 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45388/247232 objects degraded (18.358%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":227,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":123616,"data_bytes":506286833,"bytes_used":1652506624,"bytes_avail":288257785856,"bytes_total":289910292480,"degraded_objects":45388,"degraded_total":247232,"degraded_ratio":0.18358464923634482,"misplaced_objects":706,"misplaced_total":247232,"misplaced_ratio":0.0028556173958063681,"recovering_objects_per_sec":32,"recovering_bytes_per_sec":132062,"recovering_keys_per_sec":0,"num_objects_recovered":387,"num_bytes_recovered":1585152,"num_keys_recovered":0,"write_bytes_sec":1739682,"read_op_per_sec":0,"write_op_per_sec":424},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T15:27:19.616376+0000","services":{}},"progress_events":{}} 2024-09-17T15:28:10.775 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:28:10.776 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:28:11.316 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 152 20 126127 126107 3.24013 1.87109 0.0410481 0.0241047 2024-09-17T15:28:11.360 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:28:11.360 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:28:11.384 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2024-09-17T15:28:09.629917+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":506286833,"num_objects":123616,"num_object_clones":0,"num_object_copies":247232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":706,"num_objects_unfound":0,"num_objects_dirty":123616,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":83723,"num_write_kb":334886,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":13779446,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":123689,"ondisk_log_size":123689,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1613776,"kb_used_data":1000928,"kb_used_omap":13,"kb_used_meta":612786,"kb_avail":281501744,"statfs":{"total":289910292480,"available":288257785856,"internally_reserved":0,"allocated":1024950272,"data_stored":1024100243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":627493887},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,12,13,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20881408,"num_objects":5098,"num_object_clones":0,"num_object_copies":10196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-387,"num_objects_unfound":0,"num_objects_dirty":5098,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5098,"num_write_kb":20392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":387,"num_bytes_recovered":1585152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003001"},"pg_stats":[{"pgid":"2.f","version":"41'7672","reported_seq":7745,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:05.766623+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:28:05.766623+0000","last_peered":"2024-09-17T15:28:05.766623+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:28:05.766623+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":7672,"log_dups_size":0,"ondisk_log_size":7672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31424512,"num_objects":7672,"num_object_clones":0,"num_object_copies":15344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7672,"num_write_kb":30688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3602},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'7759","reported_seq":7857,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:07.043441+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:28:07.043441+0000","last_peered":"2024-09-17T15:28:07.043441+0000","last_clean":"2024-09-17T15:28:07.043441+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:28:07.043441+0000","last_undegraded":"2024-09-17T15:28:07.043441+0000","last_fullsized":"2024-09-17T15:28:07.043441+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":7759,"log_dups_size":0,"ondisk_log_size":7759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31757254,"num_objects":7755,"num_object_clones":0,"num_object_copies":15510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7755,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6116,"num_write_kb":24464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'7732","reported_seq":7852,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:07.074791+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:28:07.074791+0000","last_peered":"2024-09-17T15:28:07.074791+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:28:07.074791+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":7732,"log_dups_size":0,"ondisk_log_size":7732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31653888,"num_objects":7728,"num_object_clones":0,"num_object_copies":15456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7728,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5889,"num_write_kb":23556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":3560}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'7811","reported_seq":7876,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:07.001506+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:28:07.001506+0000","last_peered":"2024-09-17T15:28:07.001506+0000","last_clean":"2024-09-17T15:28:07.001506+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:28:07.001506+0000","last_undegraded":"2024-09-17T15:28:07.001506+0000","last_fullsized":"2024-09-17T15:28:07.001506+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":7811,"log_dups_size":0,"ondisk_log_size":7811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31993856,"num_objects":7811,"num_object_clones":0,"num_object_copies":15622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7811,"num_write_kb":31244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'7609","reported_seq":7733,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:05.694586+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:28:05.694586+0000","last_peered":"2024-09-17T15:28:05.694586+0000","last_clean":"2024-09-17T15:28:05.694586+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:28:05.694586+0000","last_undegraded":"2024-09-17T15:28:05.694586+0000","last_fullsized":"2024-09-17T15:28:05.694586+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":7609,"log_dups_size":0,"ondisk_log_size":7609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31138275,"num_objects":7603,"num_object_clones":0,"num_object_copies":15206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7603,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4591,"num_write_kb":18364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'7923","reported_seq":8053,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:05.790641+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:28:05.790641+0000","last_peered":"2024-09-17T15:28:05.790641+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:28:05.790641+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":7923,"log_dups_size":0,"ondisk_log_size":7923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32432611,"num_objects":7919,"num_object_clones":0,"num_object_copies":15838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7919,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4422,"num_write_kb":17688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3649},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'7761","reported_seq":7895,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:05.784883+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:28:05.784883+0000","last_peered":"2024-09-17T15:28:05.784883+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:28:05.784883+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":7761,"log_dups_size":0,"ondisk_log_size":7761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31765446,"num_objects":7757,"num_object_clones":0,"num_object_copies":15514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7757,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4530,"num_write_kb":18120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3583},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'7759","reported_seq":7882,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:05.684883+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:28:05.684883+0000","last_peered":"2024-09-17T15:28:05.684883+0000","last_clean":"2024-09-17T15:28:05.684883+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:28:05.684883+0000","last_undegraded":"2024-09-17T15:28:05.684883+0000","last_fullsized":"2024-09-17T15:28:05.684883+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":7759,"log_dups_size":0,"ondisk_log_size":7759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31772672,"num_objects":7757,"num_object_clones":0,"num_object_copies":15514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4713,"num_write_kb":18852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'7720","reported_seq":14576,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:28:07.066535+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:28:07.066535+0000","last_peered":"2024-09-17T15:28:07.066535+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:28:07.066535+0000","last_undegraded":"2024-09-17T15:28:07.066535+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":7720,"log_dups_size":0,"ondisk_log_size":7720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31592931,"num_objects":7714,"num_object_clones":0,"num_object_copies":15428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":706,"num_objects_unfound":0,"num_objects_dirty":7714,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5720,"num_write_kb":22880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3366,"num_bytes_recovered":13779427,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":706},{"shards":"2","objects":7008}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233600+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233600+0000","last_peered":"2024-09-17T15:26:22.233600+0000","last_clean":"2024-09-17T15:26:22.233600+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233600+0000","last_undegraded":"2024-09-17T15:26:22.233600+0000","last_fullsized":"2024-09-17T15:26:22.233600+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'7612","reported_seq":7737,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:05.797806+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:28:05.797806+0000","last_peered":"2024-09-17T15:28:05.797806+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:28:05.797806+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":7612,"log_dups_size":0,"ondisk_log_size":7612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31145984,"num_objects":7604,"num_object_clones":0,"num_object_copies":15208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4208,"num_write_kb":16832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3594},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":98,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412941+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412941+0000","last_peered":"2024-09-17T15:26:22.412941+0000","last_clean":"2024-09-17T15:26:22.412941+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412941+0000","last_undegraded":"2024-09-17T15:26:22.412941+0000","last_fullsized":"2024-09-17T15:26:22.412941+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'7661","reported_seq":7771,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:07.070499+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:28:07.070499+0000","last_peered":"2024-09-17T15:28:07.070499+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:28:07.070499+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":7661,"log_dups_size":0,"ondisk_log_size":7661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31359459,"num_objects":7657,"num_object_clones":0,"num_object_copies":15314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7657,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5556,"num_write_kb":22224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":3580}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'7763","reported_seq":7877,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:07.083662+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:28:07.083662+0000","last_peered":"2024-09-17T15:28:07.083662+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:28:07.083662+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":7763,"log_dups_size":0,"ondisk_log_size":7763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31772672,"num_objects":7757,"num_object_clones":0,"num_object_copies":15514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5525,"num_write_kb":22100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":3620}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'7784","reported_seq":7914,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:05.779973+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:28:05.779973+0000","last_peered":"2024-09-17T15:28:05.779973+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:28:05.779973+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":7784,"log_dups_size":0,"ondisk_log_size":7784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31875072,"num_objects":7782,"num_object_clones":0,"num_object_copies":15564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4332,"num_write_kb":17328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3644},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'7690","reported_seq":7826,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:05.837007+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:28:05.837007+0000","last_peered":"2024-09-17T15:28:05.837007+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:28:05.837007+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":7690,"log_dups_size":0,"ondisk_log_size":7690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31465472,"num_objects":7682,"num_object_clones":0,"num_object_copies":15364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3977,"num_write_kb":15908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3525},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229061+0000","last_peered":"2024-09-17T15:26:22.229061+0000","last_clean":"2024-09-17T15:26:22.229061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229061+0000","last_undegraded":"2024-09-17T15:26:22.229061+0000","last_fullsized":"2024-09-17T15:26:22.229061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'7735","reported_seq":7862,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:05.820387+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:28:05.820387+0000","last_peered":"2024-09-17T15:28:05.820387+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:28:05.820387+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":7735,"log_dups_size":0,"ondisk_log_size":7735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31666176,"num_objects":7731,"num_object_clones":0,"num_object_copies":15462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4522,"num_write_kb":18088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3668},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'7695","reported_seq":7833,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:05.732000+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:28:05.732000+0000","last_peered":"2024-09-17T15:28:05.732000+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:28:05.732000+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":7695,"log_dups_size":0,"ondisk_log_size":7695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31470534,"num_objects":7685,"num_object_clones":0,"num_object_copies":15370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7685,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4137,"num_write_kb":16548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3576},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":506286814,"num_objects":123614,"num_object_clones":0,"num_object_copies":247228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":706,"num_objects_unfound":0,"num_objects_dirty":123614,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":83721,"num_write_kb":334884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3366,"num_bytes_recovered":13779427,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1024446464,"data_stored":1024294718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":123686,"ondisk_log_size":123686,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051621,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":577948,"kb_used_data":379660,"kb_used_omap":4,"kb_used_meta":198267,"kb_avail":93793892,"statfs":{"total":96636764160,"available":96044945408,"internally_reserved":0,"allocated":388771840,"data_stored":388505667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":203026092},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724566,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":342552,"kb_used_data":184520,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":94029288,"statfs":{"total":96636764160,"available":96285990912,"internally_reserved":0,"allocated":188948480,"data_stored":188634874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":161803947},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,4,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247849,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":693276,"kb_used_data":436748,"kb_used_omap":4,"kb_used_meta":256507,"kb_avail":93678564,"statfs":{"total":96636764160,"available":95926849536,"internally_reserved":0,"allocated":447229952,"data_stored":446959702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":262663848},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,4,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":447057920,"data_stored":447021790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":188878848,"data_stored":188799362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":388509696,"data_stored":388473566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:28:12.316 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 153 20 126640 126620 3.23205 2.00391 0.0425978 0.0241635 2024-09-17T15:28:13.317 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 154 20 127179 127159 3.22471 2.10547 0.0220098 0.0242196 2024-09-17T15:28:14.317 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 155 20 127703 127683 3.21711 2.04688 0.0654875 0.0242767 2024-09-17T15:28:14.387 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:28:14.975 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:28:14.976 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:28:15.005 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":149,"stamp":"2024-09-17T15:28:13.630925+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":516588273,"num_objects":126131,"num_object_clones":0,"num_object_copies":252262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":485,"num_objects_unfound":0,"num_objects_dirty":126131,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":86238,"num_write_kb":344946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3588,"num_bytes_recovered":14684662,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126204,"ondisk_log_size":126204,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1638368,"kb_used_data":1021360,"kb_used_omap":13,"kb_used_meta":616946,"kb_avail":281477152,"statfs":{"total":289910292480,"available":288232603648,"internally_reserved":0,"allocated":1045872640,"data_stored":1045022611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":631753727},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,5,5,12,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24633344,"num_objects":6014,"num_object_clones":0,"num_object_copies":12028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-608,"num_objects_unfound":0,"num_objects_dirty":6014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6014,"num_write_kb":24056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":608,"num_bytes_recovered":2490368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003047"},"pg_stats":[{"pgid":"2.f","version":"41'7830","reported_seq":7903,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:10.743987+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:28:10.743987+0000","last_peered":"2024-09-17T15:28:10.743987+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:28:10.743987+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":7830,"log_dups_size":0,"ondisk_log_size":7830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32071680,"num_objects":7830,"num_object_clones":0,"num_object_copies":15660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7830,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7830,"num_write_kb":31320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3760},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'7916","reported_seq":8014,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:12.046305+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:28:12.046305+0000","last_peered":"2024-09-17T15:28:12.046305+0000","last_clean":"2024-09-17T15:28:12.046305+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:28:12.046305+0000","last_undegraded":"2024-09-17T15:28:12.046305+0000","last_fullsized":"2024-09-17T15:28:12.046305+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":7916,"log_dups_size":0,"ondisk_log_size":7916,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32400326,"num_objects":7912,"num_object_clones":0,"num_object_copies":15824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7912,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6273,"num_write_kb":25092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'7898","reported_seq":8018,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:12.057089+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:28:12.057089+0000","last_peered":"2024-09-17T15:28:12.057089+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:28:12.057089+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":7898,"log_dups_size":0,"ondisk_log_size":7898,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32333824,"num_objects":7894,"num_object_clones":0,"num_object_copies":15788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6055,"num_write_kb":24220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":3726}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'7975","reported_seq":8040,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:12.056988+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:28:12.056988+0000","last_peered":"2024-09-17T15:28:12.056988+0000","last_clean":"2024-09-17T15:28:12.056988+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:28:12.056988+0000","last_undegraded":"2024-09-17T15:28:12.056988+0000","last_fullsized":"2024-09-17T15:28:12.056988+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":7975,"log_dups_size":0,"ondisk_log_size":7975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32665600,"num_objects":7975,"num_object_clones":0,"num_object_copies":15950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7975,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7975,"num_write_kb":31900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'7742","reported_seq":7866,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:10.753258+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:28:10.753258+0000","last_peered":"2024-09-17T15:28:10.753258+0000","last_clean":"2024-09-17T15:28:10.753258+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:28:10.753258+0000","last_undegraded":"2024-09-17T15:28:10.753258+0000","last_fullsized":"2024-09-17T15:28:10.753258+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":7742,"log_dups_size":0,"ondisk_log_size":7742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31683043,"num_objects":7736,"num_object_clones":0,"num_object_copies":15472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7736,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4724,"num_write_kb":18896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'8066","reported_seq":8196,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:10.698494+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:28:10.698494+0000","last_peered":"2024-09-17T15:28:10.698494+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:28:10.698494+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":8066,"log_dups_size":0,"ondisk_log_size":8066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33018339,"num_objects":8062,"num_object_clones":0,"num_object_copies":16124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8062,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4565,"num_write_kb":18260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3792},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'7911","reported_seq":8045,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:10.705960+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:28:10.705960+0000","last_peered":"2024-09-17T15:28:10.705960+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:28:10.705960+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":7911,"log_dups_size":0,"ondisk_log_size":7911,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32379846,"num_objects":7907,"num_object_clones":0,"num_object_copies":15814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7907,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4680,"num_write_kb":18720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3733},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'7921","reported_seq":8044,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:10.764773+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:28:10.764773+0000","last_peered":"2024-09-17T15:28:10.764773+0000","last_clean":"2024-09-17T15:28:10.764773+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:28:10.764773+0000","last_undegraded":"2024-09-17T15:28:10.764773+0000","last_fullsized":"2024-09-17T15:28:10.764773+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":7921,"log_dups_size":0,"ondisk_log_size":7921,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32436224,"num_objects":7919,"num_object_clones":0,"num_object_copies":15838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4875,"num_write_kb":19500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'7875","reported_seq":15173,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:28:12.085333+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:28:12.085333+0000","last_peered":"2024-09-17T15:28:12.085333+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:28:12.085333+0000","last_undegraded":"2024-09-17T15:28:12.085333+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":7875,"log_dups_size":0,"ondisk_log_size":7875,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32227811,"num_objects":7869,"num_object_clones":0,"num_object_copies":15738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":485,"num_objects_unfound":0,"num_objects_dirty":7869,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5875,"num_write_kb":23500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":14684643,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":485},{"shards":"2","objects":7384}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233600+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233600+0000","last_peered":"2024-09-17T15:26:22.233600+0000","last_clean":"2024-09-17T15:26:22.233600+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233600+0000","last_undegraded":"2024-09-17T15:26:22.233600+0000","last_fullsized":"2024-09-17T15:26:22.233600+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'7767","reported_seq":7892,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:10.742301+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:28:10.742301+0000","last_peered":"2024-09-17T15:28:10.742301+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:28:10.742301+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":7767,"log_dups_size":0,"ondisk_log_size":7767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31780864,"num_objects":7759,"num_object_clones":0,"num_object_copies":15518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7759,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4363,"num_write_kb":17452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3749},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":98,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412941+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412941+0000","last_peered":"2024-09-17T15:26:22.412941+0000","last_clean":"2024-09-17T15:26:22.412941+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412941+0000","last_undegraded":"2024-09-17T15:26:22.412941+0000","last_fullsized":"2024-09-17T15:26:22.412941+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'7816","reported_seq":7932,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:11.971966+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:28:11.971966+0000","last_peered":"2024-09-17T15:28:11.971966+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:28:11.971966+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":7816,"log_dups_size":0,"ondisk_log_size":7816,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31994339,"num_objects":7812,"num_object_clones":0,"num_object_copies":15624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7812,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5711,"num_write_kb":22844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":3735}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'7925","reported_seq":8039,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:11.986931+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:28:11.986931+0000","last_peered":"2024-09-17T15:28:11.986931+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:28:11.986931+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":7925,"log_dups_size":0,"ondisk_log_size":7925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32436224,"num_objects":7919,"num_object_clones":0,"num_object_copies":15838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5687,"num_write_kb":22748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":3782}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'7956","reported_seq":8084,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:10.803936+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:28:10.803936+0000","last_peered":"2024-09-17T15:28:10.803936+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:28:10.803936+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":7956,"log_dups_size":0,"ondisk_log_size":7956,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32579584,"num_objects":7954,"num_object_clones":0,"num_object_copies":15908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4504,"num_write_kb":18016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3816},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'7862","reported_seq":7999,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:10.784465+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:28:10.784465+0000","last_peered":"2024-09-17T15:28:10.784465+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:28:10.784465+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":7862,"log_dups_size":0,"ondisk_log_size":7862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32169984,"num_objects":7854,"num_object_clones":0,"num_object_copies":15708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4149,"num_write_kb":16596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3697},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229061+0000","last_peered":"2024-09-17T15:26:22.229061+0000","last_clean":"2024-09-17T15:26:22.229061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229061+0000","last_undegraded":"2024-09-17T15:26:22.229061+0000","last_fullsized":"2024-09-17T15:26:22.229061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'7904","reported_seq":8030,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:10.837950+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:28:10.837950+0000","last_peered":"2024-09-17T15:28:10.837950+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:28:10.837950+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":7904,"log_dups_size":0,"ondisk_log_size":7904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32358400,"num_objects":7900,"num_object_clones":0,"num_object_copies":15800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4691,"num_write_kb":18764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3837},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'7837","reported_seq":7975,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:10.691612+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:28:10.691612+0000","last_peered":"2024-09-17T15:28:10.691612+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:28:10.691612+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":7837,"log_dups_size":0,"ondisk_log_size":7837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32052166,"num_objects":7827,"num_object_clones":0,"num_object_copies":15654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4279,"num_write_kb":17116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3718},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":516588254,"num_objects":126129,"num_object_clones":0,"num_object_copies":252258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":485,"num_objects_unfound":0,"num_objects_dirty":126129,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":86236,"num_write_kb":344944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":14684643,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1046056960,"data_stored":1045905214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126201,"ondisk_log_size":126201,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051622,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":586204,"kb_used_data":383756,"kb_used_omap":4,"kb_used_meta":202427,"kb_avail":93785636,"statfs":{"total":96636764160,"available":96036491264,"internally_reserved":0,"allocated":392966144,"data_stored":392699971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":207285932},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724567,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":359680,"kb_used_data":193328,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":94012160,"statfs":{"total":96636764160,"available":96268451840,"internally_reserved":0,"allocated":197967872,"data_stored":197654266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":170323627},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,7],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247850,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":692484,"kb_used_data":444276,"kb_used_omap":4,"kb_used_meta":248187,"kb_avail":93679356,"statfs":{"total":96636764160,"available":95927660544,"internally_reserved":0,"allocated":454938624,"data_stored":454668374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":254144168},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,3,2,4,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":454725632,"data_stored":454689502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":197689344,"data_stored":197609858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":393641984,"data_stored":393605854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:28:15.006 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:28:15.317 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 156 20 128138 128118 3.20738 1.69922 0.0778722 0.0243511 2024-09-17T15:28:15.572 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:28:15.574 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:28:15.599 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":149,"stamp":"2024-09-17T15:28:13.630925+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":516588273,"num_objects":126131,"num_object_clones":0,"num_object_copies":252262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":485,"num_objects_unfound":0,"num_objects_dirty":126131,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":86238,"num_write_kb":344946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3588,"num_bytes_recovered":14684662,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126204,"ondisk_log_size":126204,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1638368,"kb_used_data":1021360,"kb_used_omap":13,"kb_used_meta":616946,"kb_avail":281477152,"statfs":{"total":289910292480,"available":288232603648,"internally_reserved":0,"allocated":1045872640,"data_stored":1045022611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":631753727},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,5,5,12,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24633344,"num_objects":6014,"num_object_clones":0,"num_object_copies":12028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-608,"num_objects_unfound":0,"num_objects_dirty":6014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6014,"num_write_kb":24056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":608,"num_bytes_recovered":2490368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003047"},"pg_stats":[{"pgid":"2.f","version":"41'7830","reported_seq":7903,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:10.743987+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:28:10.743987+0000","last_peered":"2024-09-17T15:28:10.743987+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:28:10.743987+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":7830,"log_dups_size":0,"ondisk_log_size":7830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32071680,"num_objects":7830,"num_object_clones":0,"num_object_copies":15660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7830,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7830,"num_write_kb":31320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3760},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'7916","reported_seq":8014,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:12.046305+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:28:12.046305+0000","last_peered":"2024-09-17T15:28:12.046305+0000","last_clean":"2024-09-17T15:28:12.046305+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:28:12.046305+0000","last_undegraded":"2024-09-17T15:28:12.046305+0000","last_fullsized":"2024-09-17T15:28:12.046305+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":7916,"log_dups_size":0,"ondisk_log_size":7916,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32400326,"num_objects":7912,"num_object_clones":0,"num_object_copies":15824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7912,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6273,"num_write_kb":25092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'7898","reported_seq":8018,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:12.057089+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:28:12.057089+0000","last_peered":"2024-09-17T15:28:12.057089+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:28:12.057089+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":7898,"log_dups_size":0,"ondisk_log_size":7898,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32333824,"num_objects":7894,"num_object_clones":0,"num_object_copies":15788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6055,"num_write_kb":24220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":3726}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'7975","reported_seq":8040,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:12.056988+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:28:12.056988+0000","last_peered":"2024-09-17T15:28:12.056988+0000","last_clean":"2024-09-17T15:28:12.056988+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:28:12.056988+0000","last_undegraded":"2024-09-17T15:28:12.056988+0000","last_fullsized":"2024-09-17T15:28:12.056988+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":7975,"log_dups_size":0,"ondisk_log_size":7975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32665600,"num_objects":7975,"num_object_clones":0,"num_object_copies":15950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7975,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7975,"num_write_kb":31900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'7742","reported_seq":7866,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:10.753258+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:28:10.753258+0000","last_peered":"2024-09-17T15:28:10.753258+0000","last_clean":"2024-09-17T15:28:10.753258+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:28:10.753258+0000","last_undegraded":"2024-09-17T15:28:10.753258+0000","last_fullsized":"2024-09-17T15:28:10.753258+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":7742,"log_dups_size":0,"ondisk_log_size":7742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31683043,"num_objects":7736,"num_object_clones":0,"num_object_copies":15472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7736,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4724,"num_write_kb":18896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'8066","reported_seq":8196,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:10.698494+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:28:10.698494+0000","last_peered":"2024-09-17T15:28:10.698494+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:28:10.698494+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":8066,"log_dups_size":0,"ondisk_log_size":8066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33018339,"num_objects":8062,"num_object_clones":0,"num_object_copies":16124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8062,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4565,"num_write_kb":18260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3792},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'7911","reported_seq":8045,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:10.705960+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:28:10.705960+0000","last_peered":"2024-09-17T15:28:10.705960+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:28:10.705960+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":7911,"log_dups_size":0,"ondisk_log_size":7911,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32379846,"num_objects":7907,"num_object_clones":0,"num_object_copies":15814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7907,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4680,"num_write_kb":18720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3733},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'7921","reported_seq":8044,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:10.764773+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:28:10.764773+0000","last_peered":"2024-09-17T15:28:10.764773+0000","last_clean":"2024-09-17T15:28:10.764773+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:28:10.764773+0000","last_undegraded":"2024-09-17T15:28:10.764773+0000","last_fullsized":"2024-09-17T15:28:10.764773+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":7921,"log_dups_size":0,"ondisk_log_size":7921,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32436224,"num_objects":7919,"num_object_clones":0,"num_object_copies":15838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4875,"num_write_kb":19500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'7875","reported_seq":15173,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:28:12.085333+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:28:12.085333+0000","last_peered":"2024-09-17T15:28:12.085333+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:28:12.085333+0000","last_undegraded":"2024-09-17T15:28:12.085333+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":7875,"log_dups_size":0,"ondisk_log_size":7875,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32227811,"num_objects":7869,"num_object_clones":0,"num_object_copies":15738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":485,"num_objects_unfound":0,"num_objects_dirty":7869,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5875,"num_write_kb":23500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":14684643,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":485},{"shards":"2","objects":7384}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233600+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233600+0000","last_peered":"2024-09-17T15:26:22.233600+0000","last_clean":"2024-09-17T15:26:22.233600+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233600+0000","last_undegraded":"2024-09-17T15:26:22.233600+0000","last_fullsized":"2024-09-17T15:26:22.233600+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'7767","reported_seq":7892,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:10.742301+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:28:10.742301+0000","last_peered":"2024-09-17T15:28:10.742301+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:28:10.742301+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":7767,"log_dups_size":0,"ondisk_log_size":7767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31780864,"num_objects":7759,"num_object_clones":0,"num_object_copies":15518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7759,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4363,"num_write_kb":17452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3749},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":98,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412941+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412941+0000","last_peered":"2024-09-17T15:26:22.412941+0000","last_clean":"2024-09-17T15:26:22.412941+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412941+0000","last_undegraded":"2024-09-17T15:26:22.412941+0000","last_fullsized":"2024-09-17T15:26:22.412941+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'7816","reported_seq":7932,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:11.971966+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:28:11.971966+0000","last_peered":"2024-09-17T15:28:11.971966+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:28:11.971966+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":7816,"log_dups_size":0,"ondisk_log_size":7816,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31994339,"num_objects":7812,"num_object_clones":0,"num_object_copies":15624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7812,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5711,"num_write_kb":22844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":3735}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'7925","reported_seq":8039,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:11.986931+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:28:11.986931+0000","last_peered":"2024-09-17T15:28:11.986931+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:28:11.986931+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":7925,"log_dups_size":0,"ondisk_log_size":7925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32436224,"num_objects":7919,"num_object_clones":0,"num_object_copies":15838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5687,"num_write_kb":22748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":3782}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'7956","reported_seq":8084,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:10.803936+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:28:10.803936+0000","last_peered":"2024-09-17T15:28:10.803936+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:28:10.803936+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":7956,"log_dups_size":0,"ondisk_log_size":7956,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32579584,"num_objects":7954,"num_object_clones":0,"num_object_copies":15908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4504,"num_write_kb":18016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3816},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'7862","reported_seq":7999,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:10.784465+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:28:10.784465+0000","last_peered":"2024-09-17T15:28:10.784465+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:28:10.784465+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":7862,"log_dups_size":0,"ondisk_log_size":7862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32169984,"num_objects":7854,"num_object_clones":0,"num_object_copies":15708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4149,"num_write_kb":16596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3697},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229061+0000","last_peered":"2024-09-17T15:26:22.229061+0000","last_clean":"2024-09-17T15:26:22.229061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229061+0000","last_undegraded":"2024-09-17T15:26:22.229061+0000","last_fullsized":"2024-09-17T15:26:22.229061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'7904","reported_seq":8030,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:10.837950+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:28:10.837950+0000","last_peered":"2024-09-17T15:28:10.837950+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:28:10.837950+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":7904,"log_dups_size":0,"ondisk_log_size":7904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32358400,"num_objects":7900,"num_object_clones":0,"num_object_copies":15800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4691,"num_write_kb":18764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3837},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'7837","reported_seq":7975,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:10.691612+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:28:10.691612+0000","last_peered":"2024-09-17T15:28:10.691612+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:28:10.691612+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":7837,"log_dups_size":0,"ondisk_log_size":7837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32052166,"num_objects":7827,"num_object_clones":0,"num_object_copies":15654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4279,"num_write_kb":17116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3718},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":516588254,"num_objects":126129,"num_object_clones":0,"num_object_copies":252258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":485,"num_objects_unfound":0,"num_objects_dirty":126129,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":86236,"num_write_kb":344944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":14684643,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1046056960,"data_stored":1045905214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126201,"ondisk_log_size":126201,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051622,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":586204,"kb_used_data":383756,"kb_used_omap":4,"kb_used_meta":202427,"kb_avail":93785636,"statfs":{"total":96636764160,"available":96036491264,"internally_reserved":0,"allocated":392966144,"data_stored":392699971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":207285932},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724567,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":359680,"kb_used_data":193328,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":94012160,"statfs":{"total":96636764160,"available":96268451840,"internally_reserved":0,"allocated":197967872,"data_stored":197654266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":170323627},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,7],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247850,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":692484,"kb_used_data":444276,"kb_used_omap":4,"kb_used_meta":248187,"kb_avail":93679356,"statfs":{"total":96636764160,"available":95927660544,"internally_reserved":0,"allocated":454938624,"data_stored":454668374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":254144168},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,3,2,4,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":454725632,"data_stored":454689502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":197689344,"data_stored":197609858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":393641984,"data_stored":393605854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:28:15.600 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:28:15.600 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'7830", 'reported_seq': 7903, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:10.743987+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:28:10.743987+0000', 'last_peered': '2024-09-17T15:28:10.743987+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:28:10.743987+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 7830, 'log_dups_size': 0, 'ondisk_log_size': 7830, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32071680, 'num_objects': 7830, 'num_object_clones': 0, 'num_object_copies': 15660, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4070, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7830, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7830, 'num_write_kb': 31320, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3760}, {'shards': '0,2', 'objects': 4070}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:15.600 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:28:15.600 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'7898", 'reported_seq': 8018, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:12.057089+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:28:12.057089+0000', 'last_peered': '2024-09-17T15:28:12.057089+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:28:12.057089+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 7898, 'log_dups_size': 0, 'ondisk_log_size': 7898, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32333824, 'num_objects': 7894, 'num_object_clones': 0, 'num_object_copies': 15788, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4169, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7894, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6055, 'num_write_kb': 24220, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4168}, {'shards': '2', 'objects': 3726}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:28:15.601 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:28:15.601 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'8066", 'reported_seq': 8196, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:10.698494+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:28:10.698494+0000', 'last_peered': '2024-09-17T15:28:10.698494+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:28:10.698494+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 8066, 'log_dups_size': 0, 'ondisk_log_size': 8066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33018339, 'num_objects': 8062, 'num_object_clones': 0, 'num_object_copies': 16124, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4271, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8062, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4565, 'num_write_kb': 18260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3792}, {'shards': '0,2', 'objects': 4268}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:15.601 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:28:15.601 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'7911", 'reported_seq': 8045, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:10.705960+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:28:10.705960+0000', 'last_peered': '2024-09-17T15:28:10.705960+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:28:10.705960+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 7911, 'log_dups_size': 0, 'ondisk_log_size': 7911, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32379846, 'num_objects': 7907, 'num_object_clones': 0, 'num_object_copies': 15814, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4174, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7907, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 4680, 'num_write_kb': 18720, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3733}, {'shards': '0,2', 'objects': 4174}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:15.601 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-17T15:28:15.601 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "41'7875", 'reported_seq': 15173, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:28:12.085333+0000', 'last_change': '2024-09-17T15:26:22.588242+0000', 'last_active': '2024-09-17T15:28:12.085333+0000', 'last_peered': '2024-09-17T15:28:12.085333+0000', 'last_clean': '2024-09-17T15:26:20.155646+0000', 'last_became_active': '2024-09-17T15:26:22.557742+0000', 'last_became_peered': '2024-09-17T15:26:22.557742+0000', 'last_unstale': '2024-09-17T15:28:12.085333+0000', 'last_undegraded': '2024-09-17T15:28:12.085333+0000', 'last_fullsized': '2024-09-17T15:26:22.188850+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2016", 'last_scrub_stamp': '2024-09-17T15:25:53.959011+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:53.959011+0000', 'objects_scrubbed': 1994, 'log_size': 7875, 'log_dups_size': 0, 'ondisk_log_size': 7875, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 568, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32227811, 'num_objects': 7869, 'num_object_clones': 0, 'num_object_copies': 15738, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 485, 'num_objects_unfound': 0, 'num_objects_dirty': 7869, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5875, 'num_write_kb': 23500, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3587, 'num_bytes_recovered': 14684643, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 485}, {'shards': '2', 'objects': 7384}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:28:15.601 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:28:15.601 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'7767", 'reported_seq': 7892, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:10.742301+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:28:10.742301+0000', 'last_peered': '2024-09-17T15:28:10.742301+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:28:10.742301+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 7767, 'log_dups_size': 0, 'ondisk_log_size': 7767, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31780864, 'num_objects': 7759, 'num_object_clones': 0, 'num_object_copies': 15518, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4012, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7759, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4363, 'num_write_kb': 17452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3749}, {'shards': '0,2', 'objects': 4010}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:15.601 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:28:15.601 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'7816", 'reported_seq': 7932, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:11.971966+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:28:11.971966+0000', 'last_peered': '2024-09-17T15:28:11.971966+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:28:11.971966+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 7816, 'log_dups_size': 0, 'ondisk_log_size': 7816, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31994339, 'num_objects': 7812, 'num_object_clones': 0, 'num_object_copies': 15624, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4078, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7812, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5711, 'num_write_kb': 22844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4077}, {'shards': '2', 'objects': 3735}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:28:15.601 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:28:15.601 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'7925", 'reported_seq': 8039, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:11.986931+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:28:11.986931+0000', 'last_peered': '2024-09-17T15:28:11.986931+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:28:11.986931+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 7925, 'log_dups_size': 0, 'ondisk_log_size': 7925, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32436224, 'num_objects': 7919, 'num_object_clones': 0, 'num_object_copies': 15838, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7919, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5687, 'num_write_kb': 22748, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4137}, {'shards': '2', 'objects': 3782}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:28:15.602 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:28:15.602 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'7956", 'reported_seq': 8084, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:10.803936+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:28:10.803936+0000', 'last_peered': '2024-09-17T15:28:10.803936+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:28:10.803936+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 7956, 'log_dups_size': 0, 'ondisk_log_size': 7956, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32579584, 'num_objects': 7954, 'num_object_clones': 0, 'num_object_copies': 15908, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7954, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4504, 'num_write_kb': 18016, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3816}, {'shards': '0,2', 'objects': 4138}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:15.602 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:28:15.602 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'7862", 'reported_seq': 7999, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:10.784465+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:28:10.784465+0000', 'last_peered': '2024-09-17T15:28:10.784465+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:28:10.784465+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 7862, 'log_dups_size': 0, 'ondisk_log_size': 7862, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32169984, 'num_objects': 7854, 'num_object_clones': 0, 'num_object_copies': 15708, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4160, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7854, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4149, 'num_write_kb': 16596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3697}, {'shards': '0,2', 'objects': 4157}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:15.602 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:28:15.602 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'7904", 'reported_seq': 8030, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:10.837950+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:28:10.837950+0000', 'last_peered': '2024-09-17T15:28:10.837950+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:28:10.837950+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 7904, 'log_dups_size': 0, 'ondisk_log_size': 7904, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32358400, 'num_objects': 7900, 'num_object_clones': 0, 'num_object_copies': 15800, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4065, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7900, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4691, 'num_write_kb': 18764, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3837}, {'shards': '0,2', 'objects': 4063}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:15.602 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:28:15.602 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'7837", 'reported_seq': 7975, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:10.691612+0000', 'last_change': '2024-09-17T15:26:22.654521+0000', 'last_active': '2024-09-17T15:28:10.691612+0000', 'last_peered': '2024-09-17T15:28:10.691612+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:28:10.691612+0000', 'last_undegraded': '2024-09-17T15:26:22.307795+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 7837, 'log_dups_size': 0, 'ondisk_log_size': 7837, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32052166, 'num_objects': 7827, 'num_object_clones': 0, 'num_object_copies': 15654, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4111, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7827, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4279, 'num_write_kb': 17116, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3718}, {'shards': '0,2', 'objects': 4109}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:15.602 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:28:16.309 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:28:16.309 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45388/252262 objects degraded (17.992%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":233,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":126131,"data_bytes":516588273,"bytes_used":1673601024,"bytes_avail":288236691456,"bytes_total":289910292480,"degraded_objects":45388,"degraded_total":252262,"degraded_ratio":0.17992404722074667,"misplaced_objects":485,"misplaced_total":252262,"misplaced_ratio":0.0019226042765061722,"recovering_objects_per_sec":34,"recovering_bytes_per_sec":140935,"recovering_keys_per_sec":0,"num_objects_recovered":413,"num_bytes_recovered":1691648,"num_keys_recovered":0,"write_bytes_sec":1730122,"read_op_per_sec":0,"write_op_per_sec":422},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T15:27:19.616376+0000","services":{}},"progress_events":{}} 2024-09-17T15:28:16.317 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 157 20 128600 128580 3.19844 1.80469 0.0794204 0.0244172 2024-09-17T15:28:16.334 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:28:16.334 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:28:16.873 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:28:16.875 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:28:16.895 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":150,"stamp":"2024-09-17T15:28:15.631416+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":516588273,"num_objects":126131,"num_object_clones":0,"num_object_copies":252262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":485,"num_objects_unfound":0,"num_objects_dirty":126131,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":86238,"num_write_kb":344946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3588,"num_bytes_recovered":14684662,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126204,"ondisk_log_size":126204,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1634376,"kb_used_data":1029848,"kb_used_omap":13,"kb_used_meta":604466,"kb_avail":281481144,"statfs":{"total":289910292480,"available":288236691456,"internally_reserved":0,"allocated":1054564352,"data_stored":1053714323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":618974207},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,5,6,15,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20766720,"num_objects":5070,"num_object_clones":0,"num_object_copies":10140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-413,"num_objects_unfound":0,"num_objects_dirty":5070,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5070,"num_write_kb":20280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":413,"num_bytes_recovered":1691648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003031"},"pg_stats":[{"pgid":"2.f","version":"41'7830","reported_seq":7903,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:10.743987+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:28:10.743987+0000","last_peered":"2024-09-17T15:28:10.743987+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:28:10.743987+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":7830,"log_dups_size":0,"ondisk_log_size":7830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32071680,"num_objects":7830,"num_object_clones":0,"num_object_copies":15660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7830,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7830,"num_write_kb":31320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3760},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'7916","reported_seq":8014,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:12.046305+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:28:12.046305+0000","last_peered":"2024-09-17T15:28:12.046305+0000","last_clean":"2024-09-17T15:28:12.046305+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:28:12.046305+0000","last_undegraded":"2024-09-17T15:28:12.046305+0000","last_fullsized":"2024-09-17T15:28:12.046305+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":7916,"log_dups_size":0,"ondisk_log_size":7916,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32400326,"num_objects":7912,"num_object_clones":0,"num_object_copies":15824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7912,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6273,"num_write_kb":25092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'7898","reported_seq":8018,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:12.057089+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:28:12.057089+0000","last_peered":"2024-09-17T15:28:12.057089+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:28:12.057089+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":7898,"log_dups_size":0,"ondisk_log_size":7898,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32333824,"num_objects":7894,"num_object_clones":0,"num_object_copies":15788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6055,"num_write_kb":24220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":3726}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'7975","reported_seq":8040,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:12.056988+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:28:12.056988+0000","last_peered":"2024-09-17T15:28:12.056988+0000","last_clean":"2024-09-17T15:28:12.056988+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:28:12.056988+0000","last_undegraded":"2024-09-17T15:28:12.056988+0000","last_fullsized":"2024-09-17T15:28:12.056988+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":7975,"log_dups_size":0,"ondisk_log_size":7975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32665600,"num_objects":7975,"num_object_clones":0,"num_object_copies":15950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7975,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7975,"num_write_kb":31900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'7742","reported_seq":7866,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:10.753258+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:28:10.753258+0000","last_peered":"2024-09-17T15:28:10.753258+0000","last_clean":"2024-09-17T15:28:10.753258+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:28:10.753258+0000","last_undegraded":"2024-09-17T15:28:10.753258+0000","last_fullsized":"2024-09-17T15:28:10.753258+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":7742,"log_dups_size":0,"ondisk_log_size":7742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31683043,"num_objects":7736,"num_object_clones":0,"num_object_copies":15472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7736,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4724,"num_write_kb":18896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'8066","reported_seq":8196,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:10.698494+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:28:10.698494+0000","last_peered":"2024-09-17T15:28:10.698494+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:28:10.698494+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":8066,"log_dups_size":0,"ondisk_log_size":8066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33018339,"num_objects":8062,"num_object_clones":0,"num_object_copies":16124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8062,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4565,"num_write_kb":18260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3792},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'7911","reported_seq":8045,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:10.705960+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:28:10.705960+0000","last_peered":"2024-09-17T15:28:10.705960+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:28:10.705960+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":7911,"log_dups_size":0,"ondisk_log_size":7911,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32379846,"num_objects":7907,"num_object_clones":0,"num_object_copies":15814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7907,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4680,"num_write_kb":18720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3733},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'7921","reported_seq":8044,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:10.764773+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:28:10.764773+0000","last_peered":"2024-09-17T15:28:10.764773+0000","last_clean":"2024-09-17T15:28:10.764773+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:28:10.764773+0000","last_undegraded":"2024-09-17T15:28:10.764773+0000","last_fullsized":"2024-09-17T15:28:10.764773+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":7921,"log_dups_size":0,"ondisk_log_size":7921,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32436224,"num_objects":7919,"num_object_clones":0,"num_object_copies":15838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4875,"num_write_kb":19500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'7875","reported_seq":15173,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:28:12.085333+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:28:12.085333+0000","last_peered":"2024-09-17T15:28:12.085333+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:28:12.085333+0000","last_undegraded":"2024-09-17T15:28:12.085333+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":7875,"log_dups_size":0,"ondisk_log_size":7875,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32227811,"num_objects":7869,"num_object_clones":0,"num_object_copies":15738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":485,"num_objects_unfound":0,"num_objects_dirty":7869,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5875,"num_write_kb":23500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":14684643,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":485},{"shards":"2","objects":7384}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233600+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233600+0000","last_peered":"2024-09-17T15:26:22.233600+0000","last_clean":"2024-09-17T15:26:22.233600+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233600+0000","last_undegraded":"2024-09-17T15:26:22.233600+0000","last_fullsized":"2024-09-17T15:26:22.233600+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'7767","reported_seq":7892,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:10.742301+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:28:10.742301+0000","last_peered":"2024-09-17T15:28:10.742301+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:28:10.742301+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":7767,"log_dups_size":0,"ondisk_log_size":7767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31780864,"num_objects":7759,"num_object_clones":0,"num_object_copies":15518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7759,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4363,"num_write_kb":17452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3749},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":98,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412941+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412941+0000","last_peered":"2024-09-17T15:26:22.412941+0000","last_clean":"2024-09-17T15:26:22.412941+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412941+0000","last_undegraded":"2024-09-17T15:26:22.412941+0000","last_fullsized":"2024-09-17T15:26:22.412941+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'7816","reported_seq":7932,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:11.971966+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:28:11.971966+0000","last_peered":"2024-09-17T15:28:11.971966+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:28:11.971966+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":7816,"log_dups_size":0,"ondisk_log_size":7816,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31994339,"num_objects":7812,"num_object_clones":0,"num_object_copies":15624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7812,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5711,"num_write_kb":22844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":3735}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'7925","reported_seq":8039,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:11.986931+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:28:11.986931+0000","last_peered":"2024-09-17T15:28:11.986931+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:28:11.986931+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":7925,"log_dups_size":0,"ondisk_log_size":7925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32436224,"num_objects":7919,"num_object_clones":0,"num_object_copies":15838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5687,"num_write_kb":22748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":3782}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'7956","reported_seq":8084,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:10.803936+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:28:10.803936+0000","last_peered":"2024-09-17T15:28:10.803936+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:28:10.803936+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":7956,"log_dups_size":0,"ondisk_log_size":7956,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32579584,"num_objects":7954,"num_object_clones":0,"num_object_copies":15908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4504,"num_write_kb":18016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3816},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'7862","reported_seq":7999,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:10.784465+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:28:10.784465+0000","last_peered":"2024-09-17T15:28:10.784465+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:28:10.784465+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":7862,"log_dups_size":0,"ondisk_log_size":7862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32169984,"num_objects":7854,"num_object_clones":0,"num_object_copies":15708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4149,"num_write_kb":16596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3697},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229061+0000","last_peered":"2024-09-17T15:26:22.229061+0000","last_clean":"2024-09-17T15:26:22.229061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229061+0000","last_undegraded":"2024-09-17T15:26:22.229061+0000","last_fullsized":"2024-09-17T15:26:22.229061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'7904","reported_seq":8030,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:10.837950+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:28:10.837950+0000","last_peered":"2024-09-17T15:28:10.837950+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:28:10.837950+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":7904,"log_dups_size":0,"ondisk_log_size":7904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32358400,"num_objects":7900,"num_object_clones":0,"num_object_copies":15800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4691,"num_write_kb":18764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3837},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'7837","reported_seq":7975,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:10.691612+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:28:10.691612+0000","last_peered":"2024-09-17T15:28:10.691612+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:28:10.691612+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":7837,"log_dups_size":0,"ondisk_log_size":7837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32052166,"num_objects":7827,"num_object_clones":0,"num_object_copies":15654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4279,"num_write_kb":17116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3718},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":516588254,"num_objects":126129,"num_object_clones":0,"num_object_copies":252258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":485,"num_objects_unfound":0,"num_objects_dirty":126129,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":86236,"num_write_kb":344944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":14684643,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1054707712,"data_stored":1054555966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126201,"ondisk_log_size":126201,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051622,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":586204,"kb_used_data":383756,"kb_used_omap":4,"kb_used_meta":202427,"kb_avail":93785636,"statfs":{"total":96636764160,"available":96036491264,"internally_reserved":0,"allocated":392966144,"data_stored":392699971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":207285932},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724568,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":355688,"kb_used_data":201816,"kb_used_omap":4,"kb_used_meta":153851,"kb_avail":94016152,"statfs":{"total":96636764160,"available":96272539648,"internally_reserved":0,"allocated":206659584,"data_stored":206345978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":157544107},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,10,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247850,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":692484,"kb_used_data":444276,"kb_used_omap":4,"kb_used_meta":248187,"kb_avail":93679356,"statfs":{"total":96636764160,"available":95927660544,"internally_reserved":0,"allocated":454938624,"data_stored":454668374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":254144168},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,3,2,4,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":454725632,"data_stored":454689502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":206340096,"data_stored":206260610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":393641984,"data_stored":393605854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:28:17.317 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 158 20 129096 129076 3.19046 1.9375 0.038387 0.0244799 2024-09-17T15:28:18.318 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 159 20 129630 129610 3.18352 2.08594 0.0139084 0.0245324 2024-09-17T15:28:19.318 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout:2024-09-17T15:28:19.318136+0000 min lat: 0.00187689 max lat: 2.64156 avg lat: 0.0245868 2024-09-17T15:28:19.318 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-17T15:28:19.318 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 160 20 130167 130147 3.17673 2.09766 0.0481259 0.0245868 2024-09-17T15:28:19.897 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:28:20.318 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 161 20 130659 130639 3.16894 1.92188 0.0235429 0.0246451 2024-09-17T15:28:20.449 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:28:20.449 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:28:20.479 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":152,"stamp":"2024-09-17T15:28:19.632401+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":526836465,"num_objects":128633,"num_object_clones":0,"num_object_copies":257266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":266,"num_objects_unfound":0,"num_objects_dirty":128633,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":88740,"num_write_kb":354954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3807,"num_bytes_recovered":15581686,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128706,"ondisk_log_size":128706,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1727452,"kb_used_data":1042412,"kb_used_omap":13,"kb_used_meta":684978,"kb_avail":281388068,"statfs":{"total":289910292480,"available":288141381632,"internally_reserved":0,"allocated":1067429888,"data_stored":1066579859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":701418495},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,3,5,18,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20549632,"num_objects":5017,"num_object_clones":0,"num_object_copies":10034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-440,"num_objects_unfound":0,"num_objects_dirty":5017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5017,"num_write_kb":20068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":440,"num_bytes_recovered":1802240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002996"},"pg_stats":[{"pgid":"2.f","version":"41'7971","reported_seq":8044,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:15.889920+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:28:15.889920+0000","last_peered":"2024-09-17T15:28:15.889920+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:28:15.889920+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":7971,"log_dups_size":0,"ondisk_log_size":7971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32649216,"num_objects":7971,"num_object_clones":0,"num_object_copies":15942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7971,"num_write_kb":31884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3901},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'8069","reported_seq":8167,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:17.098713+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:28:17.098713+0000","last_peered":"2024-09-17T15:28:17.098713+0000","last_clean":"2024-09-17T15:28:17.098713+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:28:17.098713+0000","last_undegraded":"2024-09-17T15:28:17.098713+0000","last_fullsized":"2024-09-17T15:28:17.098713+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":8069,"log_dups_size":0,"ondisk_log_size":8069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33027014,"num_objects":8065,"num_object_clones":0,"num_object_copies":16130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8065,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6426,"num_write_kb":25704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'8065","reported_seq":8186,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:17.080572+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:28:17.080572+0000","last_peered":"2024-09-17T15:28:17.080572+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:28:17.080572+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":8065,"log_dups_size":0,"ondisk_log_size":8065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33017856,"num_objects":8061,"num_object_clones":0,"num_object_copies":16122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8061,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6222,"num_write_kb":24888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":3893}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'8129","reported_seq":8194,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:17.035564+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:28:17.035564+0000","last_peered":"2024-09-17T15:28:17.035564+0000","last_clean":"2024-09-17T15:28:17.035564+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:28:17.035564+0000","last_undegraded":"2024-09-17T15:28:17.035564+0000","last_fullsized":"2024-09-17T15:28:17.035564+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33296384,"num_objects":8129,"num_object_clones":0,"num_object_copies":16258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8129,"num_write_kb":32516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'7876","reported_seq":7999,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:15.886369+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:28:15.886369+0000","last_peered":"2024-09-17T15:28:15.886369+0000","last_clean":"2024-09-17T15:28:15.886369+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:28:15.886369+0000","last_undegraded":"2024-09-17T15:28:15.886369+0000","last_fullsized":"2024-09-17T15:28:15.886369+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":7876,"log_dups_size":0,"ondisk_log_size":7876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32231907,"num_objects":7870,"num_object_clones":0,"num_object_copies":15740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7870,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4858,"num_write_kb":19432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'8220","reported_seq":8347,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:15.864698+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:28:15.864698+0000","last_peered":"2024-09-17T15:28:15.864698+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:28:15.864698+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":8220,"log_dups_size":0,"ondisk_log_size":8220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33649123,"num_objects":8216,"num_object_clones":0,"num_object_copies":16432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8216,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4719,"num_write_kb":18876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3946},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'8079","reported_seq":8213,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:15.852854+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:28:15.852854+0000","last_peered":"2024-09-17T15:28:15.852854+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:28:15.852854+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":8079,"log_dups_size":0,"ondisk_log_size":8079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33067974,"num_objects":8075,"num_object_clones":0,"num_object_copies":16150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8075,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4848,"num_write_kb":19392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3901},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'8086","reported_seq":8209,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:15.845992+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:28:15.845992+0000","last_peered":"2024-09-17T15:28:15.845992+0000","last_clean":"2024-09-17T15:28:15.845992+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:28:15.845992+0000","last_undegraded":"2024-09-17T15:28:15.845992+0000","last_fullsized":"2024-09-17T15:28:15.845992+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":8086,"log_dups_size":0,"ondisk_log_size":8086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33112064,"num_objects":8084,"num_object_clones":0,"num_object_copies":16168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5040,"num_write_kb":20160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'8041","reported_seq":15777,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:28:17.110543+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:28:17.110543+0000","last_peered":"2024-09-17T15:28:17.110543+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:28:17.110543+0000","last_undegraded":"2024-09-17T15:28:17.110543+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":8041,"log_dups_size":0,"ondisk_log_size":8041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32907747,"num_objects":8035,"num_object_clones":0,"num_object_copies":16070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":266,"num_objects_unfound":0,"num_objects_dirty":8035,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6041,"num_write_kb":24164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3806,"num_bytes_recovered":15581667,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":266},{"shards":"2","objects":7769}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233600+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233600+0000","last_peered":"2024-09-17T15:26:22.233600+0000","last_clean":"2024-09-17T15:26:22.233600+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233600+0000","last_undegraded":"2024-09-17T15:26:22.233600+0000","last_fullsized":"2024-09-17T15:26:22.233600+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'7910","reported_seq":8035,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:15.817483+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:28:15.817483+0000","last_peered":"2024-09-17T15:28:15.817483+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:28:15.817483+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":7910,"log_dups_size":0,"ondisk_log_size":7910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32366592,"num_objects":7902,"num_object_clones":0,"num_object_copies":15804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4506,"num_write_kb":18024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3892},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":98,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412941+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412941+0000","last_peered":"2024-09-17T15:26:22.412941+0000","last_clean":"2024-09-17T15:26:22.412941+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412941+0000","last_undegraded":"2024-09-17T15:26:22.412941+0000","last_fullsized":"2024-09-17T15:26:22.412941+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'7978","reported_seq":8094,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:17.097766+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:28:17.097766+0000","last_peered":"2024-09-17T15:28:17.097766+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:28:17.097766+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":7978,"log_dups_size":0,"ondisk_log_size":7978,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32657891,"num_objects":7974,"num_object_clones":0,"num_object_copies":15948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7974,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5873,"num_write_kb":23492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":3897}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'8077","reported_seq":8190,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:17.094667+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:28:17.094667+0000","last_peered":"2024-09-17T15:28:17.094667+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:28:17.094667+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":8077,"log_dups_size":0,"ondisk_log_size":8077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33058816,"num_objects":8071,"num_object_clones":0,"num_object_copies":16142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5839,"num_write_kb":23356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":3934}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'8123","reported_seq":8253,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:15.897192+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:28:15.897192+0000","last_peered":"2024-09-17T15:28:15.897192+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:28:15.897192+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":8123,"log_dups_size":0,"ondisk_log_size":8123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33263616,"num_objects":8121,"num_object_clones":0,"num_object_copies":16242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4671,"num_write_kb":18684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3983},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'8022","reported_seq":8159,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:15.872941+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:28:15.872941+0000","last_peered":"2024-09-17T15:28:15.872941+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:28:15.872941+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":8022,"log_dups_size":0,"ondisk_log_size":8022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32825344,"num_objects":8014,"num_object_clones":0,"num_object_copies":16028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4309,"num_write_kb":17236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3857},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229061+0000","last_peered":"2024-09-17T15:26:22.229061+0000","last_clean":"2024-09-17T15:26:22.229061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229061+0000","last_undegraded":"2024-09-17T15:26:22.229061+0000","last_fullsized":"2024-09-17T15:26:22.229061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'8066","reported_seq":8193,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:15.870778+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:28:15.870778+0000","last_peered":"2024-09-17T15:28:15.870778+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:28:15.870778+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":8066,"log_dups_size":0,"ondisk_log_size":8066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33021952,"num_objects":8062,"num_object_clones":0,"num_object_copies":16124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4853,"num_write_kb":19412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3999},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'7991","reported_seq":8129,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:15.803869+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:28:15.803869+0000","last_peered":"2024-09-17T15:28:15.803869+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:28:15.803869+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":7991,"log_dups_size":0,"ondisk_log_size":7991,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32682950,"num_objects":7981,"num_object_clones":0,"num_object_copies":15962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4433,"num_write_kb":17732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3872},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":526836446,"num_objects":128631,"num_object_clones":0,"num_object_copies":257262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":266,"num_objects_unfound":0,"num_objects_dirty":128631,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":88738,"num_write_kb":354952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3806,"num_bytes_recovered":15581667,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1067520000,"data_stored":1067368254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128703,"ondisk_log_size":128703,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051623,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":649588,"kb_used_data":388836,"kb_used_omap":4,"kb_used_meta":260731,"kb_avail":93722252,"statfs":{"total":96636764160,"available":95971586048,"internally_reserved":0,"allocated":398168064,"data_stored":397901891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":266989228},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724568,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":355688,"kb_used_data":201816,"kb_used_omap":4,"kb_used_meta":153851,"kb_avail":94016152,"statfs":{"total":96636764160,"available":96272539648,"internally_reserved":0,"allocated":206659584,"data_stored":206345978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":157544107},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,10,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247851,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":722176,"kb_used_data":451760,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93649664,"statfs":{"total":96636764160,"available":95897255936,"internally_reserved":0,"allocated":462602240,"data_stored":462331990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":276885160},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,2,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":462389248,"data_stored":462353118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":206340096,"data_stored":206260610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":398790656,"data_stored":398754526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:28:20.481 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:28:21.039 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:28:21.042 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:28:21.066 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":152,"stamp":"2024-09-17T15:28:19.632401+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":526836465,"num_objects":128633,"num_object_clones":0,"num_object_copies":257266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":266,"num_objects_unfound":0,"num_objects_dirty":128633,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":88740,"num_write_kb":354954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3807,"num_bytes_recovered":15581686,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128706,"ondisk_log_size":128706,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1727452,"kb_used_data":1042412,"kb_used_omap":13,"kb_used_meta":684978,"kb_avail":281388068,"statfs":{"total":289910292480,"available":288141381632,"internally_reserved":0,"allocated":1067429888,"data_stored":1066579859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":701418495},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,3,5,18,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20549632,"num_objects":5017,"num_object_clones":0,"num_object_copies":10034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-440,"num_objects_unfound":0,"num_objects_dirty":5017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5017,"num_write_kb":20068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":440,"num_bytes_recovered":1802240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002996"},"pg_stats":[{"pgid":"2.f","version":"41'7971","reported_seq":8044,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:15.889920+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:28:15.889920+0000","last_peered":"2024-09-17T15:28:15.889920+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:28:15.889920+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":7971,"log_dups_size":0,"ondisk_log_size":7971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32649216,"num_objects":7971,"num_object_clones":0,"num_object_copies":15942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7971,"num_write_kb":31884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3901},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'8069","reported_seq":8167,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:17.098713+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:28:17.098713+0000","last_peered":"2024-09-17T15:28:17.098713+0000","last_clean":"2024-09-17T15:28:17.098713+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:28:17.098713+0000","last_undegraded":"2024-09-17T15:28:17.098713+0000","last_fullsized":"2024-09-17T15:28:17.098713+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":8069,"log_dups_size":0,"ondisk_log_size":8069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33027014,"num_objects":8065,"num_object_clones":0,"num_object_copies":16130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8065,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6426,"num_write_kb":25704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'8065","reported_seq":8186,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:17.080572+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:28:17.080572+0000","last_peered":"2024-09-17T15:28:17.080572+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:28:17.080572+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":8065,"log_dups_size":0,"ondisk_log_size":8065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33017856,"num_objects":8061,"num_object_clones":0,"num_object_copies":16122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8061,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6222,"num_write_kb":24888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":3893}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'8129","reported_seq":8194,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:17.035564+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:28:17.035564+0000","last_peered":"2024-09-17T15:28:17.035564+0000","last_clean":"2024-09-17T15:28:17.035564+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:28:17.035564+0000","last_undegraded":"2024-09-17T15:28:17.035564+0000","last_fullsized":"2024-09-17T15:28:17.035564+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33296384,"num_objects":8129,"num_object_clones":0,"num_object_copies":16258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8129,"num_write_kb":32516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'7876","reported_seq":7999,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:15.886369+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:28:15.886369+0000","last_peered":"2024-09-17T15:28:15.886369+0000","last_clean":"2024-09-17T15:28:15.886369+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:28:15.886369+0000","last_undegraded":"2024-09-17T15:28:15.886369+0000","last_fullsized":"2024-09-17T15:28:15.886369+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":7876,"log_dups_size":0,"ondisk_log_size":7876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32231907,"num_objects":7870,"num_object_clones":0,"num_object_copies":15740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7870,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4858,"num_write_kb":19432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'8220","reported_seq":8347,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:15.864698+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:28:15.864698+0000","last_peered":"2024-09-17T15:28:15.864698+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:28:15.864698+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":8220,"log_dups_size":0,"ondisk_log_size":8220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33649123,"num_objects":8216,"num_object_clones":0,"num_object_copies":16432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8216,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4719,"num_write_kb":18876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3946},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'8079","reported_seq":8213,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:15.852854+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:28:15.852854+0000","last_peered":"2024-09-17T15:28:15.852854+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:28:15.852854+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":8079,"log_dups_size":0,"ondisk_log_size":8079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33067974,"num_objects":8075,"num_object_clones":0,"num_object_copies":16150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8075,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4848,"num_write_kb":19392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3901},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'8086","reported_seq":8209,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:15.845992+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:28:15.845992+0000","last_peered":"2024-09-17T15:28:15.845992+0000","last_clean":"2024-09-17T15:28:15.845992+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:28:15.845992+0000","last_undegraded":"2024-09-17T15:28:15.845992+0000","last_fullsized":"2024-09-17T15:28:15.845992+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":8086,"log_dups_size":0,"ondisk_log_size":8086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33112064,"num_objects":8084,"num_object_clones":0,"num_object_copies":16168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5040,"num_write_kb":20160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.196680+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.196680+0000","last_peered":"2024-09-17T15:26:22.196680+0000","last_clean":"2024-09-17T15:26:22.196680+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.196680+0000","last_undegraded":"2024-09-17T15:26:22.196680+0000","last_fullsized":"2024-09-17T15:26:22.196680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'8041","reported_seq":15777,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:28:17.110543+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:28:17.110543+0000","last_peered":"2024-09-17T15:28:17.110543+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:28:17.110543+0000","last_undegraded":"2024-09-17T15:28:17.110543+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":8041,"log_dups_size":0,"ondisk_log_size":8041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32907747,"num_objects":8035,"num_object_clones":0,"num_object_copies":16070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":266,"num_objects_unfound":0,"num_objects_dirty":8035,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6041,"num_write_kb":24164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3806,"num_bytes_recovered":15581667,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":266},{"shards":"2","objects":7769}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233600+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233600+0000","last_peered":"2024-09-17T15:26:22.233600+0000","last_clean":"2024-09-17T15:26:22.233600+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233600+0000","last_undegraded":"2024-09-17T15:26:22.233600+0000","last_fullsized":"2024-09-17T15:26:22.233600+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'7910","reported_seq":8035,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:15.817483+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:28:15.817483+0000","last_peered":"2024-09-17T15:28:15.817483+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:28:15.817483+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":7910,"log_dups_size":0,"ondisk_log_size":7910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32366592,"num_objects":7902,"num_object_clones":0,"num_object_copies":15804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4506,"num_write_kb":18024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3892},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":98,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412941+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412941+0000","last_peered":"2024-09-17T15:26:22.412941+0000","last_clean":"2024-09-17T15:26:22.412941+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412941+0000","last_undegraded":"2024-09-17T15:26:22.412941+0000","last_fullsized":"2024-09-17T15:26:22.412941+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'7978","reported_seq":8094,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:17.097766+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:28:17.097766+0000","last_peered":"2024-09-17T15:28:17.097766+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:28:17.097766+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":7978,"log_dups_size":0,"ondisk_log_size":7978,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32657891,"num_objects":7974,"num_object_clones":0,"num_object_copies":15948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7974,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5873,"num_write_kb":23492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":3897}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'8077","reported_seq":8190,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:17.094667+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:28:17.094667+0000","last_peered":"2024-09-17T15:28:17.094667+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:28:17.094667+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":8077,"log_dups_size":0,"ondisk_log_size":8077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33058816,"num_objects":8071,"num_object_clones":0,"num_object_copies":16142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5839,"num_write_kb":23356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":3934}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'8123","reported_seq":8253,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:15.897192+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:28:15.897192+0000","last_peered":"2024-09-17T15:28:15.897192+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:28:15.897192+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":8123,"log_dups_size":0,"ondisk_log_size":8123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33263616,"num_objects":8121,"num_object_clones":0,"num_object_copies":16242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4671,"num_write_kb":18684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3983},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'8022","reported_seq":8159,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:15.872941+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:28:15.872941+0000","last_peered":"2024-09-17T15:28:15.872941+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:28:15.872941+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":8022,"log_dups_size":0,"ondisk_log_size":8022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32825344,"num_objects":8014,"num_object_clones":0,"num_object_copies":16028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4309,"num_write_kb":17236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3857},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229061+0000","last_peered":"2024-09-17T15:26:22.229061+0000","last_clean":"2024-09-17T15:26:22.229061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229061+0000","last_undegraded":"2024-09-17T15:26:22.229061+0000","last_fullsized":"2024-09-17T15:26:22.229061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'8066","reported_seq":8193,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:15.870778+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:28:15.870778+0000","last_peered":"2024-09-17T15:28:15.870778+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:28:15.870778+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":8066,"log_dups_size":0,"ondisk_log_size":8066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33021952,"num_objects":8062,"num_object_clones":0,"num_object_copies":16124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4853,"num_write_kb":19412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3999},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'7991","reported_seq":8129,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:15.803869+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:28:15.803869+0000","last_peered":"2024-09-17T15:28:15.803869+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:28:15.803869+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":7991,"log_dups_size":0,"ondisk_log_size":7991,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32682950,"num_objects":7981,"num_object_clones":0,"num_object_copies":15962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4433,"num_write_kb":17732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3872},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":526836446,"num_objects":128631,"num_object_clones":0,"num_object_copies":257262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":266,"num_objects_unfound":0,"num_objects_dirty":128631,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":88738,"num_write_kb":354952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3806,"num_bytes_recovered":15581667,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1067520000,"data_stored":1067368254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128703,"ondisk_log_size":128703,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051623,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":649588,"kb_used_data":388836,"kb_used_omap":4,"kb_used_meta":260731,"kb_avail":93722252,"statfs":{"total":96636764160,"available":95971586048,"internally_reserved":0,"allocated":398168064,"data_stored":397901891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":266989228},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724568,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":355688,"kb_used_data":201816,"kb_used_omap":4,"kb_used_meta":153851,"kb_avail":94016152,"statfs":{"total":96636764160,"available":96272539648,"internally_reserved":0,"allocated":206659584,"data_stored":206345978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":157544107},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,10,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247851,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":722176,"kb_used_data":451760,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93649664,"statfs":{"total":96636764160,"available":95897255936,"internally_reserved":0,"allocated":462602240,"data_stored":462331990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":276885160},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,2,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":462389248,"data_stored":462353118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":206340096,"data_stored":206260610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":398790656,"data_stored":398754526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:28:21.067 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:28:21.067 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'7971", 'reported_seq': 8044, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:15.889920+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:28:15.889920+0000', 'last_peered': '2024-09-17T15:28:15.889920+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:28:15.889920+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 7971, 'log_dups_size': 0, 'ondisk_log_size': 7971, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32649216, 'num_objects': 7971, 'num_object_clones': 0, 'num_object_copies': 15942, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4070, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7971, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7971, 'num_write_kb': 31884, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3901}, {'shards': '0,2', 'objects': 4070}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:21.067 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:28:21.067 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'8065", 'reported_seq': 8186, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:17.080572+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:28:17.080572+0000', 'last_peered': '2024-09-17T15:28:17.080572+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:28:17.080572+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 8065, 'log_dups_size': 0, 'ondisk_log_size': 8065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33017856, 'num_objects': 8061, 'num_object_clones': 0, 'num_object_copies': 16122, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4169, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8061, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6222, 'num_write_kb': 24888, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4168}, {'shards': '2', 'objects': 3893}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:28:21.067 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:28:21.067 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'8220", 'reported_seq': 8347, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:15.864698+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:28:15.864698+0000', 'last_peered': '2024-09-17T15:28:15.864698+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:28:15.864698+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 8220, 'log_dups_size': 0, 'ondisk_log_size': 8220, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33649123, 'num_objects': 8216, 'num_object_clones': 0, 'num_object_copies': 16432, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4271, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8216, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4719, 'num_write_kb': 18876, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3946}, {'shards': '0,2', 'objects': 4268}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:21.068 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:28:21.068 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'8079", 'reported_seq': 8213, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:15.852854+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:28:15.852854+0000', 'last_peered': '2024-09-17T15:28:15.852854+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:28:15.852854+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 8079, 'log_dups_size': 0, 'ondisk_log_size': 8079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33067974, 'num_objects': 8075, 'num_object_clones': 0, 'num_object_copies': 16150, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4174, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8075, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 4848, 'num_write_kb': 19392, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3901}, {'shards': '0,2', 'objects': 4174}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:21.068 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-17T15:28:21.068 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "41'8041", 'reported_seq': 15777, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:28:17.110543+0000', 'last_change': '2024-09-17T15:26:22.588242+0000', 'last_active': '2024-09-17T15:28:17.110543+0000', 'last_peered': '2024-09-17T15:28:17.110543+0000', 'last_clean': '2024-09-17T15:26:20.155646+0000', 'last_became_active': '2024-09-17T15:26:22.557742+0000', 'last_became_peered': '2024-09-17T15:26:22.557742+0000', 'last_unstale': '2024-09-17T15:28:17.110543+0000', 'last_undegraded': '2024-09-17T15:28:17.110543+0000', 'last_fullsized': '2024-09-17T15:26:22.188850+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2016", 'last_scrub_stamp': '2024-09-17T15:25:53.959011+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:53.959011+0000', 'objects_scrubbed': 1994, 'log_size': 8041, 'log_dups_size': 0, 'ondisk_log_size': 8041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 568, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32907747, 'num_objects': 8035, 'num_object_clones': 0, 'num_object_copies': 16070, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 266, 'num_objects_unfound': 0, 'num_objects_dirty': 8035, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6041, 'num_write_kb': 24164, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3806, 'num_bytes_recovered': 15581667, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 266}, {'shards': '2', 'objects': 7769}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:28:21.068 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:28:21.068 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'7910", 'reported_seq': 8035, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:15.817483+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:28:15.817483+0000', 'last_peered': '2024-09-17T15:28:15.817483+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:28:15.817483+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 7910, 'log_dups_size': 0, 'ondisk_log_size': 7910, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32366592, 'num_objects': 7902, 'num_object_clones': 0, 'num_object_copies': 15804, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4012, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7902, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4506, 'num_write_kb': 18024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3892}, {'shards': '0,2', 'objects': 4010}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:21.068 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:28:21.068 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'7978", 'reported_seq': 8094, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:17.097766+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:28:17.097766+0000', 'last_peered': '2024-09-17T15:28:17.097766+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:28:17.097766+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 7978, 'log_dups_size': 0, 'ondisk_log_size': 7978, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32657891, 'num_objects': 7974, 'num_object_clones': 0, 'num_object_copies': 15948, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4078, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7974, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5873, 'num_write_kb': 23492, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4077}, {'shards': '2', 'objects': 3897}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:28:21.068 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:28:21.068 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'8077", 'reported_seq': 8190, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:17.094667+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:28:17.094667+0000', 'last_peered': '2024-09-17T15:28:17.094667+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:28:17.094667+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 8077, 'log_dups_size': 0, 'ondisk_log_size': 8077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33058816, 'num_objects': 8071, 'num_object_clones': 0, 'num_object_copies': 16142, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8071, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5839, 'num_write_kb': 23356, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4137}, {'shards': '2', 'objects': 3934}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:28:21.069 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:28:21.069 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'8123", 'reported_seq': 8253, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:15.897192+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:28:15.897192+0000', 'last_peered': '2024-09-17T15:28:15.897192+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:28:15.897192+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 8123, 'log_dups_size': 0, 'ondisk_log_size': 8123, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33263616, 'num_objects': 8121, 'num_object_clones': 0, 'num_object_copies': 16242, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8121, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4671, 'num_write_kb': 18684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3983}, {'shards': '0,2', 'objects': 4138}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:21.069 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:28:21.069 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'8022", 'reported_seq': 8159, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:15.872941+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:28:15.872941+0000', 'last_peered': '2024-09-17T15:28:15.872941+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:28:15.872941+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 8022, 'log_dups_size': 0, 'ondisk_log_size': 8022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32825344, 'num_objects': 8014, 'num_object_clones': 0, 'num_object_copies': 16028, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4160, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8014, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4309, 'num_write_kb': 17236, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3857}, {'shards': '0,2', 'objects': 4157}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:21.069 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:28:21.069 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'8066", 'reported_seq': 8193, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:15.870778+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:28:15.870778+0000', 'last_peered': '2024-09-17T15:28:15.870778+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:28:15.870778+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 8066, 'log_dups_size': 0, 'ondisk_log_size': 8066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33021952, 'num_objects': 8062, 'num_object_clones': 0, 'num_object_copies': 16124, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4065, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8062, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4853, 'num_write_kb': 19412, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3999}, {'shards': '0,2', 'objects': 4063}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:21.069 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:28:21.069 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'7991", 'reported_seq': 8129, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:15.803869+0000', 'last_change': '2024-09-17T15:26:22.654521+0000', 'last_active': '2024-09-17T15:28:15.803869+0000', 'last_peered': '2024-09-17T15:28:15.803869+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:28:15.803869+0000', 'last_undegraded': '2024-09-17T15:26:22.307795+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 7991, 'log_dups_size': 0, 'ondisk_log_size': 7991, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32682950, 'num_objects': 7981, 'num_object_clones': 0, 'num_object_copies': 15962, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4111, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7981, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4433, 'num_write_kb': 17732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3872}, {'shards': '0,2', 'objects': 4109}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:21.069 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:28:21.319 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 162 20 131121 131101 3.16049 1.80469 0.0629413 0.0247131 2024-09-17T15:28:21.757 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:28:21.757 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45388/257266 objects degraded (17.642%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":238,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":128633,"data_bytes":526836465,"bytes_used":1768910848,"bytes_avail":288141381632,"bytes_total":289910292480,"degraded_objects":45388,"degraded_total":257266,"degraded_ratio":0.1764244012034237,"misplaced_objects":266,"misplaced_total":257266,"misplaced_ratio":0.0010339492976141425,"recovering_objects_per_sec":36,"recovering_bytes_per_sec":150149,"recovering_keys_per_sec":0,"num_objects_recovered":440,"num_bytes_recovered":1802240,"num_keys_recovered":0,"write_bytes_sec":1712041,"read_op_per_sec":0,"write_op_per_sec":417},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T15:27:19.616376+0000","services":{}},"progress_events":{}} 2024-09-17T15:28:21.776 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:28:21.776 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:28:22.321 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 163 20 131578 131558 3.15205 1.78516 0.0597079 0.0247772 2024-09-17T15:28:22.326 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:28:22.326 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:28:22.350 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2024-09-17T15:28:21.632855+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":533398257,"num_objects":130235,"num_object_clones":0,"num_object_copies":260470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":266,"num_objects_unfound":0,"num_objects_dirty":130235,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":90342,"num_write_kb":361362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3807,"num_bytes_recovered":15581686,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130308,"ondisk_log_size":130308,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1769736,"kb_used_data":1058324,"kb_used_omap":13,"kb_used_meta":711346,"kb_avail":281345784,"statfs":{"total":289910292480,"available":288098082816,"internally_reserved":0,"allocated":1083723776,"data_stored":1082873747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":728419327},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,3,6,11,13,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27111424,"num_objects":6619,"num_object_clones":0,"num_object_copies":13238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-440,"num_objects_unfound":0,"num_objects_dirty":6619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6619,"num_write_kb":26476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":440,"num_bytes_recovered":1802240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002937"},"pg_stats":[{"pgid":"2.f","version":"41'8129","reported_seq":8202,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:20.945097+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:28:20.945097+0000","last_peered":"2024-09-17T15:28:20.945097+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:28:20.945097+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33296384,"num_objects":8129,"num_object_clones":0,"num_object_copies":16258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8129,"num_write_kb":32516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4059},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'8069","reported_seq":8167,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:17.098713+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:28:17.098713+0000","last_peered":"2024-09-17T15:28:17.098713+0000","last_clean":"2024-09-17T15:28:17.098713+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:28:17.098713+0000","last_undegraded":"2024-09-17T15:28:17.098713+0000","last_fullsized":"2024-09-17T15:28:17.098713+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":8069,"log_dups_size":0,"ondisk_log_size":8069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33027014,"num_objects":8065,"num_object_clones":0,"num_object_copies":16130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8065,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6426,"num_write_kb":25704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'8065","reported_seq":8186,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:17.080572+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:28:17.080572+0000","last_peered":"2024-09-17T15:28:17.080572+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:28:17.080572+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":8065,"log_dups_size":0,"ondisk_log_size":8065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33017856,"num_objects":8061,"num_object_clones":0,"num_object_copies":16122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8061,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6222,"num_write_kb":24888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":3893}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'8129","reported_seq":8194,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:17.035564+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:28:17.035564+0000","last_peered":"2024-09-17T15:28:17.035564+0000","last_clean":"2024-09-17T15:28:17.035564+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:28:17.035564+0000","last_undegraded":"2024-09-17T15:28:17.035564+0000","last_fullsized":"2024-09-17T15:28:17.035564+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33296384,"num_objects":8129,"num_object_clones":0,"num_object_copies":16258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8129,"num_write_kb":32516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'8061","reported_seq":8185,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:20.922660+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:28:20.922660+0000","last_peered":"2024-09-17T15:28:20.922660+0000","last_clean":"2024-09-17T15:28:20.922660+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:28:20.922660+0000","last_undegraded":"2024-09-17T15:28:20.922660+0000","last_fullsized":"2024-09-17T15:28:20.922660+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":8061,"log_dups_size":0,"ondisk_log_size":8061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32989667,"num_objects":8055,"num_object_clones":0,"num_object_copies":16110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8055,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5043,"num_write_kb":20172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'8365","reported_seq":8495,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:20.895671+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:28:20.895671+0000","last_peered":"2024-09-17T15:28:20.895671+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:28:20.895671+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":8365,"log_dups_size":0,"ondisk_log_size":8365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34243043,"num_objects":8361,"num_object_clones":0,"num_object_copies":16722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8361,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4864,"num_write_kb":19456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4091},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'8235","reported_seq":8369,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:20.856839+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:28:20.856839+0000","last_peered":"2024-09-17T15:28:20.856839+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:28:20.856839+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":8235,"log_dups_size":0,"ondisk_log_size":8235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33706950,"num_objects":8231,"num_object_clones":0,"num_object_copies":16462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8231,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5004,"num_write_kb":20016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4057},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'8249","reported_seq":8372,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:20.916249+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:28:20.916249+0000","last_peered":"2024-09-17T15:28:20.916249+0000","last_clean":"2024-09-17T15:28:20.916249+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:28:20.916249+0000","last_undegraded":"2024-09-17T15:28:20.916249+0000","last_fullsized":"2024-09-17T15:28:20.916249+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":8249,"log_dups_size":0,"ondisk_log_size":8249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33779712,"num_objects":8247,"num_object_clones":0,"num_object_copies":16494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5203,"num_write_kb":20812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.197413+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:26:22.197413+0000","last_peered":"2024-09-17T15:26:22.197413+0000","last_clean":"2024-09-17T15:26:22.197413+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:26:22.197413+0000","last_undegraded":"2024-09-17T15:26:22.197413+0000","last_fullsized":"2024-09-17T15:26:22.197413+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'8041","reported_seq":15777,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:28:17.110543+0000","last_change":"2024-09-17T15:26:22.588242+0000","last_active":"2024-09-17T15:28:17.110543+0000","last_peered":"2024-09-17T15:28:17.110543+0000","last_clean":"2024-09-17T15:26:20.155646+0000","last_became_active":"2024-09-17T15:26:22.557742+0000","last_became_peered":"2024-09-17T15:26:22.557742+0000","last_unstale":"2024-09-17T15:28:17.110543+0000","last_undegraded":"2024-09-17T15:28:17.110543+0000","last_fullsized":"2024-09-17T15:26:22.188850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":8041,"log_dups_size":0,"ondisk_log_size":8041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32907747,"num_objects":8035,"num_object_clones":0,"num_object_copies":16070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":266,"num_objects_unfound":0,"num_objects_dirty":8035,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6041,"num_write_kb":24164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3806,"num_bytes_recovered":15581667,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":266},{"shards":"2","objects":7769}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233600+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233600+0000","last_peered":"2024-09-17T15:26:22.233600+0000","last_clean":"2024-09-17T15:26:22.233600+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233600+0000","last_undegraded":"2024-09-17T15:26:22.233600+0000","last_fullsized":"2024-09-17T15:26:22.233600+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'8070","reported_seq":8196,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:20.895799+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:28:20.895799+0000","last_peered":"2024-09-17T15:28:20.895799+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:28:20.895799+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":8070,"log_dups_size":0,"ondisk_log_size":8070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33021952,"num_objects":8062,"num_object_clones":0,"num_object_copies":16124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4666,"num_write_kb":18664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4052},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":98,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412941+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412941+0000","last_peered":"2024-09-17T15:26:22.412941+0000","last_clean":"2024-09-17T15:26:22.412941+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412941+0000","last_undegraded":"2024-09-17T15:26:22.412941+0000","last_fullsized":"2024-09-17T15:26:22.412941+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'7978","reported_seq":8094,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:17.097766+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:28:17.097766+0000","last_peered":"2024-09-17T15:28:17.097766+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:28:17.097766+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":7978,"log_dups_size":0,"ondisk_log_size":7978,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32657891,"num_objects":7974,"num_object_clones":0,"num_object_copies":15948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7974,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5873,"num_write_kb":23492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":3897}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":72,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.249666+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:26:22.249666+0000","last_peered":"2024-09-17T15:26:22.249666+0000","last_clean":"2024-09-17T15:26:22.249666+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:26:22.249666+0000","last_undegraded":"2024-09-17T15:26:22.249666+0000","last_fullsized":"2024-09-17T15:26:22.249666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'8077","reported_seq":8190,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:17.094667+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:28:17.094667+0000","last_peered":"2024-09-17T15:28:17.094667+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:28:17.094667+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":8077,"log_dups_size":0,"ondisk_log_size":8077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33058816,"num_objects":8071,"num_object_clones":0,"num_object_copies":16142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5839,"num_write_kb":23356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":3934}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'8264","reported_seq":8395,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:20.930252+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:28:20.930252+0000","last_peered":"2024-09-17T15:28:20.930252+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:28:20.930252+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":8264,"log_dups_size":0,"ondisk_log_size":8264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33841152,"num_objects":8262,"num_object_clones":0,"num_object_copies":16524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4812,"num_write_kb":19248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4124},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.387611+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:26:22.387611+0000","last_peered":"2024-09-17T15:26:22.387611+0000","last_clean":"2024-09-17T15:26:22.387611+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:26:22.387611+0000","last_undegraded":"2024-09-17T15:26:22.387611+0000","last_fullsized":"2024-09-17T15:26:22.387611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'8188","reported_seq":8325,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:20.810920+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:28:20.810920+0000","last_peered":"2024-09-17T15:28:20.810920+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:28:20.810920+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":8188,"log_dups_size":0,"ondisk_log_size":8188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33505280,"num_objects":8180,"num_object_clones":0,"num_object_copies":16360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4475,"num_write_kb":17900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4023},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229061+0000","last_peered":"2024-09-17T15:26:22.229061+0000","last_clean":"2024-09-17T15:26:22.229061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229061+0000","last_undegraded":"2024-09-17T15:26:22.229061+0000","last_fullsized":"2024-09-17T15:26:22.229061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'8239","reported_seq":8366,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:20.857494+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:28:20.857494+0000","last_peered":"2024-09-17T15:28:20.857494+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:28:20.857494+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":8239,"log_dups_size":0,"ondisk_log_size":8239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33730560,"num_objects":8235,"num_object_clones":0,"num_object_copies":16470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5026,"num_write_kb":20104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4172},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.296750+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:26:22.296750+0000","last_peered":"2024-09-17T15:26:22.296750+0000","last_clean":"2024-09-17T15:26:22.296750+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:26:22.296750+0000","last_undegraded":"2024-09-17T15:26:22.296750+0000","last_fullsized":"2024-09-17T15:26:22.296750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.192638+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:26:22.192638+0000","last_peered":"2024-09-17T15:26:22.192638+0000","last_clean":"2024-09-17T15:26:22.192638+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:26:22.192638+0000","last_undegraded":"2024-09-17T15:26:22.192638+0000","last_fullsized":"2024-09-17T15:26:22.192638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'8146","reported_seq":8284,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:20.857159+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:28:20.857159+0000","last_peered":"2024-09-17T15:28:20.857159+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:28:20.857159+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":8146,"log_dups_size":0,"ondisk_log_size":8146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33317830,"num_objects":8136,"num_object_clones":0,"num_object_copies":16272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4588,"num_write_kb":18352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4027},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":533398238,"num_objects":130233,"num_object_clones":0,"num_object_copies":260466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":266,"num_objects_unfound":0,"num_objects_dirty":130233,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":90340,"num_write_kb":361360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3806,"num_bytes_recovered":15581667,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1084063744,"data_stored":1083911998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130305,"ondisk_log_size":130305,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051623,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":649588,"kb_used_data":388836,"kb_used_omap":4,"kb_used_meta":260731,"kb_avail":93722252,"statfs":{"total":96636764160,"available":95971586048,"internally_reserved":0,"allocated":398168064,"data_stored":397901891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":266989228},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724569,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":386196,"kb_used_data":210116,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93985644,"statfs":{"total":96636764160,"available":96241299456,"internally_reserved":0,"allocated":215158784,"data_stored":214845178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":180285099},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2,4,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247852,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":733952,"kb_used_data":459372,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93637888,"statfs":{"total":96636764160,"available":95885197312,"internally_reserved":0,"allocated":470396928,"data_stored":470126678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":281145000},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,6,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":470245376,"data_stored":470209246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":215027712,"data_stored":214948226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":398790656,"data_stored":398754526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:28:23.319 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 164 20 132059 132039 3.14429 1.87891 0.0084573 0.0247992 2024-09-17T15:28:24.319 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 165 20 132349 132329 3.1321 1.13281 0.0220692 0.0249363 2024-09-17T15:28:25.319 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 166 20 132866 132846 3.1254 2.01953 0.0305896 0.0249889 2024-09-17T15:28:25.352 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:28:25.879 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:28:25.881 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:28:25.906 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2024-09-17T15:28:25.633709+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":537387761,"num_objects":131209,"num_object_clones":0,"num_object_copies":262418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":131209,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":91316,"num_write_kb":365258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4073,"num_bytes_recovered":16671222,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":131282,"ondisk_log_size":131282,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1778656,"kb_used_data":1071404,"kb_used_omap":13,"kb_used_meta":707186,"kb_avail":281336864,"statfs":{"total":289910292480,"available":288088948736,"internally_reserved":0,"allocated":1097117696,"data_stored":1096257266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":724159487},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,5,8,12,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":10551296,"num_objects":2576,"num_object_clones":0,"num_object_copies":5152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-266,"num_objects_unfound":0,"num_objects_dirty":2576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2576,"num_write_kb":10304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":1089536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001815"},"pg_stats":[{"pgid":"2.f","version":"41'8129","reported_seq":8202,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:20.945097+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:28:20.945097+0000","last_peered":"2024-09-17T15:28:20.945097+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:28:20.945097+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33296384,"num_objects":8129,"num_object_clones":0,"num_object_copies":16258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8129,"num_write_kb":32516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4059},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'8209","reported_seq":8307,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:21.957151+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:28:21.957151+0000","last_peered":"2024-09-17T15:28:21.957151+0000","last_clean":"2024-09-17T15:28:21.957151+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:28:21.957151+0000","last_undegraded":"2024-09-17T15:28:21.957151+0000","last_fullsized":"2024-09-17T15:28:21.957151+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":8209,"log_dups_size":0,"ondisk_log_size":8209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33600454,"num_objects":8205,"num_object_clones":0,"num_object_copies":16410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8205,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6566,"num_write_kb":26264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'8206","reported_seq":8327,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:22.074820+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:28:22.074820+0000","last_peered":"2024-09-17T15:28:22.074820+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:28:22.074820+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":8206,"log_dups_size":0,"ondisk_log_size":8206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33595392,"num_objects":8202,"num_object_clones":0,"num_object_copies":16404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6363,"num_write_kb":25452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":4034}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'8298","reported_seq":8363,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:22.049411+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:28:22.049411+0000","last_peered":"2024-09-17T15:28:22.049411+0000","last_clean":"2024-09-17T15:28:22.049411+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:28:22.049411+0000","last_undegraded":"2024-09-17T15:28:22.049411+0000","last_fullsized":"2024-09-17T15:28:22.049411+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":8298,"log_dups_size":0,"ondisk_log_size":8298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33988608,"num_objects":8298,"num_object_clones":0,"num_object_copies":16596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8298,"num_write_kb":33192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'8061","reported_seq":8185,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:20.922660+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:28:20.922660+0000","last_peered":"2024-09-17T15:28:20.922660+0000","last_clean":"2024-09-17T15:28:20.922660+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:28:20.922660+0000","last_undegraded":"2024-09-17T15:28:20.922660+0000","last_fullsized":"2024-09-17T15:28:20.922660+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":8061,"log_dups_size":0,"ondisk_log_size":8061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32989667,"num_objects":8055,"num_object_clones":0,"num_object_copies":16110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8055,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5043,"num_write_kb":20172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'8365","reported_seq":8495,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:20.895671+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:28:20.895671+0000","last_peered":"2024-09-17T15:28:20.895671+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:28:20.895671+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":8365,"log_dups_size":0,"ondisk_log_size":8365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34243043,"num_objects":8361,"num_object_clones":0,"num_object_copies":16722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8361,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4864,"num_write_kb":19456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4091},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'8235","reported_seq":8369,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:20.856839+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:28:20.856839+0000","last_peered":"2024-09-17T15:28:20.856839+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:28:20.856839+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":8235,"log_dups_size":0,"ondisk_log_size":8235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33706950,"num_objects":8231,"num_object_clones":0,"num_object_copies":16462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8231,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5004,"num_write_kb":20016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4057},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'8249","reported_seq":8372,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:20.916249+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:28:20.916249+0000","last_peered":"2024-09-17T15:28:20.916249+0000","last_clean":"2024-09-17T15:28:20.916249+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:28:20.916249+0000","last_undegraded":"2024-09-17T15:28:20.916249+0000","last_fullsized":"2024-09-17T15:28:20.916249+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":8249,"log_dups_size":0,"ondisk_log_size":8249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33779712,"num_objects":8247,"num_object_clones":0,"num_object_copies":16494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5203,"num_write_kb":20812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8287","reported_seq":16569,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.173008+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:28:25.173008+0000","last_peered":"2024-09-17T15:28:25.173008+0000","last_clean":"2024-09-17T15:28:25.173008+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:28:25.173008+0000","last_undegraded":"2024-09-17T15:28:25.173008+0000","last_fullsized":"2024-09-17T15:28:25.173008+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":8287,"log_dups_size":0,"ondisk_log_size":8287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33915363,"num_objects":8281,"num_object_clones":0,"num_object_copies":16562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8281,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6287,"num_write_kb":25148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233600+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233600+0000","last_peered":"2024-09-17T15:26:22.233600+0000","last_clean":"2024-09-17T15:26:22.233600+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233600+0000","last_undegraded":"2024-09-17T15:26:22.233600+0000","last_fullsized":"2024-09-17T15:26:22.233600+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'8070","reported_seq":8196,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:20.895799+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:28:20.895799+0000","last_peered":"2024-09-17T15:28:20.895799+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:28:20.895799+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":8070,"log_dups_size":0,"ondisk_log_size":8070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33021952,"num_objects":8062,"num_object_clones":0,"num_object_copies":16124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4666,"num_write_kb":18664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4052},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":98,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412941+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412941+0000","last_peered":"2024-09-17T15:26:22.412941+0000","last_clean":"2024-09-17T15:26:22.412941+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412941+0000","last_undegraded":"2024-09-17T15:26:22.412941+0000","last_fullsized":"2024-09-17T15:26:22.412941+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'8111","reported_seq":8227,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:21.960258+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:28:21.960258+0000","last_peered":"2024-09-17T15:28:21.960258+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:28:21.960258+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":8111,"log_dups_size":0,"ondisk_log_size":8111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33202659,"num_objects":8107,"num_object_clones":0,"num_object_copies":16214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8107,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6006,"num_write_kb":24024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":4030}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'8222","reported_seq":8336,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:21.912784+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:28:21.912784+0000","last_peered":"2024-09-17T15:28:21.912784+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:28:21.912784+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":8222,"log_dups_size":0,"ondisk_log_size":8222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33652736,"num_objects":8216,"num_object_clones":0,"num_object_copies":16432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5984,"num_write_kb":23936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":4079}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'8264","reported_seq":8395,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:20.930252+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:28:20.930252+0000","last_peered":"2024-09-17T15:28:20.930252+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:28:20.930252+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":8264,"log_dups_size":0,"ondisk_log_size":8264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33841152,"num_objects":8262,"num_object_clones":0,"num_object_copies":16524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4812,"num_write_kb":19248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4124},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'8188","reported_seq":8325,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:20.810920+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:28:20.810920+0000","last_peered":"2024-09-17T15:28:20.810920+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:28:20.810920+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":8188,"log_dups_size":0,"ondisk_log_size":8188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33505280,"num_objects":8180,"num_object_clones":0,"num_object_copies":16360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4475,"num_write_kb":17900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4023},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229061+0000","last_peered":"2024-09-17T15:26:22.229061+0000","last_clean":"2024-09-17T15:26:22.229061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229061+0000","last_undegraded":"2024-09-17T15:26:22.229061+0000","last_fullsized":"2024-09-17T15:26:22.229061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'8239","reported_seq":8366,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:20.857494+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:28:20.857494+0000","last_peered":"2024-09-17T15:28:20.857494+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:28:20.857494+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":8239,"log_dups_size":0,"ondisk_log_size":8239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33730560,"num_objects":8235,"num_object_clones":0,"num_object_copies":16470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5026,"num_write_kb":20104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4172},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":71,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806516+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806516+0000","last_peered":"2024-09-17T15:28:23.806516+0000","last_clean":"2024-09-17T15:28:23.806516+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806516+0000","last_undegraded":"2024-09-17T15:28:23.806516+0000","last_fullsized":"2024-09-17T15:28:23.806516+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'8146","reported_seq":8284,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:20.857159+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:28:20.857159+0000","last_peered":"2024-09-17T15:28:20.857159+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:28:20.857159+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":8146,"log_dups_size":0,"ondisk_log_size":8146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33317830,"num_objects":8136,"num_object_clones":0,"num_object_copies":16272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4588,"num_write_kb":18352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4027},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":537387742,"num_objects":131207,"num_object_clones":0,"num_object_copies":262414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":131207,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":91314,"num_write_kb":365256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1096716288,"data_stored":1096564542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":131279,"ondisk_log_size":131279,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051624,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":642960,"kb_used_data":394688,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93728880,"statfs":{"total":96636764160,"available":95978373120,"internally_reserved":0,"allocated":404160512,"data_stored":403894339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":254209708},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724570,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":401744,"kb_used_data":217344,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93970096,"statfs":{"total":96636764160,"available":96225378304,"internally_reserved":0,"allocated":222560256,"data_stored":222236249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":188804779},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,3,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247852,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":733952,"kb_used_data":459372,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93637888,"statfs":{"total":96636764160,"available":95885197312,"internally_reserved":0,"allocated":470396928,"data_stored":470126678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":281145000},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,6,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":470245376,"data_stored":470209246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":222629888,"data_stored":222550402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":403841024,"data_stored":403804894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:28:25.908 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:28:26.319 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 167 20 133318 133298 3.11726 1.76562 0.0216434 0.025055 2024-09-17T15:28:26.454 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:28:26.455 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:28:26.483 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2024-09-17T15:28:25.633709+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":537387761,"num_objects":131209,"num_object_clones":0,"num_object_copies":262418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":131209,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":91316,"num_write_kb":365258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4073,"num_bytes_recovered":16671222,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":131282,"ondisk_log_size":131282,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1778656,"kb_used_data":1071404,"kb_used_omap":13,"kb_used_meta":707186,"kb_avail":281336864,"statfs":{"total":289910292480,"available":288088948736,"internally_reserved":0,"allocated":1097117696,"data_stored":1096257266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":724159487},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,5,8,12,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":10551296,"num_objects":2576,"num_object_clones":0,"num_object_copies":5152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-266,"num_objects_unfound":0,"num_objects_dirty":2576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2576,"num_write_kb":10304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":1089536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001815"},"pg_stats":[{"pgid":"2.f","version":"41'8129","reported_seq":8202,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:20.945097+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:28:20.945097+0000","last_peered":"2024-09-17T15:28:20.945097+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:28:20.945097+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33296384,"num_objects":8129,"num_object_clones":0,"num_object_copies":16258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8129,"num_write_kb":32516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4059},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'8209","reported_seq":8307,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:21.957151+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:28:21.957151+0000","last_peered":"2024-09-17T15:28:21.957151+0000","last_clean":"2024-09-17T15:28:21.957151+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:28:21.957151+0000","last_undegraded":"2024-09-17T15:28:21.957151+0000","last_fullsized":"2024-09-17T15:28:21.957151+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":8209,"log_dups_size":0,"ondisk_log_size":8209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33600454,"num_objects":8205,"num_object_clones":0,"num_object_copies":16410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8205,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6566,"num_write_kb":26264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'8206","reported_seq":8327,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:22.074820+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:28:22.074820+0000","last_peered":"2024-09-17T15:28:22.074820+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:28:22.074820+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":8206,"log_dups_size":0,"ondisk_log_size":8206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33595392,"num_objects":8202,"num_object_clones":0,"num_object_copies":16404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6363,"num_write_kb":25452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":4034}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'8298","reported_seq":8363,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:22.049411+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:28:22.049411+0000","last_peered":"2024-09-17T15:28:22.049411+0000","last_clean":"2024-09-17T15:28:22.049411+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:28:22.049411+0000","last_undegraded":"2024-09-17T15:28:22.049411+0000","last_fullsized":"2024-09-17T15:28:22.049411+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":8298,"log_dups_size":0,"ondisk_log_size":8298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33988608,"num_objects":8298,"num_object_clones":0,"num_object_copies":16596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8298,"num_write_kb":33192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'8061","reported_seq":8185,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:20.922660+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:28:20.922660+0000","last_peered":"2024-09-17T15:28:20.922660+0000","last_clean":"2024-09-17T15:28:20.922660+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:28:20.922660+0000","last_undegraded":"2024-09-17T15:28:20.922660+0000","last_fullsized":"2024-09-17T15:28:20.922660+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":8061,"log_dups_size":0,"ondisk_log_size":8061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32989667,"num_objects":8055,"num_object_clones":0,"num_object_copies":16110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8055,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5043,"num_write_kb":20172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'8365","reported_seq":8495,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:20.895671+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:28:20.895671+0000","last_peered":"2024-09-17T15:28:20.895671+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:28:20.895671+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":8365,"log_dups_size":0,"ondisk_log_size":8365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34243043,"num_objects":8361,"num_object_clones":0,"num_object_copies":16722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8361,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4864,"num_write_kb":19456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4091},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'8235","reported_seq":8369,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:20.856839+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:28:20.856839+0000","last_peered":"2024-09-17T15:28:20.856839+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:28:20.856839+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":8235,"log_dups_size":0,"ondisk_log_size":8235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33706950,"num_objects":8231,"num_object_clones":0,"num_object_copies":16462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8231,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5004,"num_write_kb":20016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4057},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'8249","reported_seq":8372,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:28:20.916249+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:28:20.916249+0000","last_peered":"2024-09-17T15:28:20.916249+0000","last_clean":"2024-09-17T15:28:20.916249+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:28:20.916249+0000","last_undegraded":"2024-09-17T15:28:20.916249+0000","last_fullsized":"2024-09-17T15:28:20.916249+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":8249,"log_dups_size":0,"ondisk_log_size":8249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33779712,"num_objects":8247,"num_object_clones":0,"num_object_copies":16494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5203,"num_write_kb":20812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8287","reported_seq":16569,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.173008+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:28:25.173008+0000","last_peered":"2024-09-17T15:28:25.173008+0000","last_clean":"2024-09-17T15:28:25.173008+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:28:25.173008+0000","last_undegraded":"2024-09-17T15:28:25.173008+0000","last_fullsized":"2024-09-17T15:28:25.173008+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":8287,"log_dups_size":0,"ondisk_log_size":8287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33915363,"num_objects":8281,"num_object_clones":0,"num_object_copies":16562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8281,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6287,"num_write_kb":25148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.233600+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:26:22.233600+0000","last_peered":"2024-09-17T15:26:22.233600+0000","last_clean":"2024-09-17T15:26:22.233600+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:26:22.233600+0000","last_undegraded":"2024-09-17T15:26:22.233600+0000","last_fullsized":"2024-09-17T15:26:22.233600+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'8070","reported_seq":8196,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:20.895799+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:28:20.895799+0000","last_peered":"2024-09-17T15:28:20.895799+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:28:20.895799+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":8070,"log_dups_size":0,"ondisk_log_size":8070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33021952,"num_objects":8062,"num_object_clones":0,"num_object_copies":16124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4666,"num_write_kb":18664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4052},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":98,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.412941+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:26:22.412941+0000","last_peered":"2024-09-17T15:26:22.412941+0000","last_clean":"2024-09-17T15:26:22.412941+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:26:22.412941+0000","last_undegraded":"2024-09-17T15:26:22.412941+0000","last_fullsized":"2024-09-17T15:26:22.412941+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'8111","reported_seq":8227,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:21.960258+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:28:21.960258+0000","last_peered":"2024-09-17T15:28:21.960258+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:28:21.960258+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":8111,"log_dups_size":0,"ondisk_log_size":8111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33202659,"num_objects":8107,"num_object_clones":0,"num_object_copies":16214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8107,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6006,"num_write_kb":24024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":4030}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'8222","reported_seq":8336,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:21.912784+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:28:21.912784+0000","last_peered":"2024-09-17T15:28:21.912784+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:28:21.912784+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":8222,"log_dups_size":0,"ondisk_log_size":8222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33652736,"num_objects":8216,"num_object_clones":0,"num_object_copies":16432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5984,"num_write_kb":23936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":4079}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'8264","reported_seq":8395,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:20.930252+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:28:20.930252+0000","last_peered":"2024-09-17T15:28:20.930252+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:28:20.930252+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":8264,"log_dups_size":0,"ondisk_log_size":8264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33841152,"num_objects":8262,"num_object_clones":0,"num_object_copies":16524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4812,"num_write_kb":19248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4124},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'8188","reported_seq":8325,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:20.810920+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:28:20.810920+0000","last_peered":"2024-09-17T15:28:20.810920+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:28:20.810920+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":8188,"log_dups_size":0,"ondisk_log_size":8188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33505280,"num_objects":8180,"num_object_clones":0,"num_object_copies":16360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4475,"num_write_kb":17900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4023},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T15:26:22.229061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:26:22.229061+0000","last_peered":"2024-09-17T15:26:22.229061+0000","last_clean":"2024-09-17T15:26:22.229061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:26:22.229061+0000","last_undegraded":"2024-09-17T15:26:22.229061+0000","last_fullsized":"2024-09-17T15:26:22.229061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'8239","reported_seq":8366,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:20.857494+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:28:20.857494+0000","last_peered":"2024-09-17T15:28:20.857494+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:28:20.857494+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":8239,"log_dups_size":0,"ondisk_log_size":8239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33730560,"num_objects":8235,"num_object_clones":0,"num_object_copies":16470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5026,"num_write_kb":20104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4172},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":71,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806516+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806516+0000","last_peered":"2024-09-17T15:28:23.806516+0000","last_clean":"2024-09-17T15:28:23.806516+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806516+0000","last_undegraded":"2024-09-17T15:28:23.806516+0000","last_fullsized":"2024-09-17T15:28:23.806516+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'8146","reported_seq":8284,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:20.857159+0000","last_change":"2024-09-17T15:26:22.654521+0000","last_active":"2024-09-17T15:28:20.857159+0000","last_peered":"2024-09-17T15:28:20.857159+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:28:20.857159+0000","last_undegraded":"2024-09-17T15:26:22.307795+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":8146,"log_dups_size":0,"ondisk_log_size":8146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33317830,"num_objects":8136,"num_object_clones":0,"num_object_copies":16272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4588,"num_write_kb":18352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4027},{"shards":"0,2","objects":4109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":537387742,"num_objects":131207,"num_object_clones":0,"num_object_copies":262414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45388,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":131207,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":91314,"num_write_kb":365256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1096716288,"data_stored":1096564542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":131279,"ondisk_log_size":131279,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051624,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":642960,"kb_used_data":394688,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93728880,"statfs":{"total":96636764160,"available":95978373120,"internally_reserved":0,"allocated":404160512,"data_stored":403894339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":254209708},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724570,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":401744,"kb_used_data":217344,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93970096,"statfs":{"total":96636764160,"available":96225378304,"internally_reserved":0,"allocated":222560256,"data_stored":222236249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":188804779},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,3,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247852,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":733952,"kb_used_data":459372,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93637888,"statfs":{"total":96636764160,"available":95885197312,"internally_reserved":0,"allocated":470396928,"data_stored":470126678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":281145000},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,6,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":470245376,"data_stored":470209246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":222629888,"data_stored":222550402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":403841024,"data_stored":403804894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:28:26.485 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:28:26.485 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'8129", 'reported_seq': 8202, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:20.945097+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:28:20.945097+0000', 'last_peered': '2024-09-17T15:28:20.945097+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:28:20.945097+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 8129, 'log_dups_size': 0, 'ondisk_log_size': 8129, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33296384, 'num_objects': 8129, 'num_object_clones': 0, 'num_object_copies': 16258, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4070, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8129, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8129, 'num_write_kb': 32516, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4059}, {'shards': '0,2', 'objects': 4070}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:26.485 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:28:26.485 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'8206", 'reported_seq': 8327, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:22.074820+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:28:22.074820+0000', 'last_peered': '2024-09-17T15:28:22.074820+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:28:22.074820+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 8206, 'log_dups_size': 0, 'ondisk_log_size': 8206, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33595392, 'num_objects': 8202, 'num_object_clones': 0, 'num_object_copies': 16404, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4169, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8202, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6363, 'num_write_kb': 25452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4168}, {'shards': '2', 'objects': 4034}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:28:26.485 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:28:26.485 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'8365", 'reported_seq': 8495, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:20.895671+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:28:20.895671+0000', 'last_peered': '2024-09-17T15:28:20.895671+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:28:20.895671+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 8365, 'log_dups_size': 0, 'ondisk_log_size': 8365, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34243043, 'num_objects': 8361, 'num_object_clones': 0, 'num_object_copies': 16722, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4271, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8361, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4864, 'num_write_kb': 19456, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4091}, {'shards': '0,2', 'objects': 4268}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:26.485 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:28:26.485 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'8235", 'reported_seq': 8369, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:20.856839+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:28:20.856839+0000', 'last_peered': '2024-09-17T15:28:20.856839+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:28:20.856839+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 8235, 'log_dups_size': 0, 'ondisk_log_size': 8235, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33706950, 'num_objects': 8231, 'num_object_clones': 0, 'num_object_copies': 16462, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4174, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8231, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 5004, 'num_write_kb': 20016, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4057}, {'shards': '0,2', 'objects': 4174}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:26.486 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:28:26.486 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'8070", 'reported_seq': 8196, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:20.895799+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:28:20.895799+0000', 'last_peered': '2024-09-17T15:28:20.895799+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:28:20.895799+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 8070, 'log_dups_size': 0, 'ondisk_log_size': 8070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33021952, 'num_objects': 8062, 'num_object_clones': 0, 'num_object_copies': 16124, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4012, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8062, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4666, 'num_write_kb': 18664, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4052}, {'shards': '0,2', 'objects': 4010}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:26.486 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:28:26.486 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'8111", 'reported_seq': 8227, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:21.960258+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:28:21.960258+0000', 'last_peered': '2024-09-17T15:28:21.960258+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:28:21.960258+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 8111, 'log_dups_size': 0, 'ondisk_log_size': 8111, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33202659, 'num_objects': 8107, 'num_object_clones': 0, 'num_object_copies': 16214, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4078, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8107, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6006, 'num_write_kb': 24024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4077}, {'shards': '2', 'objects': 4030}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:28:26.486 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:28:26.486 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'8222", 'reported_seq': 8336, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:21.912784+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:28:21.912784+0000', 'last_peered': '2024-09-17T15:28:21.912784+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:28:21.912784+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 8222, 'log_dups_size': 0, 'ondisk_log_size': 8222, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33652736, 'num_objects': 8216, 'num_object_clones': 0, 'num_object_copies': 16432, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8216, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5984, 'num_write_kb': 23936, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4137}, {'shards': '2', 'objects': 4079}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:28:26.486 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:28:26.486 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'8264", 'reported_seq': 8395, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:20.930252+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:28:20.930252+0000', 'last_peered': '2024-09-17T15:28:20.930252+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:28:20.930252+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 8264, 'log_dups_size': 0, 'ondisk_log_size': 8264, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33841152, 'num_objects': 8262, 'num_object_clones': 0, 'num_object_copies': 16524, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8262, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4812, 'num_write_kb': 19248, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4124}, {'shards': '0,2', 'objects': 4138}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:26.486 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:28:26.486 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'8188", 'reported_seq': 8325, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:20.810920+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:28:20.810920+0000', 'last_peered': '2024-09-17T15:28:20.810920+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:28:20.810920+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 8188, 'log_dups_size': 0, 'ondisk_log_size': 8188, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33505280, 'num_objects': 8180, 'num_object_clones': 0, 'num_object_copies': 16360, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4160, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8180, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4475, 'num_write_kb': 17900, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4023}, {'shards': '0,2', 'objects': 4157}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:26.487 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:28:26.487 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'8239", 'reported_seq': 8366, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:20.857494+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:28:20.857494+0000', 'last_peered': '2024-09-17T15:28:20.857494+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:28:20.857494+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 8239, 'log_dups_size': 0, 'ondisk_log_size': 8239, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33730560, 'num_objects': 8235, 'num_object_clones': 0, 'num_object_copies': 16470, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4065, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8235, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5026, 'num_write_kb': 20104, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4172}, {'shards': '0,2', 'objects': 4063}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:26.487 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:28:26.487 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'8146", 'reported_seq': 8284, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:20.857159+0000', 'last_change': '2024-09-17T15:26:22.654521+0000', 'last_active': '2024-09-17T15:28:20.857159+0000', 'last_peered': '2024-09-17T15:28:20.857159+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:28:20.857159+0000', 'last_undegraded': '2024-09-17T15:26:22.307795+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 8146, 'log_dups_size': 0, 'ondisk_log_size': 8146, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33317830, 'num_objects': 8136, 'num_object_clones': 0, 'num_object_copies': 16272, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4111, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8136, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4588, 'num_write_kb': 18352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4027}, {'shards': '0,2', 'objects': 4109}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:26.487 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:28:27.170 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:28:27.170 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45388/262418 objects degraded (17.296%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":244,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11}],"num_pgs":24,"num_pools":2,"num_objects":131209,"data_bytes":537387761,"bytes_used":1821343744,"bytes_avail":288088948736,"bytes_total":289910292480,"degraded_objects":45388,"degraded_total":262418,"degraded_ratio":0.1729606962936994,"recovering_objects_per_sec":33,"recovering_bytes_per_sec":136161,"recovering_keys_per_sec":0,"num_objects_recovered":266,"num_bytes_recovered":1089536,"num_keys_recovered":0,"write_bytes_sec":1318612,"read_op_per_sec":0,"write_op_per_sec":321},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T15:27:19.616376+0000","services":{}},"progress_events":{}} 2024-09-17T15:28:27.189 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:28:27.189 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:28:27.319 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 168 20 133769 133749 3.10919 1.76172 0.0608188 0.0251182 2024-09-17T15:28:27.736 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:28:27.739 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:28:27.758 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":158,"stamp":"2024-09-17T15:28:27.634162+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":546009841,"num_objects":133314,"num_object_clones":0,"num_object_copies":266628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3995,"num_objects_unfound":0,"num_objects_dirty":133314,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":93421,"num_write_kb":373678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4187,"num_bytes_recovered":17138166,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133387,"ondisk_log_size":133387,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1780572,"kb_used_data":1081644,"kb_used_omap":13,"kb_used_meta":698866,"kb_avail":281334948,"statfs":{"total":289910292480,"available":288086986752,"internally_reserved":0,"allocated":1107603456,"data_stored":1106722224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":715639807},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,5,6,10,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19173376,"num_objects":4681,"num_object_clones":0,"num_object_copies":9362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-4111,"num_objects_misplaced":3729,"num_objects_unfound":0,"num_objects_dirty":4681,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4681,"num_write_kb":18724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":380,"num_bytes_recovered":1556480,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001761"},"pg_stats":[{"pgid":"2.f","version":"43'8281","reported_seq":8358,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:25.939645+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:28:25.939645+0000","last_peered":"2024-09-17T15:28:25.939645+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:28:25.939645+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":8281,"log_dups_size":0,"ondisk_log_size":8281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33918976,"num_objects":8281,"num_object_clones":0,"num_object_copies":16562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8281,"num_write_kb":33124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4211},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8345","reported_seq":8447,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:27.093476+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:28:27.093476+0000","last_peered":"2024-09-17T15:28:27.093476+0000","last_clean":"2024-09-17T15:28:27.093476+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:28:27.093476+0000","last_undegraded":"2024-09-17T15:28:27.093476+0000","last_fullsized":"2024-09-17T15:28:27.093476+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":8345,"log_dups_size":0,"ondisk_log_size":8345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34157510,"num_objects":8341,"num_object_clones":0,"num_object_copies":16682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8341,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6702,"num_write_kb":26808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8337","reported_seq":8462,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:27.087372+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:28:27.087372+0000","last_peered":"2024-09-17T15:28:27.087372+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:28:27.087372+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":8337,"log_dups_size":0,"ondisk_log_size":8337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34131968,"num_objects":8333,"num_object_clones":0,"num_object_copies":16666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6494,"num_write_kb":25976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":4165}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8445","reported_seq":8514,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:27.011065+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:28:27.011065+0000","last_peered":"2024-09-17T15:28:27.011065+0000","last_clean":"2024-09-17T15:28:27.011065+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:28:27.011065+0000","last_undegraded":"2024-09-17T15:28:27.011065+0000","last_fullsized":"2024-09-17T15:28:27.011065+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":8445,"log_dups_size":0,"ondisk_log_size":8445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34590720,"num_objects":8445,"num_object_clones":0,"num_object_copies":16890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8445,"num_write_kb":33780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8212","reported_seq":8341,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.861910+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:28:25.861910+0000","last_peered":"2024-09-17T15:28:25.861910+0000","last_clean":"2024-09-17T15:28:25.861910+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:28:25.861910+0000","last_undegraded":"2024-09-17T15:28:25.861910+0000","last_fullsized":"2024-09-17T15:28:25.861910+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":8212,"log_dups_size":0,"ondisk_log_size":8212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33608163,"num_objects":8206,"num_object_clones":0,"num_object_copies":16412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8206,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5194,"num_write_kb":20776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8514","reported_seq":8647,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:25.956358+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:28:25.956358+0000","last_peered":"2024-09-17T15:28:25.956358+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:28:25.956358+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":8514,"log_dups_size":0,"ondisk_log_size":8514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34853347,"num_objects":8510,"num_object_clones":0,"num_object_copies":17020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8510,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5013,"num_write_kb":20052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4240},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8383","reported_seq":8519,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:25.945340+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:28:25.945340+0000","last_peered":"2024-09-17T15:28:25.945340+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:28:25.945340+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":8383,"log_dups_size":0,"ondisk_log_size":8383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34313158,"num_objects":8379,"num_object_clones":0,"num_object_copies":16758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8379,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5152,"num_write_kb":20608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4205},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8378","reported_seq":8505,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.805775+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:28:25.805775+0000","last_peered":"2024-09-17T15:28:25.805775+0000","last_clean":"2024-09-17T15:28:25.805775+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:28:25.805775+0000","last_undegraded":"2024-09-17T15:28:25.805775+0000","last_fullsized":"2024-09-17T15:28:25.805775+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":8378,"log_dups_size":0,"ondisk_log_size":8378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34308096,"num_objects":8376,"num_object_clones":0,"num_object_copies":16752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8376,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5332,"num_write_kb":21328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8287","reported_seq":16569,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.173008+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:28:25.173008+0000","last_peered":"2024-09-17T15:28:25.173008+0000","last_clean":"2024-09-17T15:28:25.173008+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:28:25.173008+0000","last_undegraded":"2024-09-17T15:28:25.173008+0000","last_fullsized":"2024-09-17T15:28:25.173008+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":8287,"log_dups_size":0,"ondisk_log_size":8287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33915363,"num_objects":8281,"num_object_clones":0,"num_object_copies":16562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8281,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6287,"num_write_kb":25148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804705+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804705+0000","last_peered":"2024-09-17T15:28:23.804705+0000","last_clean":"2024-09-17T15:28:23.804705+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804705+0000","last_undegraded":"2024-09-17T15:28:23.804705+0000","last_fullsized":"2024-09-17T15:28:23.804705+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8200","reported_seq":8330,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:25.958722+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:28:25.958722+0000","last_peered":"2024-09-17T15:28:25.958722+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:28:25.958722+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":8200,"log_dups_size":0,"ondisk_log_size":8200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":8192,"num_object_clones":0,"num_object_copies":16384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4796,"num_write_kb":19184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4182},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":102,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814215+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814215+0000","last_peered":"2024-09-17T15:28:23.814215+0000","last_clean":"2024-09-17T15:28:23.814215+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814215+0000","last_undegraded":"2024-09-17T15:28:23.814215+0000","last_fullsized":"2024-09-17T15:28:23.814215+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8244","reported_seq":8361,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:27.019393+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:28:27.019393+0000","last_peered":"2024-09-17T15:28:27.019393+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:28:27.019393+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":8244,"log_dups_size":0,"ondisk_log_size":8244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33747427,"num_objects":8240,"num_object_clones":0,"num_object_copies":16480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8240,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6139,"num_write_kb":24556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":4163}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8360","reported_seq":8478,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:27.091067+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:28:27.091067+0000","last_peered":"2024-09-17T15:28:27.091067+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:28:27.091067+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":8360,"log_dups_size":0,"ondisk_log_size":8360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34217984,"num_objects":8354,"num_object_clones":0,"num_object_copies":16708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6122,"num_write_kb":24488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":4217}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8397","reported_seq":8532,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:25.914190+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:28:25.914190+0000","last_peered":"2024-09-17T15:28:25.914190+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:28:25.914190+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":8397,"log_dups_size":0,"ondisk_log_size":8397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34385920,"num_objects":8395,"num_object_clones":0,"num_object_copies":16790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4945,"num_write_kb":19780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4257},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8341","reported_seq":8482,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:25.834738+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:28:25.834738+0000","last_peered":"2024-09-17T15:28:25.834738+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:28:25.834738+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":8341,"log_dups_size":0,"ondisk_log_size":8341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34131968,"num_objects":8333,"num_object_clones":0,"num_object_copies":16666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4628,"num_write_kb":18512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4176},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805074+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805074+0000","last_peered":"2024-09-17T15:28:23.805074+0000","last_clean":"2024-09-17T15:28:23.805074+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805074+0000","last_undegraded":"2024-09-17T15:28:23.805074+0000","last_fullsized":"2024-09-17T15:28:23.805074+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8369","reported_seq":8499,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:25.885826+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:28:25.885826+0000","last_peered":"2024-09-17T15:28:25.885826+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:28:25.885826+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":8369,"log_dups_size":0,"ondisk_log_size":8369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34263040,"num_objects":8365,"num_object_clones":0,"num_object_copies":16730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8365,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5156,"num_write_kb":20624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4302},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":71,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806516+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806516+0000","last_peered":"2024-09-17T15:28:23.806516+0000","last_clean":"2024-09-17T15:28:23.806516+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806516+0000","last_undegraded":"2024-09-17T15:28:23.806516+0000","last_fullsized":"2024-09-17T15:28:23.806516+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8291","reported_seq":8660,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:28:25.956329+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:28:25.956329+0000","last_peered":"2024-09-17T15:28:25.956329+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:28:25.956329+0000","last_undegraded":"2024-09-17T15:28:25.956329+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":8291,"log_dups_size":0,"ondisk_log_size":8291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33911750,"num_objects":8281,"num_object_clones":0,"num_object_copies":16562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3995,"num_objects_unfound":0,"num_objects_dirty":8281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4733,"num_write_kb":18932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":466944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4286},{"shards":"0,2","objects":3995}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":546009822,"num_objects":133312,"num_object_clones":0,"num_object_copies":266624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3995,"num_objects_unfound":0,"num_objects_dirty":133312,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":93419,"num_write_kb":373676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4186,"num_bytes_recovered":17138147,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1108140032,"data_stored":1107988286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133384,"ondisk_log_size":133384,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051625,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":650720,"kb_used_data":398288,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93721120,"statfs":{"total":96636764160,"available":95970426880,"internally_reserved":0,"allocated":407846912,"data_stored":407570338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":258469548},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,2,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724570,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":401744,"kb_used_data":217344,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93970096,"statfs":{"total":96636764160,"available":96225378304,"internally_reserved":0,"allocated":222560256,"data_stored":222236249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":188804779},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,3,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247853,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":728108,"kb_used_data":466012,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93643732,"statfs":{"total":96636764160,"available":95891181568,"internally_reserved":0,"allocated":477196288,"data_stored":476915637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":268365480},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,5,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":477138944,"data_stored":477102814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":222629888,"data_stored":222550402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":408371200,"data_stored":408335070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:28:28.320 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 169 19 134249 134230 3.10191 1.87891 0.0373249 0.0251777 2024-09-17T15:28:29.320 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 170 20 134764 134744 3.09547 2.00781 0.0321057 0.0252312 2024-09-17T15:28:30.320 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 171 20 135275 135255 3.08904 1.99609 0.0594795 0.0252836 2024-09-17T15:28:30.760 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:28:31.313 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:28:31.318 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:28:31.320 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 172 20 135747 135727 3.0818 1.84375 0.0640881 0.0253426 2024-09-17T15:28:31.336 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":159,"stamp":"2024-09-17T15:28:29.634689+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":546009841,"num_objects":133314,"num_object_clones":0,"num_object_copies":266628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3995,"num_objects_unfound":0,"num_objects_dirty":133314,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":93421,"num_write_kb":373678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4187,"num_bytes_recovered":17138166,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133387,"ondisk_log_size":133387,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1780572,"kb_used_data":1081644,"kb_used_omap":13,"kb_used_meta":698866,"kb_avail":281334948,"statfs":{"total":289910292480,"available":288086986752,"internally_reserved":0,"allocated":1107603456,"data_stored":1106722224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":715639807},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,5,6,10,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":12611584,"num_objects":3079,"num_object_clones":0,"num_object_copies":6158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-4111,"num_objects_misplaced":3729,"num_objects_unfound":0,"num_objects_dirty":3079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3079,"num_write_kb":12316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":380,"num_bytes_recovered":1556480,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001833"},"pg_stats":[{"pgid":"2.f","version":"43'8281","reported_seq":8358,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:25.939645+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:28:25.939645+0000","last_peered":"2024-09-17T15:28:25.939645+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:28:25.939645+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":8281,"log_dups_size":0,"ondisk_log_size":8281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33918976,"num_objects":8281,"num_object_clones":0,"num_object_copies":16562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8281,"num_write_kb":33124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4211},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8345","reported_seq":8447,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:27.093476+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:28:27.093476+0000","last_peered":"2024-09-17T15:28:27.093476+0000","last_clean":"2024-09-17T15:28:27.093476+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:28:27.093476+0000","last_undegraded":"2024-09-17T15:28:27.093476+0000","last_fullsized":"2024-09-17T15:28:27.093476+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":8345,"log_dups_size":0,"ondisk_log_size":8345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34157510,"num_objects":8341,"num_object_clones":0,"num_object_copies":16682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8341,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6702,"num_write_kb":26808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8337","reported_seq":8462,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:27.087372+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:28:27.087372+0000","last_peered":"2024-09-17T15:28:27.087372+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:28:27.087372+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":8337,"log_dups_size":0,"ondisk_log_size":8337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34131968,"num_objects":8333,"num_object_clones":0,"num_object_copies":16666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6494,"num_write_kb":25976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":4165}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8445","reported_seq":8514,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:27.011065+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:28:27.011065+0000","last_peered":"2024-09-17T15:28:27.011065+0000","last_clean":"2024-09-17T15:28:27.011065+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:28:27.011065+0000","last_undegraded":"2024-09-17T15:28:27.011065+0000","last_fullsized":"2024-09-17T15:28:27.011065+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":8445,"log_dups_size":0,"ondisk_log_size":8445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34590720,"num_objects":8445,"num_object_clones":0,"num_object_copies":16890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8445,"num_write_kb":33780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8212","reported_seq":8341,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.861910+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:28:25.861910+0000","last_peered":"2024-09-17T15:28:25.861910+0000","last_clean":"2024-09-17T15:28:25.861910+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:28:25.861910+0000","last_undegraded":"2024-09-17T15:28:25.861910+0000","last_fullsized":"2024-09-17T15:28:25.861910+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":8212,"log_dups_size":0,"ondisk_log_size":8212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33608163,"num_objects":8206,"num_object_clones":0,"num_object_copies":16412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8206,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5194,"num_write_kb":20776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8514","reported_seq":8647,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:25.956358+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:28:25.956358+0000","last_peered":"2024-09-17T15:28:25.956358+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:28:25.956358+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":8514,"log_dups_size":0,"ondisk_log_size":8514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34853347,"num_objects":8510,"num_object_clones":0,"num_object_copies":17020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8510,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5013,"num_write_kb":20052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4240},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8383","reported_seq":8519,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:25.945340+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:28:25.945340+0000","last_peered":"2024-09-17T15:28:25.945340+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:28:25.945340+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":8383,"log_dups_size":0,"ondisk_log_size":8383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34313158,"num_objects":8379,"num_object_clones":0,"num_object_copies":16758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8379,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5152,"num_write_kb":20608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4205},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8378","reported_seq":8505,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.805775+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:28:25.805775+0000","last_peered":"2024-09-17T15:28:25.805775+0000","last_clean":"2024-09-17T15:28:25.805775+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:28:25.805775+0000","last_undegraded":"2024-09-17T15:28:25.805775+0000","last_fullsized":"2024-09-17T15:28:25.805775+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":8378,"log_dups_size":0,"ondisk_log_size":8378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34308096,"num_objects":8376,"num_object_clones":0,"num_object_copies":16752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8376,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5332,"num_write_kb":21328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8287","reported_seq":16569,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.173008+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:28:25.173008+0000","last_peered":"2024-09-17T15:28:25.173008+0000","last_clean":"2024-09-17T15:28:25.173008+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:28:25.173008+0000","last_undegraded":"2024-09-17T15:28:25.173008+0000","last_fullsized":"2024-09-17T15:28:25.173008+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":8287,"log_dups_size":0,"ondisk_log_size":8287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33915363,"num_objects":8281,"num_object_clones":0,"num_object_copies":16562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8281,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6287,"num_write_kb":25148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804705+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804705+0000","last_peered":"2024-09-17T15:28:23.804705+0000","last_clean":"2024-09-17T15:28:23.804705+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804705+0000","last_undegraded":"2024-09-17T15:28:23.804705+0000","last_fullsized":"2024-09-17T15:28:23.804705+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8200","reported_seq":8330,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:25.958722+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:28:25.958722+0000","last_peered":"2024-09-17T15:28:25.958722+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:28:25.958722+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":8200,"log_dups_size":0,"ondisk_log_size":8200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":8192,"num_object_clones":0,"num_object_copies":16384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4796,"num_write_kb":19184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4182},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":102,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814215+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814215+0000","last_peered":"2024-09-17T15:28:23.814215+0000","last_clean":"2024-09-17T15:28:23.814215+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814215+0000","last_undegraded":"2024-09-17T15:28:23.814215+0000","last_fullsized":"2024-09-17T15:28:23.814215+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8244","reported_seq":8361,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:27.019393+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:28:27.019393+0000","last_peered":"2024-09-17T15:28:27.019393+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:28:27.019393+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":8244,"log_dups_size":0,"ondisk_log_size":8244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33747427,"num_objects":8240,"num_object_clones":0,"num_object_copies":16480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8240,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6139,"num_write_kb":24556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":4163}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8360","reported_seq":8478,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:27.091067+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:28:27.091067+0000","last_peered":"2024-09-17T15:28:27.091067+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:28:27.091067+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":8360,"log_dups_size":0,"ondisk_log_size":8360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34217984,"num_objects":8354,"num_object_clones":0,"num_object_copies":16708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6122,"num_write_kb":24488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":4217}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8397","reported_seq":8532,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:25.914190+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:28:25.914190+0000","last_peered":"2024-09-17T15:28:25.914190+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:28:25.914190+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":8397,"log_dups_size":0,"ondisk_log_size":8397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34385920,"num_objects":8395,"num_object_clones":0,"num_object_copies":16790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4945,"num_write_kb":19780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4257},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8341","reported_seq":8482,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:25.834738+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:28:25.834738+0000","last_peered":"2024-09-17T15:28:25.834738+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:28:25.834738+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":8341,"log_dups_size":0,"ondisk_log_size":8341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34131968,"num_objects":8333,"num_object_clones":0,"num_object_copies":16666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4628,"num_write_kb":18512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4176},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805074+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805074+0000","last_peered":"2024-09-17T15:28:23.805074+0000","last_clean":"2024-09-17T15:28:23.805074+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805074+0000","last_undegraded":"2024-09-17T15:28:23.805074+0000","last_fullsized":"2024-09-17T15:28:23.805074+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8369","reported_seq":8499,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:25.885826+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:28:25.885826+0000","last_peered":"2024-09-17T15:28:25.885826+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:28:25.885826+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":8369,"log_dups_size":0,"ondisk_log_size":8369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34263040,"num_objects":8365,"num_object_clones":0,"num_object_copies":16730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8365,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5156,"num_write_kb":20624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4302},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":71,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806516+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806516+0000","last_peered":"2024-09-17T15:28:23.806516+0000","last_clean":"2024-09-17T15:28:23.806516+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806516+0000","last_undegraded":"2024-09-17T15:28:23.806516+0000","last_fullsized":"2024-09-17T15:28:23.806516+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8291","reported_seq":8660,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:28:25.956329+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:28:25.956329+0000","last_peered":"2024-09-17T15:28:25.956329+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:28:25.956329+0000","last_undegraded":"2024-09-17T15:28:25.956329+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":8291,"log_dups_size":0,"ondisk_log_size":8291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33911750,"num_objects":8281,"num_object_clones":0,"num_object_copies":16562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3995,"num_objects_unfound":0,"num_objects_dirty":8281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4733,"num_write_kb":18932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":466944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4286},{"shards":"0,2","objects":3995}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":546009822,"num_objects":133312,"num_object_clones":0,"num_object_copies":266624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3995,"num_objects_unfound":0,"num_objects_dirty":133312,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":93419,"num_write_kb":373676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4186,"num_bytes_recovered":17138147,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1108140032,"data_stored":1107988286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133384,"ondisk_log_size":133384,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051625,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":650720,"kb_used_data":398288,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93721120,"statfs":{"total":96636764160,"available":95970426880,"internally_reserved":0,"allocated":407846912,"data_stored":407570338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":258469548},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,2,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724570,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":401744,"kb_used_data":217344,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93970096,"statfs":{"total":96636764160,"available":96225378304,"internally_reserved":0,"allocated":222560256,"data_stored":222236249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":188804779},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,3,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247853,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":728108,"kb_used_data":466012,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93643732,"statfs":{"total":96636764160,"available":95891181568,"internally_reserved":0,"allocated":477196288,"data_stored":476915637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":268365480},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,5,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":477138944,"data_stored":477102814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":222629888,"data_stored":222550402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":408371200,"data_stored":408335070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:28:31.338 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:28:31.909 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:28:31.910 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:28:31.932 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2024-09-17T15:28:31.635103+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":552989425,"num_objects":135018,"num_object_clones":0,"num_object_copies":270036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3908,"num_objects_unfound":0,"num_objects_dirty":135018,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":95125,"num_write_kb":380494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4274,"num_bytes_recovered":17494518,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135091,"ondisk_log_size":135091,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1823356,"kb_used_data":1096652,"kb_used_omap":13,"kb_used_meta":726642,"kb_avail":281292164,"statfs":{"total":289910292480,"available":288043175936,"internally_reserved":0,"allocated":1122971648,"data_stored":1122090416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":744082431},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,7,4,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":15978496,"num_objects":3901,"num_object_clones":0,"num_object_copies":7802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-4111,"num_objects_misplaced":3893,"num_objects_unfound":0,"num_objects_dirty":3901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3901,"num_write_kb":15604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":884736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.826439"},"pg_stats":[{"pgid":"2.f","version":"43'8419","reported_seq":8496,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:30.948394+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:28:30.948394+0000","last_peered":"2024-09-17T15:28:30.948394+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:28:30.948394+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":8419,"log_dups_size":0,"ondisk_log_size":8419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34484224,"num_objects":8419,"num_object_clones":0,"num_object_copies":16838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8419,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8419,"num_write_kb":33676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4349},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8345","reported_seq":8447,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:27.093476+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:28:27.093476+0000","last_peered":"2024-09-17T15:28:27.093476+0000","last_clean":"2024-09-17T15:28:27.093476+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:28:27.093476+0000","last_undegraded":"2024-09-17T15:28:27.093476+0000","last_fullsized":"2024-09-17T15:28:27.093476+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":8345,"log_dups_size":0,"ondisk_log_size":8345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34157510,"num_objects":8341,"num_object_clones":0,"num_object_copies":16682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8341,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6702,"num_write_kb":26808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8337","reported_seq":8462,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:27.087372+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:28:27.087372+0000","last_peered":"2024-09-17T15:28:27.087372+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:28:27.087372+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":8337,"log_dups_size":0,"ondisk_log_size":8337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34131968,"num_objects":8333,"num_object_clones":0,"num_object_copies":16666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6494,"num_write_kb":25976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":4165}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8445","reported_seq":8514,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:27.011065+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:28:27.011065+0000","last_peered":"2024-09-17T15:28:27.011065+0000","last_clean":"2024-09-17T15:28:27.011065+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:28:27.011065+0000","last_undegraded":"2024-09-17T15:28:27.011065+0000","last_fullsized":"2024-09-17T15:28:27.011065+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":8445,"log_dups_size":0,"ondisk_log_size":8445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34590720,"num_objects":8445,"num_object_clones":0,"num_object_copies":16890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8445,"num_write_kb":33780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8364","reported_seq":8493,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:31.025418+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:28:31.025418+0000","last_peered":"2024-09-17T15:28:31.025418+0000","last_clean":"2024-09-17T15:28:31.025418+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:28:31.025418+0000","last_undegraded":"2024-09-17T15:28:31.025418+0000","last_fullsized":"2024-09-17T15:28:31.025418+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":8364,"log_dups_size":0,"ondisk_log_size":8364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34230755,"num_objects":8358,"num_object_clones":0,"num_object_copies":16716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8358,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5346,"num_write_kb":21384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8673","reported_seq":8807,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:30.986200+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:28:30.986200+0000","last_peered":"2024-09-17T15:28:30.986200+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:28:30.986200+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":8673,"log_dups_size":0,"ondisk_log_size":8673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35504611,"num_objects":8669,"num_object_clones":0,"num_object_copies":17338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8669,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5172,"num_write_kb":20688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4399},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8528","reported_seq":8666,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:30.981154+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:28:30.981154+0000","last_peered":"2024-09-17T15:28:30.981154+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:28:30.981154+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":8528,"log_dups_size":0,"ondisk_log_size":8528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34907078,"num_objects":8524,"num_object_clones":0,"num_object_copies":17048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8524,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5297,"num_write_kb":21188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4350},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8543","reported_seq":8670,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:31.050041+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:28:31.050041+0000","last_peered":"2024-09-17T15:28:31.050041+0000","last_clean":"2024-09-17T15:28:31.050041+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:28:31.050041+0000","last_undegraded":"2024-09-17T15:28:31.050041+0000","last_fullsized":"2024-09-17T15:28:31.050041+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":8543,"log_dups_size":0,"ondisk_log_size":8543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34983936,"num_objects":8541,"num_object_clones":0,"num_object_copies":17082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5497,"num_write_kb":21988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8438","reported_seq":16720,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:30.211734+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:28:30.211734+0000","last_peered":"2024-09-17T15:28:30.211734+0000","last_clean":"2024-09-17T15:28:30.211734+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:28:30.211734+0000","last_undegraded":"2024-09-17T15:28:30.211734+0000","last_fullsized":"2024-09-17T15:28:30.211734+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":8438,"log_dups_size":0,"ondisk_log_size":8438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34533859,"num_objects":8432,"num_object_clones":0,"num_object_copies":16864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8432,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6438,"num_write_kb":25752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804705+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804705+0000","last_peered":"2024-09-17T15:28:23.804705+0000","last_clean":"2024-09-17T15:28:23.804705+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804705+0000","last_undegraded":"2024-09-17T15:28:23.804705+0000","last_fullsized":"2024-09-17T15:28:23.804705+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8352","reported_seq":8482,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:31.049074+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:28:31.049074+0000","last_peered":"2024-09-17T15:28:31.049074+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:28:31.049074+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":8352,"log_dups_size":0,"ondisk_log_size":8352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34177024,"num_objects":8344,"num_object_clones":0,"num_object_copies":16688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4948,"num_write_kb":19792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4334},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":102,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814215+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814215+0000","last_peered":"2024-09-17T15:28:23.814215+0000","last_clean":"2024-09-17T15:28:23.814215+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814215+0000","last_undegraded":"2024-09-17T15:28:23.814215+0000","last_fullsized":"2024-09-17T15:28:23.814215+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8244","reported_seq":8361,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:27.019393+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:28:27.019393+0000","last_peered":"2024-09-17T15:28:27.019393+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:28:27.019393+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":8244,"log_dups_size":0,"ondisk_log_size":8244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33747427,"num_objects":8240,"num_object_clones":0,"num_object_copies":16480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8240,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6139,"num_write_kb":24556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":4163}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8360","reported_seq":8478,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:27.091067+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:28:27.091067+0000","last_peered":"2024-09-17T15:28:27.091067+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:28:27.091067+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":8360,"log_dups_size":0,"ondisk_log_size":8360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34217984,"num_objects":8354,"num_object_clones":0,"num_object_copies":16708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6122,"num_write_kb":24488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":4217}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8547","reported_seq":8682,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:31.026488+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:28:31.026488+0000","last_peered":"2024-09-17T15:28:31.026488+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:28:31.026488+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":8547,"log_dups_size":0,"ondisk_log_size":8547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35000320,"num_objects":8545,"num_object_clones":0,"num_object_copies":17090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5095,"num_write_kb":20380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4407},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8523","reported_seq":8662,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:31.036570+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:28:31.036570+0000","last_peered":"2024-09-17T15:28:31.036570+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:28:31.036570+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":8523,"log_dups_size":0,"ondisk_log_size":8523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34877440,"num_objects":8515,"num_object_clones":0,"num_object_copies":17030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4810,"num_write_kb":19240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4358},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805074+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805074+0000","last_peered":"2024-09-17T15:28:23.805074+0000","last_clean":"2024-09-17T15:28:23.805074+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805074+0000","last_undegraded":"2024-09-17T15:28:23.805074+0000","last_fullsized":"2024-09-17T15:28:23.805074+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8520","reported_seq":8651,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:31.003136+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:28:31.003136+0000","last_peered":"2024-09-17T15:28:31.003136+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:28:31.003136+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":8520,"log_dups_size":0,"ondisk_log_size":8520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34881536,"num_objects":8516,"num_object_clones":0,"num_object_copies":17032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5307,"num_write_kb":21228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4453},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8450","reported_seq":8994,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:28:31.063757+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:28:31.063757+0000","last_peered":"2024-09-17T15:28:31.063757+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:28:31.063757+0000","last_undegraded":"2024-09-17T15:28:31.063757+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":8450,"log_dups_size":0,"ondisk_log_size":8450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34563014,"num_objects":8440,"num_object_clones":0,"num_object_copies":16880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3908,"num_objects_unfound":0,"num_objects_dirty":8440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4892,"num_write_kb":19568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":823296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4532},{"shards":"0,2","objects":3908}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":552989406,"num_objects":135016,"num_object_clones":0,"num_object_copies":270032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3908,"num_objects_unfound":0,"num_objects_dirty":135016,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":95123,"num_write_kb":380492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4273,"num_bytes_recovered":17494499,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1123655680,"data_stored":1123503934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135088,"ondisk_log_size":135088,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051625,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":650720,"kb_used_data":398288,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93721120,"statfs":{"total":96636764160,"available":95970426880,"internally_reserved":0,"allocated":407846912,"data_stored":407570338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":258469548},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,2,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724571,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":415004,"kb_used_data":225036,"kb_used_omap":4,"kb_used_meta":189947,"kb_avail":93956836,"statfs":{"total":96636764160,"available":96211800064,"internally_reserved":0,"allocated":230436864,"data_stored":230112857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":194506411},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,6],"upper_bound":32},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":757632,"kb_used_data":473328,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93614208,"statfs":{"total":96636764160,"available":95860948992,"internally_reserved":0,"allocated":484687872,"data_stored":484407221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":291106472},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,2,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":484839424,"data_stored":484803294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":230445056,"data_stored":230365570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":408371200,"data_stored":408335070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:28:31.934 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:28:31.934 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'8419", 'reported_seq': 8496, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:30.948394+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:28:30.948394+0000', 'last_peered': '2024-09-17T15:28:30.948394+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:28:30.948394+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 8419, 'log_dups_size': 0, 'ondisk_log_size': 8419, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34484224, 'num_objects': 8419, 'num_object_clones': 0, 'num_object_copies': 16838, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4070, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8419, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8419, 'num_write_kb': 33676, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4349}, {'shards': '0,2', 'objects': 4070}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:31.934 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:28:31.934 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'8337", 'reported_seq': 8462, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:27.087372+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:28:27.087372+0000', 'last_peered': '2024-09-17T15:28:27.087372+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:28:27.087372+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 8337, 'log_dups_size': 0, 'ondisk_log_size': 8337, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34131968, 'num_objects': 8333, 'num_object_clones': 0, 'num_object_copies': 16666, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4169, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8333, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6494, 'num_write_kb': 25976, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4168}, {'shards': '2', 'objects': 4165}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:28:31.934 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:28:31.934 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'8673", 'reported_seq': 8807, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:30.986200+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:28:30.986200+0000', 'last_peered': '2024-09-17T15:28:30.986200+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:28:30.986200+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 8673, 'log_dups_size': 0, 'ondisk_log_size': 8673, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35504611, 'num_objects': 8669, 'num_object_clones': 0, 'num_object_copies': 17338, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4271, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8669, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5172, 'num_write_kb': 20688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4399}, {'shards': '0,2', 'objects': 4268}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:31.935 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:28:31.935 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'8528", 'reported_seq': 8666, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:30.981154+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:28:30.981154+0000', 'last_peered': '2024-09-17T15:28:30.981154+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:28:30.981154+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 8528, 'log_dups_size': 0, 'ondisk_log_size': 8528, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34907078, 'num_objects': 8524, 'num_object_clones': 0, 'num_object_copies': 17048, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4174, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8524, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 5297, 'num_write_kb': 21188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4350}, {'shards': '0,2', 'objects': 4174}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:31.935 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:28:31.935 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'8352", 'reported_seq': 8482, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:31.049074+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:28:31.049074+0000', 'last_peered': '2024-09-17T15:28:31.049074+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:28:31.049074+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 8352, 'log_dups_size': 0, 'ondisk_log_size': 8352, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34177024, 'num_objects': 8344, 'num_object_clones': 0, 'num_object_copies': 16688, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4012, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8344, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4948, 'num_write_kb': 19792, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4334}, {'shards': '0,2', 'objects': 4010}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:31.935 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:28:31.935 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'8244", 'reported_seq': 8361, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:27.019393+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:28:27.019393+0000', 'last_peered': '2024-09-17T15:28:27.019393+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:28:27.019393+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 8244, 'log_dups_size': 0, 'ondisk_log_size': 8244, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33747427, 'num_objects': 8240, 'num_object_clones': 0, 'num_object_copies': 16480, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4078, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8240, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6139, 'num_write_kb': 24556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4077}, {'shards': '2', 'objects': 4163}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:28:31.935 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:28:31.935 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'8360", 'reported_seq': 8478, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:27.091067+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:28:27.091067+0000', 'last_peered': '2024-09-17T15:28:27.091067+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:28:27.091067+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 8360, 'log_dups_size': 0, 'ondisk_log_size': 8360, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34217984, 'num_objects': 8354, 'num_object_clones': 0, 'num_object_copies': 16708, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8354, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6122, 'num_write_kb': 24488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4137}, {'shards': '2', 'objects': 4217}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:28:31.935 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:28:31.935 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'8547", 'reported_seq': 8682, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:31.026488+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:28:31.026488+0000', 'last_peered': '2024-09-17T15:28:31.026488+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:28:31.026488+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 8547, 'log_dups_size': 0, 'ondisk_log_size': 8547, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35000320, 'num_objects': 8545, 'num_object_clones': 0, 'num_object_copies': 17090, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8545, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5095, 'num_write_kb': 20380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4407}, {'shards': '0,2', 'objects': 4138}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:31.935 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:28:31.936 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'8523", 'reported_seq': 8662, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:31.036570+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:28:31.036570+0000', 'last_peered': '2024-09-17T15:28:31.036570+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:28:31.036570+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 8523, 'log_dups_size': 0, 'ondisk_log_size': 8523, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34877440, 'num_objects': 8515, 'num_object_clones': 0, 'num_object_copies': 17030, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4160, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8515, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4810, 'num_write_kb': 19240, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4358}, {'shards': '0,2', 'objects': 4157}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:31.936 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:28:31.936 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'8520", 'reported_seq': 8651, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:31.003136+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:28:31.003136+0000', 'last_peered': '2024-09-17T15:28:31.003136+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:28:31.003136+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 8520, 'log_dups_size': 0, 'ondisk_log_size': 8520, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34881536, 'num_objects': 8516, 'num_object_clones': 0, 'num_object_copies': 17032, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4065, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8516, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5307, 'num_write_kb': 21228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4453}, {'shards': '0,2', 'objects': 4063}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:31.936 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:28:31.936 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'8450", 'reported_seq': 8994, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:28:31.063757+0000', 'last_change': '2024-09-17T15:28:22.509818+0000', 'last_active': '2024-09-17T15:28:31.063757+0000', 'last_peered': '2024-09-17T15:28:31.063757+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:28:31.063757+0000', 'last_undegraded': '2024-09-17T15:28:31.063757+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 8450, 'log_dups_size': 0, 'ondisk_log_size': 8450, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34563014, 'num_objects': 8440, 'num_object_clones': 0, 'num_object_copies': 16880, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3908, 'num_objects_unfound': 0, 'num_objects_dirty': 8440, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4892, 'num_write_kb': 19568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 201, 'num_bytes_recovered': 823296, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4532}, {'shards': '0,2', 'objects': 3908}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:31.936 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:28:32.320 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 173 20 136196 136176 3.07413 1.75391 0.0367511 0.0254063 2024-09-17T15:28:32.620 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:28:32.620 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41277/270036 objects degraded (15.286%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":249,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":135018,"data_bytes":552989425,"bytes_used":1867116544,"bytes_avail":288043175936,"bytes_total":289910292480,"degraded_objects":41277,"degraded_total":270036,"degraded_ratio":0.1528573967915389,"misplaced_objects":3908,"misplaced_total":270036,"misplaced_ratio":0.014472144454813434,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":100237,"recovering_keys_per_sec":0,"num_objects_recovered":216,"num_bytes_recovered":884736,"num_keys_recovered":0,"write_bytes_sec":1810299,"read_op_per_sec":0,"write_op_per_sec":441},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T15:27:19.616376+0000","services":{}},"progress_events":{}} 2024-09-17T15:28:32.637 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:28:32.638 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:28:33.196 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:28:33.196 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:28:33.221 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2024-09-17T15:28:31.635103+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":552989425,"num_objects":135018,"num_object_clones":0,"num_object_copies":270036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3908,"num_objects_unfound":0,"num_objects_dirty":135018,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":95125,"num_write_kb":380494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4274,"num_bytes_recovered":17494518,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135091,"ondisk_log_size":135091,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1823356,"kb_used_data":1096652,"kb_used_omap":13,"kb_used_meta":726642,"kb_avail":281292164,"statfs":{"total":289910292480,"available":288043175936,"internally_reserved":0,"allocated":1122971648,"data_stored":1122090416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":744082431},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,7,4,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":15978496,"num_objects":3901,"num_object_clones":0,"num_object_copies":7802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-4111,"num_objects_misplaced":3893,"num_objects_unfound":0,"num_objects_dirty":3901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3901,"num_write_kb":15604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":884736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.826439"},"pg_stats":[{"pgid":"2.f","version":"43'8419","reported_seq":8496,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:30.948394+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:28:30.948394+0000","last_peered":"2024-09-17T15:28:30.948394+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:28:30.948394+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":8419,"log_dups_size":0,"ondisk_log_size":8419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34484224,"num_objects":8419,"num_object_clones":0,"num_object_copies":16838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8419,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8419,"num_write_kb":33676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4349},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8345","reported_seq":8447,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:27.093476+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:28:27.093476+0000","last_peered":"2024-09-17T15:28:27.093476+0000","last_clean":"2024-09-17T15:28:27.093476+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:28:27.093476+0000","last_undegraded":"2024-09-17T15:28:27.093476+0000","last_fullsized":"2024-09-17T15:28:27.093476+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":8345,"log_dups_size":0,"ondisk_log_size":8345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34157510,"num_objects":8341,"num_object_clones":0,"num_object_copies":16682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8341,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6702,"num_write_kb":26808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8337","reported_seq":8462,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:27.087372+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:28:27.087372+0000","last_peered":"2024-09-17T15:28:27.087372+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:28:27.087372+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":8337,"log_dups_size":0,"ondisk_log_size":8337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34131968,"num_objects":8333,"num_object_clones":0,"num_object_copies":16666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6494,"num_write_kb":25976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":4165}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8445","reported_seq":8514,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:27.011065+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:28:27.011065+0000","last_peered":"2024-09-17T15:28:27.011065+0000","last_clean":"2024-09-17T15:28:27.011065+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:28:27.011065+0000","last_undegraded":"2024-09-17T15:28:27.011065+0000","last_fullsized":"2024-09-17T15:28:27.011065+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":8445,"log_dups_size":0,"ondisk_log_size":8445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34590720,"num_objects":8445,"num_object_clones":0,"num_object_copies":16890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8445,"num_write_kb":33780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8364","reported_seq":8493,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:31.025418+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:28:31.025418+0000","last_peered":"2024-09-17T15:28:31.025418+0000","last_clean":"2024-09-17T15:28:31.025418+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:28:31.025418+0000","last_undegraded":"2024-09-17T15:28:31.025418+0000","last_fullsized":"2024-09-17T15:28:31.025418+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":8364,"log_dups_size":0,"ondisk_log_size":8364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34230755,"num_objects":8358,"num_object_clones":0,"num_object_copies":16716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8358,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5346,"num_write_kb":21384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8673","reported_seq":8807,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:30.986200+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:28:30.986200+0000","last_peered":"2024-09-17T15:28:30.986200+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:28:30.986200+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":8673,"log_dups_size":0,"ondisk_log_size":8673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35504611,"num_objects":8669,"num_object_clones":0,"num_object_copies":17338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8669,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5172,"num_write_kb":20688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4399},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8528","reported_seq":8666,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:30.981154+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:28:30.981154+0000","last_peered":"2024-09-17T15:28:30.981154+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:28:30.981154+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":8528,"log_dups_size":0,"ondisk_log_size":8528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34907078,"num_objects":8524,"num_object_clones":0,"num_object_copies":17048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8524,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5297,"num_write_kb":21188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4350},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8543","reported_seq":8670,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:31.050041+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:28:31.050041+0000","last_peered":"2024-09-17T15:28:31.050041+0000","last_clean":"2024-09-17T15:28:31.050041+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:28:31.050041+0000","last_undegraded":"2024-09-17T15:28:31.050041+0000","last_fullsized":"2024-09-17T15:28:31.050041+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":8543,"log_dups_size":0,"ondisk_log_size":8543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34983936,"num_objects":8541,"num_object_clones":0,"num_object_copies":17082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5497,"num_write_kb":21988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8438","reported_seq":16720,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:30.211734+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:28:30.211734+0000","last_peered":"2024-09-17T15:28:30.211734+0000","last_clean":"2024-09-17T15:28:30.211734+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:28:30.211734+0000","last_undegraded":"2024-09-17T15:28:30.211734+0000","last_fullsized":"2024-09-17T15:28:30.211734+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":8438,"log_dups_size":0,"ondisk_log_size":8438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34533859,"num_objects":8432,"num_object_clones":0,"num_object_copies":16864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8432,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6438,"num_write_kb":25752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804705+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804705+0000","last_peered":"2024-09-17T15:28:23.804705+0000","last_clean":"2024-09-17T15:28:23.804705+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804705+0000","last_undegraded":"2024-09-17T15:28:23.804705+0000","last_fullsized":"2024-09-17T15:28:23.804705+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8352","reported_seq":8482,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:31.049074+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:28:31.049074+0000","last_peered":"2024-09-17T15:28:31.049074+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:28:31.049074+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":8352,"log_dups_size":0,"ondisk_log_size":8352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34177024,"num_objects":8344,"num_object_clones":0,"num_object_copies":16688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4948,"num_write_kb":19792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4334},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":102,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814215+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814215+0000","last_peered":"2024-09-17T15:28:23.814215+0000","last_clean":"2024-09-17T15:28:23.814215+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814215+0000","last_undegraded":"2024-09-17T15:28:23.814215+0000","last_fullsized":"2024-09-17T15:28:23.814215+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8244","reported_seq":8361,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:27.019393+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:28:27.019393+0000","last_peered":"2024-09-17T15:28:27.019393+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:28:27.019393+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":8244,"log_dups_size":0,"ondisk_log_size":8244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33747427,"num_objects":8240,"num_object_clones":0,"num_object_copies":16480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8240,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6139,"num_write_kb":24556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":4163}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8360","reported_seq":8478,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:27.091067+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:28:27.091067+0000","last_peered":"2024-09-17T15:28:27.091067+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:28:27.091067+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":8360,"log_dups_size":0,"ondisk_log_size":8360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34217984,"num_objects":8354,"num_object_clones":0,"num_object_copies":16708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6122,"num_write_kb":24488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":4217}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8547","reported_seq":8682,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:31.026488+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:28:31.026488+0000","last_peered":"2024-09-17T15:28:31.026488+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:28:31.026488+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":8547,"log_dups_size":0,"ondisk_log_size":8547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35000320,"num_objects":8545,"num_object_clones":0,"num_object_copies":17090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5095,"num_write_kb":20380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4407},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8523","reported_seq":8662,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:31.036570+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:28:31.036570+0000","last_peered":"2024-09-17T15:28:31.036570+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:28:31.036570+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":8523,"log_dups_size":0,"ondisk_log_size":8523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34877440,"num_objects":8515,"num_object_clones":0,"num_object_copies":17030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4810,"num_write_kb":19240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4358},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805074+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805074+0000","last_peered":"2024-09-17T15:28:23.805074+0000","last_clean":"2024-09-17T15:28:23.805074+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805074+0000","last_undegraded":"2024-09-17T15:28:23.805074+0000","last_fullsized":"2024-09-17T15:28:23.805074+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8520","reported_seq":8651,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:31.003136+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:28:31.003136+0000","last_peered":"2024-09-17T15:28:31.003136+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:28:31.003136+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":8520,"log_dups_size":0,"ondisk_log_size":8520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34881536,"num_objects":8516,"num_object_clones":0,"num_object_copies":17032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5307,"num_write_kb":21228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4453},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8450","reported_seq":8994,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:28:31.063757+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:28:31.063757+0000","last_peered":"2024-09-17T15:28:31.063757+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:28:31.063757+0000","last_undegraded":"2024-09-17T15:28:31.063757+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":8450,"log_dups_size":0,"ondisk_log_size":8450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34563014,"num_objects":8440,"num_object_clones":0,"num_object_copies":16880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3908,"num_objects_unfound":0,"num_objects_dirty":8440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4892,"num_write_kb":19568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":823296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4532},{"shards":"0,2","objects":3908}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":552989406,"num_objects":135016,"num_object_clones":0,"num_object_copies":270032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3908,"num_objects_unfound":0,"num_objects_dirty":135016,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":95123,"num_write_kb":380492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4273,"num_bytes_recovered":17494499,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1123655680,"data_stored":1123503934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135088,"ondisk_log_size":135088,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051625,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":650720,"kb_used_data":398288,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93721120,"statfs":{"total":96636764160,"available":95970426880,"internally_reserved":0,"allocated":407846912,"data_stored":407570338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":258469548},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,2,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724571,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":415004,"kb_used_data":225036,"kb_used_omap":4,"kb_used_meta":189947,"kb_avail":93956836,"statfs":{"total":96636764160,"available":96211800064,"internally_reserved":0,"allocated":230436864,"data_stored":230112857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":194506411},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,6],"upper_bound":32},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":757632,"kb_used_data":473328,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93614208,"statfs":{"total":96636764160,"available":95860948992,"internally_reserved":0,"allocated":484687872,"data_stored":484407221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":291106472},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,2,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":484839424,"data_stored":484803294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":230445056,"data_stored":230365570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":408371200,"data_stored":408335070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:28:33.320 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 174 20 136644 136624 3.06652 1.75 0.024535 0.0254692 2024-09-17T15:28:34.320 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 175 20 137156 137136 3.06042 2 0.0566068 0.0255209 2024-09-17T15:28:35.321 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 176 20 137654 137634 3.05409 1.94531 0.0704122 0.0255722 2024-09-17T15:28:36.223 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:28:36.320 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 177 20 138164 138144 3.04809 1.99219 0.0727671 0.0256232 2024-09-17T15:28:36.764 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:28:36.765 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:28:36.805 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":162,"stamp":"2024-09-17T15:28:35.636042+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":556675825,"num_objects":135918,"num_object_clones":0,"num_object_copies":271836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3908,"num_objects_unfound":0,"num_objects_dirty":135918,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":96025,"num_write_kb":384094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4274,"num_bytes_recovered":17494518,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135991,"ondisk_log_size":135991,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1844052,"kb_used_data":1109028,"kb_used_omap":13,"kb_used_meta":734962,"kb_avail":281271468,"statfs":{"total":289910292480,"available":288021983232,"internally_reserved":0,"allocated":1135644672,"data_stored":1134763440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":752602111},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,3,8,10,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19664896,"num_objects":4801,"num_object_clones":0,"num_object_copies":9602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-4111,"num_objects_misplaced":3893,"num_objects_unfound":0,"num_objects_dirty":4801,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4801,"num_write_kb":19204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":884736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.834364"},"pg_stats":[{"pgid":"2.f","version":"43'8419","reported_seq":8496,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:30.948394+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:28:30.948394+0000","last_peered":"2024-09-17T15:28:30.948394+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:28:30.948394+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":8419,"log_dups_size":0,"ondisk_log_size":8419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34484224,"num_objects":8419,"num_object_clones":0,"num_object_copies":16838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8419,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8419,"num_write_kb":33676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4349},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8497","reported_seq":8599,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:31.912549+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:28:31.912549+0000","last_peered":"2024-09-17T15:28:31.912549+0000","last_clean":"2024-09-17T15:28:31.912549+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:28:31.912549+0000","last_undegraded":"2024-09-17T15:28:31.912549+0000","last_fullsized":"2024-09-17T15:28:31.912549+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":8497,"log_dups_size":0,"ondisk_log_size":8497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34780102,"num_objects":8493,"num_object_clones":0,"num_object_copies":16986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8493,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6854,"num_write_kb":27416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8478","reported_seq":8603,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:32.106845+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:28:32.106845+0000","last_peered":"2024-09-17T15:28:32.106845+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:28:32.106845+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":8478,"log_dups_size":0,"ondisk_log_size":8478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34709504,"num_objects":8474,"num_object_clones":0,"num_object_copies":16948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6635,"num_write_kb":26540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":4306}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8594","reported_seq":8663,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:32.063120+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:28:32.063120+0000","last_peered":"2024-09-17T15:28:32.063120+0000","last_clean":"2024-09-17T15:28:32.063120+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:28:32.063120+0000","last_undegraded":"2024-09-17T15:28:32.063120+0000","last_fullsized":"2024-09-17T15:28:32.063120+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":8594,"log_dups_size":0,"ondisk_log_size":8594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35201024,"num_objects":8594,"num_object_clones":0,"num_object_copies":17188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8594,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8594,"num_write_kb":34376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8364","reported_seq":8493,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:31.025418+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:28:31.025418+0000","last_peered":"2024-09-17T15:28:31.025418+0000","last_clean":"2024-09-17T15:28:31.025418+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:28:31.025418+0000","last_undegraded":"2024-09-17T15:28:31.025418+0000","last_fullsized":"2024-09-17T15:28:31.025418+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":8364,"log_dups_size":0,"ondisk_log_size":8364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34230755,"num_objects":8358,"num_object_clones":0,"num_object_copies":16716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8358,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5346,"num_write_kb":21384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8673","reported_seq":8807,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:30.986200+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:28:30.986200+0000","last_peered":"2024-09-17T15:28:30.986200+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:28:30.986200+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":8673,"log_dups_size":0,"ondisk_log_size":8673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35504611,"num_objects":8669,"num_object_clones":0,"num_object_copies":17338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8669,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5172,"num_write_kb":20688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4399},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8528","reported_seq":8666,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:30.981154+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:28:30.981154+0000","last_peered":"2024-09-17T15:28:30.981154+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:28:30.981154+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":8528,"log_dups_size":0,"ondisk_log_size":8528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34907078,"num_objects":8524,"num_object_clones":0,"num_object_copies":17048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8524,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5297,"num_write_kb":21188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4350},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8543","reported_seq":8670,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:31.050041+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:28:31.050041+0000","last_peered":"2024-09-17T15:28:31.050041+0000","last_clean":"2024-09-17T15:28:31.050041+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:28:31.050041+0000","last_undegraded":"2024-09-17T15:28:31.050041+0000","last_fullsized":"2024-09-17T15:28:31.050041+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":8543,"log_dups_size":0,"ondisk_log_size":8543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34983936,"num_objects":8541,"num_object_clones":0,"num_object_copies":17082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5497,"num_write_kb":21988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8587","reported_seq":16869,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:35.306522+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:28:35.306522+0000","last_peered":"2024-09-17T15:28:35.306522+0000","last_clean":"2024-09-17T15:28:35.306522+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:28:35.306522+0000","last_undegraded":"2024-09-17T15:28:35.306522+0000","last_fullsized":"2024-09-17T15:28:35.306522+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":8587,"log_dups_size":0,"ondisk_log_size":8587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35144163,"num_objects":8581,"num_object_clones":0,"num_object_copies":17162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8581,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6587,"num_write_kb":26348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804705+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804705+0000","last_peered":"2024-09-17T15:28:23.804705+0000","last_clean":"2024-09-17T15:28:23.804705+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804705+0000","last_undegraded":"2024-09-17T15:28:23.804705+0000","last_fullsized":"2024-09-17T15:28:23.804705+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8352","reported_seq":8482,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:31.049074+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:28:31.049074+0000","last_peered":"2024-09-17T15:28:31.049074+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:28:31.049074+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":8352,"log_dups_size":0,"ondisk_log_size":8352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34177024,"num_objects":8344,"num_object_clones":0,"num_object_copies":16688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4948,"num_write_kb":19792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4334},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":102,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814215+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814215+0000","last_peered":"2024-09-17T15:28:23.814215+0000","last_clean":"2024-09-17T15:28:23.814215+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814215+0000","last_undegraded":"2024-09-17T15:28:23.814215+0000","last_fullsized":"2024-09-17T15:28:23.814215+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8408","reported_seq":8528,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:32.071846+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:28:32.071846+0000","last_peered":"2024-09-17T15:28:32.071846+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:28:32.071846+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":8408,"log_dups_size":0,"ondisk_log_size":8408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34419171,"num_objects":8404,"num_object_clones":0,"num_object_copies":16808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8404,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6303,"num_write_kb":25212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":4327}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8505","reported_seq":8623,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:32.107313+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:28:32.107313+0000","last_peered":"2024-09-17T15:28:32.107313+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:28:32.107313+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":8505,"log_dups_size":0,"ondisk_log_size":8505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34811904,"num_objects":8499,"num_object_clones":0,"num_object_copies":16998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8499,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6267,"num_write_kb":25068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":4362}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8547","reported_seq":8682,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:31.026488+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:28:31.026488+0000","last_peered":"2024-09-17T15:28:31.026488+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:28:31.026488+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":8547,"log_dups_size":0,"ondisk_log_size":8547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35000320,"num_objects":8545,"num_object_clones":0,"num_object_copies":17090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5095,"num_write_kb":20380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4407},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8523","reported_seq":8662,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:31.036570+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:28:31.036570+0000","last_peered":"2024-09-17T15:28:31.036570+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:28:31.036570+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":8523,"log_dups_size":0,"ondisk_log_size":8523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34877440,"num_objects":8515,"num_object_clones":0,"num_object_copies":17030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4810,"num_write_kb":19240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4358},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805074+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805074+0000","last_peered":"2024-09-17T15:28:23.805074+0000","last_clean":"2024-09-17T15:28:23.805074+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805074+0000","last_undegraded":"2024-09-17T15:28:23.805074+0000","last_fullsized":"2024-09-17T15:28:23.805074+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8520","reported_seq":8651,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:31.003136+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:28:31.003136+0000","last_peered":"2024-09-17T15:28:31.003136+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:28:31.003136+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":8520,"log_dups_size":0,"ondisk_log_size":8520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34881536,"num_objects":8516,"num_object_clones":0,"num_object_copies":17032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5307,"num_write_kb":21228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4453},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8450","reported_seq":8994,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:28:31.063757+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:28:31.063757+0000","last_peered":"2024-09-17T15:28:31.063757+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:28:31.063757+0000","last_undegraded":"2024-09-17T15:28:31.063757+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":8450,"log_dups_size":0,"ondisk_log_size":8450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34563014,"num_objects":8440,"num_object_clones":0,"num_object_copies":16880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3908,"num_objects_unfound":0,"num_objects_dirty":8440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4892,"num_write_kb":19568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":823296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4532},{"shards":"0,2","objects":3908}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":556675806,"num_objects":135916,"num_object_clones":0,"num_object_copies":271832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3908,"num_objects_unfound":0,"num_objects_dirty":135916,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":96023,"num_write_kb":384092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4273,"num_bytes_recovered":17494499,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1136308224,"data_stored":1136156478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135988,"ondisk_log_size":135988,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051626,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":659748,"kb_used_data":403156,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93712092,"statfs":{"total":96636764160,"available":95961182208,"internally_reserved":0,"allocated":412831744,"data_stored":412555170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":262729388},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724572,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":426672,"kb_used_data":232544,"kb_used_omap":4,"kb_used_meta":194107,"kb_avail":93945168,"statfs":{"total":96636764160,"available":96199852032,"internally_reserved":0,"allocated":238125056,"data_stored":237801049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":198766251},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,5,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":757632,"kb_used_data":473328,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93614208,"statfs":{"total":96636764160,"available":95860948992,"internally_reserved":0,"allocated":484687872,"data_stored":484407221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":291106472},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,2,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":484839424,"data_stored":484803294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":238153728,"data_stored":238074242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":413315072,"data_stored":413278942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:28:36.806 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:28:37.320 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 178 20 138597 138577 3.04046 1.69141 0.0960756 0.0256872 2024-09-17T15:28:37.373 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:28:37.375 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:28:37.402 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":162,"stamp":"2024-09-17T15:28:35.636042+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":556675825,"num_objects":135918,"num_object_clones":0,"num_object_copies":271836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3908,"num_objects_unfound":0,"num_objects_dirty":135918,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":96025,"num_write_kb":384094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4274,"num_bytes_recovered":17494518,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135991,"ondisk_log_size":135991,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1844052,"kb_used_data":1109028,"kb_used_omap":13,"kb_used_meta":734962,"kb_avail":281271468,"statfs":{"total":289910292480,"available":288021983232,"internally_reserved":0,"allocated":1135644672,"data_stored":1134763440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":752602111},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,3,8,10,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19664896,"num_objects":4801,"num_object_clones":0,"num_object_copies":9602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-4111,"num_objects_misplaced":3893,"num_objects_unfound":0,"num_objects_dirty":4801,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4801,"num_write_kb":19204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":884736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.834364"},"pg_stats":[{"pgid":"2.f","version":"43'8419","reported_seq":8496,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:30.948394+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:28:30.948394+0000","last_peered":"2024-09-17T15:28:30.948394+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:28:30.948394+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":8419,"log_dups_size":0,"ondisk_log_size":8419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34484224,"num_objects":8419,"num_object_clones":0,"num_object_copies":16838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8419,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8419,"num_write_kb":33676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4349},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8497","reported_seq":8599,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:31.912549+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:28:31.912549+0000","last_peered":"2024-09-17T15:28:31.912549+0000","last_clean":"2024-09-17T15:28:31.912549+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:28:31.912549+0000","last_undegraded":"2024-09-17T15:28:31.912549+0000","last_fullsized":"2024-09-17T15:28:31.912549+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":8497,"log_dups_size":0,"ondisk_log_size":8497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34780102,"num_objects":8493,"num_object_clones":0,"num_object_copies":16986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8493,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6854,"num_write_kb":27416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8478","reported_seq":8603,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:32.106845+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:28:32.106845+0000","last_peered":"2024-09-17T15:28:32.106845+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:28:32.106845+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":8478,"log_dups_size":0,"ondisk_log_size":8478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34709504,"num_objects":8474,"num_object_clones":0,"num_object_copies":16948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6635,"num_write_kb":26540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":4306}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8594","reported_seq":8663,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:32.063120+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:28:32.063120+0000","last_peered":"2024-09-17T15:28:32.063120+0000","last_clean":"2024-09-17T15:28:32.063120+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:28:32.063120+0000","last_undegraded":"2024-09-17T15:28:32.063120+0000","last_fullsized":"2024-09-17T15:28:32.063120+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":8594,"log_dups_size":0,"ondisk_log_size":8594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35201024,"num_objects":8594,"num_object_clones":0,"num_object_copies":17188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8594,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8594,"num_write_kb":34376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8364","reported_seq":8493,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:31.025418+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:28:31.025418+0000","last_peered":"2024-09-17T15:28:31.025418+0000","last_clean":"2024-09-17T15:28:31.025418+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:28:31.025418+0000","last_undegraded":"2024-09-17T15:28:31.025418+0000","last_fullsized":"2024-09-17T15:28:31.025418+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":8364,"log_dups_size":0,"ondisk_log_size":8364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34230755,"num_objects":8358,"num_object_clones":0,"num_object_copies":16716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8358,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5346,"num_write_kb":21384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8673","reported_seq":8807,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:30.986200+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:28:30.986200+0000","last_peered":"2024-09-17T15:28:30.986200+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:28:30.986200+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":8673,"log_dups_size":0,"ondisk_log_size":8673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35504611,"num_objects":8669,"num_object_clones":0,"num_object_copies":17338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8669,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5172,"num_write_kb":20688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4399},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8528","reported_seq":8666,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:30.981154+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:28:30.981154+0000","last_peered":"2024-09-17T15:28:30.981154+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:28:30.981154+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":8528,"log_dups_size":0,"ondisk_log_size":8528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34907078,"num_objects":8524,"num_object_clones":0,"num_object_copies":17048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8524,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5297,"num_write_kb":21188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4350},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8543","reported_seq":8670,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:31.050041+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:28:31.050041+0000","last_peered":"2024-09-17T15:28:31.050041+0000","last_clean":"2024-09-17T15:28:31.050041+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:28:31.050041+0000","last_undegraded":"2024-09-17T15:28:31.050041+0000","last_fullsized":"2024-09-17T15:28:31.050041+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":8543,"log_dups_size":0,"ondisk_log_size":8543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34983936,"num_objects":8541,"num_object_clones":0,"num_object_copies":17082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5497,"num_write_kb":21988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8587","reported_seq":16869,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:35.306522+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:28:35.306522+0000","last_peered":"2024-09-17T15:28:35.306522+0000","last_clean":"2024-09-17T15:28:35.306522+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:28:35.306522+0000","last_undegraded":"2024-09-17T15:28:35.306522+0000","last_fullsized":"2024-09-17T15:28:35.306522+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":8587,"log_dups_size":0,"ondisk_log_size":8587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35144163,"num_objects":8581,"num_object_clones":0,"num_object_copies":17162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8581,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6587,"num_write_kb":26348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804705+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804705+0000","last_peered":"2024-09-17T15:28:23.804705+0000","last_clean":"2024-09-17T15:28:23.804705+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804705+0000","last_undegraded":"2024-09-17T15:28:23.804705+0000","last_fullsized":"2024-09-17T15:28:23.804705+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8352","reported_seq":8482,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:31.049074+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:28:31.049074+0000","last_peered":"2024-09-17T15:28:31.049074+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:28:31.049074+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":8352,"log_dups_size":0,"ondisk_log_size":8352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34177024,"num_objects":8344,"num_object_clones":0,"num_object_copies":16688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4948,"num_write_kb":19792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4334},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":102,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814215+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814215+0000","last_peered":"2024-09-17T15:28:23.814215+0000","last_clean":"2024-09-17T15:28:23.814215+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814215+0000","last_undegraded":"2024-09-17T15:28:23.814215+0000","last_fullsized":"2024-09-17T15:28:23.814215+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8408","reported_seq":8528,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:32.071846+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:28:32.071846+0000","last_peered":"2024-09-17T15:28:32.071846+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:28:32.071846+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":8408,"log_dups_size":0,"ondisk_log_size":8408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34419171,"num_objects":8404,"num_object_clones":0,"num_object_copies":16808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8404,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6303,"num_write_kb":25212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":4327}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8505","reported_seq":8623,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:32.107313+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:28:32.107313+0000","last_peered":"2024-09-17T15:28:32.107313+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:28:32.107313+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":8505,"log_dups_size":0,"ondisk_log_size":8505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34811904,"num_objects":8499,"num_object_clones":0,"num_object_copies":16998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8499,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6267,"num_write_kb":25068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":4362}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8547","reported_seq":8682,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:31.026488+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:28:31.026488+0000","last_peered":"2024-09-17T15:28:31.026488+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:28:31.026488+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":8547,"log_dups_size":0,"ondisk_log_size":8547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35000320,"num_objects":8545,"num_object_clones":0,"num_object_copies":17090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5095,"num_write_kb":20380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4407},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8523","reported_seq":8662,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:31.036570+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:28:31.036570+0000","last_peered":"2024-09-17T15:28:31.036570+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:28:31.036570+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":8523,"log_dups_size":0,"ondisk_log_size":8523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34877440,"num_objects":8515,"num_object_clones":0,"num_object_copies":17030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4810,"num_write_kb":19240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4358},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805074+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805074+0000","last_peered":"2024-09-17T15:28:23.805074+0000","last_clean":"2024-09-17T15:28:23.805074+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805074+0000","last_undegraded":"2024-09-17T15:28:23.805074+0000","last_fullsized":"2024-09-17T15:28:23.805074+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8520","reported_seq":8651,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:31.003136+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:28:31.003136+0000","last_peered":"2024-09-17T15:28:31.003136+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:28:31.003136+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":8520,"log_dups_size":0,"ondisk_log_size":8520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34881536,"num_objects":8516,"num_object_clones":0,"num_object_copies":17032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5307,"num_write_kb":21228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4453},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8450","reported_seq":8994,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:28:31.063757+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:28:31.063757+0000","last_peered":"2024-09-17T15:28:31.063757+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:28:31.063757+0000","last_undegraded":"2024-09-17T15:28:31.063757+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":8450,"log_dups_size":0,"ondisk_log_size":8450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34563014,"num_objects":8440,"num_object_clones":0,"num_object_copies":16880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3908,"num_objects_unfound":0,"num_objects_dirty":8440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4892,"num_write_kb":19568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":823296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4532},{"shards":"0,2","objects":3908}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":556675806,"num_objects":135916,"num_object_clones":0,"num_object_copies":271832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3908,"num_objects_unfound":0,"num_objects_dirty":135916,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":96023,"num_write_kb":384092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4273,"num_bytes_recovered":17494499,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1136308224,"data_stored":1136156478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135988,"ondisk_log_size":135988,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051626,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":659748,"kb_used_data":403156,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93712092,"statfs":{"total":96636764160,"available":95961182208,"internally_reserved":0,"allocated":412831744,"data_stored":412555170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":262729388},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724572,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":426672,"kb_used_data":232544,"kb_used_omap":4,"kb_used_meta":194107,"kb_avail":93945168,"statfs":{"total":96636764160,"available":96199852032,"internally_reserved":0,"allocated":238125056,"data_stored":237801049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":198766251},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,5,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":757632,"kb_used_data":473328,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93614208,"statfs":{"total":96636764160,"available":95860948992,"internally_reserved":0,"allocated":484687872,"data_stored":484407221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":291106472},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,2,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":484839424,"data_stored":484803294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":238153728,"data_stored":238074242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":413315072,"data_stored":413278942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:28:37.404 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:28:37.404 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'8419", 'reported_seq': 8496, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:30.948394+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:28:30.948394+0000', 'last_peered': '2024-09-17T15:28:30.948394+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:28:30.948394+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 8419, 'log_dups_size': 0, 'ondisk_log_size': 8419, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34484224, 'num_objects': 8419, 'num_object_clones': 0, 'num_object_copies': 16838, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4070, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8419, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8419, 'num_write_kb': 33676, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4349}, {'shards': '0,2', 'objects': 4070}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:37.404 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:28:37.404 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'8478", 'reported_seq': 8603, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:32.106845+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:28:32.106845+0000', 'last_peered': '2024-09-17T15:28:32.106845+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:28:32.106845+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 8478, 'log_dups_size': 0, 'ondisk_log_size': 8478, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34709504, 'num_objects': 8474, 'num_object_clones': 0, 'num_object_copies': 16948, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4169, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8474, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6635, 'num_write_kb': 26540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4168}, {'shards': '2', 'objects': 4306}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:28:37.404 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:28:37.404 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'8673", 'reported_seq': 8807, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:30.986200+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:28:30.986200+0000', 'last_peered': '2024-09-17T15:28:30.986200+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:28:30.986200+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 8673, 'log_dups_size': 0, 'ondisk_log_size': 8673, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35504611, 'num_objects': 8669, 'num_object_clones': 0, 'num_object_copies': 17338, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4271, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8669, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5172, 'num_write_kb': 20688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4399}, {'shards': '0,2', 'objects': 4268}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:37.404 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:28:37.405 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'8528", 'reported_seq': 8666, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:30.981154+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:28:30.981154+0000', 'last_peered': '2024-09-17T15:28:30.981154+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:28:30.981154+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 8528, 'log_dups_size': 0, 'ondisk_log_size': 8528, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34907078, 'num_objects': 8524, 'num_object_clones': 0, 'num_object_copies': 17048, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4174, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8524, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 5297, 'num_write_kb': 21188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4350}, {'shards': '0,2', 'objects': 4174}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:37.405 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:28:37.405 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'8352", 'reported_seq': 8482, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:31.049074+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:28:31.049074+0000', 'last_peered': '2024-09-17T15:28:31.049074+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:28:31.049074+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 8352, 'log_dups_size': 0, 'ondisk_log_size': 8352, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34177024, 'num_objects': 8344, 'num_object_clones': 0, 'num_object_copies': 16688, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4012, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8344, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4948, 'num_write_kb': 19792, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4334}, {'shards': '0,2', 'objects': 4010}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:37.405 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:28:37.405 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'8408", 'reported_seq': 8528, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:32.071846+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:28:32.071846+0000', 'last_peered': '2024-09-17T15:28:32.071846+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:28:32.071846+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 8408, 'log_dups_size': 0, 'ondisk_log_size': 8408, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34419171, 'num_objects': 8404, 'num_object_clones': 0, 'num_object_copies': 16808, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4078, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8404, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6303, 'num_write_kb': 25212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4077}, {'shards': '2', 'objects': 4327}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:28:37.405 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:28:37.405 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'8505", 'reported_seq': 8623, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:32.107313+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:28:32.107313+0000', 'last_peered': '2024-09-17T15:28:32.107313+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:28:32.107313+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 8505, 'log_dups_size': 0, 'ondisk_log_size': 8505, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34811904, 'num_objects': 8499, 'num_object_clones': 0, 'num_object_copies': 16998, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8499, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6267, 'num_write_kb': 25068, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4137}, {'shards': '2', 'objects': 4362}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:28:37.405 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:28:37.405 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'8547", 'reported_seq': 8682, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:31.026488+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:28:31.026488+0000', 'last_peered': '2024-09-17T15:28:31.026488+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:28:31.026488+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 8547, 'log_dups_size': 0, 'ondisk_log_size': 8547, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35000320, 'num_objects': 8545, 'num_object_clones': 0, 'num_object_copies': 17090, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8545, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5095, 'num_write_kb': 20380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4407}, {'shards': '0,2', 'objects': 4138}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:37.406 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:28:37.406 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'8523", 'reported_seq': 8662, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:31.036570+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:28:31.036570+0000', 'last_peered': '2024-09-17T15:28:31.036570+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:28:31.036570+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 8523, 'log_dups_size': 0, 'ondisk_log_size': 8523, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34877440, 'num_objects': 8515, 'num_object_clones': 0, 'num_object_copies': 17030, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4160, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8515, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4810, 'num_write_kb': 19240, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4358}, {'shards': '0,2', 'objects': 4157}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:37.406 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:28:37.406 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'8520", 'reported_seq': 8651, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:31.003136+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:28:31.003136+0000', 'last_peered': '2024-09-17T15:28:31.003136+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:28:31.003136+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 8520, 'log_dups_size': 0, 'ondisk_log_size': 8520, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34881536, 'num_objects': 8516, 'num_object_clones': 0, 'num_object_copies': 17032, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4065, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8516, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5307, 'num_write_kb': 21228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4453}, {'shards': '0,2', 'objects': 4063}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:37.406 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:28:37.406 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'8450", 'reported_seq': 8994, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:28:31.063757+0000', 'last_change': '2024-09-17T15:28:22.509818+0000', 'last_active': '2024-09-17T15:28:31.063757+0000', 'last_peered': '2024-09-17T15:28:31.063757+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:28:31.063757+0000', 'last_undegraded': '2024-09-17T15:28:31.063757+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 8450, 'log_dups_size': 0, 'ondisk_log_size': 8450, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34563014, 'num_objects': 8440, 'num_object_clones': 0, 'num_object_copies': 16880, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3908, 'num_objects_unfound': 0, 'num_objects_dirty': 8440, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4892, 'num_write_kb': 19568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 201, 'num_bytes_recovered': 823296, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4532}, {'shards': '0,2', 'objects': 3908}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:37.406 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:28:38.099 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:28:38.099 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41277/276350 objects degraded (14.936%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":255,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":138175,"data_bytes":565920497,"bytes_used":1927593984,"bytes_avail":287982698496,"bytes_total":289910292480,"degraded_objects":41277,"degraded_total":276350,"degraded_ratio":0.14936493576985707,"misplaced_objects":3806,"misplaced_total":276350,"misplaced_ratio":0.013772390085037091,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":103399,"recovering_keys_per_sec":0,"num_objects_recovered":303,"num_bytes_recovered":1241088,"num_keys_recovered":0,"write_bytes_sec":2377168,"read_op_per_sec":0,"write_op_per_sec":580},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T15:27:19.616376+0000","services":{}},"progress_events":{}} 2024-09-17T15:28:38.119 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:28:38.119 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:28:38.321 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 179 20 139034 139014 3.03301 1.70703 0.0295081 0.0257507 2024-09-17T15:28:38.667 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:28:38.668 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:28:38.695 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":163,"stamp":"2024-09-17T15:28:37.636534+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":565920497,"num_objects":138175,"num_object_clones":0,"num_object_copies":276350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3806,"num_objects_unfound":0,"num_objects_dirty":138175,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":98282,"num_write_kb":393122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4376,"num_bytes_recovered":17912310,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138248,"ondisk_log_size":138248,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1882416,"kb_used_data":1121024,"kb_used_omap":13,"kb_used_meta":761330,"kb_avail":281233104,"statfs":{"total":289910292480,"available":287982698496,"internally_reserved":0,"allocated":1147928576,"data_stored":1147047344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":779602943},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,2,10,13,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28532736,"num_objects":6966,"num_object_clones":0,"num_object_copies":13932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-4111,"num_objects_misplaced":3806,"num_objects_unfound":0,"num_objects_dirty":6966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6966,"num_write_kb":27864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":1241088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002825"},"pg_stats":[{"pgid":"2.f","version":"43'8573","reported_seq":8650,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:36.107895+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:28:36.107895+0000","last_peered":"2024-09-17T15:28:36.107895+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:28:36.107895+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":8573,"log_dups_size":0,"ondisk_log_size":8573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35115008,"num_objects":8573,"num_object_clones":0,"num_object_copies":17146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8573,"num_write_kb":34292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4503},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8657","reported_seq":8759,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:37.122306+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:28:37.122306+0000","last_peered":"2024-09-17T15:28:37.122306+0000","last_clean":"2024-09-17T15:28:37.122306+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:28:37.122306+0000","last_undegraded":"2024-09-17T15:28:37.122306+0000","last_fullsized":"2024-09-17T15:28:37.122306+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":8657,"log_dups_size":0,"ondisk_log_size":8657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35435462,"num_objects":8653,"num_object_clones":0,"num_object_copies":17306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8653,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7014,"num_write_kb":28056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8625","reported_seq":8747,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:37.116307+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:28:37.116307+0000","last_peered":"2024-09-17T15:28:37.116307+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:28:37.116307+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":8625,"log_dups_size":0,"ondisk_log_size":8625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35311616,"num_objects":8621,"num_object_clones":0,"num_object_copies":17242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6782,"num_write_kb":27128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":4453}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8735","reported_seq":8804,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:37.101365+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:28:37.101365+0000","last_peered":"2024-09-17T15:28:37.101365+0000","last_clean":"2024-09-17T15:28:37.101365+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:28:37.101365+0000","last_undegraded":"2024-09-17T15:28:37.101365+0000","last_fullsized":"2024-09-17T15:28:37.101365+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":8735,"log_dups_size":0,"ondisk_log_size":8735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35778560,"num_objects":8735,"num_object_clones":0,"num_object_copies":17470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8735,"num_write_kb":34940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8527","reported_seq":8656,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:35.986274+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:28:35.986274+0000","last_peered":"2024-09-17T15:28:35.986274+0000","last_clean":"2024-09-17T15:28:35.986274+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:28:35.986274+0000","last_undegraded":"2024-09-17T15:28:35.986274+0000","last_fullsized":"2024-09-17T15:28:35.986274+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":8527,"log_dups_size":0,"ondisk_log_size":8527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34898403,"num_objects":8521,"num_object_clones":0,"num_object_copies":17042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8521,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5509,"num_write_kb":22036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8819","reported_seq":8953,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:36.033789+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:28:36.033789+0000","last_peered":"2024-09-17T15:28:36.033789+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:28:36.033789+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":8819,"log_dups_size":0,"ondisk_log_size":8819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36102627,"num_objects":8815,"num_object_clones":0,"num_object_copies":17630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8815,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5318,"num_write_kb":21272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4545},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8673","reported_seq":8811,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:36.097069+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:28:36.097069+0000","last_peered":"2024-09-17T15:28:36.097069+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:28:36.097069+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":8673,"log_dups_size":0,"ondisk_log_size":8673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35500998,"num_objects":8669,"num_object_clones":0,"num_object_copies":17338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8669,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5442,"num_write_kb":21768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4495},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8702","reported_seq":8829,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:35.983977+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:28:35.983977+0000","last_peered":"2024-09-17T15:28:35.983977+0000","last_clean":"2024-09-17T15:28:35.983977+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:28:35.983977+0000","last_undegraded":"2024-09-17T15:28:35.983977+0000","last_fullsized":"2024-09-17T15:28:35.983977+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":8702,"log_dups_size":0,"ondisk_log_size":8702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35635200,"num_objects":8700,"num_object_clones":0,"num_object_copies":17400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5656,"num_write_kb":22624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8587","reported_seq":16869,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:35.306522+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:28:35.306522+0000","last_peered":"2024-09-17T15:28:35.306522+0000","last_clean":"2024-09-17T15:28:35.306522+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:28:35.306522+0000","last_undegraded":"2024-09-17T15:28:35.306522+0000","last_fullsized":"2024-09-17T15:28:35.306522+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":8587,"log_dups_size":0,"ondisk_log_size":8587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35144163,"num_objects":8581,"num_object_clones":0,"num_object_copies":17162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8581,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6587,"num_write_kb":26348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804705+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804705+0000","last_peered":"2024-09-17T15:28:23.804705+0000","last_clean":"2024-09-17T15:28:23.804705+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804705+0000","last_undegraded":"2024-09-17T15:28:23.804705+0000","last_fullsized":"2024-09-17T15:28:23.804705+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8498","reported_seq":8626,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:36.049695+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:28:36.049695+0000","last_peered":"2024-09-17T15:28:36.049695+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:28:36.049695+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":8498,"log_dups_size":0,"ondisk_log_size":8498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34775040,"num_objects":8490,"num_object_clones":0,"num_object_copies":16980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5094,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4480},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":102,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814215+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814215+0000","last_peered":"2024-09-17T15:28:23.814215+0000","last_clean":"2024-09-17T15:28:23.814215+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814215+0000","last_undegraded":"2024-09-17T15:28:23.814215+0000","last_fullsized":"2024-09-17T15:28:23.814215+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8574","reported_seq":8694,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:37.101399+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:28:37.101399+0000","last_peered":"2024-09-17T15:28:37.101399+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:28:37.101399+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":8574,"log_dups_size":0,"ondisk_log_size":8574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35099107,"num_objects":8570,"num_object_clones":0,"num_object_copies":17140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8570,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6469,"num_write_kb":25876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":4493}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8637","reported_seq":8755,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:37.068156+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:28:37.068156+0000","last_peered":"2024-09-17T15:28:37.068156+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:28:37.068156+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":8637,"log_dups_size":0,"ondisk_log_size":8637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35352576,"num_objects":8631,"num_object_clones":0,"num_object_copies":17262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6399,"num_write_kb":25596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":4494}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8712","reported_seq":8847,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:36.117153+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:28:36.117153+0000","last_peered":"2024-09-17T15:28:36.117153+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:28:36.117153+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":8712,"log_dups_size":0,"ondisk_log_size":8712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35676160,"num_objects":8710,"num_object_clones":0,"num_object_copies":17420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5260,"num_write_kb":21040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4572},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8682","reported_seq":8821,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:36.070966+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:28:36.070966+0000","last_peered":"2024-09-17T15:28:36.070966+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:28:36.070966+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":8682,"log_dups_size":0,"ondisk_log_size":8682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35528704,"num_objects":8674,"num_object_clones":0,"num_object_copies":17348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4969,"num_write_kb":19876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4517},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805074+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805074+0000","last_peered":"2024-09-17T15:28:23.805074+0000","last_clean":"2024-09-17T15:28:23.805074+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805074+0000","last_undegraded":"2024-09-17T15:28:23.805074+0000","last_fullsized":"2024-09-17T15:28:23.805074+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8667","reported_seq":8798,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:36.050979+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:28:36.050979+0000","last_peered":"2024-09-17T15:28:36.050979+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:28:36.050979+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":8667,"log_dups_size":0,"ondisk_log_size":8667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35483648,"num_objects":8663,"num_object_clones":0,"num_object_copies":17326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5454,"num_write_kb":21816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4600},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8577","reported_seq":9326,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:28:36.137159+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:28:36.137159+0000","last_peered":"2024-09-17T15:28:36.137159+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:28:36.137159+0000","last_undegraded":"2024-09-17T15:28:36.137159+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":8577,"log_dups_size":0,"ondisk_log_size":8577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35083206,"num_objects":8567,"num_object_clones":0,"num_object_copies":17134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3806,"num_objects_unfound":0,"num_objects_dirty":8567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5019,"num_write_kb":20076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":1241088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4761},{"shards":"0,2","objects":3806}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":565920478,"num_objects":138173,"num_object_clones":0,"num_object_copies":276346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3806,"num_objects_unfound":0,"num_objects_dirty":138173,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":98280,"num_write_kb":393120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4375,"num_bytes_recovered":17912291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1148768256,"data_stored":1148616510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138245,"ondisk_log_size":138245,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051627,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":682652,"kb_used_data":408012,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93689188,"statfs":{"total":96636764160,"available":95937728512,"internally_reserved":0,"allocated":417804288,"data_stored":417527714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":281210540},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724572,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":426672,"kb_used_data":232544,"kb_used_omap":4,"kb_used_meta":194107,"kb_avail":93945168,"statfs":{"total":96636764160,"available":96199852032,"internally_reserved":0,"allocated":238125056,"data_stored":237801049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":198766251},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,5,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247855,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":773092,"kb_used_data":480468,"kb_used_omap":4,"kb_used_meta":292603,"kb_avail":93598748,"statfs":{"total":96636764160,"available":95845117952,"internally_reserved":0,"allocated":491999232,"data_stored":491718581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":299626152},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,3,5,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":492294144,"data_stored":492258014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":238153728,"data_stored":238074242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":418320384,"data_stored":418284254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:28:39.321 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout:2024-09-17T15:28:39.321366+0000 min lat: 0.00187689 max lat: 2.64156 avg lat: 0.025805 2024-09-17T15:28:39.321 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-17T15:28:39.321 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 180 20 139512 139492 3.02654 1.86719 0.0418584 0.025805 2024-09-17T15:28:40.321 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 181 20 140009 139989 3.02054 1.94141 0.0261025 0.025857 2024-09-17T15:28:41.321 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 182 20 140502 140482 3.01452 1.92578 0.0323205 0.0259087 2024-09-17T15:28:41.698 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:28:42.260 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:28:42.263 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:28:42.287 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":165,"stamp":"2024-09-17T15:28:41.637480+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":572613361,"num_objects":139809,"num_object_clones":0,"num_object_copies":279618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3705,"num_objects_unfound":0,"num_objects_dirty":139809,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":99916,"num_write_kb":399658,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4477,"num_bytes_recovered":18326006,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139882,"ondisk_log_size":139882,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1888372,"kb_used_data":1135300,"kb_used_omap":13,"kb_used_meta":753010,"kb_avail":281227148,"statfs":{"total":289910292480,"available":287976599552,"internally_reserved":0,"allocated":1162547200,"data_stored":1161665968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":771083263},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,2,10,13,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26603520,"num_objects":6495,"num_object_clones":0,"num_object_copies":12990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-290,"num_objects_unfound":0,"num_objects_dirty":6495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6495,"num_write_kb":25980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":1187840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002791"},"pg_stats":[{"pgid":"2.f","version":"43'8727","reported_seq":8801,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:41.117808+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:28:41.117808+0000","last_peered":"2024-09-17T15:28:41.117808+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:28:41.117808+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":8727,"log_dups_size":0,"ondisk_log_size":8727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35745792,"num_objects":8727,"num_object_clones":0,"num_object_copies":17454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8727,"num_write_kb":34908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4657},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8657","reported_seq":8759,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:37.122306+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:28:37.122306+0000","last_peered":"2024-09-17T15:28:37.122306+0000","last_clean":"2024-09-17T15:28:37.122306+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:28:37.122306+0000","last_undegraded":"2024-09-17T15:28:37.122306+0000","last_fullsized":"2024-09-17T15:28:37.122306+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":8657,"log_dups_size":0,"ondisk_log_size":8657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35435462,"num_objects":8653,"num_object_clones":0,"num_object_copies":17306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8653,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7014,"num_write_kb":28056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8625","reported_seq":8747,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:37.116307+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:28:37.116307+0000","last_peered":"2024-09-17T15:28:37.116307+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:28:37.116307+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":8625,"log_dups_size":0,"ondisk_log_size":8625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35311616,"num_objects":8621,"num_object_clones":0,"num_object_copies":17242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6782,"num_write_kb":27128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":4453}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8735","reported_seq":8804,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:37.101365+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:28:37.101365+0000","last_peered":"2024-09-17T15:28:37.101365+0000","last_clean":"2024-09-17T15:28:37.101365+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:28:37.101365+0000","last_undegraded":"2024-09-17T15:28:37.101365+0000","last_fullsized":"2024-09-17T15:28:37.101365+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":8735,"log_dups_size":0,"ondisk_log_size":8735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35778560,"num_objects":8735,"num_object_clones":0,"num_object_copies":17470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8735,"num_write_kb":34940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8669","reported_seq":8798,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:41.081428+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:28:41.081428+0000","last_peered":"2024-09-17T15:28:41.081428+0000","last_clean":"2024-09-17T15:28:41.081428+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:28:41.081428+0000","last_undegraded":"2024-09-17T15:28:41.081428+0000","last_fullsized":"2024-09-17T15:28:41.081428+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":8669,"log_dups_size":0,"ondisk_log_size":8669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35480035,"num_objects":8663,"num_object_clones":0,"num_object_copies":17326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8663,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5651,"num_write_kb":22604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8981","reported_seq":9115,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:41.122828+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:28:41.122828+0000","last_peered":"2024-09-17T15:28:41.122828+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:28:41.122828+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":8981,"log_dups_size":0,"ondisk_log_size":8981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36766179,"num_objects":8977,"num_object_clones":0,"num_object_copies":17954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8977,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5480,"num_write_kb":21920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4707},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8817","reported_seq":8952,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:41.134380+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:28:41.134380+0000","last_peered":"2024-09-17T15:28:41.134380+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:28:41.134380+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":8817,"log_dups_size":0,"ondisk_log_size":8817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36090822,"num_objects":8813,"num_object_clones":0,"num_object_copies":17626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8813,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5586,"num_write_kb":22344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4639},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8857","reported_seq":8984,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:41.065697+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:28:41.065697+0000","last_peered":"2024-09-17T15:28:41.065697+0000","last_clean":"2024-09-17T15:28:41.065697+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:28:41.065697+0000","last_undegraded":"2024-09-17T15:28:41.065697+0000","last_fullsized":"2024-09-17T15:28:41.065697+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":8857,"log_dups_size":0,"ondisk_log_size":8857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36270080,"num_objects":8855,"num_object_clones":0,"num_object_copies":17710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5811,"num_write_kb":23244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8745","reported_seq":17027,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:40.312124+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:28:40.312124+0000","last_peered":"2024-09-17T15:28:40.312124+0000","last_clean":"2024-09-17T15:28:40.312124+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:28:40.312124+0000","last_undegraded":"2024-09-17T15:28:40.312124+0000","last_fullsized":"2024-09-17T15:28:40.312124+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":8745,"log_dups_size":0,"ondisk_log_size":8745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35791331,"num_objects":8739,"num_object_clones":0,"num_object_copies":17478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8739,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6745,"num_write_kb":26980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804705+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804705+0000","last_peered":"2024-09-17T15:28:23.804705+0000","last_clean":"2024-09-17T15:28:23.804705+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804705+0000","last_undegraded":"2024-09-17T15:28:23.804705+0000","last_fullsized":"2024-09-17T15:28:23.804705+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8642","reported_seq":8772,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:41.123377+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:28:41.123377+0000","last_peered":"2024-09-17T15:28:41.123377+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:28:41.123377+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":8642,"log_dups_size":0,"ondisk_log_size":8642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35364864,"num_objects":8634,"num_object_clones":0,"num_object_copies":17268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5238,"num_write_kb":20952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4624},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":102,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814215+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814215+0000","last_peered":"2024-09-17T15:28:23.814215+0000","last_clean":"2024-09-17T15:28:23.814215+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814215+0000","last_undegraded":"2024-09-17T15:28:23.814215+0000","last_fullsized":"2024-09-17T15:28:23.814215+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8574","reported_seq":8694,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:37.101399+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:28:37.101399+0000","last_peered":"2024-09-17T15:28:37.101399+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:28:37.101399+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":8574,"log_dups_size":0,"ondisk_log_size":8574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35099107,"num_objects":8570,"num_object_clones":0,"num_object_copies":17140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8570,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6469,"num_write_kb":25876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":4493}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8637","reported_seq":8755,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:37.068156+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:28:37.068156+0000","last_peered":"2024-09-17T15:28:37.068156+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:28:37.068156+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":8637,"log_dups_size":0,"ondisk_log_size":8637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35352576,"num_objects":8631,"num_object_clones":0,"num_object_copies":17262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6399,"num_write_kb":25596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":4494}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8852","reported_seq":8987,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:41.121520+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:28:41.121520+0000","last_peered":"2024-09-17T15:28:41.121520+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:28:41.121520+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":8852,"log_dups_size":0,"ondisk_log_size":8852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36249600,"num_objects":8850,"num_object_clones":0,"num_object_copies":17700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5400,"num_write_kb":21600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4712},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8827","reported_seq":8967,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:41.084341+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:28:41.084341+0000","last_peered":"2024-09-17T15:28:41.084341+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:28:41.084341+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":8827,"log_dups_size":0,"ondisk_log_size":8827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36122624,"num_objects":8819,"num_object_clones":0,"num_object_copies":17638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8819,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5114,"num_write_kb":20456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4662},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805074+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805074+0000","last_peered":"2024-09-17T15:28:23.805074+0000","last_clean":"2024-09-17T15:28:23.805074+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805074+0000","last_undegraded":"2024-09-17T15:28:23.805074+0000","last_fullsized":"2024-09-17T15:28:23.805074+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8825","reported_seq":8956,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:41.069135+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:28:41.069135+0000","last_peered":"2024-09-17T15:28:41.069135+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:28:41.069135+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":8825,"log_dups_size":0,"ondisk_log_size":8825,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36130816,"num_objects":8821,"num_object_clones":0,"num_object_copies":17642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8821,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5612,"num_write_kb":22448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4758},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8709","reported_seq":9660,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:28:41.159131+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:28:41.159131+0000","last_peered":"2024-09-17T15:28:41.159131+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:28:41.159131+0000","last_undegraded":"2024-09-17T15:28:41.159131+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":8709,"log_dups_size":0,"ondisk_log_size":8709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35623878,"num_objects":8699,"num_object_clones":0,"num_object_copies":17398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3705,"num_objects_unfound":0,"num_objects_dirty":8699,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5151,"num_write_kb":20604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":404,"num_bytes_recovered":1654784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4994},{"shards":"0,2","objects":3705}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":572613342,"num_objects":139807,"num_object_clones":0,"num_object_copies":279614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3705,"num_objects_unfound":0,"num_objects_dirty":139807,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":99914,"num_write_kb":399656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4476,"num_bytes_recovered":18325987,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1163681792,"data_stored":1163530046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139879,"ondisk_log_size":139879,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051627,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":682652,"kb_used_data":408012,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93689188,"statfs":{"total":96636764160,"available":95937728512,"internally_reserved":0,"allocated":417804288,"data_stored":417527714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":281210540},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724573,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":438160,"kb_used_data":239872,"kb_used_omap":4,"kb_used_meta":198267,"kb_avail":93933680,"statfs":{"total":96636764160,"available":96188088320,"internally_reserved":0,"allocated":245628928,"data_stored":245304921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":203026091},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,5,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247856,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":767560,"kb_used_data":487416,"kb_used_omap":4,"kb_used_meta":280123,"kb_avail":93604280,"statfs":{"total":96636764160,"available":95850782720,"internally_reserved":0,"allocated":499113984,"data_stored":498833333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":286846632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,3,5,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":499523584,"data_stored":499487454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":245837824,"data_stored":245758338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":418320384,"data_stored":418284254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:28:42.288 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:28:42.325 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 183 20 140962 140942 3.00787 1.79688 0.0206106 0.0259642 2024-09-17T15:28:42.865 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:28:42.865 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:28:42.888 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":165,"stamp":"2024-09-17T15:28:41.637480+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":572613361,"num_objects":139809,"num_object_clones":0,"num_object_copies":279618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3705,"num_objects_unfound":0,"num_objects_dirty":139809,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":99916,"num_write_kb":399658,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4477,"num_bytes_recovered":18326006,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139882,"ondisk_log_size":139882,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1888372,"kb_used_data":1135300,"kb_used_omap":13,"kb_used_meta":753010,"kb_avail":281227148,"statfs":{"total":289910292480,"available":287976599552,"internally_reserved":0,"allocated":1162547200,"data_stored":1161665968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":771083263},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,2,10,13,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26603520,"num_objects":6495,"num_object_clones":0,"num_object_copies":12990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-290,"num_objects_unfound":0,"num_objects_dirty":6495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6495,"num_write_kb":25980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":1187840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002791"},"pg_stats":[{"pgid":"2.f","version":"43'8727","reported_seq":8801,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:41.117808+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:28:41.117808+0000","last_peered":"2024-09-17T15:28:41.117808+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:28:41.117808+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":8727,"log_dups_size":0,"ondisk_log_size":8727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35745792,"num_objects":8727,"num_object_clones":0,"num_object_copies":17454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8727,"num_write_kb":34908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4657},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8657","reported_seq":8759,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:37.122306+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:28:37.122306+0000","last_peered":"2024-09-17T15:28:37.122306+0000","last_clean":"2024-09-17T15:28:37.122306+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:28:37.122306+0000","last_undegraded":"2024-09-17T15:28:37.122306+0000","last_fullsized":"2024-09-17T15:28:37.122306+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":8657,"log_dups_size":0,"ondisk_log_size":8657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35435462,"num_objects":8653,"num_object_clones":0,"num_object_copies":17306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8653,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7014,"num_write_kb":28056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8625","reported_seq":8747,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:37.116307+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:28:37.116307+0000","last_peered":"2024-09-17T15:28:37.116307+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:28:37.116307+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":8625,"log_dups_size":0,"ondisk_log_size":8625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35311616,"num_objects":8621,"num_object_clones":0,"num_object_copies":17242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6782,"num_write_kb":27128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":4453}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8735","reported_seq":8804,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:37.101365+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:28:37.101365+0000","last_peered":"2024-09-17T15:28:37.101365+0000","last_clean":"2024-09-17T15:28:37.101365+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:28:37.101365+0000","last_undegraded":"2024-09-17T15:28:37.101365+0000","last_fullsized":"2024-09-17T15:28:37.101365+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":8735,"log_dups_size":0,"ondisk_log_size":8735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35778560,"num_objects":8735,"num_object_clones":0,"num_object_copies":17470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8735,"num_write_kb":34940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8669","reported_seq":8798,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:41.081428+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:28:41.081428+0000","last_peered":"2024-09-17T15:28:41.081428+0000","last_clean":"2024-09-17T15:28:41.081428+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:28:41.081428+0000","last_undegraded":"2024-09-17T15:28:41.081428+0000","last_fullsized":"2024-09-17T15:28:41.081428+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":8669,"log_dups_size":0,"ondisk_log_size":8669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35480035,"num_objects":8663,"num_object_clones":0,"num_object_copies":17326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8663,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5651,"num_write_kb":22604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8981","reported_seq":9115,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:41.122828+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:28:41.122828+0000","last_peered":"2024-09-17T15:28:41.122828+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:28:41.122828+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":8981,"log_dups_size":0,"ondisk_log_size":8981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36766179,"num_objects":8977,"num_object_clones":0,"num_object_copies":17954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8977,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5480,"num_write_kb":21920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4707},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8817","reported_seq":8952,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:41.134380+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:28:41.134380+0000","last_peered":"2024-09-17T15:28:41.134380+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:28:41.134380+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":8817,"log_dups_size":0,"ondisk_log_size":8817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36090822,"num_objects":8813,"num_object_clones":0,"num_object_copies":17626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8813,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5586,"num_write_kb":22344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4639},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8857","reported_seq":8984,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:41.065697+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:28:41.065697+0000","last_peered":"2024-09-17T15:28:41.065697+0000","last_clean":"2024-09-17T15:28:41.065697+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:28:41.065697+0000","last_undegraded":"2024-09-17T15:28:41.065697+0000","last_fullsized":"2024-09-17T15:28:41.065697+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":8857,"log_dups_size":0,"ondisk_log_size":8857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36270080,"num_objects":8855,"num_object_clones":0,"num_object_copies":17710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5811,"num_write_kb":23244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8745","reported_seq":17027,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:40.312124+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:28:40.312124+0000","last_peered":"2024-09-17T15:28:40.312124+0000","last_clean":"2024-09-17T15:28:40.312124+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:28:40.312124+0000","last_undegraded":"2024-09-17T15:28:40.312124+0000","last_fullsized":"2024-09-17T15:28:40.312124+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":8745,"log_dups_size":0,"ondisk_log_size":8745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35791331,"num_objects":8739,"num_object_clones":0,"num_object_copies":17478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8739,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6745,"num_write_kb":26980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804705+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804705+0000","last_peered":"2024-09-17T15:28:23.804705+0000","last_clean":"2024-09-17T15:28:23.804705+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804705+0000","last_undegraded":"2024-09-17T15:28:23.804705+0000","last_fullsized":"2024-09-17T15:28:23.804705+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8642","reported_seq":8772,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:41.123377+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:28:41.123377+0000","last_peered":"2024-09-17T15:28:41.123377+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:28:41.123377+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":8642,"log_dups_size":0,"ondisk_log_size":8642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35364864,"num_objects":8634,"num_object_clones":0,"num_object_copies":17268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5238,"num_write_kb":20952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4624},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":102,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814215+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814215+0000","last_peered":"2024-09-17T15:28:23.814215+0000","last_clean":"2024-09-17T15:28:23.814215+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814215+0000","last_undegraded":"2024-09-17T15:28:23.814215+0000","last_fullsized":"2024-09-17T15:28:23.814215+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8574","reported_seq":8694,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:37.101399+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:28:37.101399+0000","last_peered":"2024-09-17T15:28:37.101399+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:28:37.101399+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":8574,"log_dups_size":0,"ondisk_log_size":8574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35099107,"num_objects":8570,"num_object_clones":0,"num_object_copies":17140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8570,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6469,"num_write_kb":25876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":4493}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8637","reported_seq":8755,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:37.068156+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:28:37.068156+0000","last_peered":"2024-09-17T15:28:37.068156+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:28:37.068156+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":8637,"log_dups_size":0,"ondisk_log_size":8637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35352576,"num_objects":8631,"num_object_clones":0,"num_object_copies":17262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6399,"num_write_kb":25596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":4494}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8852","reported_seq":8987,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:41.121520+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:28:41.121520+0000","last_peered":"2024-09-17T15:28:41.121520+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:28:41.121520+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":8852,"log_dups_size":0,"ondisk_log_size":8852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36249600,"num_objects":8850,"num_object_clones":0,"num_object_copies":17700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5400,"num_write_kb":21600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4712},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8827","reported_seq":8967,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:41.084341+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:28:41.084341+0000","last_peered":"2024-09-17T15:28:41.084341+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:28:41.084341+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":8827,"log_dups_size":0,"ondisk_log_size":8827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36122624,"num_objects":8819,"num_object_clones":0,"num_object_copies":17638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8819,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5114,"num_write_kb":20456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4662},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805074+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805074+0000","last_peered":"2024-09-17T15:28:23.805074+0000","last_clean":"2024-09-17T15:28:23.805074+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805074+0000","last_undegraded":"2024-09-17T15:28:23.805074+0000","last_fullsized":"2024-09-17T15:28:23.805074+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8825","reported_seq":8956,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:41.069135+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:28:41.069135+0000","last_peered":"2024-09-17T15:28:41.069135+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:28:41.069135+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":8825,"log_dups_size":0,"ondisk_log_size":8825,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36130816,"num_objects":8821,"num_object_clones":0,"num_object_copies":17642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8821,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5612,"num_write_kb":22448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4758},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8709","reported_seq":9660,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:28:41.159131+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:28:41.159131+0000","last_peered":"2024-09-17T15:28:41.159131+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:28:41.159131+0000","last_undegraded":"2024-09-17T15:28:41.159131+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":8709,"log_dups_size":0,"ondisk_log_size":8709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35623878,"num_objects":8699,"num_object_clones":0,"num_object_copies":17398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3705,"num_objects_unfound":0,"num_objects_dirty":8699,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5151,"num_write_kb":20604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":404,"num_bytes_recovered":1654784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4994},{"shards":"0,2","objects":3705}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":572613342,"num_objects":139807,"num_object_clones":0,"num_object_copies":279614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3705,"num_objects_unfound":0,"num_objects_dirty":139807,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":99914,"num_write_kb":399656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4476,"num_bytes_recovered":18325987,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1163681792,"data_stored":1163530046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139879,"ondisk_log_size":139879,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051627,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":682652,"kb_used_data":408012,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93689188,"statfs":{"total":96636764160,"available":95937728512,"internally_reserved":0,"allocated":417804288,"data_stored":417527714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":281210540},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724573,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":438160,"kb_used_data":239872,"kb_used_omap":4,"kb_used_meta":198267,"kb_avail":93933680,"statfs":{"total":96636764160,"available":96188088320,"internally_reserved":0,"allocated":245628928,"data_stored":245304921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":203026091},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,5,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247856,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":767560,"kb_used_data":487416,"kb_used_omap":4,"kb_used_meta":280123,"kb_avail":93604280,"statfs":{"total":96636764160,"available":95850782720,"internally_reserved":0,"allocated":499113984,"data_stored":498833333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":286846632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,3,5,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":499523584,"data_stored":499487454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":245837824,"data_stored":245758338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":418320384,"data_stored":418284254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:28:42.889 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:28:42.890 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'8727", 'reported_seq': 8801, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:41.117808+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:28:41.117808+0000', 'last_peered': '2024-09-17T15:28:41.117808+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:28:41.117808+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 8727, 'log_dups_size': 0, 'ondisk_log_size': 8727, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35745792, 'num_objects': 8727, 'num_object_clones': 0, 'num_object_copies': 17454, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4070, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8727, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8727, 'num_write_kb': 34908, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4657}, {'shards': '0,2', 'objects': 4070}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:42.890 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:28:42.890 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'8625", 'reported_seq': 8747, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:37.116307+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:28:37.116307+0000', 'last_peered': '2024-09-17T15:28:37.116307+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:28:37.116307+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 8625, 'log_dups_size': 0, 'ondisk_log_size': 8625, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35311616, 'num_objects': 8621, 'num_object_clones': 0, 'num_object_copies': 17242, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4169, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8621, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6782, 'num_write_kb': 27128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4168}, {'shards': '2', 'objects': 4453}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:28:42.890 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:28:42.890 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'8981", 'reported_seq': 9115, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:41.122828+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:28:41.122828+0000', 'last_peered': '2024-09-17T15:28:41.122828+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:28:41.122828+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 8981, 'log_dups_size': 0, 'ondisk_log_size': 8981, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36766179, 'num_objects': 8977, 'num_object_clones': 0, 'num_object_copies': 17954, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4271, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8977, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5480, 'num_write_kb': 21920, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4707}, {'shards': '0,2', 'objects': 4268}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:42.890 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:28:42.890 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'8817", 'reported_seq': 8952, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:41.134380+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:28:41.134380+0000', 'last_peered': '2024-09-17T15:28:41.134380+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:28:41.134380+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 8817, 'log_dups_size': 0, 'ondisk_log_size': 8817, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36090822, 'num_objects': 8813, 'num_object_clones': 0, 'num_object_copies': 17626, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4174, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8813, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 5586, 'num_write_kb': 22344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4639}, {'shards': '0,2', 'objects': 4174}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:42.890 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:28:42.890 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'8642", 'reported_seq': 8772, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:41.123377+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:28:41.123377+0000', 'last_peered': '2024-09-17T15:28:41.123377+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:28:41.123377+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 8642, 'log_dups_size': 0, 'ondisk_log_size': 8642, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35364864, 'num_objects': 8634, 'num_object_clones': 0, 'num_object_copies': 17268, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4012, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8634, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5238, 'num_write_kb': 20952, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4624}, {'shards': '0,2', 'objects': 4010}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:42.891 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:28:42.891 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'8574", 'reported_seq': 8694, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:37.101399+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:28:37.101399+0000', 'last_peered': '2024-09-17T15:28:37.101399+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:28:37.101399+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 8574, 'log_dups_size': 0, 'ondisk_log_size': 8574, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35099107, 'num_objects': 8570, 'num_object_clones': 0, 'num_object_copies': 17140, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4078, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8570, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6469, 'num_write_kb': 25876, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4077}, {'shards': '2', 'objects': 4493}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:28:42.891 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:28:42.891 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'8637", 'reported_seq': 8755, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:37.068156+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:28:37.068156+0000', 'last_peered': '2024-09-17T15:28:37.068156+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:28:37.068156+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 8637, 'log_dups_size': 0, 'ondisk_log_size': 8637, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35352576, 'num_objects': 8631, 'num_object_clones': 0, 'num_object_copies': 17262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8631, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6399, 'num_write_kb': 25596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4137}, {'shards': '2', 'objects': 4494}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:28:42.891 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:28:42.891 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'8852", 'reported_seq': 8987, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:41.121520+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:28:41.121520+0000', 'last_peered': '2024-09-17T15:28:41.121520+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:28:41.121520+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 8852, 'log_dups_size': 0, 'ondisk_log_size': 8852, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36249600, 'num_objects': 8850, 'num_object_clones': 0, 'num_object_copies': 17700, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8850, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5400, 'num_write_kb': 21600, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4712}, {'shards': '0,2', 'objects': 4138}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:42.891 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:28:42.891 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'8827", 'reported_seq': 8967, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:41.084341+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:28:41.084341+0000', 'last_peered': '2024-09-17T15:28:41.084341+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:28:41.084341+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 8827, 'log_dups_size': 0, 'ondisk_log_size': 8827, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36122624, 'num_objects': 8819, 'num_object_clones': 0, 'num_object_copies': 17638, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4160, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8819, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5114, 'num_write_kb': 20456, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4662}, {'shards': '0,2', 'objects': 4157}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:42.891 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:28:42.892 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'8825", 'reported_seq': 8956, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:41.069135+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:28:41.069135+0000', 'last_peered': '2024-09-17T15:28:41.069135+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:28:41.069135+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 8825, 'log_dups_size': 0, 'ondisk_log_size': 8825, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36130816, 'num_objects': 8821, 'num_object_clones': 0, 'num_object_copies': 17642, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4065, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8821, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5612, 'num_write_kb': 22448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4758}, {'shards': '0,2', 'objects': 4063}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:42.892 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:28:42.892 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'8709", 'reported_seq': 9660, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:28:41.159131+0000', 'last_change': '2024-09-17T15:28:22.509818+0000', 'last_active': '2024-09-17T15:28:41.159131+0000', 'last_peered': '2024-09-17T15:28:41.159131+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:28:41.159131+0000', 'last_undegraded': '2024-09-17T15:28:41.159131+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 8709, 'log_dups_size': 0, 'ondisk_log_size': 8709, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35623878, 'num_objects': 8699, 'num_object_clones': 0, 'num_object_copies': 17398, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3705, 'num_objects_unfound': 0, 'num_objects_dirty': 8699, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5151, 'num_write_kb': 20604, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 404, 'num_bytes_recovered': 1654784, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4994}, {'shards': '0,2', 'objects': 3705}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:42.892 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:28:43.321 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 184 20 141403 141383 3.00089 1.72266 0.0339704 0.026026 2024-09-17T15:28:43.621 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:28:43.621 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41277/279618 objects degraded (14.762%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":260,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":139809,"data_bytes":572613361,"bytes_used":1933692928,"bytes_avail":287976599552,"bytes_total":289910292480,"degraded_objects":41277,"degraded_total":279618,"degraded_ratio":0.1476192519794863,"misplaced_objects":3705,"misplaced_total":279618,"misplaced_ratio":0.01325021994292213,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":98963,"recovering_keys_per_sec":0,"num_objects_recovered":290,"num_bytes_recovered":1187840,"num_keys_recovered":0,"write_bytes_sec":2216444,"read_op_per_sec":0,"write_op_per_sec":541},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T15:27:19.616376+0000","services":{}},"progress_events":{}} 2024-09-17T15:28:43.640 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:28:43.640 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:28:44.191 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:28:44.191 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:28:44.212 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":166,"stamp":"2024-09-17T15:28:43.637967+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":575648497,"num_objects":140550,"num_object_clones":0,"num_object_copies":281100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3705,"num_objects_unfound":0,"num_objects_dirty":140550,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":100657,"num_write_kb":402622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4477,"num_bytes_recovered":18326006,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140623,"ondisk_log_size":140623,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1885628,"kb_used_data":1140876,"kb_used_omap":13,"kb_used_meta":744690,"kb_avail":281229892,"statfs":{"total":289910292480,"available":287979409408,"internally_reserved":0,"allocated":1168257024,"data_stored":1167375792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":762563583},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,3,2,11,10,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22659072,"num_objects":5532,"num_object_clones":0,"num_object_copies":11064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-203,"num_objects_unfound":0,"num_objects_dirty":5532,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5532,"num_write_kb":22128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":831488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002863"},"pg_stats":[{"pgid":"2.f","version":"43'8727","reported_seq":8801,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:41.117808+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:28:41.117808+0000","last_peered":"2024-09-17T15:28:41.117808+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:28:41.117808+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":8727,"log_dups_size":0,"ondisk_log_size":8727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35745792,"num_objects":8727,"num_object_clones":0,"num_object_copies":17454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8727,"num_write_kb":34908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4657},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8811","reported_seq":8913,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:42.148833+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:28:42.148833+0000","last_peered":"2024-09-17T15:28:42.148833+0000","last_clean":"2024-09-17T15:28:42.148833+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:28:42.148833+0000","last_undegraded":"2024-09-17T15:28:42.148833+0000","last_fullsized":"2024-09-17T15:28:42.148833+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":8811,"log_dups_size":0,"ondisk_log_size":8811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36066246,"num_objects":8807,"num_object_clones":0,"num_object_copies":17614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8807,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7168,"num_write_kb":28672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8790","reported_seq":8915,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:42.145853+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:28:42.145853+0000","last_peered":"2024-09-17T15:28:42.145853+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:28:42.145853+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":8790,"log_dups_size":0,"ondisk_log_size":8790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35987456,"num_objects":8786,"num_object_clones":0,"num_object_copies":17572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6947,"num_write_kb":27788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":4618}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8883","reported_seq":8951,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:42.141079+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:28:42.141079+0000","last_peered":"2024-09-17T15:28:42.141079+0000","last_clean":"2024-09-17T15:28:42.141079+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:28:42.141079+0000","last_undegraded":"2024-09-17T15:28:42.141079+0000","last_fullsized":"2024-09-17T15:28:42.141079+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":8883,"log_dups_size":0,"ondisk_log_size":8883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36384768,"num_objects":8883,"num_object_clones":0,"num_object_copies":17766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8883,"num_write_kb":35532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8669","reported_seq":8798,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:41.081428+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:28:41.081428+0000","last_peered":"2024-09-17T15:28:41.081428+0000","last_clean":"2024-09-17T15:28:41.081428+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:28:41.081428+0000","last_undegraded":"2024-09-17T15:28:41.081428+0000","last_fullsized":"2024-09-17T15:28:41.081428+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":8669,"log_dups_size":0,"ondisk_log_size":8669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35480035,"num_objects":8663,"num_object_clones":0,"num_object_copies":17326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8663,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5651,"num_write_kb":22604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8981","reported_seq":9115,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:41.122828+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:28:41.122828+0000","last_peered":"2024-09-17T15:28:41.122828+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:28:41.122828+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":8981,"log_dups_size":0,"ondisk_log_size":8981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36766179,"num_objects":8977,"num_object_clones":0,"num_object_copies":17954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8977,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5480,"num_write_kb":21920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4707},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8817","reported_seq":8952,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:41.134380+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:28:41.134380+0000","last_peered":"2024-09-17T15:28:41.134380+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:28:41.134380+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":8817,"log_dups_size":0,"ondisk_log_size":8817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36090822,"num_objects":8813,"num_object_clones":0,"num_object_copies":17626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8813,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5586,"num_write_kb":22344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4639},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8857","reported_seq":8984,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:41.065697+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:28:41.065697+0000","last_peered":"2024-09-17T15:28:41.065697+0000","last_clean":"2024-09-17T15:28:41.065697+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:28:41.065697+0000","last_undegraded":"2024-09-17T15:28:41.065697+0000","last_fullsized":"2024-09-17T15:28:41.065697+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":8857,"log_dups_size":0,"ondisk_log_size":8857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36270080,"num_objects":8855,"num_object_clones":0,"num_object_copies":17710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5811,"num_write_kb":23244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8745","reported_seq":17027,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:40.312124+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:28:40.312124+0000","last_peered":"2024-09-17T15:28:40.312124+0000","last_clean":"2024-09-17T15:28:40.312124+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:28:40.312124+0000","last_undegraded":"2024-09-17T15:28:40.312124+0000","last_fullsized":"2024-09-17T15:28:40.312124+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":8745,"log_dups_size":0,"ondisk_log_size":8745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35791331,"num_objects":8739,"num_object_clones":0,"num_object_copies":17478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8739,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6745,"num_write_kb":26980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804705+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804705+0000","last_peered":"2024-09-17T15:28:23.804705+0000","last_clean":"2024-09-17T15:28:23.804705+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804705+0000","last_undegraded":"2024-09-17T15:28:23.804705+0000","last_fullsized":"2024-09-17T15:28:23.804705+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8642","reported_seq":8772,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:41.123377+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:28:41.123377+0000","last_peered":"2024-09-17T15:28:41.123377+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:28:41.123377+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":8642,"log_dups_size":0,"ondisk_log_size":8642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35364864,"num_objects":8634,"num_object_clones":0,"num_object_copies":17268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5238,"num_write_kb":20952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4624},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":102,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814215+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814215+0000","last_peered":"2024-09-17T15:28:23.814215+0000","last_clean":"2024-09-17T15:28:23.814215+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814215+0000","last_undegraded":"2024-09-17T15:28:23.814215+0000","last_fullsized":"2024-09-17T15:28:23.814215+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8709","reported_seq":8829,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:41.917890+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:28:41.917890+0000","last_peered":"2024-09-17T15:28:41.917890+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:28:41.917890+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":8709,"log_dups_size":0,"ondisk_log_size":8709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35652067,"num_objects":8705,"num_object_clones":0,"num_object_copies":17410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8705,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6604,"num_write_kb":26416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":4628}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8776","reported_seq":8894,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:42.073461+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:28:42.073461+0000","last_peered":"2024-09-17T15:28:42.073461+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:28:42.073461+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":8776,"log_dups_size":0,"ondisk_log_size":8776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35921920,"num_objects":8770,"num_object_clones":0,"num_object_copies":17540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6538,"num_write_kb":26152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":4633}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8852","reported_seq":8987,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:41.121520+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:28:41.121520+0000","last_peered":"2024-09-17T15:28:41.121520+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:28:41.121520+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":8852,"log_dups_size":0,"ondisk_log_size":8852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36249600,"num_objects":8850,"num_object_clones":0,"num_object_copies":17700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5400,"num_write_kb":21600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4712},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8827","reported_seq":8967,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:41.084341+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:28:41.084341+0000","last_peered":"2024-09-17T15:28:41.084341+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:28:41.084341+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":8827,"log_dups_size":0,"ondisk_log_size":8827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36122624,"num_objects":8819,"num_object_clones":0,"num_object_copies":17638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8819,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5114,"num_write_kb":20456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4662},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805074+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805074+0000","last_peered":"2024-09-17T15:28:23.805074+0000","last_clean":"2024-09-17T15:28:23.805074+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805074+0000","last_undegraded":"2024-09-17T15:28:23.805074+0000","last_fullsized":"2024-09-17T15:28:23.805074+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8825","reported_seq":8956,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:41.069135+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:28:41.069135+0000","last_peered":"2024-09-17T15:28:41.069135+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:28:41.069135+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":8825,"log_dups_size":0,"ondisk_log_size":8825,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36130816,"num_objects":8821,"num_object_clones":0,"num_object_copies":17642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8821,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5612,"num_write_kb":22448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4758},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8709","reported_seq":9660,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:28:41.159131+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:28:41.159131+0000","last_peered":"2024-09-17T15:28:41.159131+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:28:41.159131+0000","last_undegraded":"2024-09-17T15:28:41.159131+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":8709,"log_dups_size":0,"ondisk_log_size":8709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35623878,"num_objects":8699,"num_object_clones":0,"num_object_copies":17398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3705,"num_objects_unfound":0,"num_objects_dirty":8699,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5151,"num_write_kb":20604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":404,"num_bytes_recovered":1654784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4994},{"shards":"0,2","objects":3705}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":575648478,"num_objects":140548,"num_object_clones":0,"num_object_copies":281096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3705,"num_objects_unfound":0,"num_objects_dirty":140548,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":100655,"num_write_kb":402620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4476,"num_bytes_recovered":18325987,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1168515072,"data_stored":1168363326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140620,"ondisk_log_size":140620,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051628,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":679908,"kb_used_data":413588,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93691932,"statfs":{"total":96636764160,"available":95940538368,"internally_reserved":0,"allocated":423514112,"data_stored":423237538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":272690860},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,3,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724573,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":438160,"kb_used_data":239872,"kb_used_omap":4,"kb_used_meta":198267,"kb_avail":93933680,"statfs":{"total":96636764160,"available":96188088320,"internally_reserved":0,"allocated":245628928,"data_stored":245304921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":203026091},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,5,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247856,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":767560,"kb_used_data":487416,"kb_used_omap":4,"kb_used_meta":280123,"kb_avail":93604280,"statfs":{"total":96636764160,"available":95850782720,"internally_reserved":0,"allocated":499113984,"data_stored":498833333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":286846632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,3,5,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":499523584,"data_stored":499487454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":245837824,"data_stored":245758338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":423153664,"data_stored":423117534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:28:44.321 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 185 20 141843 141823 2.99396 1.71875 0.0564289 0.0260867 2024-09-17T15:28:45.321 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 186 20 142341 142321 2.98832 1.94531 0.0382475 0.0261362 2024-09-17T15:28:46.321 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 187 20 142842 142822 2.9828 1.95703 0.0453116 0.0261847 2024-09-17T15:28:47.214 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:28:47.321 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 188 20 143329 143309 2.97705 1.90234 0.0698349 0.0262333 2024-09-17T15:28:47.774 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:28:47.774 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:28:47.803 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":168,"stamp":"2024-09-17T15:28:47.638866+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":585220849,"num_objects":142887,"num_object_clones":0,"num_object_copies":285774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3604,"num_objects_unfound":0,"num_objects_dirty":142887,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":102994,"num_write_kb":411970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4578,"num_bytes_recovered":18739702,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142960,"ondisk_log_size":142960,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1899800,"kb_used_data":1159208,"kb_used_omap":13,"kb_used_meta":740530,"kb_avail":281215720,"statfs":{"total":289910292480,"available":287964897280,"internally_reserved":0,"allocated":1187028992,"data_stored":1186147760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":758303743},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,12,3,11,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28545024,"num_objects":6969,"num_object_clones":0,"num_object_copies":13938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-304,"num_objects_unfound":0,"num_objects_dirty":6969,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6969,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":1245184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002824"},"pg_stats":[{"pgid":"2.f","version":"43'8882","reported_seq":8959,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:46.162661+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:28:46.162661+0000","last_peered":"2024-09-17T15:28:46.162661+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:28:46.162661+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":8882,"log_dups_size":0,"ondisk_log_size":8882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36380672,"num_objects":8882,"num_object_clones":0,"num_object_copies":17764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8882,"num_write_kb":35528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4812},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8966","reported_seq":9068,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:47.098876+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:28:47.098876+0000","last_peered":"2024-09-17T15:28:47.098876+0000","last_clean":"2024-09-17T15:28:47.098876+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:28:47.098876+0000","last_undegraded":"2024-09-17T15:28:47.098876+0000","last_fullsized":"2024-09-17T15:28:47.098876+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":8966,"log_dups_size":0,"ondisk_log_size":8966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36701126,"num_objects":8962,"num_object_clones":0,"num_object_copies":17924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8962,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7323,"num_write_kb":29292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8928","reported_seq":9051,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:47.081260+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:28:47.081260+0000","last_peered":"2024-09-17T15:28:47.081260+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:28:47.081260+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":8928,"log_dups_size":0,"ondisk_log_size":8928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36552704,"num_objects":8924,"num_object_clones":0,"num_object_copies":17848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7085,"num_write_kb":28340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":4756}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9027","reported_seq":9096,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:47.127200+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:28:47.127200+0000","last_peered":"2024-09-17T15:28:47.127200+0000","last_clean":"2024-09-17T15:28:47.127200+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:28:47.127200+0000","last_undegraded":"2024-09-17T15:28:47.127200+0000","last_fullsized":"2024-09-17T15:28:47.127200+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9027,"log_dups_size":0,"ondisk_log_size":9027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36974592,"num_objects":9027,"num_object_clones":0,"num_object_copies":18054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9027,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9027,"num_write_kb":36108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8827","reported_seq":8956,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:46.140984+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:28:46.140984+0000","last_peered":"2024-09-17T15:28:46.140984+0000","last_clean":"2024-09-17T15:28:46.140984+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:28:46.140984+0000","last_undegraded":"2024-09-17T15:28:46.140984+0000","last_fullsized":"2024-09-17T15:28:46.140984+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":8827,"log_dups_size":0,"ondisk_log_size":8827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36127203,"num_objects":8821,"num_object_clones":0,"num_object_copies":17642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8821,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5809,"num_write_kb":23236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9135","reported_seq":9268,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:46.151801+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:28:46.151801+0000","last_peered":"2024-09-17T15:28:46.151801+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:28:46.151801+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":9135,"log_dups_size":0,"ondisk_log_size":9135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37396963,"num_objects":9131,"num_object_clones":0,"num_object_copies":18262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9131,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5634,"num_write_kb":22536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4861},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8940","reported_seq":9078,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:46.154777+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:28:46.154777+0000","last_peered":"2024-09-17T15:28:46.154777+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:28:46.154777+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":8940,"log_dups_size":0,"ondisk_log_size":8940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36594630,"num_objects":8936,"num_object_clones":0,"num_object_copies":17872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8936,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5709,"num_write_kb":22836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4762},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8997","reported_seq":9124,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:46.141257+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:28:46.141257+0000","last_peered":"2024-09-17T15:28:46.141257+0000","last_clean":"2024-09-17T15:28:46.141257+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:28:46.141257+0000","last_undegraded":"2024-09-17T15:28:46.141257+0000","last_fullsized":"2024-09-17T15:28:46.141257+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":8997,"log_dups_size":0,"ondisk_log_size":8997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36843520,"num_objects":8995,"num_object_clones":0,"num_object_copies":17990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5951,"num_write_kb":23804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8894","reported_seq":17176,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:45.357044+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:28:45.357044+0000","last_peered":"2024-09-17T15:28:45.357044+0000","last_clean":"2024-09-17T15:28:45.357044+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:28:45.357044+0000","last_undegraded":"2024-09-17T15:28:45.357044+0000","last_fullsized":"2024-09-17T15:28:45.357044+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":8894,"log_dups_size":0,"ondisk_log_size":8894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36401635,"num_objects":8888,"num_object_clones":0,"num_object_copies":17776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8888,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6894,"num_write_kb":27576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804705+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804705+0000","last_peered":"2024-09-17T15:28:23.804705+0000","last_clean":"2024-09-17T15:28:23.804705+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804705+0000","last_undegraded":"2024-09-17T15:28:23.804705+0000","last_fullsized":"2024-09-17T15:28:23.804705+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8794","reported_seq":8924,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:46.206501+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:28:46.206501+0000","last_peered":"2024-09-17T15:28:46.206501+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:28:46.206501+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":8794,"log_dups_size":0,"ondisk_log_size":8794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35987456,"num_objects":8786,"num_object_clones":0,"num_object_copies":17572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5390,"num_write_kb":21560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4776},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":102,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814215+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814215+0000","last_peered":"2024-09-17T15:28:23.814215+0000","last_clean":"2024-09-17T15:28:23.814215+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814215+0000","last_undegraded":"2024-09-17T15:28:23.814215+0000","last_fullsized":"2024-09-17T15:28:23.814215+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8847","reported_seq":8967,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:46.957714+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:28:46.957714+0000","last_peered":"2024-09-17T15:28:46.957714+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:28:46.957714+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":8847,"log_dups_size":0,"ondisk_log_size":8847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36217315,"num_objects":8843,"num_object_clones":0,"num_object_copies":17686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8843,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6742,"num_write_kb":26968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":4766}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8937","reported_seq":9055,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:47.105481+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:28:47.105481+0000","last_peered":"2024-09-17T15:28:47.105481+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:28:47.105481+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":8937,"log_dups_size":0,"ondisk_log_size":8937,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36581376,"num_objects":8931,"num_object_clones":0,"num_object_copies":17862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8931,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6699,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":4794}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9005","reported_seq":9140,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:46.178225+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:28:46.178225+0000","last_peered":"2024-09-17T15:28:46.178225+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:28:46.178225+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":9005,"log_dups_size":0,"ondisk_log_size":9005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36876288,"num_objects":9003,"num_object_clones":0,"num_object_copies":18006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5553,"num_write_kb":22212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4865},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8973","reported_seq":9114,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:46.200191+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:28:46.200191+0000","last_peered":"2024-09-17T15:28:46.200191+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:28:46.200191+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":8973,"log_dups_size":0,"ondisk_log_size":8973,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36720640,"num_objects":8965,"num_object_clones":0,"num_object_copies":17930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8965,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5260,"num_write_kb":21040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4808},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805074+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805074+0000","last_peered":"2024-09-17T15:28:23.805074+0000","last_clean":"2024-09-17T15:28:23.805074+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805074+0000","last_undegraded":"2024-09-17T15:28:23.805074+0000","last_fullsized":"2024-09-17T15:28:23.805074+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8975","reported_seq":9106,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:46.108352+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:28:46.108352+0000","last_peered":"2024-09-17T15:28:46.108352+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:28:46.108352+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":8975,"log_dups_size":0,"ondisk_log_size":8975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36745216,"num_objects":8971,"num_object_clones":0,"num_object_copies":17942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5762,"num_write_kb":23048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4908},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8830","reported_seq":9983,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:28:46.177373+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:28:46.177373+0000","last_peered":"2024-09-17T15:28:46.177373+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:28:46.177373+0000","last_undegraded":"2024-09-17T15:28:46.177373+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":8830,"log_dups_size":0,"ondisk_log_size":8830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36119494,"num_objects":8820,"num_object_clones":0,"num_object_copies":17640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3604,"num_objects_unfound":0,"num_objects_dirty":8820,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5272,"num_write_kb":21088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":505,"num_bytes_recovered":2068480,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5216},{"shards":"0,2","objects":3604}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":585220830,"num_objects":142885,"num_object_clones":0,"num_object_copies":285770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3604,"num_objects_unfound":0,"num_objects_dirty":142885,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":102992,"num_write_kb":411968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4577,"num_bytes_recovered":18739683,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1188229120,"data_stored":1188077374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142957,"ondisk_log_size":142957,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051629,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":683844,"kb_used_data":417524,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93687996,"statfs":{"total":96636764160,"available":95936507904,"internally_reserved":0,"allocated":427544576,"data_stored":427268002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":272690860},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724574,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":437120,"kb_used_data":247152,"kb_used_omap":4,"kb_used_meta":189947,"kb_avail":93934720,"statfs":{"total":96636764160,"available":96189153280,"internally_reserved":0,"allocated":253083648,"data_stored":252759641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":194506411},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,3,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247857,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":778836,"kb_used_data":494532,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93593004,"statfs":{"total":96636764160,"available":95839236096,"internally_reserved":0,"allocated":506400768,"data_stored":506120117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":291106472},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,7,0,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":506716160,"data_stored":506680030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":253456384,"data_stored":253376898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":428056576,"data_stored":428020446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:28:47.804 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:28:48.321 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 189 20 143762 143742 2.97025 1.69141 0.0387419 0.0262949 2024-09-17T15:28:48.367 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:28:48.368 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:28:48.393 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":168,"stamp":"2024-09-17T15:28:47.638866+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":585220849,"num_objects":142887,"num_object_clones":0,"num_object_copies":285774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3604,"num_objects_unfound":0,"num_objects_dirty":142887,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":102994,"num_write_kb":411970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4578,"num_bytes_recovered":18739702,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142960,"ondisk_log_size":142960,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1899800,"kb_used_data":1159208,"kb_used_omap":13,"kb_used_meta":740530,"kb_avail":281215720,"statfs":{"total":289910292480,"available":287964897280,"internally_reserved":0,"allocated":1187028992,"data_stored":1186147760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":758303743},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,12,3,11,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28545024,"num_objects":6969,"num_object_clones":0,"num_object_copies":13938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-304,"num_objects_unfound":0,"num_objects_dirty":6969,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6969,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":1245184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002824"},"pg_stats":[{"pgid":"2.f","version":"43'8882","reported_seq":8959,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:46.162661+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:28:46.162661+0000","last_peered":"2024-09-17T15:28:46.162661+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:28:46.162661+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":8882,"log_dups_size":0,"ondisk_log_size":8882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36380672,"num_objects":8882,"num_object_clones":0,"num_object_copies":17764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8882,"num_write_kb":35528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4812},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8966","reported_seq":9068,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:47.098876+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:28:47.098876+0000","last_peered":"2024-09-17T15:28:47.098876+0000","last_clean":"2024-09-17T15:28:47.098876+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:28:47.098876+0000","last_undegraded":"2024-09-17T15:28:47.098876+0000","last_fullsized":"2024-09-17T15:28:47.098876+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":8966,"log_dups_size":0,"ondisk_log_size":8966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36701126,"num_objects":8962,"num_object_clones":0,"num_object_copies":17924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8962,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7323,"num_write_kb":29292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8928","reported_seq":9051,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:47.081260+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:28:47.081260+0000","last_peered":"2024-09-17T15:28:47.081260+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:28:47.081260+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":8928,"log_dups_size":0,"ondisk_log_size":8928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36552704,"num_objects":8924,"num_object_clones":0,"num_object_copies":17848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7085,"num_write_kb":28340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":4756}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9027","reported_seq":9096,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:47.127200+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:28:47.127200+0000","last_peered":"2024-09-17T15:28:47.127200+0000","last_clean":"2024-09-17T15:28:47.127200+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:28:47.127200+0000","last_undegraded":"2024-09-17T15:28:47.127200+0000","last_fullsized":"2024-09-17T15:28:47.127200+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9027,"log_dups_size":0,"ondisk_log_size":9027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36974592,"num_objects":9027,"num_object_clones":0,"num_object_copies":18054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9027,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9027,"num_write_kb":36108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8827","reported_seq":8956,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:46.140984+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:28:46.140984+0000","last_peered":"2024-09-17T15:28:46.140984+0000","last_clean":"2024-09-17T15:28:46.140984+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:28:46.140984+0000","last_undegraded":"2024-09-17T15:28:46.140984+0000","last_fullsized":"2024-09-17T15:28:46.140984+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":8827,"log_dups_size":0,"ondisk_log_size":8827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36127203,"num_objects":8821,"num_object_clones":0,"num_object_copies":17642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8821,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5809,"num_write_kb":23236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9135","reported_seq":9268,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:46.151801+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:28:46.151801+0000","last_peered":"2024-09-17T15:28:46.151801+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:28:46.151801+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":9135,"log_dups_size":0,"ondisk_log_size":9135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37396963,"num_objects":9131,"num_object_clones":0,"num_object_copies":18262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9131,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5634,"num_write_kb":22536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4861},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8940","reported_seq":9078,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:46.154777+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:28:46.154777+0000","last_peered":"2024-09-17T15:28:46.154777+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:28:46.154777+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":8940,"log_dups_size":0,"ondisk_log_size":8940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36594630,"num_objects":8936,"num_object_clones":0,"num_object_copies":17872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8936,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5709,"num_write_kb":22836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4762},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8997","reported_seq":9124,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:46.141257+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:28:46.141257+0000","last_peered":"2024-09-17T15:28:46.141257+0000","last_clean":"2024-09-17T15:28:46.141257+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:28:46.141257+0000","last_undegraded":"2024-09-17T15:28:46.141257+0000","last_fullsized":"2024-09-17T15:28:46.141257+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":8997,"log_dups_size":0,"ondisk_log_size":8997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36843520,"num_objects":8995,"num_object_clones":0,"num_object_copies":17990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5951,"num_write_kb":23804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8894","reported_seq":17176,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:45.357044+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:28:45.357044+0000","last_peered":"2024-09-17T15:28:45.357044+0000","last_clean":"2024-09-17T15:28:45.357044+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:28:45.357044+0000","last_undegraded":"2024-09-17T15:28:45.357044+0000","last_fullsized":"2024-09-17T15:28:45.357044+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":8894,"log_dups_size":0,"ondisk_log_size":8894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36401635,"num_objects":8888,"num_object_clones":0,"num_object_copies":17776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8888,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6894,"num_write_kb":27576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804705+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804705+0000","last_peered":"2024-09-17T15:28:23.804705+0000","last_clean":"2024-09-17T15:28:23.804705+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804705+0000","last_undegraded":"2024-09-17T15:28:23.804705+0000","last_fullsized":"2024-09-17T15:28:23.804705+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8794","reported_seq":8924,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:46.206501+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:28:46.206501+0000","last_peered":"2024-09-17T15:28:46.206501+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:28:46.206501+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":8794,"log_dups_size":0,"ondisk_log_size":8794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35987456,"num_objects":8786,"num_object_clones":0,"num_object_copies":17572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5390,"num_write_kb":21560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4776},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":102,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814215+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814215+0000","last_peered":"2024-09-17T15:28:23.814215+0000","last_clean":"2024-09-17T15:28:23.814215+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814215+0000","last_undegraded":"2024-09-17T15:28:23.814215+0000","last_fullsized":"2024-09-17T15:28:23.814215+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8847","reported_seq":8967,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:46.957714+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:28:46.957714+0000","last_peered":"2024-09-17T15:28:46.957714+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:28:46.957714+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":8847,"log_dups_size":0,"ondisk_log_size":8847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36217315,"num_objects":8843,"num_object_clones":0,"num_object_copies":17686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8843,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6742,"num_write_kb":26968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":4766}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8937","reported_seq":9055,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:47.105481+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:28:47.105481+0000","last_peered":"2024-09-17T15:28:47.105481+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:28:47.105481+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":8937,"log_dups_size":0,"ondisk_log_size":8937,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36581376,"num_objects":8931,"num_object_clones":0,"num_object_copies":17862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8931,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6699,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":4794}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9005","reported_seq":9140,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:46.178225+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:28:46.178225+0000","last_peered":"2024-09-17T15:28:46.178225+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:28:46.178225+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":9005,"log_dups_size":0,"ondisk_log_size":9005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36876288,"num_objects":9003,"num_object_clones":0,"num_object_copies":18006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5553,"num_write_kb":22212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4865},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8973","reported_seq":9114,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:46.200191+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:28:46.200191+0000","last_peered":"2024-09-17T15:28:46.200191+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:28:46.200191+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":8973,"log_dups_size":0,"ondisk_log_size":8973,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36720640,"num_objects":8965,"num_object_clones":0,"num_object_copies":17930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8965,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5260,"num_write_kb":21040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4808},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805074+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805074+0000","last_peered":"2024-09-17T15:28:23.805074+0000","last_clean":"2024-09-17T15:28:23.805074+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805074+0000","last_undegraded":"2024-09-17T15:28:23.805074+0000","last_fullsized":"2024-09-17T15:28:23.805074+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8975","reported_seq":9106,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:46.108352+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:28:46.108352+0000","last_peered":"2024-09-17T15:28:46.108352+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:28:46.108352+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":8975,"log_dups_size":0,"ondisk_log_size":8975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36745216,"num_objects":8971,"num_object_clones":0,"num_object_copies":17942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5762,"num_write_kb":23048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4908},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8830","reported_seq":9983,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:28:46.177373+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:28:46.177373+0000","last_peered":"2024-09-17T15:28:46.177373+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:28:46.177373+0000","last_undegraded":"2024-09-17T15:28:46.177373+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":8830,"log_dups_size":0,"ondisk_log_size":8830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36119494,"num_objects":8820,"num_object_clones":0,"num_object_copies":17640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3604,"num_objects_unfound":0,"num_objects_dirty":8820,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5272,"num_write_kb":21088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":505,"num_bytes_recovered":2068480,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5216},{"shards":"0,2","objects":3604}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":585220830,"num_objects":142885,"num_object_clones":0,"num_object_copies":285770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3604,"num_objects_unfound":0,"num_objects_dirty":142885,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":102992,"num_write_kb":411968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4577,"num_bytes_recovered":18739683,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1188229120,"data_stored":1188077374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142957,"ondisk_log_size":142957,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051629,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":683844,"kb_used_data":417524,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93687996,"statfs":{"total":96636764160,"available":95936507904,"internally_reserved":0,"allocated":427544576,"data_stored":427268002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":272690860},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724574,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":437120,"kb_used_data":247152,"kb_used_omap":4,"kb_used_meta":189947,"kb_avail":93934720,"statfs":{"total":96636764160,"available":96189153280,"internally_reserved":0,"allocated":253083648,"data_stored":252759641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":194506411},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,3,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247857,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":778836,"kb_used_data":494532,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93593004,"statfs":{"total":96636764160,"available":95839236096,"internally_reserved":0,"allocated":506400768,"data_stored":506120117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":291106472},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,7,0,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":506716160,"data_stored":506680030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":253456384,"data_stored":253376898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":428056576,"data_stored":428020446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:28:48.395 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:28:48.395 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'8882", 'reported_seq': 8959, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:46.162661+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:28:46.162661+0000', 'last_peered': '2024-09-17T15:28:46.162661+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:28:46.162661+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 8882, 'log_dups_size': 0, 'ondisk_log_size': 8882, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36380672, 'num_objects': 8882, 'num_object_clones': 0, 'num_object_copies': 17764, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4070, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8882, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8882, 'num_write_kb': 35528, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4812}, {'shards': '0,2', 'objects': 4070}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:48.395 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:28:48.395 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'8928", 'reported_seq': 9051, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:47.081260+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:28:47.081260+0000', 'last_peered': '2024-09-17T15:28:47.081260+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:28:47.081260+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 8928, 'log_dups_size': 0, 'ondisk_log_size': 8928, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36552704, 'num_objects': 8924, 'num_object_clones': 0, 'num_object_copies': 17848, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4169, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8924, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7085, 'num_write_kb': 28340, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4168}, {'shards': '2', 'objects': 4756}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:28:48.395 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:28:48.395 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'9135", 'reported_seq': 9268, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:46.151801+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:28:46.151801+0000', 'last_peered': '2024-09-17T15:28:46.151801+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:28:46.151801+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 9135, 'log_dups_size': 0, 'ondisk_log_size': 9135, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37396963, 'num_objects': 9131, 'num_object_clones': 0, 'num_object_copies': 18262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4271, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9131, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5634, 'num_write_kb': 22536, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4861}, {'shards': '0,2', 'objects': 4268}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:48.396 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:28:48.396 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'8940", 'reported_seq': 9078, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:46.154777+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:28:46.154777+0000', 'last_peered': '2024-09-17T15:28:46.154777+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:28:46.154777+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 8940, 'log_dups_size': 0, 'ondisk_log_size': 8940, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36594630, 'num_objects': 8936, 'num_object_clones': 0, 'num_object_copies': 17872, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4174, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8936, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 5709, 'num_write_kb': 22836, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4762}, {'shards': '0,2', 'objects': 4174}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:48.396 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:28:48.397 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'8794", 'reported_seq': 8924, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:46.206501+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:28:46.206501+0000', 'last_peered': '2024-09-17T15:28:46.206501+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:28:46.206501+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 8794, 'log_dups_size': 0, 'ondisk_log_size': 8794, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35987456, 'num_objects': 8786, 'num_object_clones': 0, 'num_object_copies': 17572, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4012, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8786, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5390, 'num_write_kb': 21560, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4776}, {'shards': '0,2', 'objects': 4010}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:48.397 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:28:48.397 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'8847", 'reported_seq': 8967, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:46.957714+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:28:46.957714+0000', 'last_peered': '2024-09-17T15:28:46.957714+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:28:46.957714+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 8847, 'log_dups_size': 0, 'ondisk_log_size': 8847, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36217315, 'num_objects': 8843, 'num_object_clones': 0, 'num_object_copies': 17686, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4078, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8843, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6742, 'num_write_kb': 26968, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4077}, {'shards': '2', 'objects': 4766}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:28:48.397 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:28:48.397 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'8937", 'reported_seq': 9055, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:47.105481+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:28:47.105481+0000', 'last_peered': '2024-09-17T15:28:47.105481+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:28:47.105481+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 8937, 'log_dups_size': 0, 'ondisk_log_size': 8937, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36581376, 'num_objects': 8931, 'num_object_clones': 0, 'num_object_copies': 17862, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8931, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6699, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4137}, {'shards': '2', 'objects': 4794}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:28:48.397 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:28:48.397 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'9005", 'reported_seq': 9140, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:46.178225+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:28:46.178225+0000', 'last_peered': '2024-09-17T15:28:46.178225+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:28:46.178225+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 9005, 'log_dups_size': 0, 'ondisk_log_size': 9005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36876288, 'num_objects': 9003, 'num_object_clones': 0, 'num_object_copies': 18006, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9003, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5553, 'num_write_kb': 22212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4865}, {'shards': '0,2', 'objects': 4138}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:48.397 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:28:48.397 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'8973", 'reported_seq': 9114, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:46.200191+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:28:46.200191+0000', 'last_peered': '2024-09-17T15:28:46.200191+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:28:46.200191+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 8973, 'log_dups_size': 0, 'ondisk_log_size': 8973, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36720640, 'num_objects': 8965, 'num_object_clones': 0, 'num_object_copies': 17930, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4160, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8965, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5260, 'num_write_kb': 21040, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4808}, {'shards': '0,2', 'objects': 4157}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:48.398 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:28:48.398 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'8975", 'reported_seq': 9106, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:46.108352+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:28:46.108352+0000', 'last_peered': '2024-09-17T15:28:46.108352+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:28:46.108352+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 8975, 'log_dups_size': 0, 'ondisk_log_size': 8975, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36745216, 'num_objects': 8971, 'num_object_clones': 0, 'num_object_copies': 17942, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4065, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8971, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5762, 'num_write_kb': 23048, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4908}, {'shards': '0,2', 'objects': 4063}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:48.398 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:28:48.398 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'8830", 'reported_seq': 9983, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:28:46.177373+0000', 'last_change': '2024-09-17T15:28:22.509818+0000', 'last_active': '2024-09-17T15:28:46.177373+0000', 'last_peered': '2024-09-17T15:28:46.177373+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:28:46.177373+0000', 'last_undegraded': '2024-09-17T15:28:46.177373+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 8830, 'log_dups_size': 0, 'ondisk_log_size': 8830, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36119494, 'num_objects': 8820, 'num_object_clones': 0, 'num_object_copies': 17640, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3604, 'num_objects_unfound': 0, 'num_objects_dirty': 8820, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5272, 'num_write_kb': 21088, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 505, 'num_bytes_recovered': 2068480, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5216}, {'shards': '0,2', 'objects': 3604}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:48.398 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:28:49.086 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:28:49.086 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41277/285774 objects degraded (14.444%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":266,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":142887,"data_bytes":585220849,"bytes_used":1945395200,"bytes_avail":287964897280,"bytes_total":289910292480,"degraded_objects":41277,"degraded_total":285774,"degraded_ratio":0.14443931218375358,"misplaced_objects":3604,"misplaced_total":285774,"misplaced_ratio":0.012611364224876999,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":103740,"recovering_keys_per_sec":0,"num_objects_recovered":304,"num_bytes_recovered":1245184,"num_keys_recovered":0,"write_bytes_sec":2378192,"read_op_per_sec":0,"write_op_per_sec":580},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T15:27:19.616376+0000","services":{}},"progress_events":{}} 2024-09-17T15:28:49.111 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:28:49.111 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:28:49.321 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 190 20 144198 144178 2.96358 1.70312 0.0426173 0.0263541 2024-09-17T15:28:49.701 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:28:49.701 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:28:49.730 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":169,"stamp":"2024-09-17T15:28:49.639371+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":585220849,"num_objects":142887,"num_object_clones":0,"num_object_copies":285774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3604,"num_objects_unfound":0,"num_objects_dirty":142887,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":102994,"num_write_kb":411970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4578,"num_bytes_recovered":18739702,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142960,"ondisk_log_size":142960,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1899800,"kb_used_data":1159208,"kb_used_omap":13,"kb_used_meta":740530,"kb_avail":281215720,"statfs":{"total":289910292480,"available":287964897280,"internally_reserved":0,"allocated":1187028992,"data_stored":1186147760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":758303743},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,12,3,11,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19300352,"num_objects":4712,"num_object_clones":0,"num_object_copies":9424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-202,"num_objects_unfound":0,"num_objects_dirty":4712,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4712,"num_write_kb":18848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":827392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002836"},"pg_stats":[{"pgid":"2.f","version":"43'8882","reported_seq":8959,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:46.162661+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:28:46.162661+0000","last_peered":"2024-09-17T15:28:46.162661+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:28:46.162661+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":8882,"log_dups_size":0,"ondisk_log_size":8882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36380672,"num_objects":8882,"num_object_clones":0,"num_object_copies":17764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8882,"num_write_kb":35528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4812},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8966","reported_seq":9068,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:47.098876+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:28:47.098876+0000","last_peered":"2024-09-17T15:28:47.098876+0000","last_clean":"2024-09-17T15:28:47.098876+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:28:47.098876+0000","last_undegraded":"2024-09-17T15:28:47.098876+0000","last_fullsized":"2024-09-17T15:28:47.098876+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":8966,"log_dups_size":0,"ondisk_log_size":8966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36701126,"num_objects":8962,"num_object_clones":0,"num_object_copies":17924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8962,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7323,"num_write_kb":29292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8928","reported_seq":9051,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:47.081260+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:28:47.081260+0000","last_peered":"2024-09-17T15:28:47.081260+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:28:47.081260+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":8928,"log_dups_size":0,"ondisk_log_size":8928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36552704,"num_objects":8924,"num_object_clones":0,"num_object_copies":17848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7085,"num_write_kb":28340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":4756}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9027","reported_seq":9096,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:47.127200+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:28:47.127200+0000","last_peered":"2024-09-17T15:28:47.127200+0000","last_clean":"2024-09-17T15:28:47.127200+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:28:47.127200+0000","last_undegraded":"2024-09-17T15:28:47.127200+0000","last_fullsized":"2024-09-17T15:28:47.127200+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9027,"log_dups_size":0,"ondisk_log_size":9027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36974592,"num_objects":9027,"num_object_clones":0,"num_object_copies":18054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9027,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9027,"num_write_kb":36108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8827","reported_seq":8956,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:46.140984+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:28:46.140984+0000","last_peered":"2024-09-17T15:28:46.140984+0000","last_clean":"2024-09-17T15:28:46.140984+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:28:46.140984+0000","last_undegraded":"2024-09-17T15:28:46.140984+0000","last_fullsized":"2024-09-17T15:28:46.140984+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":8827,"log_dups_size":0,"ondisk_log_size":8827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36127203,"num_objects":8821,"num_object_clones":0,"num_object_copies":17642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8821,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5809,"num_write_kb":23236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9135","reported_seq":9268,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:46.151801+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:28:46.151801+0000","last_peered":"2024-09-17T15:28:46.151801+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:28:46.151801+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":9135,"log_dups_size":0,"ondisk_log_size":9135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37396963,"num_objects":9131,"num_object_clones":0,"num_object_copies":18262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9131,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5634,"num_write_kb":22536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4861},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8940","reported_seq":9078,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:46.154777+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:28:46.154777+0000","last_peered":"2024-09-17T15:28:46.154777+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:28:46.154777+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":8940,"log_dups_size":0,"ondisk_log_size":8940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36594630,"num_objects":8936,"num_object_clones":0,"num_object_copies":17872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8936,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5709,"num_write_kb":22836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4762},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8997","reported_seq":9124,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:46.141257+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:28:46.141257+0000","last_peered":"2024-09-17T15:28:46.141257+0000","last_clean":"2024-09-17T15:28:46.141257+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:28:46.141257+0000","last_undegraded":"2024-09-17T15:28:46.141257+0000","last_fullsized":"2024-09-17T15:28:46.141257+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":8997,"log_dups_size":0,"ondisk_log_size":8997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36843520,"num_objects":8995,"num_object_clones":0,"num_object_copies":17990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5951,"num_write_kb":23804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8894","reported_seq":17176,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:45.357044+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:28:45.357044+0000","last_peered":"2024-09-17T15:28:45.357044+0000","last_clean":"2024-09-17T15:28:45.357044+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:28:45.357044+0000","last_undegraded":"2024-09-17T15:28:45.357044+0000","last_fullsized":"2024-09-17T15:28:45.357044+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":8894,"log_dups_size":0,"ondisk_log_size":8894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36401635,"num_objects":8888,"num_object_clones":0,"num_object_copies":17776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8888,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6894,"num_write_kb":27576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804705+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804705+0000","last_peered":"2024-09-17T15:28:23.804705+0000","last_clean":"2024-09-17T15:28:23.804705+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804705+0000","last_undegraded":"2024-09-17T15:28:23.804705+0000","last_fullsized":"2024-09-17T15:28:23.804705+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8794","reported_seq":8924,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:46.206501+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:28:46.206501+0000","last_peered":"2024-09-17T15:28:46.206501+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:28:46.206501+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":8794,"log_dups_size":0,"ondisk_log_size":8794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35987456,"num_objects":8786,"num_object_clones":0,"num_object_copies":17572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5390,"num_write_kb":21560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4776},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":102,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814215+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814215+0000","last_peered":"2024-09-17T15:28:23.814215+0000","last_clean":"2024-09-17T15:28:23.814215+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814215+0000","last_undegraded":"2024-09-17T15:28:23.814215+0000","last_fullsized":"2024-09-17T15:28:23.814215+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8847","reported_seq":8967,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:46.957714+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:28:46.957714+0000","last_peered":"2024-09-17T15:28:46.957714+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:28:46.957714+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":8847,"log_dups_size":0,"ondisk_log_size":8847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36217315,"num_objects":8843,"num_object_clones":0,"num_object_copies":17686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8843,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6742,"num_write_kb":26968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":4766}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8937","reported_seq":9055,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:47.105481+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:28:47.105481+0000","last_peered":"2024-09-17T15:28:47.105481+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:28:47.105481+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":8937,"log_dups_size":0,"ondisk_log_size":8937,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36581376,"num_objects":8931,"num_object_clones":0,"num_object_copies":17862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8931,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6699,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":4794}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9005","reported_seq":9140,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:46.178225+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:28:46.178225+0000","last_peered":"2024-09-17T15:28:46.178225+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:28:46.178225+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":9005,"log_dups_size":0,"ondisk_log_size":9005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36876288,"num_objects":9003,"num_object_clones":0,"num_object_copies":18006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5553,"num_write_kb":22212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4865},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8973","reported_seq":9114,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:46.200191+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:28:46.200191+0000","last_peered":"2024-09-17T15:28:46.200191+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:28:46.200191+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":8973,"log_dups_size":0,"ondisk_log_size":8973,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36720640,"num_objects":8965,"num_object_clones":0,"num_object_copies":17930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8965,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5260,"num_write_kb":21040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4808},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805074+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805074+0000","last_peered":"2024-09-17T15:28:23.805074+0000","last_clean":"2024-09-17T15:28:23.805074+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805074+0000","last_undegraded":"2024-09-17T15:28:23.805074+0000","last_fullsized":"2024-09-17T15:28:23.805074+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8975","reported_seq":9106,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:46.108352+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:28:46.108352+0000","last_peered":"2024-09-17T15:28:46.108352+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:28:46.108352+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":8975,"log_dups_size":0,"ondisk_log_size":8975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36745216,"num_objects":8971,"num_object_clones":0,"num_object_copies":17942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5762,"num_write_kb":23048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4908},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8830","reported_seq":9983,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:28:46.177373+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:28:46.177373+0000","last_peered":"2024-09-17T15:28:46.177373+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:28:46.177373+0000","last_undegraded":"2024-09-17T15:28:46.177373+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":8830,"log_dups_size":0,"ondisk_log_size":8830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36119494,"num_objects":8820,"num_object_clones":0,"num_object_copies":17640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3604,"num_objects_unfound":0,"num_objects_dirty":8820,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5272,"num_write_kb":21088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":505,"num_bytes_recovered":2068480,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5216},{"shards":"0,2","objects":3604}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":585220830,"num_objects":142885,"num_object_clones":0,"num_object_copies":285770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3604,"num_objects_unfound":0,"num_objects_dirty":142885,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":102992,"num_write_kb":411968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4577,"num_bytes_recovered":18739683,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1188229120,"data_stored":1188077374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142957,"ondisk_log_size":142957,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051629,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":683844,"kb_used_data":417524,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93687996,"statfs":{"total":96636764160,"available":95936507904,"internally_reserved":0,"allocated":427544576,"data_stored":427268002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":272690860},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724574,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":437120,"kb_used_data":247152,"kb_used_omap":4,"kb_used_meta":189947,"kb_avail":93934720,"statfs":{"total":96636764160,"available":96189153280,"internally_reserved":0,"allocated":253083648,"data_stored":252759641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":194506411},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,3,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247857,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":778836,"kb_used_data":494532,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93593004,"statfs":{"total":96636764160,"available":95839236096,"internally_reserved":0,"allocated":506400768,"data_stored":506120117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":291106472},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,7,0,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":506716160,"data_stored":506680030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":253456384,"data_stored":253376898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":428056576,"data_stored":428020446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:28:50.322 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 191 20 144669 144649 2.9577 1.83984 0.0196264 0.0264066 2024-09-17T15:28:51.322 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 192 20 145172 145152 2.95253 1.96484 0.0301142 0.0264525 2024-09-17T15:28:52.324 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 193 20 145656 145636 2.94703 1.89062 0.0262538 0.0265014 2024-09-17T15:28:52.733 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:28:53.288 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:28:53.289 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:28:53.313 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":170,"stamp":"2024-09-17T15:28:51.639962+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591852273,"num_objects":144506,"num_object_clones":0,"num_object_copies":289012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3507,"num_objects_unfound":0,"num_objects_dirty":144506,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":104613,"num_write_kb":418446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4675,"num_bytes_recovered":19137014,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144579,"ondisk_log_size":144579,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1922896,"kb_used_data":1173984,"kb_used_omap":13,"kb_used_meta":748850,"kb_avail":281192624,"statfs":{"total":289910292480,"available":287941246976,"internally_reserved":0,"allocated":1202159616,"data_stored":1201278384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":766823423},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,5,8,9,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25931776,"num_objects":6331,"num_object_clones":0,"num_object_copies":12662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-299,"num_objects_unfound":0,"num_objects_dirty":6331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6331,"num_write_kb":25324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":299,"num_bytes_recovered":1224704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002904"},"pg_stats":[{"pgid":"2.f","version":"43'9016","reported_seq":9093,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:51.236938+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:28:51.236938+0000","last_peered":"2024-09-17T15:28:51.236938+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:28:51.236938+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9016,"log_dups_size":0,"ondisk_log_size":9016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36929536,"num_objects":9016,"num_object_clones":0,"num_object_copies":18032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9016,"num_write_kb":36064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4946},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8966","reported_seq":9068,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:47.098876+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:28:47.098876+0000","last_peered":"2024-09-17T15:28:47.098876+0000","last_clean":"2024-09-17T15:28:47.098876+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:28:47.098876+0000","last_undegraded":"2024-09-17T15:28:47.098876+0000","last_fullsized":"2024-09-17T15:28:47.098876+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":8966,"log_dups_size":0,"ondisk_log_size":8966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36701126,"num_objects":8962,"num_object_clones":0,"num_object_copies":17924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8962,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7323,"num_write_kb":29292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8928","reported_seq":9051,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:47.081260+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:28:47.081260+0000","last_peered":"2024-09-17T15:28:47.081260+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:28:47.081260+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":8928,"log_dups_size":0,"ondisk_log_size":8928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36552704,"num_objects":8924,"num_object_clones":0,"num_object_copies":17848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7085,"num_write_kb":28340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":4756}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9027","reported_seq":9096,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:47.127200+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:28:47.127200+0000","last_peered":"2024-09-17T15:28:47.127200+0000","last_clean":"2024-09-17T15:28:47.127200+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:28:47.127200+0000","last_undegraded":"2024-09-17T15:28:47.127200+0000","last_fullsized":"2024-09-17T15:28:47.127200+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9027,"log_dups_size":0,"ondisk_log_size":9027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36974592,"num_objects":9027,"num_object_clones":0,"num_object_copies":18054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9027,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9027,"num_write_kb":36108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8985","reported_seq":9114,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:51.185950+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:28:51.185950+0000","last_peered":"2024-09-17T15:28:51.185950+0000","last_clean":"2024-09-17T15:28:51.185950+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:28:51.185950+0000","last_undegraded":"2024-09-17T15:28:51.185950+0000","last_fullsized":"2024-09-17T15:28:51.185950+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":8985,"log_dups_size":0,"ondisk_log_size":8985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36774371,"num_objects":8979,"num_object_clones":0,"num_object_copies":17958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8979,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5967,"num_write_kb":23868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9293","reported_seq":9427,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:51.197429+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:28:51.197429+0000","last_peered":"2024-09-17T15:28:51.197429+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:28:51.197429+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":9293,"log_dups_size":0,"ondisk_log_size":9293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38044131,"num_objects":9289,"num_object_clones":0,"num_object_copies":18578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9289,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5792,"num_write_kb":23168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5019},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9078","reported_seq":9216,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:51.047414+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:28:51.047414+0000","last_peered":"2024-09-17T15:28:51.047414+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:28:51.047414+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":9078,"log_dups_size":0,"ondisk_log_size":9078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37159878,"num_objects":9074,"num_object_clones":0,"num_object_copies":18148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9074,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5847,"num_write_kb":23388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4900},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9148","reported_seq":9275,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:51.135000+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:28:51.135000+0000","last_peered":"2024-09-17T15:28:51.135000+0000","last_clean":"2024-09-17T15:28:51.135000+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:28:51.135000+0000","last_undegraded":"2024-09-17T15:28:51.135000+0000","last_fullsized":"2024-09-17T15:28:51.135000+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":9148,"log_dups_size":0,"ondisk_log_size":9148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37462016,"num_objects":9146,"num_object_clones":0,"num_object_copies":18292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6102,"num_write_kb":24408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'9045","reported_seq":17327,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:50.436632+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:28:50.436632+0000","last_peered":"2024-09-17T15:28:50.436632+0000","last_clean":"2024-09-17T15:28:50.436632+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:28:50.436632+0000","last_undegraded":"2024-09-17T15:28:50.436632+0000","last_fullsized":"2024-09-17T15:28:50.436632+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":9045,"log_dups_size":0,"ondisk_log_size":9045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37020131,"num_objects":9039,"num_object_clones":0,"num_object_copies":18078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9039,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7045,"num_write_kb":28180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804705+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804705+0000","last_peered":"2024-09-17T15:28:23.804705+0000","last_clean":"2024-09-17T15:28:23.804705+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804705+0000","last_undegraded":"2024-09-17T15:28:23.804705+0000","last_fullsized":"2024-09-17T15:28:23.804705+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8947","reported_seq":9077,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:51.222586+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:28:51.222586+0000","last_peered":"2024-09-17T15:28:51.222586+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:28:51.222586+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":8947,"log_dups_size":0,"ondisk_log_size":8947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36614144,"num_objects":8939,"num_object_clones":0,"num_object_copies":17878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8939,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5543,"num_write_kb":22172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4929},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":102,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814215+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814215+0000","last_peered":"2024-09-17T15:28:23.814215+0000","last_clean":"2024-09-17T15:28:23.814215+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814215+0000","last_undegraded":"2024-09-17T15:28:23.814215+0000","last_fullsized":"2024-09-17T15:28:23.814215+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8847","reported_seq":8967,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:46.957714+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:28:46.957714+0000","last_peered":"2024-09-17T15:28:46.957714+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:28:46.957714+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":8847,"log_dups_size":0,"ondisk_log_size":8847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36217315,"num_objects":8843,"num_object_clones":0,"num_object_copies":17686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8843,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6742,"num_write_kb":26968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":4766}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8937","reported_seq":9055,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:47.105481+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:28:47.105481+0000","last_peered":"2024-09-17T15:28:47.105481+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:28:47.105481+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":8937,"log_dups_size":0,"ondisk_log_size":8937,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36581376,"num_objects":8931,"num_object_clones":0,"num_object_copies":17862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8931,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6699,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":4794}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9146","reported_seq":9281,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:51.212063+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:28:51.212063+0000","last_peered":"2024-09-17T15:28:51.212063+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:28:51.212063+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":9146,"log_dups_size":0,"ondisk_log_size":9146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37453824,"num_objects":9144,"num_object_clones":0,"num_object_copies":18288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5694,"num_write_kb":22776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5006},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9104","reported_seq":9245,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:51.114585+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:28:51.114585+0000","last_peered":"2024-09-17T15:28:51.114585+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:28:51.114585+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":9104,"log_dups_size":0,"ondisk_log_size":9104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37257216,"num_objects":9096,"num_object_clones":0,"num_object_copies":18192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5391,"num_write_kb":21564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4939},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805074+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805074+0000","last_peered":"2024-09-17T15:28:23.805074+0000","last_clean":"2024-09-17T15:28:23.805074+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805074+0000","last_undegraded":"2024-09-17T15:28:23.805074+0000","last_fullsized":"2024-09-17T15:28:23.805074+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9127","reported_seq":9258,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:51.153238+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:28:51.153238+0000","last_peered":"2024-09-17T15:28:51.153238+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:28:51.153238+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":9127,"log_dups_size":0,"ondisk_log_size":9127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37367808,"num_objects":9123,"num_object_clones":0,"num_object_copies":18246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5914,"num_write_kb":23656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5060},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8982","reported_seq":10329,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:28:51.243205+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:28:51.243205+0000","last_peered":"2024-09-17T15:28:51.243205+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:28:51.243205+0000","last_undegraded":"2024-09-17T15:28:51.243205+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":8982,"log_dups_size":0,"ondisk_log_size":8982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36742086,"num_objects":8972,"num_object_clones":0,"num_object_copies":17944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3507,"num_objects_unfound":0,"num_objects_dirty":8972,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5424,"num_write_kb":21696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":602,"num_bytes_recovered":2465792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5465},{"shards":"0,2","objects":3507}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":591852254,"num_objects":144504,"num_object_clones":0,"num_object_copies":289008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3507,"num_objects_unfound":0,"num_objects_dirty":144504,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":104611,"num_write_kb":418444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4674,"num_bytes_recovered":19136995,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1203040256,"data_stored":1202888510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144576,"ondisk_log_size":144576,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051629,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":683844,"kb_used_data":417524,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93687996,"statfs":{"total":96636764160,"available":95936507904,"internally_reserved":0,"allocated":427544576,"data_stored":427268002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":272690860},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724575,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":448780,"kb_used_data":254652,"kb_used_omap":4,"kb_used_meta":194107,"kb_avail":93923060,"statfs":{"total":96636764160,"available":96177213440,"internally_reserved":0,"allocated":260763648,"data_stored":260439641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":198766251},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247858,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":790272,"kb_used_data":501808,"kb_used_omap":4,"kb_used_meta":288443,"kb_avail":93581568,"statfs":{"total":96636764160,"available":95827525632,"internally_reserved":0,"allocated":513851392,"data_stored":513570741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":295366312},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,2,5,1,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":513970176,"data_stored":513934046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":261013504,"data_stored":260934018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":428056576,"data_stored":428020446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:28:53.314 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:28:53.322 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 194 20 146122 146102 2.94122 1.82031 0.0433409 0.0265559 2024-09-17T15:28:53.879 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:28:53.880 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:28:53.901 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":171,"stamp":"2024-09-17T15:28:53.640481+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594924273,"num_objects":145256,"num_object_clones":0,"num_object_copies":290512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3507,"num_objects_unfound":0,"num_objects_dirty":145256,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":105363,"num_write_kb":421446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4675,"num_bytes_recovered":19137014,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145329,"ondisk_log_size":145329,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1936804,"kb_used_data":1179572,"kb_used_omap":13,"kb_used_meta":757170,"kb_avail":281178716,"statfs":{"total":289910292480,"available":287927005184,"internally_reserved":0,"allocated":1207881728,"data_stored":1207000496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":775343103},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,3,8,9,12],"upper_bound":128},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22310912,"num_objects":5447,"num_object_clones":0,"num_object_copies":10894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-198,"num_objects_unfound":0,"num_objects_dirty":5447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5447,"num_write_kb":21788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":811008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003000"},"pg_stats":[{"pgid":"2.f","version":"43'9016","reported_seq":9093,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:51.236938+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:28:51.236938+0000","last_peered":"2024-09-17T15:28:51.236938+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:28:51.236938+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9016,"log_dups_size":0,"ondisk_log_size":9016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36929536,"num_objects":9016,"num_object_clones":0,"num_object_copies":18032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9016,"num_write_kb":36064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4946},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9112","reported_seq":9213,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:52.202108+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:28:52.202108+0000","last_peered":"2024-09-17T15:28:52.202108+0000","last_clean":"2024-09-17T15:28:52.202108+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:28:52.202108+0000","last_undegraded":"2024-09-17T15:28:52.202108+0000","last_fullsized":"2024-09-17T15:28:52.202108+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":9112,"log_dups_size":0,"ondisk_log_size":9112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37299142,"num_objects":9108,"num_object_clones":0,"num_object_copies":18216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9108,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7469,"num_write_kb":29876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9090","reported_seq":9215,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:52.202763+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:28:52.202763+0000","last_peered":"2024-09-17T15:28:52.202763+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:28:52.202763+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":9090,"log_dups_size":0,"ondisk_log_size":9090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37216256,"num_objects":9086,"num_object_clones":0,"num_object_copies":18172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7247,"num_write_kb":28988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":4918}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9187","reported_seq":9256,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:52.182200+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:28:52.182200+0000","last_peered":"2024-09-17T15:28:52.182200+0000","last_clean":"2024-09-17T15:28:52.182200+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:28:52.182200+0000","last_undegraded":"2024-09-17T15:28:52.182200+0000","last_fullsized":"2024-09-17T15:28:52.182200+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9187,"log_dups_size":0,"ondisk_log_size":9187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629952,"num_objects":9187,"num_object_clones":0,"num_object_copies":18374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9187,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9187,"num_write_kb":36748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8985","reported_seq":9114,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:51.185950+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:28:51.185950+0000","last_peered":"2024-09-17T15:28:51.185950+0000","last_clean":"2024-09-17T15:28:51.185950+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:28:51.185950+0000","last_undegraded":"2024-09-17T15:28:51.185950+0000","last_fullsized":"2024-09-17T15:28:51.185950+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":8985,"log_dups_size":0,"ondisk_log_size":8985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36774371,"num_objects":8979,"num_object_clones":0,"num_object_copies":17958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8979,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5967,"num_write_kb":23868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9293","reported_seq":9427,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:51.197429+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:28:51.197429+0000","last_peered":"2024-09-17T15:28:51.197429+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:28:51.197429+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":9293,"log_dups_size":0,"ondisk_log_size":9293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38044131,"num_objects":9289,"num_object_clones":0,"num_object_copies":18578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9289,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5792,"num_write_kb":23168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5019},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9078","reported_seq":9216,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:51.047414+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:28:51.047414+0000","last_peered":"2024-09-17T15:28:51.047414+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:28:51.047414+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":9078,"log_dups_size":0,"ondisk_log_size":9078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37159878,"num_objects":9074,"num_object_clones":0,"num_object_copies":18148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9074,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5847,"num_write_kb":23388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4900},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9148","reported_seq":9275,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:51.135000+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:28:51.135000+0000","last_peered":"2024-09-17T15:28:51.135000+0000","last_clean":"2024-09-17T15:28:51.135000+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:28:51.135000+0000","last_undegraded":"2024-09-17T15:28:51.135000+0000","last_fullsized":"2024-09-17T15:28:51.135000+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":9148,"log_dups_size":0,"ondisk_log_size":9148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37462016,"num_objects":9146,"num_object_clones":0,"num_object_copies":18292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6102,"num_write_kb":24408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'9045","reported_seq":17327,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:50.436632+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:28:50.436632+0000","last_peered":"2024-09-17T15:28:50.436632+0000","last_clean":"2024-09-17T15:28:50.436632+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:28:50.436632+0000","last_undegraded":"2024-09-17T15:28:50.436632+0000","last_fullsized":"2024-09-17T15:28:50.436632+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":9045,"log_dups_size":0,"ondisk_log_size":9045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37020131,"num_objects":9039,"num_object_clones":0,"num_object_copies":18078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9039,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7045,"num_write_kb":28180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804705+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804705+0000","last_peered":"2024-09-17T15:28:23.804705+0000","last_clean":"2024-09-17T15:28:23.804705+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804705+0000","last_undegraded":"2024-09-17T15:28:23.804705+0000","last_fullsized":"2024-09-17T15:28:23.804705+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8947","reported_seq":9077,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:51.222586+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:28:51.222586+0000","last_peered":"2024-09-17T15:28:51.222586+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:28:51.222586+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":8947,"log_dups_size":0,"ondisk_log_size":8947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36614144,"num_objects":8939,"num_object_clones":0,"num_object_copies":17878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8939,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5543,"num_write_kb":22172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4929},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":102,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814215+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814215+0000","last_peered":"2024-09-17T15:28:23.814215+0000","last_clean":"2024-09-17T15:28:23.814215+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814215+0000","last_undegraded":"2024-09-17T15:28:23.814215+0000","last_fullsized":"2024-09-17T15:28:23.814215+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8992","reported_seq":9112,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:52.108853+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:28:52.108853+0000","last_peered":"2024-09-17T15:28:52.108853+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:28:52.108853+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":8992,"log_dups_size":0,"ondisk_log_size":8992,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36811235,"num_objects":8988,"num_object_clones":0,"num_object_copies":17976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8988,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6887,"num_write_kb":27548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":4911}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9074","reported_seq":9192,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:52.115575+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:28:52.115575+0000","last_peered":"2024-09-17T15:28:52.115575+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:28:52.115575+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":9074,"log_dups_size":0,"ondisk_log_size":9074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37142528,"num_objects":9068,"num_object_clones":0,"num_object_copies":18136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6836,"num_write_kb":27344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":4931}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9146","reported_seq":9281,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:51.212063+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:28:51.212063+0000","last_peered":"2024-09-17T15:28:51.212063+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:28:51.212063+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":9146,"log_dups_size":0,"ondisk_log_size":9146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37453824,"num_objects":9144,"num_object_clones":0,"num_object_copies":18288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5694,"num_write_kb":22776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5006},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9104","reported_seq":9245,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:51.114585+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:28:51.114585+0000","last_peered":"2024-09-17T15:28:51.114585+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:28:51.114585+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":9104,"log_dups_size":0,"ondisk_log_size":9104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37257216,"num_objects":9096,"num_object_clones":0,"num_object_copies":18192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5391,"num_write_kb":21564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4939},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805074+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805074+0000","last_peered":"2024-09-17T15:28:23.805074+0000","last_clean":"2024-09-17T15:28:23.805074+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805074+0000","last_undegraded":"2024-09-17T15:28:23.805074+0000","last_fullsized":"2024-09-17T15:28:23.805074+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9127","reported_seq":9258,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:51.153238+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:28:51.153238+0000","last_peered":"2024-09-17T15:28:51.153238+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:28:51.153238+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":9127,"log_dups_size":0,"ondisk_log_size":9127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37367808,"num_objects":9123,"num_object_clones":0,"num_object_copies":18246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5914,"num_write_kb":23656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5060},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8982","reported_seq":10329,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:28:51.243205+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:28:51.243205+0000","last_peered":"2024-09-17T15:28:51.243205+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:28:51.243205+0000","last_undegraded":"2024-09-17T15:28:51.243205+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":8982,"log_dups_size":0,"ondisk_log_size":8982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36742086,"num_objects":8972,"num_object_clones":0,"num_object_copies":17944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3507,"num_objects_unfound":0,"num_objects_dirty":8972,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5424,"num_write_kb":21696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":602,"num_bytes_recovered":2465792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5465},{"shards":"0,2","objects":3507}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":594924254,"num_objects":145254,"num_object_clones":0,"num_object_copies":290508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3507,"num_objects_unfound":0,"num_objects_dirty":145254,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":105361,"num_write_kb":421444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4674,"num_bytes_recovered":19136995,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1208012800,"data_stored":1207861054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145326,"ondisk_log_size":145326,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051630,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":697752,"kb_used_data":423112,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93674088,"statfs":{"total":96636764160,"available":95922266112,"internally_reserved":0,"allocated":433266688,"data_stored":432990114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":281210540},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724575,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":448780,"kb_used_data":254652,"kb_used_omap":4,"kb_used_meta":194107,"kb_avail":93923060,"statfs":{"total":96636764160,"available":96177213440,"internally_reserved":0,"allocated":260763648,"data_stored":260439641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":198766251},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247858,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":790272,"kb_used_data":501808,"kb_used_omap":4,"kb_used_meta":288443,"kb_avail":93581568,"statfs":{"total":96636764160,"available":95827525632,"internally_reserved":0,"allocated":513851392,"data_stored":513570741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":295366312},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,2,5,1,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":513970176,"data_stored":513934046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":261013504,"data_stored":260934018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":433029120,"data_stored":432992990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:28:53.902 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:28:53.902 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'9016", 'reported_seq': 9093, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:51.236938+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:28:51.236938+0000', 'last_peered': '2024-09-17T15:28:51.236938+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:28:51.236938+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 9016, 'log_dups_size': 0, 'ondisk_log_size': 9016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36929536, 'num_objects': 9016, 'num_object_clones': 0, 'num_object_copies': 18032, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4070, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9016, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9016, 'num_write_kb': 36064, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4946}, {'shards': '0,2', 'objects': 4070}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:53.902 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:28:53.903 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'9090", 'reported_seq': 9215, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:52.202763+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:28:52.202763+0000', 'last_peered': '2024-09-17T15:28:52.202763+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:28:52.202763+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 9090, 'log_dups_size': 0, 'ondisk_log_size': 9090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37216256, 'num_objects': 9086, 'num_object_clones': 0, 'num_object_copies': 18172, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4169, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9086, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7247, 'num_write_kb': 28988, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4168}, {'shards': '2', 'objects': 4918}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:28:53.903 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:28:53.903 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'9293", 'reported_seq': 9427, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:51.197429+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:28:51.197429+0000', 'last_peered': '2024-09-17T15:28:51.197429+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:28:51.197429+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 9293, 'log_dups_size': 0, 'ondisk_log_size': 9293, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38044131, 'num_objects': 9289, 'num_object_clones': 0, 'num_object_copies': 18578, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4271, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9289, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5792, 'num_write_kb': 23168, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5019}, {'shards': '0,2', 'objects': 4268}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:53.903 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:28:53.903 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'9078", 'reported_seq': 9216, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:51.047414+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:28:51.047414+0000', 'last_peered': '2024-09-17T15:28:51.047414+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:28:51.047414+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 9078, 'log_dups_size': 0, 'ondisk_log_size': 9078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37159878, 'num_objects': 9074, 'num_object_clones': 0, 'num_object_copies': 18148, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4174, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9074, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 5847, 'num_write_kb': 23388, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4900}, {'shards': '0,2', 'objects': 4174}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:53.903 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:28:53.903 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'8947", 'reported_seq': 9077, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:51.222586+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:28:51.222586+0000', 'last_peered': '2024-09-17T15:28:51.222586+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:28:51.222586+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 8947, 'log_dups_size': 0, 'ondisk_log_size': 8947, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36614144, 'num_objects': 8939, 'num_object_clones': 0, 'num_object_copies': 17878, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4012, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8939, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5543, 'num_write_kb': 22172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4929}, {'shards': '0,2', 'objects': 4010}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:53.903 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:28:53.903 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'8992", 'reported_seq': 9112, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:52.108853+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:28:52.108853+0000', 'last_peered': '2024-09-17T15:28:52.108853+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:28:52.108853+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 8992, 'log_dups_size': 0, 'ondisk_log_size': 8992, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36811235, 'num_objects': 8988, 'num_object_clones': 0, 'num_object_copies': 17976, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4078, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8988, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6887, 'num_write_kb': 27548, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4077}, {'shards': '2', 'objects': 4911}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:28:53.904 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:28:53.904 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'9074", 'reported_seq': 9192, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:52.115575+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:28:52.115575+0000', 'last_peered': '2024-09-17T15:28:52.115575+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:28:52.115575+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 9074, 'log_dups_size': 0, 'ondisk_log_size': 9074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37142528, 'num_objects': 9068, 'num_object_clones': 0, 'num_object_copies': 18136, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9068, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6836, 'num_write_kb': 27344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4137}, {'shards': '2', 'objects': 4931}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:28:53.904 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:28:53.904 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'9146", 'reported_seq': 9281, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:51.212063+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:28:51.212063+0000', 'last_peered': '2024-09-17T15:28:51.212063+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:28:51.212063+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 9146, 'log_dups_size': 0, 'ondisk_log_size': 9146, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37453824, 'num_objects': 9144, 'num_object_clones': 0, 'num_object_copies': 18288, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9144, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5694, 'num_write_kb': 22776, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5006}, {'shards': '0,2', 'objects': 4138}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:53.904 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:28:53.904 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'9104", 'reported_seq': 9245, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:51.114585+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:28:51.114585+0000', 'last_peered': '2024-09-17T15:28:51.114585+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:28:51.114585+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 9104, 'log_dups_size': 0, 'ondisk_log_size': 9104, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37257216, 'num_objects': 9096, 'num_object_clones': 0, 'num_object_copies': 18192, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4160, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9096, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5391, 'num_write_kb': 21564, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4939}, {'shards': '0,2', 'objects': 4157}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:53.904 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:28:53.904 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'9127", 'reported_seq': 9258, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:51.153238+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:28:51.153238+0000', 'last_peered': '2024-09-17T15:28:51.153238+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:28:51.153238+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 9127, 'log_dups_size': 0, 'ondisk_log_size': 9127, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37367808, 'num_objects': 9123, 'num_object_clones': 0, 'num_object_copies': 18246, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4065, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9123, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5914, 'num_write_kb': 23656, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5060}, {'shards': '0,2', 'objects': 4063}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:53.904 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:28:53.904 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'8982", 'reported_seq': 10329, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:28:51.243205+0000', 'last_change': '2024-09-17T15:28:22.509818+0000', 'last_active': '2024-09-17T15:28:51.243205+0000', 'last_peered': '2024-09-17T15:28:51.243205+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:28:51.243205+0000', 'last_undegraded': '2024-09-17T15:28:51.243205+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 8982, 'log_dups_size': 0, 'ondisk_log_size': 8982, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36742086, 'num_objects': 8972, 'num_object_clones': 0, 'num_object_copies': 17944, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3507, 'num_objects_unfound': 0, 'num_objects_dirty': 8972, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5424, 'num_write_kb': 21696, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 602, 'num_bytes_recovered': 2465792, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5465}, {'shards': '0,2', 'objects': 3507}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:53.905 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:28:54.323 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 195 20 146515 146495 2.934 1.53516 0.0150542 0.0266189 2024-09-17T15:28:54.620 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:28:54.621 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41277/290512 objects degraded (14.208%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":271,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":145256,"data_bytes":594924273,"bytes_used":1983287296,"bytes_avail":287927005184,"bytes_total":289910292480,"degraded_objects":41277,"degraded_total":290512,"degraded_ratio":0.14208363165721211,"misplaced_objects":3507,"misplaced_total":290512,"misplaced_ratio":0.012071790494024343,"recovering_objects_per_sec":16,"recovering_bytes_per_sec":67567,"recovering_keys_per_sec":0,"num_objects_recovered":198,"num_bytes_recovered":811008,"num_keys_recovered":0,"write_bytes_sec":1858777,"read_op_per_sec":0,"write_op_per_sec":453},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T15:27:19.616376+0000","services":{}},"progress_events":{}} 2024-09-17T15:28:54.643 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:28:54.643 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:28:55.184 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:28:55.184 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:28:55.210 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":171,"stamp":"2024-09-17T15:28:53.640481+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594924273,"num_objects":145256,"num_object_clones":0,"num_object_copies":290512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3507,"num_objects_unfound":0,"num_objects_dirty":145256,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":105363,"num_write_kb":421446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4675,"num_bytes_recovered":19137014,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145329,"ondisk_log_size":145329,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1936804,"kb_used_data":1179572,"kb_used_omap":13,"kb_used_meta":757170,"kb_avail":281178716,"statfs":{"total":289910292480,"available":287927005184,"internally_reserved":0,"allocated":1207881728,"data_stored":1207000496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":775343103},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,3,8,9,12],"upper_bound":128},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22310912,"num_objects":5447,"num_object_clones":0,"num_object_copies":10894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-198,"num_objects_unfound":0,"num_objects_dirty":5447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5447,"num_write_kb":21788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":811008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003000"},"pg_stats":[{"pgid":"2.f","version":"43'9016","reported_seq":9093,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:51.236938+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:28:51.236938+0000","last_peered":"2024-09-17T15:28:51.236938+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:28:51.236938+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9016,"log_dups_size":0,"ondisk_log_size":9016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36929536,"num_objects":9016,"num_object_clones":0,"num_object_copies":18032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9016,"num_write_kb":36064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4946},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9112","reported_seq":9213,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:52.202108+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:28:52.202108+0000","last_peered":"2024-09-17T15:28:52.202108+0000","last_clean":"2024-09-17T15:28:52.202108+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:28:52.202108+0000","last_undegraded":"2024-09-17T15:28:52.202108+0000","last_fullsized":"2024-09-17T15:28:52.202108+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":9112,"log_dups_size":0,"ondisk_log_size":9112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37299142,"num_objects":9108,"num_object_clones":0,"num_object_copies":18216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9108,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7469,"num_write_kb":29876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9090","reported_seq":9215,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:52.202763+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:28:52.202763+0000","last_peered":"2024-09-17T15:28:52.202763+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:28:52.202763+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":9090,"log_dups_size":0,"ondisk_log_size":9090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37216256,"num_objects":9086,"num_object_clones":0,"num_object_copies":18172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7247,"num_write_kb":28988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":4918}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9187","reported_seq":9256,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:52.182200+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:28:52.182200+0000","last_peered":"2024-09-17T15:28:52.182200+0000","last_clean":"2024-09-17T15:28:52.182200+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:28:52.182200+0000","last_undegraded":"2024-09-17T15:28:52.182200+0000","last_fullsized":"2024-09-17T15:28:52.182200+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9187,"log_dups_size":0,"ondisk_log_size":9187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629952,"num_objects":9187,"num_object_clones":0,"num_object_copies":18374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9187,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9187,"num_write_kb":36748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8985","reported_seq":9114,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:51.185950+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:28:51.185950+0000","last_peered":"2024-09-17T15:28:51.185950+0000","last_clean":"2024-09-17T15:28:51.185950+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:28:51.185950+0000","last_undegraded":"2024-09-17T15:28:51.185950+0000","last_fullsized":"2024-09-17T15:28:51.185950+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":8985,"log_dups_size":0,"ondisk_log_size":8985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36774371,"num_objects":8979,"num_object_clones":0,"num_object_copies":17958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8979,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5967,"num_write_kb":23868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9293","reported_seq":9427,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:51.197429+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:28:51.197429+0000","last_peered":"2024-09-17T15:28:51.197429+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:28:51.197429+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":9293,"log_dups_size":0,"ondisk_log_size":9293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38044131,"num_objects":9289,"num_object_clones":0,"num_object_copies":18578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9289,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5792,"num_write_kb":23168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5019},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9078","reported_seq":9216,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:51.047414+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:28:51.047414+0000","last_peered":"2024-09-17T15:28:51.047414+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:28:51.047414+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":9078,"log_dups_size":0,"ondisk_log_size":9078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37159878,"num_objects":9074,"num_object_clones":0,"num_object_copies":18148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9074,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5847,"num_write_kb":23388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4900},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9148","reported_seq":9275,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:51.135000+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:28:51.135000+0000","last_peered":"2024-09-17T15:28:51.135000+0000","last_clean":"2024-09-17T15:28:51.135000+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:28:51.135000+0000","last_undegraded":"2024-09-17T15:28:51.135000+0000","last_fullsized":"2024-09-17T15:28:51.135000+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":9148,"log_dups_size":0,"ondisk_log_size":9148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37462016,"num_objects":9146,"num_object_clones":0,"num_object_copies":18292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6102,"num_write_kb":24408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'9045","reported_seq":17327,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:50.436632+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:28:50.436632+0000","last_peered":"2024-09-17T15:28:50.436632+0000","last_clean":"2024-09-17T15:28:50.436632+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:28:50.436632+0000","last_undegraded":"2024-09-17T15:28:50.436632+0000","last_fullsized":"2024-09-17T15:28:50.436632+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":9045,"log_dups_size":0,"ondisk_log_size":9045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37020131,"num_objects":9039,"num_object_clones":0,"num_object_copies":18078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9039,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7045,"num_write_kb":28180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804705+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804705+0000","last_peered":"2024-09-17T15:28:23.804705+0000","last_clean":"2024-09-17T15:28:23.804705+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804705+0000","last_undegraded":"2024-09-17T15:28:23.804705+0000","last_fullsized":"2024-09-17T15:28:23.804705+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8947","reported_seq":9077,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:51.222586+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:28:51.222586+0000","last_peered":"2024-09-17T15:28:51.222586+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:28:51.222586+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":8947,"log_dups_size":0,"ondisk_log_size":8947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36614144,"num_objects":8939,"num_object_clones":0,"num_object_copies":17878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8939,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5543,"num_write_kb":22172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4929},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":102,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814215+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814215+0000","last_peered":"2024-09-17T15:28:23.814215+0000","last_clean":"2024-09-17T15:28:23.814215+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814215+0000","last_undegraded":"2024-09-17T15:28:23.814215+0000","last_fullsized":"2024-09-17T15:28:23.814215+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8992","reported_seq":9112,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:52.108853+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:28:52.108853+0000","last_peered":"2024-09-17T15:28:52.108853+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:28:52.108853+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":8992,"log_dups_size":0,"ondisk_log_size":8992,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36811235,"num_objects":8988,"num_object_clones":0,"num_object_copies":17976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8988,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6887,"num_write_kb":27548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":4911}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9074","reported_seq":9192,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:52.115575+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:28:52.115575+0000","last_peered":"2024-09-17T15:28:52.115575+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:28:52.115575+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":9074,"log_dups_size":0,"ondisk_log_size":9074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37142528,"num_objects":9068,"num_object_clones":0,"num_object_copies":18136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6836,"num_write_kb":27344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":4931}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9146","reported_seq":9281,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:51.212063+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:28:51.212063+0000","last_peered":"2024-09-17T15:28:51.212063+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:28:51.212063+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":9146,"log_dups_size":0,"ondisk_log_size":9146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37453824,"num_objects":9144,"num_object_clones":0,"num_object_copies":18288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5694,"num_write_kb":22776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5006},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9104","reported_seq":9245,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:51.114585+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:28:51.114585+0000","last_peered":"2024-09-17T15:28:51.114585+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:28:51.114585+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":9104,"log_dups_size":0,"ondisk_log_size":9104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37257216,"num_objects":9096,"num_object_clones":0,"num_object_copies":18192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5391,"num_write_kb":21564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4939},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805074+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805074+0000","last_peered":"2024-09-17T15:28:23.805074+0000","last_clean":"2024-09-17T15:28:23.805074+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805074+0000","last_undegraded":"2024-09-17T15:28:23.805074+0000","last_fullsized":"2024-09-17T15:28:23.805074+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9127","reported_seq":9258,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:51.153238+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:28:51.153238+0000","last_peered":"2024-09-17T15:28:51.153238+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:28:51.153238+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":9127,"log_dups_size":0,"ondisk_log_size":9127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37367808,"num_objects":9123,"num_object_clones":0,"num_object_copies":18246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5914,"num_write_kb":23656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5060},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8982","reported_seq":10329,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:28:51.243205+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:28:51.243205+0000","last_peered":"2024-09-17T15:28:51.243205+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:28:51.243205+0000","last_undegraded":"2024-09-17T15:28:51.243205+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":8982,"log_dups_size":0,"ondisk_log_size":8982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36742086,"num_objects":8972,"num_object_clones":0,"num_object_copies":17944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3507,"num_objects_unfound":0,"num_objects_dirty":8972,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5424,"num_write_kb":21696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":602,"num_bytes_recovered":2465792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5465},{"shards":"0,2","objects":3507}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":594924254,"num_objects":145254,"num_object_clones":0,"num_object_copies":290508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3507,"num_objects_unfound":0,"num_objects_dirty":145254,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":105361,"num_write_kb":421444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4674,"num_bytes_recovered":19136995,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1208012800,"data_stored":1207861054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145326,"ondisk_log_size":145326,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051630,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":697752,"kb_used_data":423112,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93674088,"statfs":{"total":96636764160,"available":95922266112,"internally_reserved":0,"allocated":433266688,"data_stored":432990114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":281210540},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724575,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":448780,"kb_used_data":254652,"kb_used_omap":4,"kb_used_meta":194107,"kb_avail":93923060,"statfs":{"total":96636764160,"available":96177213440,"internally_reserved":0,"allocated":260763648,"data_stored":260439641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":198766251},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247858,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":790272,"kb_used_data":501808,"kb_used_omap":4,"kb_used_meta":288443,"kb_avail":93581568,"statfs":{"total":96636764160,"available":95827525632,"internally_reserved":0,"allocated":513851392,"data_stored":513570741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":295366312},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,2,5,1,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":513970176,"data_stored":513934046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":261013504,"data_stored":260934018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":433029120,"data_stored":432992990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:28:55.323 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 196 20 146953 146933 2.92776 1.71094 0.115797 0.0266765 2024-09-17T15:28:56.323 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 197 20 147440 147420 2.92255 1.90234 0.0237119 0.0267245 2024-09-17T15:28:57.323 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 198 20 147931 147911 2.91748 1.91797 0.062768 0.026771 2024-09-17T15:28:58.212 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:28:58.323 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 199 20 148326 148306 2.91057 1.54297 0.0332955 0.0268332 2024-09-17T15:28:58.752 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:28:58.753 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:28:58.775 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":173,"stamp":"2024-09-17T15:28:57.641420+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":604332785,"num_objects":147553,"num_object_clones":0,"num_object_copies":295106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3406,"num_objects_unfound":0,"num_objects_dirty":147553,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":107660,"num_write_kb":430634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4776,"num_bytes_recovered":19550710,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147626,"ondisk_log_size":147626,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1962996,"kb_used_data":1197444,"kb_used_omap":13,"kb_used_meta":765490,"kb_avail":281152524,"statfs":{"total":289910292480,"available":287900184576,"internally_reserved":0,"allocated":1226182656,"data_stored":1225301424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":783862783},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,3,3,10,10,12],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28073984,"num_objects":6854,"num_object_clones":0,"num_object_copies":13708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-299,"num_objects_unfound":0,"num_objects_dirty":6854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6854,"num_write_kb":27416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":299,"num_bytes_recovered":1224704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002980"},"pg_stats":[{"pgid":"2.f","version":"43'9148","reported_seq":9225,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:56.156564+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:28:56.156564+0000","last_peered":"2024-09-17T15:28:56.156564+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:28:56.156564+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9148,"log_dups_size":0,"ondisk_log_size":9148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37470208,"num_objects":9148,"num_object_clones":0,"num_object_copies":18296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9148,"num_write_kb":36592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5078},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9255","reported_seq":9355,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:57.248144+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:28:57.248144+0000","last_peered":"2024-09-17T15:28:57.248144+0000","last_clean":"2024-09-17T15:28:57.248144+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:28:57.248144+0000","last_undegraded":"2024-09-17T15:28:57.248144+0000","last_fullsized":"2024-09-17T15:28:57.248144+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":9255,"log_dups_size":0,"ondisk_log_size":9255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37884870,"num_objects":9251,"num_object_clones":0,"num_object_copies":18502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9251,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7612,"num_write_kb":30448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9240","reported_seq":9365,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:57.219880+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:28:57.219880+0000","last_peered":"2024-09-17T15:28:57.219880+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:28:57.219880+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":9240,"log_dups_size":0,"ondisk_log_size":9240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37830656,"num_objects":9236,"num_object_clones":0,"num_object_copies":18472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7397,"num_write_kb":29588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":5068}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9367","reported_seq":9436,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:57.198554+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:28:57.198554+0000","last_peered":"2024-09-17T15:28:57.198554+0000","last_clean":"2024-09-17T15:28:57.198554+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:28:57.198554+0000","last_undegraded":"2024-09-17T15:28:57.198554+0000","last_fullsized":"2024-09-17T15:28:57.198554+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9367,"log_dups_size":0,"ondisk_log_size":9367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38367232,"num_objects":9367,"num_object_clones":0,"num_object_copies":18734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9367,"num_write_kb":37468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9140","reported_seq":9269,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:56.291854+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:28:56.291854+0000","last_peered":"2024-09-17T15:28:56.291854+0000","last_clean":"2024-09-17T15:28:56.291854+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:28:56.291854+0000","last_undegraded":"2024-09-17T15:28:56.291854+0000","last_fullsized":"2024-09-17T15:28:56.291854+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":9140,"log_dups_size":0,"ondisk_log_size":9140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37409251,"num_objects":9134,"num_object_clones":0,"num_object_copies":18268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9134,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6122,"num_write_kb":24488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9448","reported_seq":9582,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:56.233747+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:28:56.233747+0000","last_peered":"2024-09-17T15:28:56.233747+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:28:56.233747+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":9448,"log_dups_size":0,"ondisk_log_size":9448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38679011,"num_objects":9444,"num_object_clones":0,"num_object_copies":18888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9444,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5947,"num_write_kb":23788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5174},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9205","reported_seq":9343,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:56.222649+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:28:56.222649+0000","last_peered":"2024-09-17T15:28:56.222649+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:28:56.222649+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":9205,"log_dups_size":0,"ondisk_log_size":9205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37680070,"num_objects":9201,"num_object_clones":0,"num_object_copies":18402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9201,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5974,"num_write_kb":23896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5027},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9293","reported_seq":9419,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:56.286198+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:28:56.286198+0000","last_peered":"2024-09-17T15:28:56.286198+0000","last_clean":"2024-09-17T15:28:56.286198+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:28:56.286198+0000","last_undegraded":"2024-09-17T15:28:56.286198+0000","last_fullsized":"2024-09-17T15:28:56.286198+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":9293,"log_dups_size":0,"ondisk_log_size":9293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38055936,"num_objects":9291,"num_object_clones":0,"num_object_copies":18582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9291,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6247,"num_write_kb":24988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'9186","reported_seq":17468,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:55.380295+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:28:55.380295+0000","last_peered":"2024-09-17T15:28:55.380295+0000","last_clean":"2024-09-17T15:28:55.380295+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:28:55.380295+0000","last_undegraded":"2024-09-17T15:28:55.380295+0000","last_fullsized":"2024-09-17T15:28:55.380295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":9186,"log_dups_size":0,"ondisk_log_size":9186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37597667,"num_objects":9180,"num_object_clones":0,"num_object_copies":18360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9180,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7186,"num_write_kb":28744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804705+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804705+0000","last_peered":"2024-09-17T15:28:23.804705+0000","last_clean":"2024-09-17T15:28:23.804705+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804705+0000","last_undegraded":"2024-09-17T15:28:23.804705+0000","last_fullsized":"2024-09-17T15:28:23.804705+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9095","reported_seq":9224,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:56.261396+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:28:56.261396+0000","last_peered":"2024-09-17T15:28:56.261396+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:28:56.261396+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":9095,"log_dups_size":0,"ondisk_log_size":9095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37220352,"num_objects":9087,"num_object_clones":0,"num_object_copies":18174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9087,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5691,"num_write_kb":22764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5077},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":102,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814215+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814215+0000","last_peered":"2024-09-17T15:28:23.814215+0000","last_clean":"2024-09-17T15:28:23.814215+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814215+0000","last_undegraded":"2024-09-17T15:28:23.814215+0000","last_fullsized":"2024-09-17T15:28:23.814215+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9126","reported_seq":9246,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:57.183507+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:28:57.183507+0000","last_peered":"2024-09-17T15:28:57.183507+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:28:57.183507+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":9126,"log_dups_size":0,"ondisk_log_size":9126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37360099,"num_objects":9122,"num_object_clones":0,"num_object_copies":18244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9122,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7021,"num_write_kb":28084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":5045}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9209","reported_seq":9327,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:57.188119+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:28:57.188119+0000","last_peered":"2024-09-17T15:28:57.188119+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:28:57.188119+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":9209,"log_dups_size":0,"ondisk_log_size":9209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37695488,"num_objects":9203,"num_object_clones":0,"num_object_copies":18406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6971,"num_write_kb":27884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":5066}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9284","reported_seq":9419,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:56.265449+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:28:56.265449+0000","last_peered":"2024-09-17T15:28:56.265449+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:28:56.265449+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":9284,"log_dups_size":0,"ondisk_log_size":9284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38019072,"num_objects":9282,"num_object_clones":0,"num_object_copies":18564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5832,"num_write_kb":23328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5144},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9242","reported_seq":9383,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:56.271923+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:28:56.271923+0000","last_peered":"2024-09-17T15:28:56.271923+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:28:56.271923+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":9242,"log_dups_size":0,"ondisk_log_size":9242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37822464,"num_objects":9234,"num_object_clones":0,"num_object_copies":18468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5529,"num_write_kb":22116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5077},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805074+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805074+0000","last_peered":"2024-09-17T15:28:23.805074+0000","last_clean":"2024-09-17T15:28:23.805074+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805074+0000","last_undegraded":"2024-09-17T15:28:23.805074+0000","last_fullsized":"2024-09-17T15:28:23.805074+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9273","reported_seq":9404,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:56.296023+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:28:56.296023+0000","last_peered":"2024-09-17T15:28:56.296023+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:28:56.296023+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":9273,"log_dups_size":0,"ondisk_log_size":9273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37965824,"num_objects":9269,"num_object_clones":0,"num_object_copies":18538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9269,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6060,"num_write_kb":24240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5206},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9112","reported_seq":10661,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:28:56.277009+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:28:56.277009+0000","last_peered":"2024-09-17T15:28:56.277009+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:28:56.277009+0000","last_undegraded":"2024-09-17T15:28:56.277009+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":9112,"log_dups_size":0,"ondisk_log_size":9112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37274566,"num_objects":9102,"num_object_clones":0,"num_object_copies":18204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3406,"num_objects_unfound":0,"num_objects_dirty":9102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5554,"num_write_kb":22216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":703,"num_bytes_recovered":2879488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5696},{"shards":"0,2","objects":3406}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":604332766,"num_objects":147551,"num_object_clones":0,"num_object_copies":295102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3406,"num_objects_unfound":0,"num_objects_dirty":147551,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":107658,"num_write_kb":430632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4775,"num_bytes_recovered":19550691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1227231232,"data_stored":1227079486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147623,"ondisk_log_size":147623,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051631,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":701580,"kb_used_data":426940,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93670260,"statfs":{"total":96636764160,"available":95918346240,"internally_reserved":0,"allocated":437186560,"data_stored":436909986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":281210540},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,0,4,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724576,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":460036,"kb_used_data":261748,"kb_used_omap":4,"kb_used_meta":198267,"kb_avail":93911804,"statfs":{"total":96636764160,"available":96165687296,"internally_reserved":0,"allocated":268029952,"data_stored":267705945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":203026091},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,0,1,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247859,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":801380,"kb_used_data":508756,"kb_used_omap":4,"kb_used_meta":292603,"kb_avail":93570460,"statfs":{"total":96636764160,"available":95816151040,"internally_reserved":0,"allocated":520966144,"data_stored":520685493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":299626152},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,5,6,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":521043968,"data_stored":521007838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":268382208,"data_stored":268302722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":437805056,"data_stored":437768926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:28:58.777 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:28:59.321 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:28:59.323 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:28:59.323 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout:2024-09-17T15:28:59.324104+0000 min lat: 0.00187689 max lat: 2.64156 avg lat: 0.0268924 2024-09-17T15:28:59.323 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-17T15:28:59.323 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 200 20 148747 148727 2.90424 1.64453 0.0541651 0.0268924 2024-09-17T15:28:59.343 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":173,"stamp":"2024-09-17T15:28:57.641420+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":604332785,"num_objects":147553,"num_object_clones":0,"num_object_copies":295106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3406,"num_objects_unfound":0,"num_objects_dirty":147553,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":107660,"num_write_kb":430634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4776,"num_bytes_recovered":19550710,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147626,"ondisk_log_size":147626,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1962996,"kb_used_data":1197444,"kb_used_omap":13,"kb_used_meta":765490,"kb_avail":281152524,"statfs":{"total":289910292480,"available":287900184576,"internally_reserved":0,"allocated":1226182656,"data_stored":1225301424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":783862783},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,3,3,10,10,12],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28073984,"num_objects":6854,"num_object_clones":0,"num_object_copies":13708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-299,"num_objects_unfound":0,"num_objects_dirty":6854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6854,"num_write_kb":27416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":299,"num_bytes_recovered":1224704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002980"},"pg_stats":[{"pgid":"2.f","version":"43'9148","reported_seq":9225,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:56.156564+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:28:56.156564+0000","last_peered":"2024-09-17T15:28:56.156564+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:28:56.156564+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9148,"log_dups_size":0,"ondisk_log_size":9148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37470208,"num_objects":9148,"num_object_clones":0,"num_object_copies":18296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9148,"num_write_kb":36592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5078},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9255","reported_seq":9355,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:57.248144+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:28:57.248144+0000","last_peered":"2024-09-17T15:28:57.248144+0000","last_clean":"2024-09-17T15:28:57.248144+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:28:57.248144+0000","last_undegraded":"2024-09-17T15:28:57.248144+0000","last_fullsized":"2024-09-17T15:28:57.248144+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":9255,"log_dups_size":0,"ondisk_log_size":9255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37884870,"num_objects":9251,"num_object_clones":0,"num_object_copies":18502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9251,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7612,"num_write_kb":30448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9240","reported_seq":9365,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:57.219880+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:28:57.219880+0000","last_peered":"2024-09-17T15:28:57.219880+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:28:57.219880+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":9240,"log_dups_size":0,"ondisk_log_size":9240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37830656,"num_objects":9236,"num_object_clones":0,"num_object_copies":18472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7397,"num_write_kb":29588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":5068}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9367","reported_seq":9436,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:57.198554+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:28:57.198554+0000","last_peered":"2024-09-17T15:28:57.198554+0000","last_clean":"2024-09-17T15:28:57.198554+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:28:57.198554+0000","last_undegraded":"2024-09-17T15:28:57.198554+0000","last_fullsized":"2024-09-17T15:28:57.198554+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9367,"log_dups_size":0,"ondisk_log_size":9367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38367232,"num_objects":9367,"num_object_clones":0,"num_object_copies":18734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9367,"num_write_kb":37468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9140","reported_seq":9269,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:56.291854+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:28:56.291854+0000","last_peered":"2024-09-17T15:28:56.291854+0000","last_clean":"2024-09-17T15:28:56.291854+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:28:56.291854+0000","last_undegraded":"2024-09-17T15:28:56.291854+0000","last_fullsized":"2024-09-17T15:28:56.291854+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":9140,"log_dups_size":0,"ondisk_log_size":9140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37409251,"num_objects":9134,"num_object_clones":0,"num_object_copies":18268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9134,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6122,"num_write_kb":24488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9448","reported_seq":9582,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:56.233747+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:28:56.233747+0000","last_peered":"2024-09-17T15:28:56.233747+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:28:56.233747+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":9448,"log_dups_size":0,"ondisk_log_size":9448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38679011,"num_objects":9444,"num_object_clones":0,"num_object_copies":18888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9444,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5947,"num_write_kb":23788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5174},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9205","reported_seq":9343,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:56.222649+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:28:56.222649+0000","last_peered":"2024-09-17T15:28:56.222649+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:28:56.222649+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":9205,"log_dups_size":0,"ondisk_log_size":9205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37680070,"num_objects":9201,"num_object_clones":0,"num_object_copies":18402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9201,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5974,"num_write_kb":23896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5027},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9293","reported_seq":9419,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:56.286198+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:28:56.286198+0000","last_peered":"2024-09-17T15:28:56.286198+0000","last_clean":"2024-09-17T15:28:56.286198+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:28:56.286198+0000","last_undegraded":"2024-09-17T15:28:56.286198+0000","last_fullsized":"2024-09-17T15:28:56.286198+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":9293,"log_dups_size":0,"ondisk_log_size":9293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38055936,"num_objects":9291,"num_object_clones":0,"num_object_copies":18582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9291,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6247,"num_write_kb":24988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'9186","reported_seq":17468,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:55.380295+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:28:55.380295+0000","last_peered":"2024-09-17T15:28:55.380295+0000","last_clean":"2024-09-17T15:28:55.380295+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:28:55.380295+0000","last_undegraded":"2024-09-17T15:28:55.380295+0000","last_fullsized":"2024-09-17T15:28:55.380295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":9186,"log_dups_size":0,"ondisk_log_size":9186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37597667,"num_objects":9180,"num_object_clones":0,"num_object_copies":18360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9180,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7186,"num_write_kb":28744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804705+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804705+0000","last_peered":"2024-09-17T15:28:23.804705+0000","last_clean":"2024-09-17T15:28:23.804705+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804705+0000","last_undegraded":"2024-09-17T15:28:23.804705+0000","last_fullsized":"2024-09-17T15:28:23.804705+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9095","reported_seq":9224,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:56.261396+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:28:56.261396+0000","last_peered":"2024-09-17T15:28:56.261396+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:28:56.261396+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":9095,"log_dups_size":0,"ondisk_log_size":9095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37220352,"num_objects":9087,"num_object_clones":0,"num_object_copies":18174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9087,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5691,"num_write_kb":22764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5077},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":102,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814215+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814215+0000","last_peered":"2024-09-17T15:28:23.814215+0000","last_clean":"2024-09-17T15:28:23.814215+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814215+0000","last_undegraded":"2024-09-17T15:28:23.814215+0000","last_fullsized":"2024-09-17T15:28:23.814215+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9126","reported_seq":9246,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:57.183507+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:28:57.183507+0000","last_peered":"2024-09-17T15:28:57.183507+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:28:57.183507+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":9126,"log_dups_size":0,"ondisk_log_size":9126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37360099,"num_objects":9122,"num_object_clones":0,"num_object_copies":18244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9122,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7021,"num_write_kb":28084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":5045}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9209","reported_seq":9327,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:57.188119+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:28:57.188119+0000","last_peered":"2024-09-17T15:28:57.188119+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:28:57.188119+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":9209,"log_dups_size":0,"ondisk_log_size":9209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37695488,"num_objects":9203,"num_object_clones":0,"num_object_copies":18406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6971,"num_write_kb":27884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":5066}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9284","reported_seq":9419,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:56.265449+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:28:56.265449+0000","last_peered":"2024-09-17T15:28:56.265449+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:28:56.265449+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":9284,"log_dups_size":0,"ondisk_log_size":9284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38019072,"num_objects":9282,"num_object_clones":0,"num_object_copies":18564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5832,"num_write_kb":23328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5144},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9242","reported_seq":9383,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:56.271923+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:28:56.271923+0000","last_peered":"2024-09-17T15:28:56.271923+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:28:56.271923+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":9242,"log_dups_size":0,"ondisk_log_size":9242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37822464,"num_objects":9234,"num_object_clones":0,"num_object_copies":18468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5529,"num_write_kb":22116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5077},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805074+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805074+0000","last_peered":"2024-09-17T15:28:23.805074+0000","last_clean":"2024-09-17T15:28:23.805074+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805074+0000","last_undegraded":"2024-09-17T15:28:23.805074+0000","last_fullsized":"2024-09-17T15:28:23.805074+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9273","reported_seq":9404,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:56.296023+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:28:56.296023+0000","last_peered":"2024-09-17T15:28:56.296023+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:28:56.296023+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":9273,"log_dups_size":0,"ondisk_log_size":9273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37965824,"num_objects":9269,"num_object_clones":0,"num_object_copies":18538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9269,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6060,"num_write_kb":24240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5206},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9112","reported_seq":10661,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:28:56.277009+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:28:56.277009+0000","last_peered":"2024-09-17T15:28:56.277009+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:28:56.277009+0000","last_undegraded":"2024-09-17T15:28:56.277009+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":9112,"log_dups_size":0,"ondisk_log_size":9112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37274566,"num_objects":9102,"num_object_clones":0,"num_object_copies":18204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3406,"num_objects_unfound":0,"num_objects_dirty":9102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5554,"num_write_kb":22216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":703,"num_bytes_recovered":2879488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5696},{"shards":"0,2","objects":3406}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":604332766,"num_objects":147551,"num_object_clones":0,"num_object_copies":295102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3406,"num_objects_unfound":0,"num_objects_dirty":147551,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":107658,"num_write_kb":430632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4775,"num_bytes_recovered":19550691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1227231232,"data_stored":1227079486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147623,"ondisk_log_size":147623,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051631,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":701580,"kb_used_data":426940,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93670260,"statfs":{"total":96636764160,"available":95918346240,"internally_reserved":0,"allocated":437186560,"data_stored":436909986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":281210540},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,0,4,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724576,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":460036,"kb_used_data":261748,"kb_used_omap":4,"kb_used_meta":198267,"kb_avail":93911804,"statfs":{"total":96636764160,"available":96165687296,"internally_reserved":0,"allocated":268029952,"data_stored":267705945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":203026091},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,0,1,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247859,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":801380,"kb_used_data":508756,"kb_used_omap":4,"kb_used_meta":292603,"kb_avail":93570460,"statfs":{"total":96636764160,"available":95816151040,"internally_reserved":0,"allocated":520966144,"data_stored":520685493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":299626152},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,5,6,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":521043968,"data_stored":521007838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":268382208,"data_stored":268302722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":437805056,"data_stored":437768926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:28:59.344 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:28:59.344 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'9148", 'reported_seq': 9225, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:56.156564+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:28:56.156564+0000', 'last_peered': '2024-09-17T15:28:56.156564+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:28:56.156564+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 9148, 'log_dups_size': 0, 'ondisk_log_size': 9148, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37470208, 'num_objects': 9148, 'num_object_clones': 0, 'num_object_copies': 18296, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4070, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9148, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9148, 'num_write_kb': 36592, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5078}, {'shards': '0,2', 'objects': 4070}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:59.344 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:28:59.344 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'9240", 'reported_seq': 9365, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:57.219880+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:28:57.219880+0000', 'last_peered': '2024-09-17T15:28:57.219880+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:28:57.219880+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 9240, 'log_dups_size': 0, 'ondisk_log_size': 9240, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37830656, 'num_objects': 9236, 'num_object_clones': 0, 'num_object_copies': 18472, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4169, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9236, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7397, 'num_write_kb': 29588, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4168}, {'shards': '2', 'objects': 5068}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:28:59.344 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:28:59.344 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'9448", 'reported_seq': 9582, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:56.233747+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:28:56.233747+0000', 'last_peered': '2024-09-17T15:28:56.233747+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:28:56.233747+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 9448, 'log_dups_size': 0, 'ondisk_log_size': 9448, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38679011, 'num_objects': 9444, 'num_object_clones': 0, 'num_object_copies': 18888, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4271, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9444, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5947, 'num_write_kb': 23788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5174}, {'shards': '0,2', 'objects': 4268}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:59.345 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:28:59.345 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'9205", 'reported_seq': 9343, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:56.222649+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:28:56.222649+0000', 'last_peered': '2024-09-17T15:28:56.222649+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:28:56.222649+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 9205, 'log_dups_size': 0, 'ondisk_log_size': 9205, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37680070, 'num_objects': 9201, 'num_object_clones': 0, 'num_object_copies': 18402, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4174, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9201, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 5974, 'num_write_kb': 23896, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5027}, {'shards': '0,2', 'objects': 4174}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:59.345 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:28:59.345 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'9095", 'reported_seq': 9224, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:56.261396+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:28:56.261396+0000', 'last_peered': '2024-09-17T15:28:56.261396+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:28:56.261396+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 9095, 'log_dups_size': 0, 'ondisk_log_size': 9095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37220352, 'num_objects': 9087, 'num_object_clones': 0, 'num_object_copies': 18174, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4012, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9087, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5691, 'num_write_kb': 22764, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5077}, {'shards': '0,2', 'objects': 4010}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:59.345 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:28:59.345 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'9126", 'reported_seq': 9246, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:57.183507+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:28:57.183507+0000', 'last_peered': '2024-09-17T15:28:57.183507+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:28:57.183507+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 9126, 'log_dups_size': 0, 'ondisk_log_size': 9126, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37360099, 'num_objects': 9122, 'num_object_clones': 0, 'num_object_copies': 18244, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4078, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9122, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7021, 'num_write_kb': 28084, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4077}, {'shards': '2', 'objects': 5045}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:28:59.345 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:28:59.345 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'9209", 'reported_seq': 9327, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:57.188119+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:28:57.188119+0000', 'last_peered': '2024-09-17T15:28:57.188119+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:28:57.188119+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 9209, 'log_dups_size': 0, 'ondisk_log_size': 9209, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37695488, 'num_objects': 9203, 'num_object_clones': 0, 'num_object_copies': 18406, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9203, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6971, 'num_write_kb': 27884, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4137}, {'shards': '2', 'objects': 5066}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:28:59.345 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:28:59.345 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'9284", 'reported_seq': 9419, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:56.265449+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:28:56.265449+0000', 'last_peered': '2024-09-17T15:28:56.265449+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:28:56.265449+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 9284, 'log_dups_size': 0, 'ondisk_log_size': 9284, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38019072, 'num_objects': 9282, 'num_object_clones': 0, 'num_object_copies': 18564, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9282, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5832, 'num_write_kb': 23328, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5144}, {'shards': '0,2', 'objects': 4138}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:59.345 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:28:59.346 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'9242", 'reported_seq': 9383, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:56.271923+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:28:56.271923+0000', 'last_peered': '2024-09-17T15:28:56.271923+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:28:56.271923+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 9242, 'log_dups_size': 0, 'ondisk_log_size': 9242, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37822464, 'num_objects': 9234, 'num_object_clones': 0, 'num_object_copies': 18468, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4160, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9234, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5529, 'num_write_kb': 22116, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5077}, {'shards': '0,2', 'objects': 4157}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:59.346 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:28:59.346 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'9273", 'reported_seq': 9404, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:28:56.296023+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:28:56.296023+0000', 'last_peered': '2024-09-17T15:28:56.296023+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:28:56.296023+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 9273, 'log_dups_size': 0, 'ondisk_log_size': 9273, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37965824, 'num_objects': 9269, 'num_object_clones': 0, 'num_object_copies': 18538, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4065, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9269, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6060, 'num_write_kb': 24240, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5206}, {'shards': '0,2', 'objects': 4063}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:59.346 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:28:59.346 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'9112", 'reported_seq': 10661, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:28:56.277009+0000', 'last_change': '2024-09-17T15:28:22.509818+0000', 'last_active': '2024-09-17T15:28:56.277009+0000', 'last_peered': '2024-09-17T15:28:56.277009+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:28:56.277009+0000', 'last_undegraded': '2024-09-17T15:28:56.277009+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 9112, 'log_dups_size': 0, 'ondisk_log_size': 9112, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37274566, 'num_objects': 9102, 'num_object_clones': 0, 'num_object_copies': 18204, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3406, 'num_objects_unfound': 0, 'num_objects_dirty': 9102, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5554, 'num_write_kb': 22216, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 703, 'num_bytes_recovered': 2879488, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5696}, {'shards': '0,2', 'objects': 3406}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:28:59.346 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:29:00.043 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:29:00.044 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41277/295106 objects degraded (13.987%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":277,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":147553,"data_bytes":604332785,"bytes_used":2010107904,"bytes_avail":287900184576,"bytes_total":289910292480,"degraded_objects":41277,"degraded_total":295106,"degraded_ratio":0.13987177488766747,"misplaced_objects":3406,"misplaced_total":295106,"misplaced_ratio":0.01154161555508868,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":102033,"recovering_keys_per_sec":0,"num_objects_recovered":299,"num_bytes_recovered":1224704,"num_keys_recovered":0,"write_bytes_sec":2338917,"read_op_per_sec":0,"write_op_per_sec":571},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T15:27:19.616376+0000","services":{}},"progress_events":{}} 2024-09-17T15:29:00.069 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:29:00.069 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:29:00.325 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 201 20 149174 149154 2.89806 1.66797 0.101 0.0269491 2024-09-17T15:29:00.613 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:29:00.615 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:29:00.640 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":174,"stamp":"2024-09-17T15:28:59.643240+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":604332785,"num_objects":147553,"num_object_clones":0,"num_object_copies":295106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3406,"num_objects_unfound":0,"num_objects_dirty":147553,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":107660,"num_write_kb":430634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4776,"num_bytes_recovered":19550710,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147626,"ondisk_log_size":147626,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1962996,"kb_used_data":1197444,"kb_used_omap":13,"kb_used_meta":765490,"kb_avail":281152524,"statfs":{"total":289910292480,"available":287900184576,"internally_reserved":0,"allocated":1226182656,"data_stored":1225301424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":783862783},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,3,3,10,10,12],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19111936,"num_objects":4666,"num_object_clones":0,"num_object_copies":9332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-198,"num_objects_unfound":0,"num_objects_dirty":4666,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4666,"num_write_kb":18664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":811008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004374"},"pg_stats":[{"pgid":"2.f","version":"43'9148","reported_seq":9225,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:56.156564+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:28:56.156564+0000","last_peered":"2024-09-17T15:28:56.156564+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:28:56.156564+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9148,"log_dups_size":0,"ondisk_log_size":9148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37470208,"num_objects":9148,"num_object_clones":0,"num_object_copies":18296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9148,"num_write_kb":36592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5078},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9255","reported_seq":9355,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:57.248144+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:28:57.248144+0000","last_peered":"2024-09-17T15:28:57.248144+0000","last_clean":"2024-09-17T15:28:57.248144+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:28:57.248144+0000","last_undegraded":"2024-09-17T15:28:57.248144+0000","last_fullsized":"2024-09-17T15:28:57.248144+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":9255,"log_dups_size":0,"ondisk_log_size":9255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37884870,"num_objects":9251,"num_object_clones":0,"num_object_copies":18502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9251,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7612,"num_write_kb":30448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9240","reported_seq":9365,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:57.219880+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:28:57.219880+0000","last_peered":"2024-09-17T15:28:57.219880+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:28:57.219880+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":9240,"log_dups_size":0,"ondisk_log_size":9240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37830656,"num_objects":9236,"num_object_clones":0,"num_object_copies":18472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7397,"num_write_kb":29588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":5068}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9367","reported_seq":9436,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:57.198554+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:28:57.198554+0000","last_peered":"2024-09-17T15:28:57.198554+0000","last_clean":"2024-09-17T15:28:57.198554+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:28:57.198554+0000","last_undegraded":"2024-09-17T15:28:57.198554+0000","last_fullsized":"2024-09-17T15:28:57.198554+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9367,"log_dups_size":0,"ondisk_log_size":9367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38367232,"num_objects":9367,"num_object_clones":0,"num_object_copies":18734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9367,"num_write_kb":37468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9140","reported_seq":9269,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:56.291854+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:28:56.291854+0000","last_peered":"2024-09-17T15:28:56.291854+0000","last_clean":"2024-09-17T15:28:56.291854+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:28:56.291854+0000","last_undegraded":"2024-09-17T15:28:56.291854+0000","last_fullsized":"2024-09-17T15:28:56.291854+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":9140,"log_dups_size":0,"ondisk_log_size":9140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37409251,"num_objects":9134,"num_object_clones":0,"num_object_copies":18268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9134,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6122,"num_write_kb":24488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9448","reported_seq":9582,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:56.233747+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:28:56.233747+0000","last_peered":"2024-09-17T15:28:56.233747+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:28:56.233747+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":9448,"log_dups_size":0,"ondisk_log_size":9448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38679011,"num_objects":9444,"num_object_clones":0,"num_object_copies":18888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9444,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5947,"num_write_kb":23788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5174},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9205","reported_seq":9343,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:56.222649+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:28:56.222649+0000","last_peered":"2024-09-17T15:28:56.222649+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:28:56.222649+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":9205,"log_dups_size":0,"ondisk_log_size":9205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37680070,"num_objects":9201,"num_object_clones":0,"num_object_copies":18402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9201,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5974,"num_write_kb":23896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5027},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9293","reported_seq":9419,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:56.286198+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:28:56.286198+0000","last_peered":"2024-09-17T15:28:56.286198+0000","last_clean":"2024-09-17T15:28:56.286198+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:28:56.286198+0000","last_undegraded":"2024-09-17T15:28:56.286198+0000","last_fullsized":"2024-09-17T15:28:56.286198+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":9293,"log_dups_size":0,"ondisk_log_size":9293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38055936,"num_objects":9291,"num_object_clones":0,"num_object_copies":18582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9291,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6247,"num_write_kb":24988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'9186","reported_seq":17468,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:55.380295+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:28:55.380295+0000","last_peered":"2024-09-17T15:28:55.380295+0000","last_clean":"2024-09-17T15:28:55.380295+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:28:55.380295+0000","last_undegraded":"2024-09-17T15:28:55.380295+0000","last_fullsized":"2024-09-17T15:28:55.380295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":9186,"log_dups_size":0,"ondisk_log_size":9186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37597667,"num_objects":9180,"num_object_clones":0,"num_object_copies":18360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9180,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7186,"num_write_kb":28744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804705+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804705+0000","last_peered":"2024-09-17T15:28:23.804705+0000","last_clean":"2024-09-17T15:28:23.804705+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804705+0000","last_undegraded":"2024-09-17T15:28:23.804705+0000","last_fullsized":"2024-09-17T15:28:23.804705+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9095","reported_seq":9224,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:56.261396+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:28:56.261396+0000","last_peered":"2024-09-17T15:28:56.261396+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:28:56.261396+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":9095,"log_dups_size":0,"ondisk_log_size":9095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37220352,"num_objects":9087,"num_object_clones":0,"num_object_copies":18174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9087,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5691,"num_write_kb":22764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5077},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":102,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814215+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814215+0000","last_peered":"2024-09-17T15:28:23.814215+0000","last_clean":"2024-09-17T15:28:23.814215+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814215+0000","last_undegraded":"2024-09-17T15:28:23.814215+0000","last_fullsized":"2024-09-17T15:28:23.814215+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9126","reported_seq":9246,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:57.183507+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:28:57.183507+0000","last_peered":"2024-09-17T15:28:57.183507+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:28:57.183507+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":9126,"log_dups_size":0,"ondisk_log_size":9126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37360099,"num_objects":9122,"num_object_clones":0,"num_object_copies":18244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9122,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7021,"num_write_kb":28084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":5045}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9209","reported_seq":9327,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:57.188119+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:28:57.188119+0000","last_peered":"2024-09-17T15:28:57.188119+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:28:57.188119+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":9209,"log_dups_size":0,"ondisk_log_size":9209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37695488,"num_objects":9203,"num_object_clones":0,"num_object_copies":18406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6971,"num_write_kb":27884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":5066}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9284","reported_seq":9419,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:56.265449+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:28:56.265449+0000","last_peered":"2024-09-17T15:28:56.265449+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:28:56.265449+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":9284,"log_dups_size":0,"ondisk_log_size":9284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38019072,"num_objects":9282,"num_object_clones":0,"num_object_copies":18564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5832,"num_write_kb":23328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5144},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9242","reported_seq":9383,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:56.271923+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:28:56.271923+0000","last_peered":"2024-09-17T15:28:56.271923+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:28:56.271923+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":9242,"log_dups_size":0,"ondisk_log_size":9242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37822464,"num_objects":9234,"num_object_clones":0,"num_object_copies":18468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5529,"num_write_kb":22116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5077},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805074+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805074+0000","last_peered":"2024-09-17T15:28:23.805074+0000","last_clean":"2024-09-17T15:28:23.805074+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805074+0000","last_undegraded":"2024-09-17T15:28:23.805074+0000","last_fullsized":"2024-09-17T15:28:23.805074+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9273","reported_seq":9404,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:28:56.296023+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:28:56.296023+0000","last_peered":"2024-09-17T15:28:56.296023+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:28:56.296023+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":9273,"log_dups_size":0,"ondisk_log_size":9273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37965824,"num_objects":9269,"num_object_clones":0,"num_object_copies":18538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9269,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6060,"num_write_kb":24240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5206},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9112","reported_seq":10661,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:28:56.277009+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:28:56.277009+0000","last_peered":"2024-09-17T15:28:56.277009+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:28:56.277009+0000","last_undegraded":"2024-09-17T15:28:56.277009+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":9112,"log_dups_size":0,"ondisk_log_size":9112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37274566,"num_objects":9102,"num_object_clones":0,"num_object_copies":18204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3406,"num_objects_unfound":0,"num_objects_dirty":9102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5554,"num_write_kb":22216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":703,"num_bytes_recovered":2879488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5696},{"shards":"0,2","objects":3406}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":604332766,"num_objects":147551,"num_object_clones":0,"num_object_copies":295102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3406,"num_objects_unfound":0,"num_objects_dirty":147551,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":107658,"num_write_kb":430632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4775,"num_bytes_recovered":19550691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1227231232,"data_stored":1227079486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147623,"ondisk_log_size":147623,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051631,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":701580,"kb_used_data":426940,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93670260,"statfs":{"total":96636764160,"available":95918346240,"internally_reserved":0,"allocated":437186560,"data_stored":436909986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":281210540},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,0,4,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724576,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":460036,"kb_used_data":261748,"kb_used_omap":4,"kb_used_meta":198267,"kb_avail":93911804,"statfs":{"total":96636764160,"available":96165687296,"internally_reserved":0,"allocated":268029952,"data_stored":267705945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":203026091},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,0,1,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247859,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":801380,"kb_used_data":508756,"kb_used_omap":4,"kb_used_meta":292603,"kb_avail":93570460,"statfs":{"total":96636764160,"available":95816151040,"internally_reserved":0,"allocated":520966144,"data_stored":520685493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":299626152},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,5,6,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":521043968,"data_stored":521007838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":268382208,"data_stored":268302722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":437805056,"data_stored":437768926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:29:01.325 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 202 20 149647 149627 2.89286 1.84766 0.0273644 0.0269998 2024-09-17T15:29:02.325 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 203 20 150130 150110 2.88791 1.88672 0.0554408 0.0270454 2024-09-17T15:29:03.325 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 204 20 150592 150572 2.8826 1.80469 0.0620488 0.0270956 2024-09-17T15:29:03.642 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:29:04.213 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:29:04.213 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:29:04.237 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":176,"stamp":"2024-09-17T15:29:03.644623+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":613401329,"num_objects":149767,"num_object_clones":0,"num_object_copies":299534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3293,"num_objects_unfound":0,"num_objects_dirty":149767,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":109874,"num_write_kb":439490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4889,"num_bytes_recovered":20013558,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149840,"ondisk_log_size":149840,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1982508,"kb_used_data":1215548,"kb_used_omap":13,"kb_used_meta":766898,"kb_avail":281133012,"statfs":{"total":289910292480,"available":287880204288,"internally_reserved":0,"allocated":1244721152,"data_stored":1243839920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":785304575},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,3,9,8,12,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21549056,"num_objects":5261,"num_object_clones":0,"num_object_copies":10522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-214,"num_objects_unfound":0,"num_objects_dirty":5261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5261,"num_write_kb":21044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":214,"num_bytes_recovered":876544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004661"},"pg_stats":[{"pgid":"2.f","version":"43'9293","reported_seq":9366,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:01.224281+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:29:01.224281+0000","last_peered":"2024-09-17T15:29:01.224281+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:29:01.224281+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9293,"log_dups_size":0,"ondisk_log_size":9293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38064128,"num_objects":9293,"num_object_clones":0,"num_object_copies":18586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9293,"num_write_kb":37172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5223},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9399","reported_seq":9501,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:02.233704+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:29:02.233704+0000","last_peered":"2024-09-17T15:29:02.233704+0000","last_clean":"2024-09-17T15:29:02.233704+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:29:02.233704+0000","last_undegraded":"2024-09-17T15:29:02.233704+0000","last_fullsized":"2024-09-17T15:29:02.233704+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":9399,"log_dups_size":0,"ondisk_log_size":9399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38474694,"num_objects":9395,"num_object_clones":0,"num_object_copies":18790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9395,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7756,"num_write_kb":31024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9406","reported_seq":9531,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:02.254469+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:29:02.254469+0000","last_peered":"2024-09-17T15:29:02.254469+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:29:02.254469+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":9406,"log_dups_size":0,"ondisk_log_size":9406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38510592,"num_objects":9402,"num_object_clones":0,"num_object_copies":18804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7563,"num_write_kb":30252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":5234}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9502","reported_seq":9571,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:02.162722+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:29:02.162722+0000","last_peered":"2024-09-17T15:29:02.162722+0000","last_clean":"2024-09-17T15:29:02.162722+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:29:02.162722+0000","last_undegraded":"2024-09-17T15:29:02.162722+0000","last_fullsized":"2024-09-17T15:29:02.162722+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9502,"log_dups_size":0,"ondisk_log_size":9502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38920192,"num_objects":9502,"num_object_clones":0,"num_object_copies":19004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9502,"num_write_kb":38008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9261","reported_seq":9390,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:01.291601+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:29:01.291601+0000","last_peered":"2024-09-17T15:29:01.291601+0000","last_clean":"2024-09-17T15:29:01.291601+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:29:01.291601+0000","last_undegraded":"2024-09-17T15:29:01.291601+0000","last_fullsized":"2024-09-17T15:29:01.291601+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":9261,"log_dups_size":0,"ondisk_log_size":9261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37904867,"num_objects":9255,"num_object_clones":0,"num_object_copies":18510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9255,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6243,"num_write_kb":24972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9562","reported_seq":9696,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:01.154139+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:29:01.154139+0000","last_peered":"2024-09-17T15:29:01.154139+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:29:01.154139+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":9562,"log_dups_size":0,"ondisk_log_size":9562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39145955,"num_objects":9558,"num_object_clones":0,"num_object_copies":19116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9558,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6061,"num_write_kb":24244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5288},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9376","reported_seq":9514,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:01.291975+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:29:01.291975+0000","last_peered":"2024-09-17T15:29:01.291975+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:29:01.291975+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":9376,"log_dups_size":0,"ondisk_log_size":9376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38380486,"num_objects":9372,"num_object_clones":0,"num_object_copies":18744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9372,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6145,"num_write_kb":24580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5198},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9432","reported_seq":9559,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:01.262327+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:29:01.262327+0000","last_peered":"2024-09-17T15:29:01.262327+0000","last_clean":"2024-09-17T15:29:01.262327+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:29:01.262327+0000","last_undegraded":"2024-09-17T15:29:01.262327+0000","last_fullsized":"2024-09-17T15:29:01.262327+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":9432,"log_dups_size":0,"ondisk_log_size":9432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38625280,"num_objects":9430,"num_object_clones":0,"num_object_copies":18860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6386,"num_write_kb":25544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'9328","reported_seq":17610,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:00.580979+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:29:00.580979+0000","last_peered":"2024-09-17T15:29:00.580979+0000","last_clean":"2024-09-17T15:29:00.580979+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:29:00.580979+0000","last_undegraded":"2024-09-17T15:29:00.580979+0000","last_fullsized":"2024-09-17T15:29:00.580979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":9328,"log_dups_size":0,"ondisk_log_size":9328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38179299,"num_objects":9322,"num_object_clones":0,"num_object_copies":18644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9322,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7328,"num_write_kb":29312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804705+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804705+0000","last_peered":"2024-09-17T15:28:23.804705+0000","last_clean":"2024-09-17T15:28:23.804705+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804705+0000","last_undegraded":"2024-09-17T15:28:23.804705+0000","last_fullsized":"2024-09-17T15:28:23.804705+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9232","reported_seq":9362,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:01.224283+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:29:01.224283+0000","last_peered":"2024-09-17T15:29:01.224283+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:29:01.224283+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":9232,"log_dups_size":0,"ondisk_log_size":9232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37781504,"num_objects":9224,"num_object_clones":0,"num_object_copies":18448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5828,"num_write_kb":23312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5214},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":102,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814215+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814215+0000","last_peered":"2024-09-17T15:28:23.814215+0000","last_clean":"2024-09-17T15:28:23.814215+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814215+0000","last_undegraded":"2024-09-17T15:28:23.814215+0000","last_fullsized":"2024-09-17T15:28:23.814215+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9258","reported_seq":9378,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:02.244111+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:29:02.244111+0000","last_peered":"2024-09-17T15:29:02.244111+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:29:02.244111+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":9258,"log_dups_size":0,"ondisk_log_size":9258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37900771,"num_objects":9254,"num_object_clones":0,"num_object_copies":18508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9254,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7153,"num_write_kb":28612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":5177}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9344","reported_seq":9462,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:02.185378+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:29:02.185378+0000","last_peered":"2024-09-17T15:29:02.185378+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:29:02.185378+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":9344,"log_dups_size":0,"ondisk_log_size":9344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38248448,"num_objects":9338,"num_object_clones":0,"num_object_copies":18676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7106,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":5201}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9428","reported_seq":9561,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:01.318563+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:29:01.318563+0000","last_peered":"2024-09-17T15:29:01.318563+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:29:01.318563+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":9428,"log_dups_size":0,"ondisk_log_size":9428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38608896,"num_objects":9426,"num_object_clones":0,"num_object_copies":18852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5976,"num_write_kb":23904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5288},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9381","reported_seq":9522,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:01.185186+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:29:01.185186+0000","last_peered":"2024-09-17T15:29:01.185186+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:29:01.185186+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":9381,"log_dups_size":0,"ondisk_log_size":9381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38391808,"num_objects":9373,"num_object_clones":0,"num_object_copies":18746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5668,"num_write_kb":22672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5216},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805074+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805074+0000","last_peered":"2024-09-17T15:28:23.805074+0000","last_clean":"2024-09-17T15:28:23.805074+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805074+0000","last_undegraded":"2024-09-17T15:28:23.805074+0000","last_fullsized":"2024-09-17T15:28:23.805074+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9394","reported_seq":9525,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:01.206351+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:29:01.206351+0000","last_peered":"2024-09-17T15:29:01.206351+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:29:01.206351+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":9394,"log_dups_size":0,"ondisk_log_size":9394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38461440,"num_objects":9390,"num_object_clones":0,"num_object_copies":18780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6181,"num_write_kb":24724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5327},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9241","reported_seq":11016,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:29:01.311971+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:29:01.311971+0000","last_peered":"2024-09-17T15:29:01.311971+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:29:01.311971+0000","last_undegraded":"2024-09-17T15:29:01.311971+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":9241,"log_dups_size":0,"ondisk_log_size":9241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37802950,"num_objects":9231,"num_object_clones":0,"num_object_copies":18462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3293,"num_objects_unfound":0,"num_objects_dirty":9231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5683,"num_write_kb":22732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":816,"num_bytes_recovered":3342336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5938},{"shards":"0,2","objects":3293}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":613401310,"num_objects":149765,"num_object_clones":0,"num_object_copies":299530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3293,"num_objects_unfound":0,"num_objects_dirty":149765,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":109872,"num_write_kb":439488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4888,"num_bytes_recovered":20013539,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1245822976,"data_stored":1245671230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149837,"ondisk_log_size":149837,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051632,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":693492,"kb_used_data":431332,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93678348,"statfs":{"total":96636764160,"available":95926628352,"internally_reserved":0,"allocated":441683968,"data_stored":441407394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":268431020},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,0,3,1,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724577,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":489276,"kb_used_data":268780,"kb_used_omap":4,"kb_used_meta":220475,"kb_avail":93882564,"statfs":{"total":96636764160,"available":96135745536,"internally_reserved":0,"allocated":275230720,"data_stored":274906713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":225767083},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,7,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247860,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":799740,"kb_used_data":515436,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93572100,"statfs":{"total":96636764160,"available":95817830400,"internally_reserved":0,"allocated":527806464,"data_stored":527525813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":291106472},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,0,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":527781888,"data_stored":527745758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":275664896,"data_stored":275585410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":442376192,"data_stored":442340062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:29:04.238 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:29:04.325 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 205 20 151045 151025 2.87717 1.76953 0.0300509 0.0271461 2024-09-17T15:29:04.795 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:29:04.796 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:29:04.819 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":176,"stamp":"2024-09-17T15:29:03.644623+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":613401329,"num_objects":149767,"num_object_clones":0,"num_object_copies":299534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3293,"num_objects_unfound":0,"num_objects_dirty":149767,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":109874,"num_write_kb":439490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4889,"num_bytes_recovered":20013558,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149840,"ondisk_log_size":149840,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1982508,"kb_used_data":1215548,"kb_used_omap":13,"kb_used_meta":766898,"kb_avail":281133012,"statfs":{"total":289910292480,"available":287880204288,"internally_reserved":0,"allocated":1244721152,"data_stored":1243839920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":785304575},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,3,9,8,12,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21549056,"num_objects":5261,"num_object_clones":0,"num_object_copies":10522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-214,"num_objects_unfound":0,"num_objects_dirty":5261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5261,"num_write_kb":21044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":214,"num_bytes_recovered":876544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004661"},"pg_stats":[{"pgid":"2.f","version":"43'9293","reported_seq":9366,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:01.224281+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:29:01.224281+0000","last_peered":"2024-09-17T15:29:01.224281+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:29:01.224281+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9293,"log_dups_size":0,"ondisk_log_size":9293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38064128,"num_objects":9293,"num_object_clones":0,"num_object_copies":18586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9293,"num_write_kb":37172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5223},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9399","reported_seq":9501,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:02.233704+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:29:02.233704+0000","last_peered":"2024-09-17T15:29:02.233704+0000","last_clean":"2024-09-17T15:29:02.233704+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:29:02.233704+0000","last_undegraded":"2024-09-17T15:29:02.233704+0000","last_fullsized":"2024-09-17T15:29:02.233704+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":9399,"log_dups_size":0,"ondisk_log_size":9399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38474694,"num_objects":9395,"num_object_clones":0,"num_object_copies":18790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9395,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7756,"num_write_kb":31024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9406","reported_seq":9531,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:02.254469+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:29:02.254469+0000","last_peered":"2024-09-17T15:29:02.254469+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:29:02.254469+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":9406,"log_dups_size":0,"ondisk_log_size":9406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38510592,"num_objects":9402,"num_object_clones":0,"num_object_copies":18804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7563,"num_write_kb":30252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":5234}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9502","reported_seq":9571,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:02.162722+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:29:02.162722+0000","last_peered":"2024-09-17T15:29:02.162722+0000","last_clean":"2024-09-17T15:29:02.162722+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:29:02.162722+0000","last_undegraded":"2024-09-17T15:29:02.162722+0000","last_fullsized":"2024-09-17T15:29:02.162722+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9502,"log_dups_size":0,"ondisk_log_size":9502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38920192,"num_objects":9502,"num_object_clones":0,"num_object_copies":19004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9502,"num_write_kb":38008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9261","reported_seq":9390,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:01.291601+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:29:01.291601+0000","last_peered":"2024-09-17T15:29:01.291601+0000","last_clean":"2024-09-17T15:29:01.291601+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:29:01.291601+0000","last_undegraded":"2024-09-17T15:29:01.291601+0000","last_fullsized":"2024-09-17T15:29:01.291601+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":9261,"log_dups_size":0,"ondisk_log_size":9261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37904867,"num_objects":9255,"num_object_clones":0,"num_object_copies":18510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9255,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6243,"num_write_kb":24972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9562","reported_seq":9696,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:01.154139+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:29:01.154139+0000","last_peered":"2024-09-17T15:29:01.154139+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:29:01.154139+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":9562,"log_dups_size":0,"ondisk_log_size":9562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39145955,"num_objects":9558,"num_object_clones":0,"num_object_copies":19116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9558,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6061,"num_write_kb":24244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5288},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9376","reported_seq":9514,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:01.291975+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:29:01.291975+0000","last_peered":"2024-09-17T15:29:01.291975+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:29:01.291975+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":9376,"log_dups_size":0,"ondisk_log_size":9376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38380486,"num_objects":9372,"num_object_clones":0,"num_object_copies":18744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9372,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6145,"num_write_kb":24580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5198},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9432","reported_seq":9559,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:01.262327+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:29:01.262327+0000","last_peered":"2024-09-17T15:29:01.262327+0000","last_clean":"2024-09-17T15:29:01.262327+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:29:01.262327+0000","last_undegraded":"2024-09-17T15:29:01.262327+0000","last_fullsized":"2024-09-17T15:29:01.262327+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":9432,"log_dups_size":0,"ondisk_log_size":9432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38625280,"num_objects":9430,"num_object_clones":0,"num_object_copies":18860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6386,"num_write_kb":25544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'9328","reported_seq":17610,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:00.580979+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:29:00.580979+0000","last_peered":"2024-09-17T15:29:00.580979+0000","last_clean":"2024-09-17T15:29:00.580979+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:29:00.580979+0000","last_undegraded":"2024-09-17T15:29:00.580979+0000","last_fullsized":"2024-09-17T15:29:00.580979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":9328,"log_dups_size":0,"ondisk_log_size":9328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38179299,"num_objects":9322,"num_object_clones":0,"num_object_copies":18644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9322,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7328,"num_write_kb":29312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804705+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804705+0000","last_peered":"2024-09-17T15:28:23.804705+0000","last_clean":"2024-09-17T15:28:23.804705+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804705+0000","last_undegraded":"2024-09-17T15:28:23.804705+0000","last_fullsized":"2024-09-17T15:28:23.804705+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9232","reported_seq":9362,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:01.224283+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:29:01.224283+0000","last_peered":"2024-09-17T15:29:01.224283+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:29:01.224283+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":9232,"log_dups_size":0,"ondisk_log_size":9232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37781504,"num_objects":9224,"num_object_clones":0,"num_object_copies":18448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5828,"num_write_kb":23312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5214},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":102,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814215+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814215+0000","last_peered":"2024-09-17T15:28:23.814215+0000","last_clean":"2024-09-17T15:28:23.814215+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814215+0000","last_undegraded":"2024-09-17T15:28:23.814215+0000","last_fullsized":"2024-09-17T15:28:23.814215+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9258","reported_seq":9378,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:02.244111+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:29:02.244111+0000","last_peered":"2024-09-17T15:29:02.244111+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:29:02.244111+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":9258,"log_dups_size":0,"ondisk_log_size":9258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37900771,"num_objects":9254,"num_object_clones":0,"num_object_copies":18508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9254,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7153,"num_write_kb":28612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":5177}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9344","reported_seq":9462,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:02.185378+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:29:02.185378+0000","last_peered":"2024-09-17T15:29:02.185378+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:29:02.185378+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":9344,"log_dups_size":0,"ondisk_log_size":9344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38248448,"num_objects":9338,"num_object_clones":0,"num_object_copies":18676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7106,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":5201}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9428","reported_seq":9561,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:01.318563+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:29:01.318563+0000","last_peered":"2024-09-17T15:29:01.318563+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:29:01.318563+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":9428,"log_dups_size":0,"ondisk_log_size":9428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38608896,"num_objects":9426,"num_object_clones":0,"num_object_copies":18852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5976,"num_write_kb":23904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5288},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9381","reported_seq":9522,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:01.185186+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:29:01.185186+0000","last_peered":"2024-09-17T15:29:01.185186+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:29:01.185186+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":9381,"log_dups_size":0,"ondisk_log_size":9381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38391808,"num_objects":9373,"num_object_clones":0,"num_object_copies":18746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5668,"num_write_kb":22672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5216},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805074+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805074+0000","last_peered":"2024-09-17T15:28:23.805074+0000","last_clean":"2024-09-17T15:28:23.805074+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805074+0000","last_undegraded":"2024-09-17T15:28:23.805074+0000","last_fullsized":"2024-09-17T15:28:23.805074+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9394","reported_seq":9525,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:01.206351+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:29:01.206351+0000","last_peered":"2024-09-17T15:29:01.206351+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:29:01.206351+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":9394,"log_dups_size":0,"ondisk_log_size":9394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38461440,"num_objects":9390,"num_object_clones":0,"num_object_copies":18780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6181,"num_write_kb":24724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5327},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9241","reported_seq":11016,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:29:01.311971+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:29:01.311971+0000","last_peered":"2024-09-17T15:29:01.311971+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:29:01.311971+0000","last_undegraded":"2024-09-17T15:29:01.311971+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":9241,"log_dups_size":0,"ondisk_log_size":9241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37802950,"num_objects":9231,"num_object_clones":0,"num_object_copies":18462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3293,"num_objects_unfound":0,"num_objects_dirty":9231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5683,"num_write_kb":22732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":816,"num_bytes_recovered":3342336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5938},{"shards":"0,2","objects":3293}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":613401310,"num_objects":149765,"num_object_clones":0,"num_object_copies":299530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3293,"num_objects_unfound":0,"num_objects_dirty":149765,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":109872,"num_write_kb":439488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4888,"num_bytes_recovered":20013539,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1245822976,"data_stored":1245671230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149837,"ondisk_log_size":149837,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051632,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":693492,"kb_used_data":431332,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93678348,"statfs":{"total":96636764160,"available":95926628352,"internally_reserved":0,"allocated":441683968,"data_stored":441407394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":268431020},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,0,3,1,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724577,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":489276,"kb_used_data":268780,"kb_used_omap":4,"kb_used_meta":220475,"kb_avail":93882564,"statfs":{"total":96636764160,"available":96135745536,"internally_reserved":0,"allocated":275230720,"data_stored":274906713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":225767083},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,7,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247860,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":799740,"kb_used_data":515436,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93572100,"statfs":{"total":96636764160,"available":95817830400,"internally_reserved":0,"allocated":527806464,"data_stored":527525813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":291106472},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,0,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":527781888,"data_stored":527745758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":275664896,"data_stored":275585410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":442376192,"data_stored":442340062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:29:04.820 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:29:04.820 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'9293", 'reported_seq': 9366, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:01.224281+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:29:01.224281+0000', 'last_peered': '2024-09-17T15:29:01.224281+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:29:01.224281+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 9293, 'log_dups_size': 0, 'ondisk_log_size': 9293, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38064128, 'num_objects': 9293, 'num_object_clones': 0, 'num_object_copies': 18586, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4070, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9293, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9293, 'num_write_kb': 37172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5223}, {'shards': '0,2', 'objects': 4070}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:04.821 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:29:04.821 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'9406", 'reported_seq': 9531, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:02.254469+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:29:02.254469+0000', 'last_peered': '2024-09-17T15:29:02.254469+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:29:02.254469+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 9406, 'log_dups_size': 0, 'ondisk_log_size': 9406, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38510592, 'num_objects': 9402, 'num_object_clones': 0, 'num_object_copies': 18804, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4169, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9402, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7563, 'num_write_kb': 30252, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4168}, {'shards': '2', 'objects': 5234}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:29:04.821 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:29:04.821 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'9562", 'reported_seq': 9696, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:01.154139+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:29:01.154139+0000', 'last_peered': '2024-09-17T15:29:01.154139+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:29:01.154139+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 9562, 'log_dups_size': 0, 'ondisk_log_size': 9562, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39145955, 'num_objects': 9558, 'num_object_clones': 0, 'num_object_copies': 19116, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4271, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9558, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6061, 'num_write_kb': 24244, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5288}, {'shards': '0,2', 'objects': 4268}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:04.821 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:29:04.821 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'9376", 'reported_seq': 9514, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:01.291975+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:29:01.291975+0000', 'last_peered': '2024-09-17T15:29:01.291975+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:29:01.291975+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 9376, 'log_dups_size': 0, 'ondisk_log_size': 9376, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38380486, 'num_objects': 9372, 'num_object_clones': 0, 'num_object_copies': 18744, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4174, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9372, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 6145, 'num_write_kb': 24580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5198}, {'shards': '0,2', 'objects': 4174}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:04.821 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:29:04.821 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'9232", 'reported_seq': 9362, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:01.224283+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:29:01.224283+0000', 'last_peered': '2024-09-17T15:29:01.224283+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:29:01.224283+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 9232, 'log_dups_size': 0, 'ondisk_log_size': 9232, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37781504, 'num_objects': 9224, 'num_object_clones': 0, 'num_object_copies': 18448, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4012, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9224, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5828, 'num_write_kb': 23312, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5214}, {'shards': '0,2', 'objects': 4010}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:04.821 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:29:04.821 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'9258", 'reported_seq': 9378, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:02.244111+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:29:02.244111+0000', 'last_peered': '2024-09-17T15:29:02.244111+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:29:02.244111+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 9258, 'log_dups_size': 0, 'ondisk_log_size': 9258, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37900771, 'num_objects': 9254, 'num_object_clones': 0, 'num_object_copies': 18508, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4078, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9254, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7153, 'num_write_kb': 28612, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4077}, {'shards': '2', 'objects': 5177}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:29:04.822 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:29:04.822 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'9344", 'reported_seq': 9462, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:02.185378+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:29:02.185378+0000', 'last_peered': '2024-09-17T15:29:02.185378+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:29:02.185378+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 9344, 'log_dups_size': 0, 'ondisk_log_size': 9344, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38248448, 'num_objects': 9338, 'num_object_clones': 0, 'num_object_copies': 18676, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9338, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7106, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4137}, {'shards': '2', 'objects': 5201}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:29:04.822 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:29:04.822 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'9428", 'reported_seq': 9561, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:01.318563+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:29:01.318563+0000', 'last_peered': '2024-09-17T15:29:01.318563+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:29:01.318563+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 9428, 'log_dups_size': 0, 'ondisk_log_size': 9428, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38608896, 'num_objects': 9426, 'num_object_clones': 0, 'num_object_copies': 18852, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9426, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5976, 'num_write_kb': 23904, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5288}, {'shards': '0,2', 'objects': 4138}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:04.822 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:29:04.822 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'9381", 'reported_seq': 9522, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:01.185186+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:29:01.185186+0000', 'last_peered': '2024-09-17T15:29:01.185186+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:29:01.185186+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 9381, 'log_dups_size': 0, 'ondisk_log_size': 9381, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38391808, 'num_objects': 9373, 'num_object_clones': 0, 'num_object_copies': 18746, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4160, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9373, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5668, 'num_write_kb': 22672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5216}, {'shards': '0,2', 'objects': 4157}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:04.822 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:29:04.822 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'9394", 'reported_seq': 9525, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:01.206351+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:29:01.206351+0000', 'last_peered': '2024-09-17T15:29:01.206351+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:29:01.206351+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 9394, 'log_dups_size': 0, 'ondisk_log_size': 9394, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38461440, 'num_objects': 9390, 'num_object_clones': 0, 'num_object_copies': 18780, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4065, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9390, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6181, 'num_write_kb': 24724, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5327}, {'shards': '0,2', 'objects': 4063}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:04.822 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:29:04.822 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'9241", 'reported_seq': 11016, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:29:01.311971+0000', 'last_change': '2024-09-17T15:28:22.509818+0000', 'last_active': '2024-09-17T15:29:01.311971+0000', 'last_peered': '2024-09-17T15:29:01.311971+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:29:01.311971+0000', 'last_undegraded': '2024-09-17T15:29:01.311971+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 9241, 'log_dups_size': 0, 'ondisk_log_size': 9241, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37802950, 'num_objects': 9231, 'num_object_clones': 0, 'num_object_copies': 18462, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3293, 'num_objects_unfound': 0, 'num_objects_dirty': 9231, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5683, 'num_write_kb': 22732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 816, 'num_bytes_recovered': 3342336, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5938}, {'shards': '0,2', 'objects': 3293}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:04.823 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:29:05.326 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 206 20 151461 151441 2.87109 1.625 0.0556501 0.0272026 2024-09-17T15:29:05.493 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:29:05.493 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41277/299534 objects degraded (13.780%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":282,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":149767,"data_bytes":613401329,"bytes_used":2030088192,"bytes_avail":287880204288,"bytes_total":289910292480,"degraded_objects":41277,"degraded_total":299534,"degraded_ratio":0.1378040556330834,"misplaced_objects":3293,"misplaced_total":299534,"misplaced_ratio":0.010993743615082095,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":73016,"recovering_keys_per_sec":0,"num_objects_recovered":214,"num_bytes_recovered":876544,"num_keys_recovered":0,"write_bytes_sec":1795057,"read_op_per_sec":0,"write_op_per_sec":438},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T15:27:19.616376+0000","services":{}},"progress_events":{}} 2024-09-17T15:29:05.517 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:29:05.517 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:29:06.071 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:29:06.072 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:29:06.093 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":177,"stamp":"2024-09-17T15:29:05.645256+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":613401329,"num_objects":149767,"num_object_clones":0,"num_object_copies":299534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3293,"num_objects_unfound":0,"num_objects_dirty":149767,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":109874,"num_write_kb":439490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4889,"num_bytes_recovered":20013558,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149840,"ondisk_log_size":149840,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1982508,"kb_used_data":1215548,"kb_used_omap":13,"kb_used_meta":766898,"kb_avail":281133012,"statfs":{"total":289910292480,"available":287880204288,"internally_reserved":0,"allocated":1244721152,"data_stored":1243839920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":785304575},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,3,9,8,12,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18477056,"num_objects":4511,"num_object_clones":0,"num_object_copies":9022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-214,"num_objects_unfound":0,"num_objects_dirty":4511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4511,"num_write_kb":18044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":214,"num_bytes_recovered":876544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004774"},"pg_stats":[{"pgid":"2.f","version":"43'9293","reported_seq":9366,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:01.224281+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:29:01.224281+0000","last_peered":"2024-09-17T15:29:01.224281+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:29:01.224281+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9293,"log_dups_size":0,"ondisk_log_size":9293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38064128,"num_objects":9293,"num_object_clones":0,"num_object_copies":18586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9293,"num_write_kb":37172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5223},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9399","reported_seq":9501,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:02.233704+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:29:02.233704+0000","last_peered":"2024-09-17T15:29:02.233704+0000","last_clean":"2024-09-17T15:29:02.233704+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:29:02.233704+0000","last_undegraded":"2024-09-17T15:29:02.233704+0000","last_fullsized":"2024-09-17T15:29:02.233704+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":9399,"log_dups_size":0,"ondisk_log_size":9399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38474694,"num_objects":9395,"num_object_clones":0,"num_object_copies":18790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9395,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7756,"num_write_kb":31024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9406","reported_seq":9531,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:02.254469+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:29:02.254469+0000","last_peered":"2024-09-17T15:29:02.254469+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:29:02.254469+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":9406,"log_dups_size":0,"ondisk_log_size":9406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38510592,"num_objects":9402,"num_object_clones":0,"num_object_copies":18804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7563,"num_write_kb":30252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":5234}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9502","reported_seq":9571,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:02.162722+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:29:02.162722+0000","last_peered":"2024-09-17T15:29:02.162722+0000","last_clean":"2024-09-17T15:29:02.162722+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:29:02.162722+0000","last_undegraded":"2024-09-17T15:29:02.162722+0000","last_fullsized":"2024-09-17T15:29:02.162722+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9502,"log_dups_size":0,"ondisk_log_size":9502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38920192,"num_objects":9502,"num_object_clones":0,"num_object_copies":19004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9502,"num_write_kb":38008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9261","reported_seq":9390,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:01.291601+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:29:01.291601+0000","last_peered":"2024-09-17T15:29:01.291601+0000","last_clean":"2024-09-17T15:29:01.291601+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:29:01.291601+0000","last_undegraded":"2024-09-17T15:29:01.291601+0000","last_fullsized":"2024-09-17T15:29:01.291601+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":9261,"log_dups_size":0,"ondisk_log_size":9261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37904867,"num_objects":9255,"num_object_clones":0,"num_object_copies":18510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9255,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6243,"num_write_kb":24972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9562","reported_seq":9696,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:01.154139+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:29:01.154139+0000","last_peered":"2024-09-17T15:29:01.154139+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:29:01.154139+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":9562,"log_dups_size":0,"ondisk_log_size":9562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39145955,"num_objects":9558,"num_object_clones":0,"num_object_copies":19116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9558,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6061,"num_write_kb":24244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5288},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9376","reported_seq":9514,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:01.291975+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:29:01.291975+0000","last_peered":"2024-09-17T15:29:01.291975+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:29:01.291975+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":9376,"log_dups_size":0,"ondisk_log_size":9376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38380486,"num_objects":9372,"num_object_clones":0,"num_object_copies":18744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9372,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6145,"num_write_kb":24580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5198},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9432","reported_seq":9559,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:01.262327+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:29:01.262327+0000","last_peered":"2024-09-17T15:29:01.262327+0000","last_clean":"2024-09-17T15:29:01.262327+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:29:01.262327+0000","last_undegraded":"2024-09-17T15:29:01.262327+0000","last_fullsized":"2024-09-17T15:29:01.262327+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":9432,"log_dups_size":0,"ondisk_log_size":9432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38625280,"num_objects":9430,"num_object_clones":0,"num_object_copies":18860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6386,"num_write_kb":25544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'9328","reported_seq":17610,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:00.580979+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:29:00.580979+0000","last_peered":"2024-09-17T15:29:00.580979+0000","last_clean":"2024-09-17T15:29:00.580979+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:29:00.580979+0000","last_undegraded":"2024-09-17T15:29:00.580979+0000","last_fullsized":"2024-09-17T15:29:00.580979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":9328,"log_dups_size":0,"ondisk_log_size":9328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38179299,"num_objects":9322,"num_object_clones":0,"num_object_copies":18644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9322,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7328,"num_write_kb":29312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804705+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804705+0000","last_peered":"2024-09-17T15:28:23.804705+0000","last_clean":"2024-09-17T15:28:23.804705+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804705+0000","last_undegraded":"2024-09-17T15:28:23.804705+0000","last_fullsized":"2024-09-17T15:28:23.804705+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9232","reported_seq":9362,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:01.224283+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:29:01.224283+0000","last_peered":"2024-09-17T15:29:01.224283+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:29:01.224283+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":9232,"log_dups_size":0,"ondisk_log_size":9232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37781504,"num_objects":9224,"num_object_clones":0,"num_object_copies":18448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5828,"num_write_kb":23312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5214},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":102,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814215+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814215+0000","last_peered":"2024-09-17T15:28:23.814215+0000","last_clean":"2024-09-17T15:28:23.814215+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814215+0000","last_undegraded":"2024-09-17T15:28:23.814215+0000","last_fullsized":"2024-09-17T15:28:23.814215+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9258","reported_seq":9378,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:02.244111+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:29:02.244111+0000","last_peered":"2024-09-17T15:29:02.244111+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:29:02.244111+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":9258,"log_dups_size":0,"ondisk_log_size":9258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37900771,"num_objects":9254,"num_object_clones":0,"num_object_copies":18508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9254,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7153,"num_write_kb":28612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":5177}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9344","reported_seq":9462,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:02.185378+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:29:02.185378+0000","last_peered":"2024-09-17T15:29:02.185378+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:29:02.185378+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":9344,"log_dups_size":0,"ondisk_log_size":9344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38248448,"num_objects":9338,"num_object_clones":0,"num_object_copies":18676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7106,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":5201}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9428","reported_seq":9561,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:01.318563+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:29:01.318563+0000","last_peered":"2024-09-17T15:29:01.318563+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:29:01.318563+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":9428,"log_dups_size":0,"ondisk_log_size":9428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38608896,"num_objects":9426,"num_object_clones":0,"num_object_copies":18852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5976,"num_write_kb":23904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5288},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9381","reported_seq":9522,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:01.185186+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:29:01.185186+0000","last_peered":"2024-09-17T15:29:01.185186+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:29:01.185186+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":9381,"log_dups_size":0,"ondisk_log_size":9381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38391808,"num_objects":9373,"num_object_clones":0,"num_object_copies":18746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5668,"num_write_kb":22672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5216},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805074+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805074+0000","last_peered":"2024-09-17T15:28:23.805074+0000","last_clean":"2024-09-17T15:28:23.805074+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805074+0000","last_undegraded":"2024-09-17T15:28:23.805074+0000","last_fullsized":"2024-09-17T15:28:23.805074+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9394","reported_seq":9525,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:01.206351+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:29:01.206351+0000","last_peered":"2024-09-17T15:29:01.206351+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:29:01.206351+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":9394,"log_dups_size":0,"ondisk_log_size":9394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38461440,"num_objects":9390,"num_object_clones":0,"num_object_copies":18780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6181,"num_write_kb":24724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5327},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9241","reported_seq":11016,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:29:01.311971+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:29:01.311971+0000","last_peered":"2024-09-17T15:29:01.311971+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:29:01.311971+0000","last_undegraded":"2024-09-17T15:29:01.311971+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":9241,"log_dups_size":0,"ondisk_log_size":9241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37802950,"num_objects":9231,"num_object_clones":0,"num_object_copies":18462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3293,"num_objects_unfound":0,"num_objects_dirty":9231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5683,"num_write_kb":22732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":816,"num_bytes_recovered":3342336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5938},{"shards":"0,2","objects":3293}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":613401310,"num_objects":149765,"num_object_clones":0,"num_object_copies":299530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3293,"num_objects_unfound":0,"num_objects_dirty":149765,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":109872,"num_write_kb":439488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4888,"num_bytes_recovered":20013539,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1245822976,"data_stored":1245671230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149837,"ondisk_log_size":149837,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051632,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":693492,"kb_used_data":431332,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93678348,"statfs":{"total":96636764160,"available":95926628352,"internally_reserved":0,"allocated":441683968,"data_stored":441407394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":268431020},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,0,3,1,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724577,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":489276,"kb_used_data":268780,"kb_used_omap":4,"kb_used_meta":220475,"kb_avail":93882564,"statfs":{"total":96636764160,"available":96135745536,"internally_reserved":0,"allocated":275230720,"data_stored":274906713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":225767083},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,7,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247860,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":799740,"kb_used_data":515436,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93572100,"statfs":{"total":96636764160,"available":95817830400,"internally_reserved":0,"allocated":527806464,"data_stored":527525813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":291106472},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,0,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":527781888,"data_stored":527745758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":275664896,"data_stored":275585410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":442376192,"data_stored":442340062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:29:06.326 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 207 20 151904 151884 2.86558 1.73047 0.0362135 0.0272573 2024-09-17T15:29:07.326 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 208 20 152387 152367 2.86087 1.88672 0.0658564 0.0273016 2024-09-17T15:29:08.328 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 209 20 152781 152761 2.85455 1.53906 0.0442939 0.0273603 2024-09-17T15:29:09.095 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:29:09.326 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 210 20 153249 153229 2.84966 1.82812 0.0719778 0.0274077 2024-09-17T15:29:09.651 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:29:09.651 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:29:09.674 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":179,"stamp":"2024-09-17T15:29:09.646568+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":622723825,"num_objects":152043,"num_object_clones":0,"num_object_copies":304086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3201,"num_objects_unfound":0,"num_objects_dirty":152043,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":112150,"num_write_kb":448594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4981,"num_bytes_recovered":20390390,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152116,"ondisk_log_size":152116,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2001096,"kb_used_data":1234136,"kb_used_omap":13,"kb_used_meta":766898,"kb_avail":281114424,"statfs":{"total":289910292480,"available":287861170176,"internally_reserved":0,"allocated":1263755264,"data_stored":1262874032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":785304575},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,4,1,9,1,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18391040,"num_objects":4490,"num_object_clones":0,"num_object_copies":8980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-205,"num_objects_unfound":0,"num_objects_dirty":4490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4490,"num_write_kb":17960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":205,"num_bytes_recovered":839680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005147"},"pg_stats":[{"pgid":"2.f","version":"43'9426","reported_seq":9502,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:06.375941+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:29:06.375941+0000","last_peered":"2024-09-17T15:29:06.375941+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:29:06.375941+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9426,"log_dups_size":0,"ondisk_log_size":9426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38608896,"num_objects":9426,"num_object_clones":0,"num_object_copies":18852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9426,"num_write_kb":37704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5356},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9544","reported_seq":9646,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:07.241201+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:29:07.241201+0000","last_peered":"2024-09-17T15:29:07.241201+0000","last_clean":"2024-09-17T15:29:07.241201+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:29:07.241201+0000","last_undegraded":"2024-09-17T15:29:07.241201+0000","last_fullsized":"2024-09-17T15:29:07.241201+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":9544,"log_dups_size":0,"ondisk_log_size":9544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39068614,"num_objects":9540,"num_object_clones":0,"num_object_copies":19080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9540,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7901,"num_write_kb":31604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9547","reported_seq":9672,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:07.189893+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:29:07.189893+0000","last_peered":"2024-09-17T15:29:07.189893+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:29:07.189893+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":9547,"log_dups_size":0,"ondisk_log_size":9547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39088128,"num_objects":9543,"num_object_clones":0,"num_object_copies":19086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7704,"num_write_kb":30816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":5375}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9618","reported_seq":9687,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:07.208219+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:29:07.208219+0000","last_peered":"2024-09-17T15:29:07.208219+0000","last_clean":"2024-09-17T15:29:07.208219+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:29:07.208219+0000","last_undegraded":"2024-09-17T15:29:07.208219+0000","last_fullsized":"2024-09-17T15:29:07.208219+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9618,"log_dups_size":0,"ondisk_log_size":9618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39395328,"num_objects":9618,"num_object_clones":0,"num_object_copies":19236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9618,"num_write_kb":38472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9415","reported_seq":9544,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:06.328575+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:29:06.328575+0000","last_peered":"2024-09-17T15:29:06.328575+0000","last_clean":"2024-09-17T15:29:06.328575+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:29:06.328575+0000","last_undegraded":"2024-09-17T15:29:06.328575+0000","last_fullsized":"2024-09-17T15:29:06.328575+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":9415,"log_dups_size":0,"ondisk_log_size":9415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38535651,"num_objects":9409,"num_object_clones":0,"num_object_copies":18818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9409,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6397,"num_write_kb":25588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9703","reported_seq":9837,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:06.304777+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:29:06.304777+0000","last_peered":"2024-09-17T15:29:06.304777+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:29:06.304777+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":9703,"log_dups_size":0,"ondisk_log_size":9703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39723491,"num_objects":9699,"num_object_clones":0,"num_object_copies":19398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9699,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6202,"num_write_kb":24808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5429},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9508","reported_seq":9646,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:06.368635+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:29:06.368635+0000","last_peered":"2024-09-17T15:29:06.368635+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:29:06.368635+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":9508,"log_dups_size":0,"ondisk_log_size":9508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38921158,"num_objects":9504,"num_object_clones":0,"num_object_copies":19008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9504,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6277,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5330},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9570","reported_seq":9697,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:06.404572+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:29:06.404572+0000","last_peered":"2024-09-17T15:29:06.404572+0000","last_clean":"2024-09-17T15:29:06.404572+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:29:06.404572+0000","last_undegraded":"2024-09-17T15:29:06.404572+0000","last_fullsized":"2024-09-17T15:29:06.404572+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":9570,"log_dups_size":0,"ondisk_log_size":9570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39190528,"num_objects":9568,"num_object_clones":0,"num_object_copies":19136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6524,"num_write_kb":26096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'9484","reported_seq":17766,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:05.542142+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:29:05.542142+0000","last_peered":"2024-09-17T15:29:05.542142+0000","last_clean":"2024-09-17T15:29:05.542142+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:29:05.542142+0000","last_undegraded":"2024-09-17T15:29:05.542142+0000","last_fullsized":"2024-09-17T15:29:05.542142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":9484,"log_dups_size":0,"ondisk_log_size":9484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38818275,"num_objects":9478,"num_object_clones":0,"num_object_copies":18956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9478,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7484,"num_write_kb":29936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804705+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804705+0000","last_peered":"2024-09-17T15:28:23.804705+0000","last_clean":"2024-09-17T15:28:23.804705+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804705+0000","last_undegraded":"2024-09-17T15:28:23.804705+0000","last_fullsized":"2024-09-17T15:28:23.804705+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9363","reported_seq":9493,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:06.376444+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:29:06.376444+0000","last_peered":"2024-09-17T15:29:06.376444+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:29:06.376444+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":9363,"log_dups_size":0,"ondisk_log_size":9363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38318080,"num_objects":9355,"num_object_clones":0,"num_object_copies":18710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9355,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5959,"num_write_kb":23836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5345},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":102,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814215+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814215+0000","last_peered":"2024-09-17T15:28:23.814215+0000","last_clean":"2024-09-17T15:28:23.814215+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814215+0000","last_undegraded":"2024-09-17T15:28:23.814215+0000","last_fullsized":"2024-09-17T15:28:23.814215+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9401","reported_seq":9521,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:07.210287+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:29:07.210287+0000","last_peered":"2024-09-17T15:29:07.210287+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:29:07.210287+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":9401,"log_dups_size":0,"ondisk_log_size":9401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38486499,"num_objects":9397,"num_object_clones":0,"num_object_copies":18794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9397,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7296,"num_write_kb":29184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":5320}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9489","reported_seq":9607,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:07.247500+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:29:07.247500+0000","last_peered":"2024-09-17T15:29:07.247500+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:29:07.247500+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":9489,"log_dups_size":0,"ondisk_log_size":9489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38842368,"num_objects":9483,"num_object_clones":0,"num_object_copies":18966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7251,"num_write_kb":29004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":5346}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9581","reported_seq":9716,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:06.331829+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:29:06.331829+0000","last_peered":"2024-09-17T15:29:06.331829+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:29:06.331829+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":9581,"log_dups_size":0,"ondisk_log_size":9581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39235584,"num_objects":9579,"num_object_clones":0,"num_object_copies":19158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6129,"num_write_kb":24516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5441},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9512","reported_seq":9653,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:06.337380+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:29:06.337380+0000","last_peered":"2024-09-17T15:29:06.337380+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:29:06.337380+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":9512,"log_dups_size":0,"ondisk_log_size":9512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38928384,"num_objects":9504,"num_object_clones":0,"num_object_copies":19008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9504,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5799,"num_write_kb":23196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5347},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805074+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805074+0000","last_peered":"2024-09-17T15:28:23.805074+0000","last_clean":"2024-09-17T15:28:23.805074+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805074+0000","last_undegraded":"2024-09-17T15:28:23.805074+0000","last_fullsized":"2024-09-17T15:28:23.805074+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9553","reported_seq":9684,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:06.362393+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:29:06.362393+0000","last_peered":"2024-09-17T15:29:06.362393+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:29:06.362393+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":9553,"log_dups_size":0,"ondisk_log_size":9553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39112704,"num_objects":9549,"num_object_clones":0,"num_object_copies":19098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6340,"num_write_kb":25360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5486},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9399","reported_seq":11358,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:29:06.404262+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:29:06.404262+0000","last_peered":"2024-09-17T15:29:06.404262+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:29:06.404262+0000","last_undegraded":"2024-09-17T15:29:06.404262+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":9399,"log_dups_size":0,"ondisk_log_size":9399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38450118,"num_objects":9389,"num_object_clones":0,"num_object_copies":18778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3201,"num_objects_unfound":0,"num_objects_dirty":9389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5841,"num_write_kb":23364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":908,"num_bytes_recovered":3719168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6188},{"shards":"0,2","objects":3201}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":622723806,"num_objects":152041,"num_object_clones":0,"num_object_copies":304082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3201,"num_objects_unfound":0,"num_objects_dirty":152041,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":112148,"num_write_kb":448592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4980,"num_bytes_recovered":20390371,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1265020928,"data_stored":1264869182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152113,"ondisk_log_size":152113,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051633,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":702336,"kb_used_data":436016,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93669504,"statfs":{"total":96636764160,"available":95917572096,"internally_reserved":0,"allocated":446480384,"data_stored":446203810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":272690860},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724578,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488276,"kb_used_data":276100,"kb_used_omap":4,"kb_used_meta":212155,"kb_avail":93883564,"statfs":{"total":96636764160,"available":96136769536,"internally_reserved":0,"allocated":282726400,"data_stored":282402393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":217247403},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,0,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247861,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":810484,"kb_used_data":522020,"kb_used_omap":4,"kb_used_meta":288443,"kb_avail":93561356,"statfs":{"total":96636764160,"available":95806828544,"internally_reserved":0,"allocated":534548480,"data_stored":534267829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":295366312},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,5,0,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":534786048,"data_stored":534749918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":283209728,"data_stored":283130242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":447025152,"data_stored":446989022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:29:09.676 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:29:10.238 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:29:10.239 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:29:10.260 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":179,"stamp":"2024-09-17T15:29:09.646568+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":622723825,"num_objects":152043,"num_object_clones":0,"num_object_copies":304086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3201,"num_objects_unfound":0,"num_objects_dirty":152043,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":112150,"num_write_kb":448594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4981,"num_bytes_recovered":20390390,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152116,"ondisk_log_size":152116,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2001096,"kb_used_data":1234136,"kb_used_omap":13,"kb_used_meta":766898,"kb_avail":281114424,"statfs":{"total":289910292480,"available":287861170176,"internally_reserved":0,"allocated":1263755264,"data_stored":1262874032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":785304575},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,4,1,9,1,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18391040,"num_objects":4490,"num_object_clones":0,"num_object_copies":8980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-205,"num_objects_unfound":0,"num_objects_dirty":4490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4490,"num_write_kb":17960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":205,"num_bytes_recovered":839680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005147"},"pg_stats":[{"pgid":"2.f","version":"43'9426","reported_seq":9502,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:06.375941+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:29:06.375941+0000","last_peered":"2024-09-17T15:29:06.375941+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:29:06.375941+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9426,"log_dups_size":0,"ondisk_log_size":9426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38608896,"num_objects":9426,"num_object_clones":0,"num_object_copies":18852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9426,"num_write_kb":37704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5356},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9544","reported_seq":9646,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:07.241201+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:29:07.241201+0000","last_peered":"2024-09-17T15:29:07.241201+0000","last_clean":"2024-09-17T15:29:07.241201+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:29:07.241201+0000","last_undegraded":"2024-09-17T15:29:07.241201+0000","last_fullsized":"2024-09-17T15:29:07.241201+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":9544,"log_dups_size":0,"ondisk_log_size":9544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39068614,"num_objects":9540,"num_object_clones":0,"num_object_copies":19080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9540,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7901,"num_write_kb":31604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9547","reported_seq":9672,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:07.189893+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:29:07.189893+0000","last_peered":"2024-09-17T15:29:07.189893+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:29:07.189893+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":9547,"log_dups_size":0,"ondisk_log_size":9547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39088128,"num_objects":9543,"num_object_clones":0,"num_object_copies":19086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7704,"num_write_kb":30816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":5375}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9618","reported_seq":9687,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:07.208219+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:29:07.208219+0000","last_peered":"2024-09-17T15:29:07.208219+0000","last_clean":"2024-09-17T15:29:07.208219+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:29:07.208219+0000","last_undegraded":"2024-09-17T15:29:07.208219+0000","last_fullsized":"2024-09-17T15:29:07.208219+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9618,"log_dups_size":0,"ondisk_log_size":9618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39395328,"num_objects":9618,"num_object_clones":0,"num_object_copies":19236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9618,"num_write_kb":38472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9415","reported_seq":9544,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:06.328575+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:29:06.328575+0000","last_peered":"2024-09-17T15:29:06.328575+0000","last_clean":"2024-09-17T15:29:06.328575+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:29:06.328575+0000","last_undegraded":"2024-09-17T15:29:06.328575+0000","last_fullsized":"2024-09-17T15:29:06.328575+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":9415,"log_dups_size":0,"ondisk_log_size":9415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38535651,"num_objects":9409,"num_object_clones":0,"num_object_copies":18818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9409,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6397,"num_write_kb":25588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9703","reported_seq":9837,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:06.304777+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:29:06.304777+0000","last_peered":"2024-09-17T15:29:06.304777+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:29:06.304777+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":9703,"log_dups_size":0,"ondisk_log_size":9703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39723491,"num_objects":9699,"num_object_clones":0,"num_object_copies":19398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9699,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6202,"num_write_kb":24808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5429},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9508","reported_seq":9646,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:06.368635+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:29:06.368635+0000","last_peered":"2024-09-17T15:29:06.368635+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:29:06.368635+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":9508,"log_dups_size":0,"ondisk_log_size":9508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38921158,"num_objects":9504,"num_object_clones":0,"num_object_copies":19008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9504,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6277,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5330},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9570","reported_seq":9697,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:06.404572+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:29:06.404572+0000","last_peered":"2024-09-17T15:29:06.404572+0000","last_clean":"2024-09-17T15:29:06.404572+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:29:06.404572+0000","last_undegraded":"2024-09-17T15:29:06.404572+0000","last_fullsized":"2024-09-17T15:29:06.404572+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":9570,"log_dups_size":0,"ondisk_log_size":9570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39190528,"num_objects":9568,"num_object_clones":0,"num_object_copies":19136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6524,"num_write_kb":26096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'9484","reported_seq":17766,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:05.542142+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:29:05.542142+0000","last_peered":"2024-09-17T15:29:05.542142+0000","last_clean":"2024-09-17T15:29:05.542142+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:29:05.542142+0000","last_undegraded":"2024-09-17T15:29:05.542142+0000","last_fullsized":"2024-09-17T15:29:05.542142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":9484,"log_dups_size":0,"ondisk_log_size":9484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38818275,"num_objects":9478,"num_object_clones":0,"num_object_copies":18956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9478,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7484,"num_write_kb":29936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804705+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804705+0000","last_peered":"2024-09-17T15:28:23.804705+0000","last_clean":"2024-09-17T15:28:23.804705+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804705+0000","last_undegraded":"2024-09-17T15:28:23.804705+0000","last_fullsized":"2024-09-17T15:28:23.804705+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9363","reported_seq":9493,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:06.376444+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:29:06.376444+0000","last_peered":"2024-09-17T15:29:06.376444+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:29:06.376444+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":9363,"log_dups_size":0,"ondisk_log_size":9363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38318080,"num_objects":9355,"num_object_clones":0,"num_object_copies":18710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9355,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5959,"num_write_kb":23836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5345},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":102,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814215+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814215+0000","last_peered":"2024-09-17T15:28:23.814215+0000","last_clean":"2024-09-17T15:28:23.814215+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814215+0000","last_undegraded":"2024-09-17T15:28:23.814215+0000","last_fullsized":"2024-09-17T15:28:23.814215+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9401","reported_seq":9521,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:07.210287+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:29:07.210287+0000","last_peered":"2024-09-17T15:29:07.210287+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:29:07.210287+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":9401,"log_dups_size":0,"ondisk_log_size":9401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38486499,"num_objects":9397,"num_object_clones":0,"num_object_copies":18794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9397,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7296,"num_write_kb":29184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":5320}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9489","reported_seq":9607,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:07.247500+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:29:07.247500+0000","last_peered":"2024-09-17T15:29:07.247500+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:29:07.247500+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":9489,"log_dups_size":0,"ondisk_log_size":9489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38842368,"num_objects":9483,"num_object_clones":0,"num_object_copies":18966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7251,"num_write_kb":29004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":5346}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9581","reported_seq":9716,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:06.331829+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:29:06.331829+0000","last_peered":"2024-09-17T15:29:06.331829+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:29:06.331829+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":9581,"log_dups_size":0,"ondisk_log_size":9581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39235584,"num_objects":9579,"num_object_clones":0,"num_object_copies":19158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6129,"num_write_kb":24516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5441},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9512","reported_seq":9653,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:06.337380+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:29:06.337380+0000","last_peered":"2024-09-17T15:29:06.337380+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:29:06.337380+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":9512,"log_dups_size":0,"ondisk_log_size":9512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38928384,"num_objects":9504,"num_object_clones":0,"num_object_copies":19008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9504,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5799,"num_write_kb":23196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5347},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805074+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805074+0000","last_peered":"2024-09-17T15:28:23.805074+0000","last_clean":"2024-09-17T15:28:23.805074+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805074+0000","last_undegraded":"2024-09-17T15:28:23.805074+0000","last_fullsized":"2024-09-17T15:28:23.805074+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9553","reported_seq":9684,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:06.362393+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:29:06.362393+0000","last_peered":"2024-09-17T15:29:06.362393+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:29:06.362393+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":9553,"log_dups_size":0,"ondisk_log_size":9553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39112704,"num_objects":9549,"num_object_clones":0,"num_object_copies":19098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6340,"num_write_kb":25360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5486},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9399","reported_seq":11358,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:29:06.404262+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:29:06.404262+0000","last_peered":"2024-09-17T15:29:06.404262+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:29:06.404262+0000","last_undegraded":"2024-09-17T15:29:06.404262+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":9399,"log_dups_size":0,"ondisk_log_size":9399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38450118,"num_objects":9389,"num_object_clones":0,"num_object_copies":18778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3201,"num_objects_unfound":0,"num_objects_dirty":9389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5841,"num_write_kb":23364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":908,"num_bytes_recovered":3719168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6188},{"shards":"0,2","objects":3201}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":622723806,"num_objects":152041,"num_object_clones":0,"num_object_copies":304082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3201,"num_objects_unfound":0,"num_objects_dirty":152041,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":112148,"num_write_kb":448592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4980,"num_bytes_recovered":20390371,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1265020928,"data_stored":1264869182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152113,"ondisk_log_size":152113,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051633,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":702336,"kb_used_data":436016,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93669504,"statfs":{"total":96636764160,"available":95917572096,"internally_reserved":0,"allocated":446480384,"data_stored":446203810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":272690860},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724578,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488276,"kb_used_data":276100,"kb_used_omap":4,"kb_used_meta":212155,"kb_avail":93883564,"statfs":{"total":96636764160,"available":96136769536,"internally_reserved":0,"allocated":282726400,"data_stored":282402393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":217247403},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,0,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247861,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":810484,"kb_used_data":522020,"kb_used_omap":4,"kb_used_meta":288443,"kb_avail":93561356,"statfs":{"total":96636764160,"available":95806828544,"internally_reserved":0,"allocated":534548480,"data_stored":534267829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":295366312},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,5,0,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":534786048,"data_stored":534749918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":283209728,"data_stored":283130242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":447025152,"data_stored":446989022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:29:10.261 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:29:10.261 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'9426", 'reported_seq': 9502, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:06.375941+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:29:06.375941+0000', 'last_peered': '2024-09-17T15:29:06.375941+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:29:06.375941+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 9426, 'log_dups_size': 0, 'ondisk_log_size': 9426, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38608896, 'num_objects': 9426, 'num_object_clones': 0, 'num_object_copies': 18852, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4070, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9426, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9426, 'num_write_kb': 37704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5356}, {'shards': '0,2', 'objects': 4070}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:10.261 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:29:10.262 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'9547", 'reported_seq': 9672, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:07.189893+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:29:07.189893+0000', 'last_peered': '2024-09-17T15:29:07.189893+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:29:07.189893+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 9547, 'log_dups_size': 0, 'ondisk_log_size': 9547, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39088128, 'num_objects': 9543, 'num_object_clones': 0, 'num_object_copies': 19086, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4169, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9543, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7704, 'num_write_kb': 30816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4168}, {'shards': '2', 'objects': 5375}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:29:10.262 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:29:10.262 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'9703", 'reported_seq': 9837, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:06.304777+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:29:06.304777+0000', 'last_peered': '2024-09-17T15:29:06.304777+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:29:06.304777+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 9703, 'log_dups_size': 0, 'ondisk_log_size': 9703, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39723491, 'num_objects': 9699, 'num_object_clones': 0, 'num_object_copies': 19398, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4271, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9699, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6202, 'num_write_kb': 24808, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5429}, {'shards': '0,2', 'objects': 4268}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:10.262 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:29:10.262 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'9508", 'reported_seq': 9646, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:06.368635+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:29:06.368635+0000', 'last_peered': '2024-09-17T15:29:06.368635+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:29:06.368635+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 9508, 'log_dups_size': 0, 'ondisk_log_size': 9508, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38921158, 'num_objects': 9504, 'num_object_clones': 0, 'num_object_copies': 19008, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4174, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9504, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 6277, 'num_write_kb': 25108, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5330}, {'shards': '0,2', 'objects': 4174}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:10.262 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:29:10.262 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'9363", 'reported_seq': 9493, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:06.376444+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:29:06.376444+0000', 'last_peered': '2024-09-17T15:29:06.376444+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:29:06.376444+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 9363, 'log_dups_size': 0, 'ondisk_log_size': 9363, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38318080, 'num_objects': 9355, 'num_object_clones': 0, 'num_object_copies': 18710, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4012, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9355, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5959, 'num_write_kb': 23836, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5345}, {'shards': '0,2', 'objects': 4010}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:10.262 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:29:10.262 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'9401", 'reported_seq': 9521, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:07.210287+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:29:07.210287+0000', 'last_peered': '2024-09-17T15:29:07.210287+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:29:07.210287+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 9401, 'log_dups_size': 0, 'ondisk_log_size': 9401, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38486499, 'num_objects': 9397, 'num_object_clones': 0, 'num_object_copies': 18794, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4078, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9397, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7296, 'num_write_kb': 29184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4077}, {'shards': '2', 'objects': 5320}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:29:10.262 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:29:10.263 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'9489", 'reported_seq': 9607, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:07.247500+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:29:07.247500+0000', 'last_peered': '2024-09-17T15:29:07.247500+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:29:07.247500+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 9489, 'log_dups_size': 0, 'ondisk_log_size': 9489, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38842368, 'num_objects': 9483, 'num_object_clones': 0, 'num_object_copies': 18966, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9483, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7251, 'num_write_kb': 29004, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4137}, {'shards': '2', 'objects': 5346}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:29:10.263 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:29:10.263 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'9581", 'reported_seq': 9716, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:06.331829+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:29:06.331829+0000', 'last_peered': '2024-09-17T15:29:06.331829+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:29:06.331829+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 9581, 'log_dups_size': 0, 'ondisk_log_size': 9581, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39235584, 'num_objects': 9579, 'num_object_clones': 0, 'num_object_copies': 19158, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9579, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6129, 'num_write_kb': 24516, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5441}, {'shards': '0,2', 'objects': 4138}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:10.263 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:29:10.263 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'9512", 'reported_seq': 9653, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:06.337380+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:29:06.337380+0000', 'last_peered': '2024-09-17T15:29:06.337380+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:29:06.337380+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 9512, 'log_dups_size': 0, 'ondisk_log_size': 9512, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38928384, 'num_objects': 9504, 'num_object_clones': 0, 'num_object_copies': 19008, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4160, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9504, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5799, 'num_write_kb': 23196, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5347}, {'shards': '0,2', 'objects': 4157}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:10.263 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:29:10.263 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'9553", 'reported_seq': 9684, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:06.362393+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:29:06.362393+0000', 'last_peered': '2024-09-17T15:29:06.362393+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:29:06.362393+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 9553, 'log_dups_size': 0, 'ondisk_log_size': 9553, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39112704, 'num_objects': 9549, 'num_object_clones': 0, 'num_object_copies': 19098, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4065, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9549, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6340, 'num_write_kb': 25360, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5486}, {'shards': '0,2', 'objects': 4063}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:10.263 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:29:10.263 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'9399", 'reported_seq': 11358, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:29:06.404262+0000', 'last_change': '2024-09-17T15:28:22.509818+0000', 'last_active': '2024-09-17T15:29:06.404262+0000', 'last_peered': '2024-09-17T15:29:06.404262+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:29:06.404262+0000', 'last_undegraded': '2024-09-17T15:29:06.404262+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 9399, 'log_dups_size': 0, 'ondisk_log_size': 9399, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38450118, 'num_objects': 9389, 'num_object_clones': 0, 'num_object_copies': 18778, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3201, 'num_objects_unfound': 0, 'num_objects_dirty': 9389, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5841, 'num_write_kb': 23364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 908, 'num_bytes_recovered': 3719168, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6188}, {'shards': '0,2', 'objects': 3201}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:10.264 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:29:10.326 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 211 20 153670 153650 2.84395 1.64453 0.058874 0.0274626 2024-09-17T15:29:10.973 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:29:10.973 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41277/304086 objects degraded (13.574%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":287,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":152043,"data_bytes":622723825,"bytes_used":2049122304,"bytes_avail":287861170176,"bytes_total":289910292480,"degraded_objects":41277,"degraded_total":304086,"degraded_ratio":0.13574120479075,"misplaced_objects":3201,"misplaced_total":304086,"misplaced_ratio":0.010526627335687931,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":69943,"recovering_keys_per_sec":0,"num_objects_recovered":205,"num_bytes_recovered":839680,"num_keys_recovered":0,"write_bytes_sec":1531929,"read_op_per_sec":0,"write_op_per_sec":374},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T15:27:19.616376+0000","services":{}},"progress_events":{}} 2024-09-17T15:29:11.003 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:29:11.003 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:29:11.326 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 212 20 154090 154070 2.83827 1.64062 0.0283048 0.027516 2024-09-17T15:29:11.556 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:29:11.559 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:29:11.578 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":179,"stamp":"2024-09-17T15:29:09.646568+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":622723825,"num_objects":152043,"num_object_clones":0,"num_object_copies":304086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3201,"num_objects_unfound":0,"num_objects_dirty":152043,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":112150,"num_write_kb":448594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4981,"num_bytes_recovered":20390390,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152116,"ondisk_log_size":152116,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2001096,"kb_used_data":1234136,"kb_used_omap":13,"kb_used_meta":766898,"kb_avail":281114424,"statfs":{"total":289910292480,"available":287861170176,"internally_reserved":0,"allocated":1263755264,"data_stored":1262874032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":785304575},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,4,1,9,1,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18391040,"num_objects":4490,"num_object_clones":0,"num_object_copies":8980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-205,"num_objects_unfound":0,"num_objects_dirty":4490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4490,"num_write_kb":17960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":205,"num_bytes_recovered":839680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005147"},"pg_stats":[{"pgid":"2.f","version":"43'9426","reported_seq":9502,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:06.375941+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:29:06.375941+0000","last_peered":"2024-09-17T15:29:06.375941+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:29:06.375941+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9426,"log_dups_size":0,"ondisk_log_size":9426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38608896,"num_objects":9426,"num_object_clones":0,"num_object_copies":18852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9426,"num_write_kb":37704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5356},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9544","reported_seq":9646,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:07.241201+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:29:07.241201+0000","last_peered":"2024-09-17T15:29:07.241201+0000","last_clean":"2024-09-17T15:29:07.241201+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:29:07.241201+0000","last_undegraded":"2024-09-17T15:29:07.241201+0000","last_fullsized":"2024-09-17T15:29:07.241201+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":9544,"log_dups_size":0,"ondisk_log_size":9544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39068614,"num_objects":9540,"num_object_clones":0,"num_object_copies":19080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9540,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7901,"num_write_kb":31604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9547","reported_seq":9672,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:07.189893+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:29:07.189893+0000","last_peered":"2024-09-17T15:29:07.189893+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:29:07.189893+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":9547,"log_dups_size":0,"ondisk_log_size":9547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39088128,"num_objects":9543,"num_object_clones":0,"num_object_copies":19086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7704,"num_write_kb":30816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":5375}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9618","reported_seq":9687,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:07.208219+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:29:07.208219+0000","last_peered":"2024-09-17T15:29:07.208219+0000","last_clean":"2024-09-17T15:29:07.208219+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:29:07.208219+0000","last_undegraded":"2024-09-17T15:29:07.208219+0000","last_fullsized":"2024-09-17T15:29:07.208219+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9618,"log_dups_size":0,"ondisk_log_size":9618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39395328,"num_objects":9618,"num_object_clones":0,"num_object_copies":19236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9618,"num_write_kb":38472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9415","reported_seq":9544,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:06.328575+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:29:06.328575+0000","last_peered":"2024-09-17T15:29:06.328575+0000","last_clean":"2024-09-17T15:29:06.328575+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:29:06.328575+0000","last_undegraded":"2024-09-17T15:29:06.328575+0000","last_fullsized":"2024-09-17T15:29:06.328575+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":9415,"log_dups_size":0,"ondisk_log_size":9415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38535651,"num_objects":9409,"num_object_clones":0,"num_object_copies":18818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9409,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6397,"num_write_kb":25588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9703","reported_seq":9837,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:06.304777+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:29:06.304777+0000","last_peered":"2024-09-17T15:29:06.304777+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:29:06.304777+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":9703,"log_dups_size":0,"ondisk_log_size":9703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39723491,"num_objects":9699,"num_object_clones":0,"num_object_copies":19398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9699,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6202,"num_write_kb":24808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5429},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9508","reported_seq":9646,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:06.368635+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:29:06.368635+0000","last_peered":"2024-09-17T15:29:06.368635+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:29:06.368635+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":9508,"log_dups_size":0,"ondisk_log_size":9508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38921158,"num_objects":9504,"num_object_clones":0,"num_object_copies":19008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9504,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6277,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5330},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9570","reported_seq":9697,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:06.404572+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:29:06.404572+0000","last_peered":"2024-09-17T15:29:06.404572+0000","last_clean":"2024-09-17T15:29:06.404572+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:29:06.404572+0000","last_undegraded":"2024-09-17T15:29:06.404572+0000","last_fullsized":"2024-09-17T15:29:06.404572+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":9570,"log_dups_size":0,"ondisk_log_size":9570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39190528,"num_objects":9568,"num_object_clones":0,"num_object_copies":19136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6524,"num_write_kb":26096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'9484","reported_seq":17766,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:05.542142+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:29:05.542142+0000","last_peered":"2024-09-17T15:29:05.542142+0000","last_clean":"2024-09-17T15:29:05.542142+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:29:05.542142+0000","last_undegraded":"2024-09-17T15:29:05.542142+0000","last_fullsized":"2024-09-17T15:29:05.542142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":9484,"log_dups_size":0,"ondisk_log_size":9484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38818275,"num_objects":9478,"num_object_clones":0,"num_object_copies":18956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9478,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7484,"num_write_kb":29936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804705+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804705+0000","last_peered":"2024-09-17T15:28:23.804705+0000","last_clean":"2024-09-17T15:28:23.804705+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804705+0000","last_undegraded":"2024-09-17T15:28:23.804705+0000","last_fullsized":"2024-09-17T15:28:23.804705+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9363","reported_seq":9493,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:06.376444+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:29:06.376444+0000","last_peered":"2024-09-17T15:29:06.376444+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:29:06.376444+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":9363,"log_dups_size":0,"ondisk_log_size":9363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38318080,"num_objects":9355,"num_object_clones":0,"num_object_copies":18710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9355,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5959,"num_write_kb":23836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5345},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":102,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814215+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814215+0000","last_peered":"2024-09-17T15:28:23.814215+0000","last_clean":"2024-09-17T15:28:23.814215+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814215+0000","last_undegraded":"2024-09-17T15:28:23.814215+0000","last_fullsized":"2024-09-17T15:28:23.814215+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9401","reported_seq":9521,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:07.210287+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:29:07.210287+0000","last_peered":"2024-09-17T15:29:07.210287+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:29:07.210287+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":9401,"log_dups_size":0,"ondisk_log_size":9401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38486499,"num_objects":9397,"num_object_clones":0,"num_object_copies":18794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9397,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7296,"num_write_kb":29184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":5320}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9489","reported_seq":9607,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:07.247500+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:29:07.247500+0000","last_peered":"2024-09-17T15:29:07.247500+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:29:07.247500+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":9489,"log_dups_size":0,"ondisk_log_size":9489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38842368,"num_objects":9483,"num_object_clones":0,"num_object_copies":18966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7251,"num_write_kb":29004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":5346}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9581","reported_seq":9716,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:06.331829+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:29:06.331829+0000","last_peered":"2024-09-17T15:29:06.331829+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:29:06.331829+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":9581,"log_dups_size":0,"ondisk_log_size":9581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39235584,"num_objects":9579,"num_object_clones":0,"num_object_copies":19158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6129,"num_write_kb":24516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5441},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9512","reported_seq":9653,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:06.337380+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:29:06.337380+0000","last_peered":"2024-09-17T15:29:06.337380+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:29:06.337380+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":9512,"log_dups_size":0,"ondisk_log_size":9512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38928384,"num_objects":9504,"num_object_clones":0,"num_object_copies":19008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9504,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5799,"num_write_kb":23196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5347},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805074+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805074+0000","last_peered":"2024-09-17T15:28:23.805074+0000","last_clean":"2024-09-17T15:28:23.805074+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805074+0000","last_undegraded":"2024-09-17T15:28:23.805074+0000","last_fullsized":"2024-09-17T15:28:23.805074+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9553","reported_seq":9684,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:06.362393+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:29:06.362393+0000","last_peered":"2024-09-17T15:29:06.362393+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:29:06.362393+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":9553,"log_dups_size":0,"ondisk_log_size":9553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39112704,"num_objects":9549,"num_object_clones":0,"num_object_copies":19098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6340,"num_write_kb":25360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5486},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9399","reported_seq":11358,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:29:06.404262+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:29:06.404262+0000","last_peered":"2024-09-17T15:29:06.404262+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:29:06.404262+0000","last_undegraded":"2024-09-17T15:29:06.404262+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":9399,"log_dups_size":0,"ondisk_log_size":9399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38450118,"num_objects":9389,"num_object_clones":0,"num_object_copies":18778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3201,"num_objects_unfound":0,"num_objects_dirty":9389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5841,"num_write_kb":23364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":908,"num_bytes_recovered":3719168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6188},{"shards":"0,2","objects":3201}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":622723806,"num_objects":152041,"num_object_clones":0,"num_object_copies":304082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3201,"num_objects_unfound":0,"num_objects_dirty":152041,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":112148,"num_write_kb":448592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4980,"num_bytes_recovered":20390371,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1265020928,"data_stored":1264869182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152113,"ondisk_log_size":152113,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051633,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":702336,"kb_used_data":436016,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93669504,"statfs":{"total":96636764160,"available":95917572096,"internally_reserved":0,"allocated":446480384,"data_stored":446203810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":272690860},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724578,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488276,"kb_used_data":276100,"kb_used_omap":4,"kb_used_meta":212155,"kb_avail":93883564,"statfs":{"total":96636764160,"available":96136769536,"internally_reserved":0,"allocated":282726400,"data_stored":282402393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":217247403},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,0,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247861,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":810484,"kb_used_data":522020,"kb_used_omap":4,"kb_used_meta":288443,"kb_avail":93561356,"statfs":{"total":96636764160,"available":95806828544,"internally_reserved":0,"allocated":534548480,"data_stored":534267829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":295366312},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,5,0,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":534786048,"data_stored":534749918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":283209728,"data_stored":283130242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":447025152,"data_stored":446989022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:29:12.326 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 213 19 154558 154539 2.83355 1.83203 0.0430568 0.0275641 2024-09-17T15:29:13.326 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 214 20 155040 155020 2.82909 1.87891 0.0254959 0.0276077 2024-09-17T15:29:14.326 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 215 20 155510 155490 2.82447 1.83594 0.026797 0.0276527 2024-09-17T15:29:14.580 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:29:15.163 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:29:15.166 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:29:15.187 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":181,"stamp":"2024-09-17T15:29:13.648128+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":631706353,"num_objects":154236,"num_object_clones":0,"num_object_copies":308472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3110,"num_objects_unfound":0,"num_objects_dirty":154236,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":114343,"num_write_kb":457366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5072,"num_bytes_recovered":20763126,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154309,"ondisk_log_size":154309,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2049720,"kb_used_data":1252232,"kb_used_omap":13,"kb_used_meta":797426,"kb_avail":281065800,"statfs":{"total":289910292480,"available":287811379200,"internally_reserved":0,"allocated":1282285568,"data_stored":1281404336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":816565247},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,8,9,7,11,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21221376,"num_objects":5181,"num_object_clones":0,"num_object_copies":10362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-183,"num_objects_unfound":0,"num_objects_dirty":5181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5181,"num_write_kb":20724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":749568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004270"},"pg_stats":[{"pgid":"2.f","version":"43'9549","reported_seq":9626,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:11.426203+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:29:11.426203+0000","last_peered":"2024-09-17T15:29:11.426203+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:29:11.426203+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9549,"log_dups_size":0,"ondisk_log_size":9549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39112704,"num_objects":9549,"num_object_clones":0,"num_object_copies":19098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9549,"num_write_kb":38196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5479},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9701","reported_seq":9803,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:12.191345+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:29:12.191345+0000","last_peered":"2024-09-17T15:29:12.191345+0000","last_clean":"2024-09-17T15:29:12.191345+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:29:12.191345+0000","last_undegraded":"2024-09-17T15:29:12.191345+0000","last_fullsized":"2024-09-17T15:29:12.191345+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":9701,"log_dups_size":0,"ondisk_log_size":9701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39711686,"num_objects":9697,"num_object_clones":0,"num_object_copies":19394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9697,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8058,"num_write_kb":32232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9699","reported_seq":9824,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:12.174079+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:29:12.174079+0000","last_peered":"2024-09-17T15:29:12.174079+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:29:12.174079+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":9699,"log_dups_size":0,"ondisk_log_size":9699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39710720,"num_objects":9695,"num_object_clones":0,"num_object_copies":19390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7856,"num_write_kb":31424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":5527}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9756","reported_seq":9825,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:12.234553+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:29:12.234553+0000","last_peered":"2024-09-17T15:29:12.234553+0000","last_clean":"2024-09-17T15:29:12.234553+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:29:12.234553+0000","last_undegraded":"2024-09-17T15:29:12.234553+0000","last_fullsized":"2024-09-17T15:29:12.234553+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9756,"log_dups_size":0,"ondisk_log_size":9756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39960576,"num_objects":9756,"num_object_clones":0,"num_object_copies":19512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9756,"num_write_kb":39024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9573","reported_seq":9701,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:11.417142+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:29:11.417142+0000","last_peered":"2024-09-17T15:29:11.417142+0000","last_clean":"2024-09-17T15:29:11.417142+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:29:11.417142+0000","last_undegraded":"2024-09-17T15:29:11.417142+0000","last_fullsized":"2024-09-17T15:29:11.417142+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":9573,"log_dups_size":0,"ondisk_log_size":9573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39182819,"num_objects":9567,"num_object_clones":0,"num_object_copies":19134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9567,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6555,"num_write_kb":26220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9826","reported_seq":9960,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:11.342265+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:29:11.342265+0000","last_peered":"2024-09-17T15:29:11.342265+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:29:11.342265+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":9826,"log_dups_size":0,"ondisk_log_size":9826,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40227299,"num_objects":9822,"num_object_clones":0,"num_object_copies":19644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9822,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6325,"num_write_kb":25300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5552},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9647","reported_seq":9785,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:11.398866+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:29:11.398866+0000","last_peered":"2024-09-17T15:29:11.398866+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:29:11.398866+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":9647,"log_dups_size":0,"ondisk_log_size":9647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39490502,"num_objects":9643,"num_object_clones":0,"num_object_copies":19286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9643,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6416,"num_write_kb":25664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5469},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9698","reported_seq":9824,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:11.444923+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:29:11.444923+0000","last_peered":"2024-09-17T15:29:11.444923+0000","last_clean":"2024-09-17T15:29:11.444923+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:29:11.444923+0000","last_undegraded":"2024-09-17T15:29:11.444923+0000","last_fullsized":"2024-09-17T15:29:11.444923+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":9698,"log_dups_size":0,"ondisk_log_size":9698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39714816,"num_objects":9696,"num_object_clones":0,"num_object_copies":19392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9696,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6652,"num_write_kb":26608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'9619","reported_seq":17901,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:10.573288+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:29:10.573288+0000","last_peered":"2024-09-17T15:29:10.573288+0000","last_clean":"2024-09-17T15:29:10.573288+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:29:10.573288+0000","last_undegraded":"2024-09-17T15:29:10.573288+0000","last_fullsized":"2024-09-17T15:29:10.573288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":9619,"log_dups_size":0,"ondisk_log_size":9619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39371235,"num_objects":9613,"num_object_clones":0,"num_object_copies":19226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9613,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7619,"num_write_kb":30476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804705+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804705+0000","last_peered":"2024-09-17T15:28:23.804705+0000","last_clean":"2024-09-17T15:28:23.804705+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804705+0000","last_undegraded":"2024-09-17T15:28:23.804705+0000","last_fullsized":"2024-09-17T15:28:23.804705+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9520","reported_seq":9650,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:11.424455+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:29:11.424455+0000","last_peered":"2024-09-17T15:29:11.424455+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:29:11.424455+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":9520,"log_dups_size":0,"ondisk_log_size":9520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38961152,"num_objects":9512,"num_object_clones":0,"num_object_copies":19024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6116,"num_write_kb":24464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5502},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":102,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814215+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814215+0000","last_peered":"2024-09-17T15:28:23.814215+0000","last_clean":"2024-09-17T15:28:23.814215+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814215+0000","last_undegraded":"2024-09-17T15:28:23.814215+0000","last_fullsized":"2024-09-17T15:28:23.814215+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9528","reported_seq":9648,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:12.266195+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:29:12.266195+0000","last_peered":"2024-09-17T15:29:12.266195+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:29:12.266195+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":9528,"log_dups_size":0,"ondisk_log_size":9528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39006691,"num_objects":9524,"num_object_clones":0,"num_object_copies":19048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9524,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7423,"num_write_kb":29692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":5447}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9626","reported_seq":9743,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:12.240300+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:29:12.240300+0000","last_peered":"2024-09-17T15:29:12.240300+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:29:12.240300+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":9626,"log_dups_size":0,"ondisk_log_size":9626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39403520,"num_objects":9620,"num_object_clones":0,"num_object_copies":19240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7388,"num_write_kb":29552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":5483}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9697","reported_seq":9832,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:11.439966+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:29:11.439966+0000","last_peered":"2024-09-17T15:29:11.439966+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:29:11.439966+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":9697,"log_dups_size":0,"ondisk_log_size":9697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39710720,"num_objects":9695,"num_object_clones":0,"num_object_copies":19390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6245,"num_write_kb":24980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5557},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9657","reported_seq":9797,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:11.368801+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:29:11.368801+0000","last_peered":"2024-09-17T15:29:11.368801+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:29:11.368801+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":9657,"log_dups_size":0,"ondisk_log_size":9657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39522304,"num_objects":9649,"num_object_clones":0,"num_object_copies":19298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5944,"num_write_kb":23776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5492},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805074+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805074+0000","last_peered":"2024-09-17T15:28:23.805074+0000","last_clean":"2024-09-17T15:28:23.805074+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805074+0000","last_undegraded":"2024-09-17T15:28:23.805074+0000","last_fullsized":"2024-09-17T15:28:23.805074+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9677","reported_seq":9808,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:11.338793+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:29:11.338793+0000","last_peered":"2024-09-17T15:29:11.338793+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:29:11.338793+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":9677,"log_dups_size":0,"ondisk_log_size":9677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39620608,"num_objects":9673,"num_object_clones":0,"num_object_copies":19346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6464,"num_write_kb":25856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5610},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9533","reported_seq":11674,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:29:11.376774+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:29:11.376774+0000","last_peered":"2024-09-17T15:29:11.376774+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:29:11.376774+0000","last_undegraded":"2024-09-17T15:29:11.376774+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":9533,"log_dups_size":0,"ondisk_log_size":9533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38998982,"num_objects":9523,"num_object_clones":0,"num_object_copies":19046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3110,"num_objects_unfound":0,"num_objects_dirty":9523,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5975,"num_write_kb":23900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":999,"num_bytes_recovered":4091904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6413},{"shards":"0,2","objects":3110}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":631706334,"num_objects":154234,"num_object_clones":0,"num_object_copies":308468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3110,"num_objects_unfound":0,"num_objects_dirty":154234,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":114341,"num_write_kb":457364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5071,"num_bytes_recovered":20763107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1283465216,"data_stored":1283313470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154306,"ondisk_log_size":154306,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051634,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":710992,"kb_used_data":440512,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93660848,"statfs":{"total":96636764160,"available":95908708352,"internally_reserved":0,"allocated":451084288,"data_stored":450807714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":276950700},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,1,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724579,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":499352,"kb_used_data":283016,"kb_used_omap":4,"kb_used_meta":216315,"kb_avail":93872488,"statfs":{"total":96636764160,"available":96125427712,"internally_reserved":0,"allocated":289808384,"data_stored":289484377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":221507243},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,2,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247862,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":839376,"kb_used_data":528704,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93532464,"statfs":{"total":96636764160,"available":95777243136,"internally_reserved":0,"allocated":541392896,"data_stored":541112245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":318107304},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,4,4,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":541478912,"data_stored":541442782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":290340864,"data_stored":290261378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":451645440,"data_stored":451609310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:29:15.188 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:29:15.326 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 216 20 155933 155913 2.81904 1.65234 0.111094 0.0277036 2024-09-17T15:29:15.719 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:29:15.719 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:29:15.746 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":182,"stamp":"2024-09-17T15:29:15.648975+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":631706353,"num_objects":154236,"num_object_clones":0,"num_object_copies":308472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3110,"num_objects_unfound":0,"num_objects_dirty":154236,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":114343,"num_write_kb":457366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5072,"num_bytes_recovered":20763126,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154309,"ondisk_log_size":154309,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2049720,"kb_used_data":1252232,"kb_used_omap":13,"kb_used_meta":797426,"kb_avail":281065800,"statfs":{"total":289910292480,"available":287811379200,"internally_reserved":0,"allocated":1282285568,"data_stored":1281404336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":816565247},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,8,9,7,11,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18305024,"num_objects":4469,"num_object_clones":0,"num_object_copies":8938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-183,"num_objects_unfound":0,"num_objects_dirty":4469,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4469,"num_write_kb":17876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":749568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004351"},"pg_stats":[{"pgid":"2.f","version":"43'9549","reported_seq":9626,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:11.426203+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:29:11.426203+0000","last_peered":"2024-09-17T15:29:11.426203+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:29:11.426203+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9549,"log_dups_size":0,"ondisk_log_size":9549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39112704,"num_objects":9549,"num_object_clones":0,"num_object_copies":19098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9549,"num_write_kb":38196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5479},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9701","reported_seq":9803,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:12.191345+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:29:12.191345+0000","last_peered":"2024-09-17T15:29:12.191345+0000","last_clean":"2024-09-17T15:29:12.191345+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:29:12.191345+0000","last_undegraded":"2024-09-17T15:29:12.191345+0000","last_fullsized":"2024-09-17T15:29:12.191345+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":9701,"log_dups_size":0,"ondisk_log_size":9701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39711686,"num_objects":9697,"num_object_clones":0,"num_object_copies":19394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9697,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8058,"num_write_kb":32232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9699","reported_seq":9824,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:12.174079+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:29:12.174079+0000","last_peered":"2024-09-17T15:29:12.174079+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:29:12.174079+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":9699,"log_dups_size":0,"ondisk_log_size":9699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39710720,"num_objects":9695,"num_object_clones":0,"num_object_copies":19390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7856,"num_write_kb":31424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":5527}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9756","reported_seq":9825,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:12.234553+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:29:12.234553+0000","last_peered":"2024-09-17T15:29:12.234553+0000","last_clean":"2024-09-17T15:29:12.234553+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:29:12.234553+0000","last_undegraded":"2024-09-17T15:29:12.234553+0000","last_fullsized":"2024-09-17T15:29:12.234553+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9756,"log_dups_size":0,"ondisk_log_size":9756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39960576,"num_objects":9756,"num_object_clones":0,"num_object_copies":19512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9756,"num_write_kb":39024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9573","reported_seq":9701,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:11.417142+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:29:11.417142+0000","last_peered":"2024-09-17T15:29:11.417142+0000","last_clean":"2024-09-17T15:29:11.417142+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:29:11.417142+0000","last_undegraded":"2024-09-17T15:29:11.417142+0000","last_fullsized":"2024-09-17T15:29:11.417142+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":9573,"log_dups_size":0,"ondisk_log_size":9573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39182819,"num_objects":9567,"num_object_clones":0,"num_object_copies":19134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9567,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6555,"num_write_kb":26220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9826","reported_seq":9960,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:11.342265+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:29:11.342265+0000","last_peered":"2024-09-17T15:29:11.342265+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:29:11.342265+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":9826,"log_dups_size":0,"ondisk_log_size":9826,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40227299,"num_objects":9822,"num_object_clones":0,"num_object_copies":19644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9822,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6325,"num_write_kb":25300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5552},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9647","reported_seq":9785,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:11.398866+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:29:11.398866+0000","last_peered":"2024-09-17T15:29:11.398866+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:29:11.398866+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":9647,"log_dups_size":0,"ondisk_log_size":9647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39490502,"num_objects":9643,"num_object_clones":0,"num_object_copies":19286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9643,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6416,"num_write_kb":25664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5469},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9698","reported_seq":9824,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:11.444923+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:29:11.444923+0000","last_peered":"2024-09-17T15:29:11.444923+0000","last_clean":"2024-09-17T15:29:11.444923+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:29:11.444923+0000","last_undegraded":"2024-09-17T15:29:11.444923+0000","last_fullsized":"2024-09-17T15:29:11.444923+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":9698,"log_dups_size":0,"ondisk_log_size":9698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39714816,"num_objects":9696,"num_object_clones":0,"num_object_copies":19392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9696,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6652,"num_write_kb":26608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'9619","reported_seq":17901,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:10.573288+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:29:10.573288+0000","last_peered":"2024-09-17T15:29:10.573288+0000","last_clean":"2024-09-17T15:29:10.573288+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:29:10.573288+0000","last_undegraded":"2024-09-17T15:29:10.573288+0000","last_fullsized":"2024-09-17T15:29:10.573288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":9619,"log_dups_size":0,"ondisk_log_size":9619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39371235,"num_objects":9613,"num_object_clones":0,"num_object_copies":19226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9613,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7619,"num_write_kb":30476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804705+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804705+0000","last_peered":"2024-09-17T15:28:23.804705+0000","last_clean":"2024-09-17T15:28:23.804705+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804705+0000","last_undegraded":"2024-09-17T15:28:23.804705+0000","last_fullsized":"2024-09-17T15:28:23.804705+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9520","reported_seq":9650,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:11.424455+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:29:11.424455+0000","last_peered":"2024-09-17T15:29:11.424455+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:29:11.424455+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":9520,"log_dups_size":0,"ondisk_log_size":9520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38961152,"num_objects":9512,"num_object_clones":0,"num_object_copies":19024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6116,"num_write_kb":24464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5502},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":102,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814215+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814215+0000","last_peered":"2024-09-17T15:28:23.814215+0000","last_clean":"2024-09-17T15:28:23.814215+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814215+0000","last_undegraded":"2024-09-17T15:28:23.814215+0000","last_fullsized":"2024-09-17T15:28:23.814215+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9528","reported_seq":9648,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:12.266195+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:29:12.266195+0000","last_peered":"2024-09-17T15:29:12.266195+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:29:12.266195+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":9528,"log_dups_size":0,"ondisk_log_size":9528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39006691,"num_objects":9524,"num_object_clones":0,"num_object_copies":19048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9524,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7423,"num_write_kb":29692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":5447}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9626","reported_seq":9743,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:12.240300+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:29:12.240300+0000","last_peered":"2024-09-17T15:29:12.240300+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:29:12.240300+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":9626,"log_dups_size":0,"ondisk_log_size":9626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39403520,"num_objects":9620,"num_object_clones":0,"num_object_copies":19240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7388,"num_write_kb":29552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":5483}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9697","reported_seq":9832,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:11.439966+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:29:11.439966+0000","last_peered":"2024-09-17T15:29:11.439966+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:29:11.439966+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":9697,"log_dups_size":0,"ondisk_log_size":9697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39710720,"num_objects":9695,"num_object_clones":0,"num_object_copies":19390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6245,"num_write_kb":24980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5557},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9657","reported_seq":9797,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:11.368801+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:29:11.368801+0000","last_peered":"2024-09-17T15:29:11.368801+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:29:11.368801+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":9657,"log_dups_size":0,"ondisk_log_size":9657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39522304,"num_objects":9649,"num_object_clones":0,"num_object_copies":19298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5944,"num_write_kb":23776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5492},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805074+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805074+0000","last_peered":"2024-09-17T15:28:23.805074+0000","last_clean":"2024-09-17T15:28:23.805074+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805074+0000","last_undegraded":"2024-09-17T15:28:23.805074+0000","last_fullsized":"2024-09-17T15:28:23.805074+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9677","reported_seq":9808,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:11.338793+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:29:11.338793+0000","last_peered":"2024-09-17T15:29:11.338793+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:29:11.338793+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":9677,"log_dups_size":0,"ondisk_log_size":9677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39620608,"num_objects":9673,"num_object_clones":0,"num_object_copies":19346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6464,"num_write_kb":25856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5610},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9533","reported_seq":11674,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:29:11.376774+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:29:11.376774+0000","last_peered":"2024-09-17T15:29:11.376774+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:29:11.376774+0000","last_undegraded":"2024-09-17T15:29:11.376774+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":9533,"log_dups_size":0,"ondisk_log_size":9533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38998982,"num_objects":9523,"num_object_clones":0,"num_object_copies":19046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3110,"num_objects_unfound":0,"num_objects_dirty":9523,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5975,"num_write_kb":23900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":999,"num_bytes_recovered":4091904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6413},{"shards":"0,2","objects":3110}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":631706334,"num_objects":154234,"num_object_clones":0,"num_object_copies":308468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3110,"num_objects_unfound":0,"num_objects_dirty":154234,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":114341,"num_write_kb":457364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5071,"num_bytes_recovered":20763107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1283465216,"data_stored":1283313470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154306,"ondisk_log_size":154306,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051634,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":710992,"kb_used_data":440512,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93660848,"statfs":{"total":96636764160,"available":95908708352,"internally_reserved":0,"allocated":451084288,"data_stored":450807714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":276950700},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,1,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724579,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":499352,"kb_used_data":283016,"kb_used_omap":4,"kb_used_meta":216315,"kb_avail":93872488,"statfs":{"total":96636764160,"available":96125427712,"internally_reserved":0,"allocated":289808384,"data_stored":289484377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":221507243},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,2,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247862,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":839376,"kb_used_data":528704,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93532464,"statfs":{"total":96636764160,"available":95777243136,"internally_reserved":0,"allocated":541392896,"data_stored":541112245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":318107304},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,4,4,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":541478912,"data_stored":541442782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":290340864,"data_stored":290261378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":451645440,"data_stored":451609310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:29:15.747 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:29:15.747 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'9549", 'reported_seq': 9626, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:11.426203+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:29:11.426203+0000', 'last_peered': '2024-09-17T15:29:11.426203+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:29:11.426203+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 9549, 'log_dups_size': 0, 'ondisk_log_size': 9549, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39112704, 'num_objects': 9549, 'num_object_clones': 0, 'num_object_copies': 19098, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4070, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9549, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9549, 'num_write_kb': 38196, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5479}, {'shards': '0,2', 'objects': 4070}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:15.748 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:29:15.748 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'9699", 'reported_seq': 9824, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:12.174079+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:29:12.174079+0000', 'last_peered': '2024-09-17T15:29:12.174079+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:29:12.174079+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 9699, 'log_dups_size': 0, 'ondisk_log_size': 9699, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39710720, 'num_objects': 9695, 'num_object_clones': 0, 'num_object_copies': 19390, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4169, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9695, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7856, 'num_write_kb': 31424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4168}, {'shards': '2', 'objects': 5527}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:29:15.748 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:29:15.748 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'9826", 'reported_seq': 9960, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:11.342265+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:29:11.342265+0000', 'last_peered': '2024-09-17T15:29:11.342265+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:29:11.342265+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 9826, 'log_dups_size': 0, 'ondisk_log_size': 9826, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40227299, 'num_objects': 9822, 'num_object_clones': 0, 'num_object_copies': 19644, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4271, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9822, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6325, 'num_write_kb': 25300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5552}, {'shards': '0,2', 'objects': 4268}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:15.748 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:29:15.748 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'9647", 'reported_seq': 9785, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:11.398866+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:29:11.398866+0000', 'last_peered': '2024-09-17T15:29:11.398866+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:29:11.398866+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 9647, 'log_dups_size': 0, 'ondisk_log_size': 9647, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39490502, 'num_objects': 9643, 'num_object_clones': 0, 'num_object_copies': 19286, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4174, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9643, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 6416, 'num_write_kb': 25664, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5469}, {'shards': '0,2', 'objects': 4174}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:15.748 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:29:15.748 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'9520", 'reported_seq': 9650, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:11.424455+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:29:11.424455+0000', 'last_peered': '2024-09-17T15:29:11.424455+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:29:11.424455+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 9520, 'log_dups_size': 0, 'ondisk_log_size': 9520, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38961152, 'num_objects': 9512, 'num_object_clones': 0, 'num_object_copies': 19024, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4012, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9512, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6116, 'num_write_kb': 24464, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5502}, {'shards': '0,2', 'objects': 4010}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:15.748 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:29:15.748 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'9528", 'reported_seq': 9648, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:12.266195+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:29:12.266195+0000', 'last_peered': '2024-09-17T15:29:12.266195+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:29:12.266195+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 9528, 'log_dups_size': 0, 'ondisk_log_size': 9528, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39006691, 'num_objects': 9524, 'num_object_clones': 0, 'num_object_copies': 19048, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4078, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9524, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7423, 'num_write_kb': 29692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4077}, {'shards': '2', 'objects': 5447}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:29:15.749 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:29:15.749 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'9626", 'reported_seq': 9743, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:12.240300+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:29:12.240300+0000', 'last_peered': '2024-09-17T15:29:12.240300+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:29:12.240300+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 9626, 'log_dups_size': 0, 'ondisk_log_size': 9626, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39403520, 'num_objects': 9620, 'num_object_clones': 0, 'num_object_copies': 19240, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9620, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7388, 'num_write_kb': 29552, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4137}, {'shards': '2', 'objects': 5483}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:29:15.749 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:29:15.749 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'9697", 'reported_seq': 9832, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:11.439966+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:29:11.439966+0000', 'last_peered': '2024-09-17T15:29:11.439966+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:29:11.439966+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 9697, 'log_dups_size': 0, 'ondisk_log_size': 9697, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39710720, 'num_objects': 9695, 'num_object_clones': 0, 'num_object_copies': 19390, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9695, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6245, 'num_write_kb': 24980, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5557}, {'shards': '0,2', 'objects': 4138}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:15.749 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:29:15.749 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'9657", 'reported_seq': 9797, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:11.368801+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:29:11.368801+0000', 'last_peered': '2024-09-17T15:29:11.368801+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:29:11.368801+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 9657, 'log_dups_size': 0, 'ondisk_log_size': 9657, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39522304, 'num_objects': 9649, 'num_object_clones': 0, 'num_object_copies': 19298, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4160, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9649, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5944, 'num_write_kb': 23776, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5492}, {'shards': '0,2', 'objects': 4157}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:15.749 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:29:15.749 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'9677", 'reported_seq': 9808, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:11.338793+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:29:11.338793+0000', 'last_peered': '2024-09-17T15:29:11.338793+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:29:11.338793+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 9677, 'log_dups_size': 0, 'ondisk_log_size': 9677, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39620608, 'num_objects': 9673, 'num_object_clones': 0, 'num_object_copies': 19346, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4065, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9673, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6464, 'num_write_kb': 25856, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5610}, {'shards': '0,2', 'objects': 4063}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:15.749 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:29:15.749 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'9533", 'reported_seq': 11674, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:29:11.376774+0000', 'last_change': '2024-09-17T15:28:22.509818+0000', 'last_active': '2024-09-17T15:29:11.376774+0000', 'last_peered': '2024-09-17T15:29:11.376774+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:29:11.376774+0000', 'last_undegraded': '2024-09-17T15:29:11.376774+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 9533, 'log_dups_size': 0, 'ondisk_log_size': 9533, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38998982, 'num_objects': 9523, 'num_object_clones': 0, 'num_object_copies': 19046, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3110, 'num_objects_unfound': 0, 'num_objects_dirty': 9523, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5975, 'num_write_kb': 23900, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 999, 'num_bytes_recovered': 4091904, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6413}, {'shards': '0,2', 'objects': 3110}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:15.750 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:29:16.326 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 217 20 156339 156319 2.81336 1.58594 0.111755 0.0277606 2024-09-17T15:29:16.469 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:29:16.469 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41277/308472 objects degraded (13.381%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":293,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":154236,"data_bytes":631706353,"bytes_used":2098913280,"bytes_avail":287811379200,"bytes_total":289910292480,"degraded_objects":41277,"degraded_total":308472,"degraded_ratio":0.13381117248891308,"misplaced_objects":3110,"misplaced_total":308472,"misplaced_ratio":0.010081952332788713,"recovering_objects_per_sec":15,"recovering_bytes_per_sec":62441,"recovering_keys_per_sec":0,"num_objects_recovered":183,"num_bytes_recovered":749568,"num_keys_recovered":0,"write_bytes_sec":1524865,"read_op_per_sec":0,"write_op_per_sec":372},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T15:27:19.616376+0000","services":{}},"progress_events":{}} 2024-09-17T15:29:16.497 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:29:16.497 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:29:17.067 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:29:17.068 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:29:17.101 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":182,"stamp":"2024-09-17T15:29:15.648975+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":631706353,"num_objects":154236,"num_object_clones":0,"num_object_copies":308472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3110,"num_objects_unfound":0,"num_objects_dirty":154236,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":114343,"num_write_kb":457366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5072,"num_bytes_recovered":20763126,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154309,"ondisk_log_size":154309,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2049720,"kb_used_data":1252232,"kb_used_omap":13,"kb_used_meta":797426,"kb_avail":281065800,"statfs":{"total":289910292480,"available":287811379200,"internally_reserved":0,"allocated":1282285568,"data_stored":1281404336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":816565247},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,8,9,7,11,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18305024,"num_objects":4469,"num_object_clones":0,"num_object_copies":8938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-183,"num_objects_unfound":0,"num_objects_dirty":4469,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4469,"num_write_kb":17876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":749568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004351"},"pg_stats":[{"pgid":"2.f","version":"43'9549","reported_seq":9626,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:11.426203+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:29:11.426203+0000","last_peered":"2024-09-17T15:29:11.426203+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:29:11.426203+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9549,"log_dups_size":0,"ondisk_log_size":9549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39112704,"num_objects":9549,"num_object_clones":0,"num_object_copies":19098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9549,"num_write_kb":38196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5479},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9701","reported_seq":9803,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:12.191345+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:29:12.191345+0000","last_peered":"2024-09-17T15:29:12.191345+0000","last_clean":"2024-09-17T15:29:12.191345+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:29:12.191345+0000","last_undegraded":"2024-09-17T15:29:12.191345+0000","last_fullsized":"2024-09-17T15:29:12.191345+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":9701,"log_dups_size":0,"ondisk_log_size":9701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39711686,"num_objects":9697,"num_object_clones":0,"num_object_copies":19394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9697,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8058,"num_write_kb":32232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9699","reported_seq":9824,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:12.174079+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:29:12.174079+0000","last_peered":"2024-09-17T15:29:12.174079+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:29:12.174079+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":9699,"log_dups_size":0,"ondisk_log_size":9699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39710720,"num_objects":9695,"num_object_clones":0,"num_object_copies":19390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7856,"num_write_kb":31424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":5527}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9756","reported_seq":9825,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:12.234553+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:29:12.234553+0000","last_peered":"2024-09-17T15:29:12.234553+0000","last_clean":"2024-09-17T15:29:12.234553+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:29:12.234553+0000","last_undegraded":"2024-09-17T15:29:12.234553+0000","last_fullsized":"2024-09-17T15:29:12.234553+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9756,"log_dups_size":0,"ondisk_log_size":9756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39960576,"num_objects":9756,"num_object_clones":0,"num_object_copies":19512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9756,"num_write_kb":39024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9573","reported_seq":9701,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:11.417142+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:29:11.417142+0000","last_peered":"2024-09-17T15:29:11.417142+0000","last_clean":"2024-09-17T15:29:11.417142+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:29:11.417142+0000","last_undegraded":"2024-09-17T15:29:11.417142+0000","last_fullsized":"2024-09-17T15:29:11.417142+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":9573,"log_dups_size":0,"ondisk_log_size":9573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39182819,"num_objects":9567,"num_object_clones":0,"num_object_copies":19134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9567,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6555,"num_write_kb":26220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9826","reported_seq":9960,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:11.342265+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:29:11.342265+0000","last_peered":"2024-09-17T15:29:11.342265+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:29:11.342265+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":9826,"log_dups_size":0,"ondisk_log_size":9826,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40227299,"num_objects":9822,"num_object_clones":0,"num_object_copies":19644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9822,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6325,"num_write_kb":25300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5552},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9647","reported_seq":9785,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:11.398866+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:29:11.398866+0000","last_peered":"2024-09-17T15:29:11.398866+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:29:11.398866+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":9647,"log_dups_size":0,"ondisk_log_size":9647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39490502,"num_objects":9643,"num_object_clones":0,"num_object_copies":19286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9643,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6416,"num_write_kb":25664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5469},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9698","reported_seq":9824,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:11.444923+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:29:11.444923+0000","last_peered":"2024-09-17T15:29:11.444923+0000","last_clean":"2024-09-17T15:29:11.444923+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:29:11.444923+0000","last_undegraded":"2024-09-17T15:29:11.444923+0000","last_fullsized":"2024-09-17T15:29:11.444923+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":9698,"log_dups_size":0,"ondisk_log_size":9698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39714816,"num_objects":9696,"num_object_clones":0,"num_object_copies":19392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9696,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6652,"num_write_kb":26608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'9619","reported_seq":17901,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:10.573288+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:29:10.573288+0000","last_peered":"2024-09-17T15:29:10.573288+0000","last_clean":"2024-09-17T15:29:10.573288+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:29:10.573288+0000","last_undegraded":"2024-09-17T15:29:10.573288+0000","last_fullsized":"2024-09-17T15:29:10.573288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":9619,"log_dups_size":0,"ondisk_log_size":9619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39371235,"num_objects":9613,"num_object_clones":0,"num_object_copies":19226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9613,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7619,"num_write_kb":30476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804705+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804705+0000","last_peered":"2024-09-17T15:28:23.804705+0000","last_clean":"2024-09-17T15:28:23.804705+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804705+0000","last_undegraded":"2024-09-17T15:28:23.804705+0000","last_fullsized":"2024-09-17T15:28:23.804705+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9520","reported_seq":9650,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:11.424455+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:29:11.424455+0000","last_peered":"2024-09-17T15:29:11.424455+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:29:11.424455+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":9520,"log_dups_size":0,"ondisk_log_size":9520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38961152,"num_objects":9512,"num_object_clones":0,"num_object_copies":19024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6116,"num_write_kb":24464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5502},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":102,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814215+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814215+0000","last_peered":"2024-09-17T15:28:23.814215+0000","last_clean":"2024-09-17T15:28:23.814215+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814215+0000","last_undegraded":"2024-09-17T15:28:23.814215+0000","last_fullsized":"2024-09-17T15:28:23.814215+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9528","reported_seq":9648,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:12.266195+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:29:12.266195+0000","last_peered":"2024-09-17T15:29:12.266195+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:29:12.266195+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":9528,"log_dups_size":0,"ondisk_log_size":9528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39006691,"num_objects":9524,"num_object_clones":0,"num_object_copies":19048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9524,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7423,"num_write_kb":29692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":5447}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9626","reported_seq":9743,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:12.240300+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:29:12.240300+0000","last_peered":"2024-09-17T15:29:12.240300+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:29:12.240300+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":9626,"log_dups_size":0,"ondisk_log_size":9626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39403520,"num_objects":9620,"num_object_clones":0,"num_object_copies":19240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7388,"num_write_kb":29552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":5483}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9697","reported_seq":9832,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:11.439966+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:29:11.439966+0000","last_peered":"2024-09-17T15:29:11.439966+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:29:11.439966+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":9697,"log_dups_size":0,"ondisk_log_size":9697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39710720,"num_objects":9695,"num_object_clones":0,"num_object_copies":19390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6245,"num_write_kb":24980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5557},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9657","reported_seq":9797,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:11.368801+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:29:11.368801+0000","last_peered":"2024-09-17T15:29:11.368801+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:29:11.368801+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":9657,"log_dups_size":0,"ondisk_log_size":9657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39522304,"num_objects":9649,"num_object_clones":0,"num_object_copies":19298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5944,"num_write_kb":23776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5492},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805074+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805074+0000","last_peered":"2024-09-17T15:28:23.805074+0000","last_clean":"2024-09-17T15:28:23.805074+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805074+0000","last_undegraded":"2024-09-17T15:28:23.805074+0000","last_fullsized":"2024-09-17T15:28:23.805074+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9677","reported_seq":9808,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:11.338793+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:29:11.338793+0000","last_peered":"2024-09-17T15:29:11.338793+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:29:11.338793+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":9677,"log_dups_size":0,"ondisk_log_size":9677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39620608,"num_objects":9673,"num_object_clones":0,"num_object_copies":19346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6464,"num_write_kb":25856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5610},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9533","reported_seq":11674,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:29:11.376774+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:29:11.376774+0000","last_peered":"2024-09-17T15:29:11.376774+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:29:11.376774+0000","last_undegraded":"2024-09-17T15:29:11.376774+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":9533,"log_dups_size":0,"ondisk_log_size":9533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38998982,"num_objects":9523,"num_object_clones":0,"num_object_copies":19046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3110,"num_objects_unfound":0,"num_objects_dirty":9523,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5975,"num_write_kb":23900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":999,"num_bytes_recovered":4091904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6413},{"shards":"0,2","objects":3110}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":631706334,"num_objects":154234,"num_object_clones":0,"num_object_copies":308468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3110,"num_objects_unfound":0,"num_objects_dirty":154234,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":114341,"num_write_kb":457364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5071,"num_bytes_recovered":20763107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1283465216,"data_stored":1283313470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154306,"ondisk_log_size":154306,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051634,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":710992,"kb_used_data":440512,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93660848,"statfs":{"total":96636764160,"available":95908708352,"internally_reserved":0,"allocated":451084288,"data_stored":450807714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":276950700},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,1,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724579,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":499352,"kb_used_data":283016,"kb_used_omap":4,"kb_used_meta":216315,"kb_avail":93872488,"statfs":{"total":96636764160,"available":96125427712,"internally_reserved":0,"allocated":289808384,"data_stored":289484377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":221507243},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,2,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247862,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":839376,"kb_used_data":528704,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93532464,"statfs":{"total":96636764160,"available":95777243136,"internally_reserved":0,"allocated":541392896,"data_stored":541112245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":318107304},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,4,4,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":541478912,"data_stored":541442782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":290340864,"data_stored":290261378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":451645440,"data_stored":451609310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:29:17.328 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 218 20 156765 156745 2.80809 1.66406 0.022883 0.027814 2024-09-17T15:29:18.327 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 219 20 157237 157217 2.80368 1.84375 0.0201631 0.027857 2024-09-17T15:29:19.327 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout:2024-09-17T15:29:19.327233+0000 min lat: 0.00187689 max lat: 2.64156 avg lat: 0.0279023 2024-09-17T15:29:19.327 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-17T15:29:19.327 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 220 20 157707 157687 2.79929 1.83594 0.0240272 0.0279023 2024-09-17T15:29:20.103 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:29:20.327 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 221 20 158168 158148 2.79477 1.80078 0.0212483 0.0279463 2024-09-17T15:29:20.667 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:29:20.669 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:29:20.692 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":184,"stamp":"2024-09-17T15:29:19.650564+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":641000177,"num_objects":156505,"num_object_clones":0,"num_object_copies":313010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3013,"num_objects_unfound":0,"num_objects_dirty":156505,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":116612,"num_write_kb":466442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5169,"num_bytes_recovered":21160438,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156578,"ondisk_log_size":156578,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2082584,"kb_used_data":1271208,"kb_used_omap":13,"kb_used_meta":811314,"kb_avail":281032936,"statfs":{"total":289910292480,"available":287777726464,"internally_reserved":0,"allocated":1301716992,"data_stored":1300835760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":830786559},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,6,2,2,1,6,9,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18276352,"num_objects":4462,"num_object_clones":0,"num_object_copies":8924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-188,"num_objects_unfound":0,"num_objects_dirty":4462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4462,"num_write_kb":17848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":770048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004664"},"pg_stats":[{"pgid":"2.f","version":"43'9685","reported_seq":9761,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:16.485157+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:29:16.485157+0000","last_peered":"2024-09-17T15:29:16.485157+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:29:16.485157+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9685,"log_dups_size":0,"ondisk_log_size":9685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39669760,"num_objects":9685,"num_object_clones":0,"num_object_copies":19370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9685,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9685,"num_write_kb":38740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5615},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9831","reported_seq":9933,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:17.222297+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:29:17.222297+0000","last_peered":"2024-09-17T15:29:17.222297+0000","last_clean":"2024-09-17T15:29:17.222297+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:29:17.222297+0000","last_undegraded":"2024-09-17T15:29:17.222297+0000","last_fullsized":"2024-09-17T15:29:17.222297+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":9831,"log_dups_size":0,"ondisk_log_size":9831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40244166,"num_objects":9827,"num_object_clones":0,"num_object_copies":19654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9827,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8188,"num_write_kb":32752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9835","reported_seq":9960,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:17.251506+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:29:17.251506+0000","last_peered":"2024-09-17T15:29:17.251506+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:29:17.251506+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":9835,"log_dups_size":0,"ondisk_log_size":9835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40267776,"num_objects":9831,"num_object_clones":0,"num_object_copies":19662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9831,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7992,"num_write_kb":31968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":5663}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9891","reported_seq":9960,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:17.304488+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:29:17.304488+0000","last_peered":"2024-09-17T15:29:17.304488+0000","last_clean":"2024-09-17T15:29:17.304488+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:29:17.304488+0000","last_undegraded":"2024-09-17T15:29:17.304488+0000","last_fullsized":"2024-09-17T15:29:17.304488+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9891,"log_dups_size":0,"ondisk_log_size":9891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40513536,"num_objects":9891,"num_object_clones":0,"num_object_copies":19782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9891,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9891,"num_write_kb":39564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9709","reported_seq":9838,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:16.485208+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:29:16.485208+0000","last_peered":"2024-09-17T15:29:16.485208+0000","last_clean":"2024-09-17T15:29:16.485208+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:29:16.485208+0000","last_undegraded":"2024-09-17T15:29:16.485208+0000","last_fullsized":"2024-09-17T15:29:16.485208+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":9709,"log_dups_size":0,"ondisk_log_size":9709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39739875,"num_objects":9703,"num_object_clones":0,"num_object_copies":19406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9703,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6691,"num_write_kb":26764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9987","reported_seq":10121,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:16.394586+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:29:16.394586+0000","last_peered":"2024-09-17T15:29:16.394586+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:29:16.394586+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":9987,"log_dups_size":0,"ondisk_log_size":9987,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40886755,"num_objects":9983,"num_object_clones":0,"num_object_copies":19966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9983,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6486,"num_write_kb":25944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5713},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9786","reported_seq":9924,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:16.381524+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:29:16.381524+0000","last_peered":"2024-09-17T15:29:16.381524+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:29:16.381524+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":9786,"log_dups_size":0,"ondisk_log_size":9786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40059846,"num_objects":9782,"num_object_clones":0,"num_object_copies":19564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9782,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6555,"num_write_kb":26220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5608},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9843","reported_seq":9970,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:16.444568+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:29:16.444568+0000","last_peered":"2024-09-17T15:29:16.444568+0000","last_clean":"2024-09-17T15:29:16.444568+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:29:16.444568+0000","last_undegraded":"2024-09-17T15:29:16.444568+0000","last_fullsized":"2024-09-17T15:29:16.444568+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":9843,"log_dups_size":0,"ondisk_log_size":9843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40308736,"num_objects":9841,"num_object_clones":0,"num_object_copies":19682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6797,"num_write_kb":27188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'9762","reported_seq":18044,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:15.752059+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:29:15.752059+0000","last_peered":"2024-09-17T15:29:15.752059+0000","last_clean":"2024-09-17T15:29:15.752059+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:29:15.752059+0000","last_undegraded":"2024-09-17T15:29:15.752059+0000","last_fullsized":"2024-09-17T15:29:15.752059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":9762,"log_dups_size":0,"ondisk_log_size":9762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39956963,"num_objects":9756,"num_object_clones":0,"num_object_copies":19512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9756,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7762,"num_write_kb":31048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804705+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804705+0000","last_peered":"2024-09-17T15:28:23.804705+0000","last_clean":"2024-09-17T15:28:23.804705+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804705+0000","last_undegraded":"2024-09-17T15:28:23.804705+0000","last_fullsized":"2024-09-17T15:28:23.804705+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9658","reported_seq":9788,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:16.383298+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:29:16.383298+0000","last_peered":"2024-09-17T15:29:16.383298+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:29:16.383298+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":9658,"log_dups_size":0,"ondisk_log_size":9658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39526400,"num_objects":9650,"num_object_clones":0,"num_object_copies":19300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6254,"num_write_kb":25016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5640},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9669","reported_seq":9789,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:17.302603+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:29:17.302603+0000","last_peered":"2024-09-17T15:29:17.302603+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:29:17.302603+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":9669,"log_dups_size":0,"ondisk_log_size":9669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39584227,"num_objects":9665,"num_object_clones":0,"num_object_copies":19330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9665,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7564,"num_write_kb":30256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":5588}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9774","reported_seq":9892,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:17.213960+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:29:17.213960+0000","last_peered":"2024-09-17T15:29:17.213960+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:29:17.213960+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":9774,"log_dups_size":0,"ondisk_log_size":9774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40009728,"num_objects":9768,"num_object_clones":0,"num_object_copies":19536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7536,"num_write_kb":30144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":5631}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9853","reported_seq":9988,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:16.455761+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:29:16.455761+0000","last_peered":"2024-09-17T15:29:16.455761+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:29:16.455761+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":9853,"log_dups_size":0,"ondisk_log_size":9853,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40349696,"num_objects":9851,"num_object_clones":0,"num_object_copies":19702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6401,"num_write_kb":25604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5713},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9777","reported_seq":9918,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:16.463990+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:29:16.463990+0000","last_peered":"2024-09-17T15:29:16.463990+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:29:16.463990+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":9777,"log_dups_size":0,"ondisk_log_size":9777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40013824,"num_objects":9769,"num_object_clones":0,"num_object_copies":19538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6064,"num_write_kb":24256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5612},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805074+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805074+0000","last_peered":"2024-09-17T15:28:23.805074+0000","last_clean":"2024-09-17T15:28:23.805074+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805074+0000","last_undegraded":"2024-09-17T15:28:23.805074+0000","last_fullsized":"2024-09-17T15:28:23.805074+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9839","reported_seq":9970,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:16.424775+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:29:16.424775+0000","last_peered":"2024-09-17T15:29:16.424775+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:29:16.424775+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":9839,"log_dups_size":0,"ondisk_log_size":9839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40284160,"num_objects":9835,"num_object_clones":0,"num_object_copies":19670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9835,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6626,"num_write_kb":26504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5772},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9676","reported_seq":12011,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:29:16.480472+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:29:16.480472+0000","last_peered":"2024-09-17T15:29:16.480472+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:29:16.480472+0000","last_undegraded":"2024-09-17T15:29:16.480472+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":9676,"log_dups_size":0,"ondisk_log_size":9676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39584710,"num_objects":9666,"num_object_clones":0,"num_object_copies":19332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3013,"num_objects_unfound":0,"num_objects_dirty":9666,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6118,"num_write_kb":24472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1096,"num_bytes_recovered":4489216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6653},{"shards":"0,2","objects":3013}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":641000158,"num_objects":156503,"num_object_clones":0,"num_object_copies":313006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3013,"num_objects_unfound":0,"num_objects_dirty":156503,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":116610,"num_write_kb":466440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5168,"num_bytes_recovered":21160419,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1302413312,"data_stored":1302261566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156575,"ondisk_log_size":156575,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051635,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":720332,"kb_used_data":445692,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93651508,"statfs":{"total":96636764160,"available":95899144192,"internally_reserved":0,"allocated":456388608,"data_stored":456112034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":281210540},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,0,1,3,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724580,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":528756,"kb_used_data":290212,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93843084,"statfs":{"total":96636764160,"available":96095318016,"internally_reserved":0,"allocated":297177088,"data_stored":296853081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":244248235},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,0,0,3,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247863,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":833496,"kb_used_data":535304,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93538344,"statfs":{"total":96636764160,"available":95783264256,"internally_reserved":0,"allocated":548151296,"data_stored":547870645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":305327784},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,1,1,2,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":548519936,"data_stored":548483806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":297771008,"data_stored":297691522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":456122368,"data_stored":456086238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:29:20.694 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:29:21.244 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:29:21.245 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:29:21.273 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":184,"stamp":"2024-09-17T15:29:19.650564+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":641000177,"num_objects":156505,"num_object_clones":0,"num_object_copies":313010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3013,"num_objects_unfound":0,"num_objects_dirty":156505,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":116612,"num_write_kb":466442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5169,"num_bytes_recovered":21160438,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156578,"ondisk_log_size":156578,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2082584,"kb_used_data":1271208,"kb_used_omap":13,"kb_used_meta":811314,"kb_avail":281032936,"statfs":{"total":289910292480,"available":287777726464,"internally_reserved":0,"allocated":1301716992,"data_stored":1300835760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":830786559},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,6,2,2,1,6,9,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18276352,"num_objects":4462,"num_object_clones":0,"num_object_copies":8924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-188,"num_objects_unfound":0,"num_objects_dirty":4462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4462,"num_write_kb":17848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":770048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004664"},"pg_stats":[{"pgid":"2.f","version":"43'9685","reported_seq":9761,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:16.485157+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:29:16.485157+0000","last_peered":"2024-09-17T15:29:16.485157+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:29:16.485157+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9685,"log_dups_size":0,"ondisk_log_size":9685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39669760,"num_objects":9685,"num_object_clones":0,"num_object_copies":19370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9685,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9685,"num_write_kb":38740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5615},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9831","reported_seq":9933,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:17.222297+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:29:17.222297+0000","last_peered":"2024-09-17T15:29:17.222297+0000","last_clean":"2024-09-17T15:29:17.222297+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:29:17.222297+0000","last_undegraded":"2024-09-17T15:29:17.222297+0000","last_fullsized":"2024-09-17T15:29:17.222297+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":9831,"log_dups_size":0,"ondisk_log_size":9831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40244166,"num_objects":9827,"num_object_clones":0,"num_object_copies":19654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9827,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8188,"num_write_kb":32752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9835","reported_seq":9960,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:17.251506+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:29:17.251506+0000","last_peered":"2024-09-17T15:29:17.251506+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:29:17.251506+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":9835,"log_dups_size":0,"ondisk_log_size":9835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40267776,"num_objects":9831,"num_object_clones":0,"num_object_copies":19662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9831,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7992,"num_write_kb":31968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":5663}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9891","reported_seq":9960,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:17.304488+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:29:17.304488+0000","last_peered":"2024-09-17T15:29:17.304488+0000","last_clean":"2024-09-17T15:29:17.304488+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:29:17.304488+0000","last_undegraded":"2024-09-17T15:29:17.304488+0000","last_fullsized":"2024-09-17T15:29:17.304488+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9891,"log_dups_size":0,"ondisk_log_size":9891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40513536,"num_objects":9891,"num_object_clones":0,"num_object_copies":19782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9891,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9891,"num_write_kb":39564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9709","reported_seq":9838,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:16.485208+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:29:16.485208+0000","last_peered":"2024-09-17T15:29:16.485208+0000","last_clean":"2024-09-17T15:29:16.485208+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:29:16.485208+0000","last_undegraded":"2024-09-17T15:29:16.485208+0000","last_fullsized":"2024-09-17T15:29:16.485208+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":9709,"log_dups_size":0,"ondisk_log_size":9709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39739875,"num_objects":9703,"num_object_clones":0,"num_object_copies":19406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9703,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6691,"num_write_kb":26764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9987","reported_seq":10121,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:16.394586+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:29:16.394586+0000","last_peered":"2024-09-17T15:29:16.394586+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:29:16.394586+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":9987,"log_dups_size":0,"ondisk_log_size":9987,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40886755,"num_objects":9983,"num_object_clones":0,"num_object_copies":19966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9983,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6486,"num_write_kb":25944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5713},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9786","reported_seq":9924,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:16.381524+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:29:16.381524+0000","last_peered":"2024-09-17T15:29:16.381524+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:29:16.381524+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":9786,"log_dups_size":0,"ondisk_log_size":9786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40059846,"num_objects":9782,"num_object_clones":0,"num_object_copies":19564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9782,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6555,"num_write_kb":26220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5608},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9843","reported_seq":9970,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:16.444568+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:29:16.444568+0000","last_peered":"2024-09-17T15:29:16.444568+0000","last_clean":"2024-09-17T15:29:16.444568+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:29:16.444568+0000","last_undegraded":"2024-09-17T15:29:16.444568+0000","last_fullsized":"2024-09-17T15:29:16.444568+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":9843,"log_dups_size":0,"ondisk_log_size":9843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40308736,"num_objects":9841,"num_object_clones":0,"num_object_copies":19682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6797,"num_write_kb":27188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'9762","reported_seq":18044,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:15.752059+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:29:15.752059+0000","last_peered":"2024-09-17T15:29:15.752059+0000","last_clean":"2024-09-17T15:29:15.752059+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:29:15.752059+0000","last_undegraded":"2024-09-17T15:29:15.752059+0000","last_fullsized":"2024-09-17T15:29:15.752059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":9762,"log_dups_size":0,"ondisk_log_size":9762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39956963,"num_objects":9756,"num_object_clones":0,"num_object_copies":19512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9756,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7762,"num_write_kb":31048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804705+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804705+0000","last_peered":"2024-09-17T15:28:23.804705+0000","last_clean":"2024-09-17T15:28:23.804705+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804705+0000","last_undegraded":"2024-09-17T15:28:23.804705+0000","last_fullsized":"2024-09-17T15:28:23.804705+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9658","reported_seq":9788,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:16.383298+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:29:16.383298+0000","last_peered":"2024-09-17T15:29:16.383298+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:29:16.383298+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":9658,"log_dups_size":0,"ondisk_log_size":9658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39526400,"num_objects":9650,"num_object_clones":0,"num_object_copies":19300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6254,"num_write_kb":25016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5640},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9669","reported_seq":9789,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:17.302603+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:29:17.302603+0000","last_peered":"2024-09-17T15:29:17.302603+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:29:17.302603+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":9669,"log_dups_size":0,"ondisk_log_size":9669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39584227,"num_objects":9665,"num_object_clones":0,"num_object_copies":19330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9665,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7564,"num_write_kb":30256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":5588}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9774","reported_seq":9892,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:17.213960+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:29:17.213960+0000","last_peered":"2024-09-17T15:29:17.213960+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:29:17.213960+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":9774,"log_dups_size":0,"ondisk_log_size":9774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40009728,"num_objects":9768,"num_object_clones":0,"num_object_copies":19536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7536,"num_write_kb":30144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":5631}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9853","reported_seq":9988,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:16.455761+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:29:16.455761+0000","last_peered":"2024-09-17T15:29:16.455761+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:29:16.455761+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":9853,"log_dups_size":0,"ondisk_log_size":9853,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40349696,"num_objects":9851,"num_object_clones":0,"num_object_copies":19702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6401,"num_write_kb":25604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5713},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9777","reported_seq":9918,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:16.463990+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:29:16.463990+0000","last_peered":"2024-09-17T15:29:16.463990+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:29:16.463990+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":9777,"log_dups_size":0,"ondisk_log_size":9777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40013824,"num_objects":9769,"num_object_clones":0,"num_object_copies":19538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6064,"num_write_kb":24256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5612},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805074+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805074+0000","last_peered":"2024-09-17T15:28:23.805074+0000","last_clean":"2024-09-17T15:28:23.805074+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805074+0000","last_undegraded":"2024-09-17T15:28:23.805074+0000","last_fullsized":"2024-09-17T15:28:23.805074+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9839","reported_seq":9970,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:16.424775+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:29:16.424775+0000","last_peered":"2024-09-17T15:29:16.424775+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:29:16.424775+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":9839,"log_dups_size":0,"ondisk_log_size":9839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40284160,"num_objects":9835,"num_object_clones":0,"num_object_copies":19670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9835,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6626,"num_write_kb":26504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5772},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9676","reported_seq":12011,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:29:16.480472+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:29:16.480472+0000","last_peered":"2024-09-17T15:29:16.480472+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:29:16.480472+0000","last_undegraded":"2024-09-17T15:29:16.480472+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":9676,"log_dups_size":0,"ondisk_log_size":9676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39584710,"num_objects":9666,"num_object_clones":0,"num_object_copies":19332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3013,"num_objects_unfound":0,"num_objects_dirty":9666,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6118,"num_write_kb":24472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1096,"num_bytes_recovered":4489216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6653},{"shards":"0,2","objects":3013}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":641000158,"num_objects":156503,"num_object_clones":0,"num_object_copies":313006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3013,"num_objects_unfound":0,"num_objects_dirty":156503,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":116610,"num_write_kb":466440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5168,"num_bytes_recovered":21160419,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1302413312,"data_stored":1302261566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156575,"ondisk_log_size":156575,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051635,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":720332,"kb_used_data":445692,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93651508,"statfs":{"total":96636764160,"available":95899144192,"internally_reserved":0,"allocated":456388608,"data_stored":456112034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":281210540},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,0,1,3,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724580,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":528756,"kb_used_data":290212,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93843084,"statfs":{"total":96636764160,"available":96095318016,"internally_reserved":0,"allocated":297177088,"data_stored":296853081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":244248235},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,0,0,3,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247863,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":833496,"kb_used_data":535304,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93538344,"statfs":{"total":96636764160,"available":95783264256,"internally_reserved":0,"allocated":548151296,"data_stored":547870645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":305327784},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,1,1,2,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":548519936,"data_stored":548483806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":297771008,"data_stored":297691522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":456122368,"data_stored":456086238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:29:21.275 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:29:21.275 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'9685", 'reported_seq': 9761, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:16.485157+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:29:16.485157+0000', 'last_peered': '2024-09-17T15:29:16.485157+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:29:16.485157+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 9685, 'log_dups_size': 0, 'ondisk_log_size': 9685, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39669760, 'num_objects': 9685, 'num_object_clones': 0, 'num_object_copies': 19370, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4070, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9685, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9685, 'num_write_kb': 38740, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5615}, {'shards': '0,2', 'objects': 4070}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:21.275 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:29:21.275 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'9835", 'reported_seq': 9960, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:17.251506+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:29:17.251506+0000', 'last_peered': '2024-09-17T15:29:17.251506+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:29:17.251506+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 9835, 'log_dups_size': 0, 'ondisk_log_size': 9835, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40267776, 'num_objects': 9831, 'num_object_clones': 0, 'num_object_copies': 19662, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4169, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9831, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7992, 'num_write_kb': 31968, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4168}, {'shards': '2', 'objects': 5663}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:29:21.275 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:29:21.275 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'9987", 'reported_seq': 10121, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:16.394586+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:29:16.394586+0000', 'last_peered': '2024-09-17T15:29:16.394586+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:29:16.394586+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 9987, 'log_dups_size': 0, 'ondisk_log_size': 9987, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40886755, 'num_objects': 9983, 'num_object_clones': 0, 'num_object_copies': 19966, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4271, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9983, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6486, 'num_write_kb': 25944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5713}, {'shards': '0,2', 'objects': 4268}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:21.275 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:29:21.275 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'9786", 'reported_seq': 9924, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:16.381524+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:29:16.381524+0000', 'last_peered': '2024-09-17T15:29:16.381524+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:29:16.381524+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 9786, 'log_dups_size': 0, 'ondisk_log_size': 9786, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40059846, 'num_objects': 9782, 'num_object_clones': 0, 'num_object_copies': 19564, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4174, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9782, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 6555, 'num_write_kb': 26220, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5608}, {'shards': '0,2', 'objects': 4174}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:21.275 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:29:21.276 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'9658", 'reported_seq': 9788, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:16.383298+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:29:16.383298+0000', 'last_peered': '2024-09-17T15:29:16.383298+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:29:16.383298+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 9658, 'log_dups_size': 0, 'ondisk_log_size': 9658, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39526400, 'num_objects': 9650, 'num_object_clones': 0, 'num_object_copies': 19300, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4012, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9650, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6254, 'num_write_kb': 25016, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5640}, {'shards': '0,2', 'objects': 4010}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:21.276 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:29:21.276 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'9669", 'reported_seq': 9789, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:17.302603+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:29:17.302603+0000', 'last_peered': '2024-09-17T15:29:17.302603+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:29:17.302603+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 9669, 'log_dups_size': 0, 'ondisk_log_size': 9669, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39584227, 'num_objects': 9665, 'num_object_clones': 0, 'num_object_copies': 19330, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4078, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9665, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7564, 'num_write_kb': 30256, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4077}, {'shards': '2', 'objects': 5588}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:29:21.276 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:29:21.276 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'9774", 'reported_seq': 9892, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:17.213960+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:29:17.213960+0000', 'last_peered': '2024-09-17T15:29:17.213960+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:29:17.213960+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 9774, 'log_dups_size': 0, 'ondisk_log_size': 9774, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40009728, 'num_objects': 9768, 'num_object_clones': 0, 'num_object_copies': 19536, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9768, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7536, 'num_write_kb': 30144, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4137}, {'shards': '2', 'objects': 5631}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:29:21.276 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:29:21.276 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'9853", 'reported_seq': 9988, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:16.455761+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:29:16.455761+0000', 'last_peered': '2024-09-17T15:29:16.455761+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:29:16.455761+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 9853, 'log_dups_size': 0, 'ondisk_log_size': 9853, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40349696, 'num_objects': 9851, 'num_object_clones': 0, 'num_object_copies': 19702, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9851, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6401, 'num_write_kb': 25604, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5713}, {'shards': '0,2', 'objects': 4138}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:21.276 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:29:21.276 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'9777", 'reported_seq': 9918, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:16.463990+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:29:16.463990+0000', 'last_peered': '2024-09-17T15:29:16.463990+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:29:16.463990+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 9777, 'log_dups_size': 0, 'ondisk_log_size': 9777, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40013824, 'num_objects': 9769, 'num_object_clones': 0, 'num_object_copies': 19538, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4160, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9769, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6064, 'num_write_kb': 24256, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5612}, {'shards': '0,2', 'objects': 4157}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:21.277 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:29:21.277 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'9839", 'reported_seq': 9970, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:16.424775+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:29:16.424775+0000', 'last_peered': '2024-09-17T15:29:16.424775+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:29:16.424775+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 9839, 'log_dups_size': 0, 'ondisk_log_size': 9839, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40284160, 'num_objects': 9835, 'num_object_clones': 0, 'num_object_copies': 19670, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4065, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9835, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6626, 'num_write_kb': 26504, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5772}, {'shards': '0,2', 'objects': 4063}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:21.277 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:29:21.277 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'9676", 'reported_seq': 12011, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:29:16.480472+0000', 'last_change': '2024-09-17T15:28:22.509818+0000', 'last_active': '2024-09-17T15:29:16.480472+0000', 'last_peered': '2024-09-17T15:29:16.480472+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:29:16.480472+0000', 'last_undegraded': '2024-09-17T15:29:16.480472+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 9676, 'log_dups_size': 0, 'ondisk_log_size': 9676, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39584710, 'num_objects': 9666, 'num_object_clones': 0, 'num_object_copies': 19332, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3013, 'num_objects_unfound': 0, 'num_objects_dirty': 9666, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6118, 'num_write_kb': 24472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1096, 'num_bytes_recovered': 4489216, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6653}, {'shards': '0,2', 'objects': 3013}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:21.277 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:29:21.327 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 222 20 158582 158562 2.78946 1.61719 0.0221444 0.0280006 2024-09-17T15:29:21.983 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:29:21.983 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41277/313010 objects degraded (13.187%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":298,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":156505,"data_bytes":641000177,"bytes_used":2132566016,"bytes_avail":287777726464,"bytes_total":289910292480,"degraded_objects":41277,"degraded_total":313010,"degraded_ratio":0.13187118622408231,"misplaced_objects":3013,"misplaced_total":313010,"misplaced_ratio":0.0096258905466279025,"recovering_objects_per_sec":15,"recovering_bytes_per_sec":64145,"recovering_keys_per_sec":0,"num_objects_recovered":188,"num_bytes_recovered":770048,"num_keys_recovered":0,"write_bytes_sec":1522437,"read_op_per_sec":0,"write_op_per_sec":371},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T15:27:19.616376+0000","services":{}},"progress_events":{}} 2024-09-17T15:29:22.015 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:29:22.015 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:29:22.327 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 223 20 158993 158973 2.78415 1.60547 0.0697636 0.0280532 2024-09-17T15:29:22.586 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:29:22.588 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:29:22.610 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":185,"stamp":"2024-09-17T15:29:21.651023+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":647279345,"num_objects":158038,"num_object_clones":0,"num_object_copies":316076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":2908,"num_objects_unfound":0,"num_objects_dirty":158038,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":118145,"num_write_kb":472574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5274,"num_bytes_recovered":21590518,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158011,"ondisk_log_size":158011,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2088248,"kb_used_data":1285192,"kb_used_omap":13,"kb_used_meta":802994,"kb_avail":281027272,"statfs":{"total":289910292480,"available":287771926528,"internally_reserved":0,"allocated":1316036608,"data_stored":1315155376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":822266879},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,2,3,4,5,5,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24555520,"num_objects":5995,"num_object_clones":0,"num_object_copies":11990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-293,"num_objects_unfound":0,"num_objects_dirty":5995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5995,"num_write_kb":23980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":1200128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004455"},"pg_stats":[{"pgid":"2.f","version":"43'9830","reported_seq":9907,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:21.521279+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:29:21.521279+0000","last_peered":"2024-09-17T15:29:21.521279+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:29:21.521279+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9830,"log_dups_size":0,"ondisk_log_size":9830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40263680,"num_objects":9830,"num_object_clones":0,"num_object_copies":19660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9830,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9830,"num_write_kb":39320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5760},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9831","reported_seq":9933,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:17.222297+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:29:17.222297+0000","last_peered":"2024-09-17T15:29:17.222297+0000","last_clean":"2024-09-17T15:29:17.222297+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:29:17.222297+0000","last_undegraded":"2024-09-17T15:29:17.222297+0000","last_fullsized":"2024-09-17T15:29:17.222297+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":9831,"log_dups_size":0,"ondisk_log_size":9831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40244166,"num_objects":9827,"num_object_clones":0,"num_object_copies":19654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9827,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8188,"num_write_kb":32752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9835","reported_seq":9960,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:17.251506+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:29:17.251506+0000","last_peered":"2024-09-17T15:29:17.251506+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:29:17.251506+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":9835,"log_dups_size":0,"ondisk_log_size":9835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40267776,"num_objects":9831,"num_object_clones":0,"num_object_copies":19662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9831,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7992,"num_write_kb":31968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":5663}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9891","reported_seq":9960,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:17.304488+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:29:17.304488+0000","last_peered":"2024-09-17T15:29:17.304488+0000","last_clean":"2024-09-17T15:29:17.304488+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:29:17.304488+0000","last_undegraded":"2024-09-17T15:29:17.304488+0000","last_fullsized":"2024-09-17T15:29:17.304488+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9891,"log_dups_size":0,"ondisk_log_size":9891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40513536,"num_objects":9891,"num_object_clones":0,"num_object_copies":19782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9891,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9891,"num_write_kb":39564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9827","reported_seq":9956,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:21.586150+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:29:21.586150+0000","last_peered":"2024-09-17T15:29:21.586150+0000","last_clean":"2024-09-17T15:29:21.586150+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:29:21.586150+0000","last_undegraded":"2024-09-17T15:29:21.586150+0000","last_fullsized":"2024-09-17T15:29:21.586150+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":9827,"log_dups_size":0,"ondisk_log_size":9827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40223203,"num_objects":9821,"num_object_clones":0,"num_object_copies":19642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9821,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6809,"num_write_kb":27236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'10123","reported_seq":10257,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:21.361205+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:29:21.361205+0000","last_peered":"2024-09-17T15:29:21.361205+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:29:21.361205+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'100","ondisk_log_start":"34'100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41443811,"num_objects":10119,"num_object_clones":0,"num_object_copies":20238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10119,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6622,"num_write_kb":26488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5849},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9925","reported_seq":10063,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:21.545831+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:29:21.545831+0000","last_peered":"2024-09-17T15:29:21.545831+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:29:21.545831+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":9925,"log_dups_size":0,"ondisk_log_size":9925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40629190,"num_objects":9921,"num_object_clones":0,"num_object_copies":19842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9921,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6694,"num_write_kb":26776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5747},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9978","reported_seq":10105,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:21.507548+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:29:21.507548+0000","last_peered":"2024-09-17T15:29:21.507548+0000","last_clean":"2024-09-17T15:29:21.507548+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:29:21.507548+0000","last_undegraded":"2024-09-17T15:29:21.507548+0000","last_fullsized":"2024-09-17T15:29:21.507548+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":9978,"log_dups_size":0,"ondisk_log_size":9978,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40861696,"num_objects":9976,"num_object_clones":0,"num_object_copies":19952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6932,"num_write_kb":27728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'9906","reported_seq":18188,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:20.748186+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:29:20.748186+0000","last_peered":"2024-09-17T15:29:20.748186+0000","last_clean":"2024-09-17T15:29:20.748186+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:29:20.748186+0000","last_undegraded":"2024-09-17T15:29:20.748186+0000","last_fullsized":"2024-09-17T15:29:20.748186+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":9906,"log_dups_size":0,"ondisk_log_size":9906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40546787,"num_objects":9900,"num_object_clones":0,"num_object_copies":19800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9900,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7906,"num_write_kb":31624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804705+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804705+0000","last_peered":"2024-09-17T15:28:23.804705+0000","last_clean":"2024-09-17T15:28:23.804705+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804705+0000","last_undegraded":"2024-09-17T15:28:23.804705+0000","last_fullsized":"2024-09-17T15:28:23.804705+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9801","reported_seq":9931,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:21.569570+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:29:21.569570+0000","last_peered":"2024-09-17T15:29:21.569570+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:29:21.569570+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":9801,"log_dups_size":0,"ondisk_log_size":9801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40112128,"num_objects":9793,"num_object_clones":0,"num_object_copies":19586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6397,"num_write_kb":25588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5783},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9669","reported_seq":9789,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:17.302603+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:29:17.302603+0000","last_peered":"2024-09-17T15:29:17.302603+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:29:17.302603+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":9669,"log_dups_size":0,"ondisk_log_size":9669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39584227,"num_objects":9665,"num_object_clones":0,"num_object_copies":19330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9665,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7564,"num_write_kb":30256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":5588}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9774","reported_seq":9892,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:17.213960+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:29:17.213960+0000","last_peered":"2024-09-17T15:29:17.213960+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:29:17.213960+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":9774,"log_dups_size":0,"ondisk_log_size":9774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40009728,"num_objects":9768,"num_object_clones":0,"num_object_copies":19536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7536,"num_write_kb":30144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":5631}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'10002","reported_seq":10137,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:21.581622+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:29:21.581622+0000","last_peered":"2024-09-17T15:29:21.581622+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:29:21.581622+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40960000,"num_objects":10000,"num_object_clones":0,"num_object_copies":20000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10000,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6550,"num_write_kb":26200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5862},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9908","reported_seq":10049,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:21.547476+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:29:21.547476+0000","last_peered":"2024-09-17T15:29:21.547476+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:29:21.547476+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":9908,"log_dups_size":0,"ondisk_log_size":9908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40550400,"num_objects":9900,"num_object_clones":0,"num_object_copies":19800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6195,"num_write_kb":24780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5743},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805074+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805074+0000","last_peered":"2024-09-17T15:28:23.805074+0000","last_clean":"2024-09-17T15:28:23.805074+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805074+0000","last_undegraded":"2024-09-17T15:28:23.805074+0000","last_fullsized":"2024-09-17T15:28:23.805074+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9981","reported_seq":10110,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:21.475154+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:29:21.475154+0000","last_peered":"2024-09-17T15:29:21.475154+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:29:21.475154+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":9981,"log_dups_size":0,"ondisk_log_size":9981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40865792,"num_objects":9977,"num_object_clones":0,"num_object_copies":19954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6768,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5914},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9827","reported_seq":12372,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:29:21.605389+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:29:21.605389+0000","last_peered":"2024-09-17T15:29:21.605389+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:29:21.605389+0000","last_undegraded":"2024-09-17T15:29:21.605389+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":9827,"log_dups_size":0,"ondisk_log_size":9827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40203206,"num_objects":9817,"num_object_clones":0,"num_object_copies":19634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2908,"num_objects_unfound":0,"num_objects_dirty":9817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6269,"num_write_kb":25076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1201,"num_bytes_recovered":4919296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6909},{"shards":"0,2","objects":2908}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":647279326,"num_objects":158036,"num_object_clones":0,"num_object_copies":316072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":2908,"num_objects_unfound":0,"num_objects_dirty":158036,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":118143,"num_write_kb":472572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5273,"num_bytes_recovered":21590499,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1316720640,"data_stored":1316568894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158008,"ondisk_log_size":158008,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051635,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":720332,"kb_used_data":445692,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93651508,"statfs":{"total":96636764160,"available":95899144192,"internally_reserved":0,"allocated":456388608,"data_stored":456112034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":281210540},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,0,1,3,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724581,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523592,"kb_used_data":297528,"kb_used_omap":4,"kb_used_meta":226043,"kb_avail":93848248,"statfs":{"total":96636764160,"available":96100605952,"internally_reserved":0,"allocated":304668672,"data_stored":304344665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":231468715},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247864,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":844324,"kb_used_data":541972,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93527516,"statfs":{"total":96636764160,"available":95772176384,"internally_reserved":0,"allocated":554979328,"data_stored":554698677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":309587624},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,2,1,4,4,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":555401216,"data_stored":555365086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":305197056,"data_stored":305117570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":456122368,"data_stored":456086238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:29:23.329 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 224 20 159433 159413 2.77939 1.71875 0.0251954 0.0281007 2024-09-17T15:29:24.327 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 225 20 159894 159874 2.77505 1.80078 0.0404587 0.0281451 2024-09-17T15:29:25.327 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 226 20 160371 160351 2.77101 1.86328 0.0171186 0.0281864 2024-09-17T15:29:25.612 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:29:26.186 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:29:26.188 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:29:26.210 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":187,"stamp":"2024-09-17T15:29:25.652229+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650220273,"num_objects":158756,"num_object_clones":0,"num_object_copies":317512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":2908,"num_objects_unfound":0,"num_objects_dirty":158756,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":118863,"num_write_kb":475446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5274,"num_bytes_recovered":21590518,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158729,"ondisk_log_size":158729,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2080256,"kb_used_data":1289680,"kb_used_omap":13,"kb_used_meta":790514,"kb_avail":281035264,"statfs":{"total":289910292480,"available":287780110336,"internally_reserved":0,"allocated":1320632320,"data_stored":1319751088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":809487359},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,3,3,5,6,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18513920,"num_objects":4520,"num_object_clones":0,"num_object_copies":9040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-202,"num_objects_unfound":0,"num_objects_dirty":4520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4520,"num_write_kb":18080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":827392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004100"},"pg_stats":[{"pgid":"2.f","version":"43'9830","reported_seq":9907,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:21.521279+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:29:21.521279+0000","last_peered":"2024-09-17T15:29:21.521279+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:29:21.521279+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9830,"log_dups_size":0,"ondisk_log_size":9830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40263680,"num_objects":9830,"num_object_clones":0,"num_object_copies":19660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9830,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9830,"num_write_kb":39320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5760},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9971","reported_seq":10073,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:22.203102+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:29:22.203102+0000","last_peered":"2024-09-17T15:29:22.203102+0000","last_clean":"2024-09-17T15:29:22.203102+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:29:22.203102+0000","last_undegraded":"2024-09-17T15:29:22.203102+0000","last_fullsized":"2024-09-17T15:29:22.203102+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":9971,"log_dups_size":0,"ondisk_log_size":9971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40817606,"num_objects":9967,"num_object_clones":0,"num_object_copies":19934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9967,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8328,"num_write_kb":33312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9982","reported_seq":10107,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:22.220453+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:29:22.220453+0000","last_peered":"2024-09-17T15:29:22.220453+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:29:22.220453+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":9982,"log_dups_size":0,"ondisk_log_size":9982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40869888,"num_objects":9978,"num_object_clones":0,"num_object_copies":19956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8139,"num_write_kb":32556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":5810}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'10049","reported_seq":10118,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:22.300733+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:29:22.300733+0000","last_peered":"2024-09-17T15:29:22.300733+0000","last_clean":"2024-09-17T15:29:22.300733+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:29:22.300733+0000","last_undegraded":"2024-09-17T15:29:22.300733+0000","last_fullsized":"2024-09-17T15:29:22.300733+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41160704,"num_objects":10049,"num_object_clones":0,"num_object_copies":20098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10049,"num_write_kb":40196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9827","reported_seq":9956,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:21.586150+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:29:21.586150+0000","last_peered":"2024-09-17T15:29:21.586150+0000","last_clean":"2024-09-17T15:29:21.586150+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:29:21.586150+0000","last_undegraded":"2024-09-17T15:29:21.586150+0000","last_fullsized":"2024-09-17T15:29:21.586150+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":9827,"log_dups_size":0,"ondisk_log_size":9827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40223203,"num_objects":9821,"num_object_clones":0,"num_object_copies":19642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9821,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6809,"num_write_kb":27236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'10123","reported_seq":10257,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:21.361205+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:29:21.361205+0000","last_peered":"2024-09-17T15:29:21.361205+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:29:21.361205+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'100","ondisk_log_start":"34'100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41443811,"num_objects":10119,"num_object_clones":0,"num_object_copies":20238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10119,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6622,"num_write_kb":26488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5849},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9925","reported_seq":10063,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:21.545831+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:29:21.545831+0000","last_peered":"2024-09-17T15:29:21.545831+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:29:21.545831+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":9925,"log_dups_size":0,"ondisk_log_size":9925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40629190,"num_objects":9921,"num_object_clones":0,"num_object_copies":19842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9921,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6694,"num_write_kb":26776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5747},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9978","reported_seq":10105,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:21.507548+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:29:21.507548+0000","last_peered":"2024-09-17T15:29:21.507548+0000","last_clean":"2024-09-17T15:29:21.507548+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:29:21.507548+0000","last_undegraded":"2024-09-17T15:29:21.507548+0000","last_fullsized":"2024-09-17T15:29:21.507548+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":9978,"log_dups_size":0,"ondisk_log_size":9978,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40861696,"num_objects":9976,"num_object_clones":0,"num_object_copies":19952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6932,"num_write_kb":27728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'9906","reported_seq":18188,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:20.748186+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:29:20.748186+0000","last_peered":"2024-09-17T15:29:20.748186+0000","last_clean":"2024-09-17T15:29:20.748186+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:29:20.748186+0000","last_undegraded":"2024-09-17T15:29:20.748186+0000","last_fullsized":"2024-09-17T15:29:20.748186+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":9906,"log_dups_size":0,"ondisk_log_size":9906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40546787,"num_objects":9900,"num_object_clones":0,"num_object_copies":19800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9900,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7906,"num_write_kb":31624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804705+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804705+0000","last_peered":"2024-09-17T15:28:23.804705+0000","last_clean":"2024-09-17T15:28:23.804705+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804705+0000","last_undegraded":"2024-09-17T15:28:23.804705+0000","last_fullsized":"2024-09-17T15:28:23.804705+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9801","reported_seq":9931,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:21.569570+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:29:21.569570+0000","last_peered":"2024-09-17T15:29:21.569570+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:29:21.569570+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":9801,"log_dups_size":0,"ondisk_log_size":9801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40112128,"num_objects":9793,"num_object_clones":0,"num_object_copies":19586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6397,"num_write_kb":25588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5783},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9813","reported_seq":9933,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:22.340194+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:29:22.340194+0000","last_peered":"2024-09-17T15:29:22.340194+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:29:22.340194+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":9813,"log_dups_size":0,"ondisk_log_size":9813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40174051,"num_objects":9809,"num_object_clones":0,"num_object_copies":19618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9809,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7708,"num_write_kb":30832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":5732}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9903","reported_seq":10020,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:22.307847+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:29:22.307847+0000","last_peered":"2024-09-17T15:29:22.307847+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:29:22.307847+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":9903,"log_dups_size":0,"ondisk_log_size":9903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40538112,"num_objects":9897,"num_object_clones":0,"num_object_copies":19794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7665,"num_write_kb":30660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":5760}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'10002","reported_seq":10137,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:21.581622+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:29:21.581622+0000","last_peered":"2024-09-17T15:29:21.581622+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:29:21.581622+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40960000,"num_objects":10000,"num_object_clones":0,"num_object_copies":20000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10000,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6550,"num_write_kb":26200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5862},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9908","reported_seq":10049,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:21.547476+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:29:21.547476+0000","last_peered":"2024-09-17T15:29:21.547476+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:29:21.547476+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":9908,"log_dups_size":0,"ondisk_log_size":9908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40550400,"num_objects":9900,"num_object_clones":0,"num_object_copies":19800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6195,"num_write_kb":24780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5743},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805074+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805074+0000","last_peered":"2024-09-17T15:28:23.805074+0000","last_clean":"2024-09-17T15:28:23.805074+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805074+0000","last_undegraded":"2024-09-17T15:28:23.805074+0000","last_fullsized":"2024-09-17T15:28:23.805074+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9981","reported_seq":10110,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:21.475154+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:29:21.475154+0000","last_peered":"2024-09-17T15:29:21.475154+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:29:21.475154+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":9981,"log_dups_size":0,"ondisk_log_size":9981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40865792,"num_objects":9977,"num_object_clones":0,"num_object_copies":19954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6768,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5914},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9827","reported_seq":12372,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:29:21.605389+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:29:21.605389+0000","last_peered":"2024-09-17T15:29:21.605389+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:29:21.605389+0000","last_undegraded":"2024-09-17T15:29:21.605389+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":9827,"log_dups_size":0,"ondisk_log_size":9827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40203206,"num_objects":9817,"num_object_clones":0,"num_object_copies":19634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2908,"num_objects_unfound":0,"num_objects_dirty":9817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6269,"num_write_kb":25076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1201,"num_bytes_recovered":4919296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6909},{"shards":"0,2","objects":2908}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":650220254,"num_objects":158754,"num_object_clones":0,"num_object_copies":317508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":2908,"num_objects_unfound":0,"num_objects_dirty":158754,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":118861,"num_write_kb":475444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5273,"num_bytes_recovered":21590499,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1321287680,"data_stored":1321135934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158726,"ondisk_log_size":158726,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051636,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":712340,"kb_used_data":450180,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93659500,"statfs":{"total":96636764160,"available":95907328000,"internally_reserved":0,"allocated":460984320,"data_stored":460707746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":268431020},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724581,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523592,"kb_used_data":297528,"kb_used_omap":4,"kb_used_meta":226043,"kb_avail":93848248,"statfs":{"total":96636764160,"available":96100605952,"internally_reserved":0,"allocated":304668672,"data_stored":304344665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":231468715},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247864,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":844324,"kb_used_data":541972,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93527516,"statfs":{"total":96636764160,"available":95772176384,"internally_reserved":0,"allocated":554979328,"data_stored":554698677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":309587624},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,2,1,4,4,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":555401216,"data_stored":555365086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":305197056,"data_stored":305117570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":460689408,"data_stored":460653278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:29:26.211 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:29:26.333 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 227 20 160801 160781 2.76617 1.67969 0.043515 0.0282356 2024-09-17T15:29:26.756 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:29:26.758 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:29:26.778 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":187,"stamp":"2024-09-17T15:29:25.652229+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650220273,"num_objects":158756,"num_object_clones":0,"num_object_copies":317512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":2908,"num_objects_unfound":0,"num_objects_dirty":158756,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":118863,"num_write_kb":475446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5274,"num_bytes_recovered":21590518,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158729,"ondisk_log_size":158729,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2080256,"kb_used_data":1289680,"kb_used_omap":13,"kb_used_meta":790514,"kb_avail":281035264,"statfs":{"total":289910292480,"available":287780110336,"internally_reserved":0,"allocated":1320632320,"data_stored":1319751088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":809487359},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,3,3,5,6,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18513920,"num_objects":4520,"num_object_clones":0,"num_object_copies":9040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-202,"num_objects_unfound":0,"num_objects_dirty":4520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4520,"num_write_kb":18080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":827392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004100"},"pg_stats":[{"pgid":"2.f","version":"43'9830","reported_seq":9907,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:21.521279+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:29:21.521279+0000","last_peered":"2024-09-17T15:29:21.521279+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:29:21.521279+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9830,"log_dups_size":0,"ondisk_log_size":9830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40263680,"num_objects":9830,"num_object_clones":0,"num_object_copies":19660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9830,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9830,"num_write_kb":39320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5760},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9971","reported_seq":10073,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:22.203102+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:29:22.203102+0000","last_peered":"2024-09-17T15:29:22.203102+0000","last_clean":"2024-09-17T15:29:22.203102+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:29:22.203102+0000","last_undegraded":"2024-09-17T15:29:22.203102+0000","last_fullsized":"2024-09-17T15:29:22.203102+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":9971,"log_dups_size":0,"ondisk_log_size":9971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40817606,"num_objects":9967,"num_object_clones":0,"num_object_copies":19934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9967,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8328,"num_write_kb":33312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9982","reported_seq":10107,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:22.220453+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:29:22.220453+0000","last_peered":"2024-09-17T15:29:22.220453+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:29:22.220453+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":9982,"log_dups_size":0,"ondisk_log_size":9982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40869888,"num_objects":9978,"num_object_clones":0,"num_object_copies":19956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8139,"num_write_kb":32556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":5810}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'10049","reported_seq":10118,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:22.300733+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:29:22.300733+0000","last_peered":"2024-09-17T15:29:22.300733+0000","last_clean":"2024-09-17T15:29:22.300733+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:29:22.300733+0000","last_undegraded":"2024-09-17T15:29:22.300733+0000","last_fullsized":"2024-09-17T15:29:22.300733+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41160704,"num_objects":10049,"num_object_clones":0,"num_object_copies":20098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10049,"num_write_kb":40196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9827","reported_seq":9956,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:21.586150+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:29:21.586150+0000","last_peered":"2024-09-17T15:29:21.586150+0000","last_clean":"2024-09-17T15:29:21.586150+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:29:21.586150+0000","last_undegraded":"2024-09-17T15:29:21.586150+0000","last_fullsized":"2024-09-17T15:29:21.586150+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":9827,"log_dups_size":0,"ondisk_log_size":9827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40223203,"num_objects":9821,"num_object_clones":0,"num_object_copies":19642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9821,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6809,"num_write_kb":27236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'10123","reported_seq":10257,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:21.361205+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:29:21.361205+0000","last_peered":"2024-09-17T15:29:21.361205+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:29:21.361205+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'100","ondisk_log_start":"34'100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41443811,"num_objects":10119,"num_object_clones":0,"num_object_copies":20238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10119,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6622,"num_write_kb":26488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5849},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9925","reported_seq":10063,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:21.545831+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:29:21.545831+0000","last_peered":"2024-09-17T15:29:21.545831+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:29:21.545831+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":9925,"log_dups_size":0,"ondisk_log_size":9925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40629190,"num_objects":9921,"num_object_clones":0,"num_object_copies":19842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9921,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6694,"num_write_kb":26776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5747},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9978","reported_seq":10105,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:21.507548+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:29:21.507548+0000","last_peered":"2024-09-17T15:29:21.507548+0000","last_clean":"2024-09-17T15:29:21.507548+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:29:21.507548+0000","last_undegraded":"2024-09-17T15:29:21.507548+0000","last_fullsized":"2024-09-17T15:29:21.507548+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":9978,"log_dups_size":0,"ondisk_log_size":9978,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40861696,"num_objects":9976,"num_object_clones":0,"num_object_copies":19952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6932,"num_write_kb":27728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'9906","reported_seq":18188,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:20.748186+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:29:20.748186+0000","last_peered":"2024-09-17T15:29:20.748186+0000","last_clean":"2024-09-17T15:29:20.748186+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:29:20.748186+0000","last_undegraded":"2024-09-17T15:29:20.748186+0000","last_fullsized":"2024-09-17T15:29:20.748186+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":9906,"log_dups_size":0,"ondisk_log_size":9906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40546787,"num_objects":9900,"num_object_clones":0,"num_object_copies":19800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9900,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7906,"num_write_kb":31624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804705+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804705+0000","last_peered":"2024-09-17T15:28:23.804705+0000","last_clean":"2024-09-17T15:28:23.804705+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804705+0000","last_undegraded":"2024-09-17T15:28:23.804705+0000","last_fullsized":"2024-09-17T15:28:23.804705+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9801","reported_seq":9931,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:21.569570+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:29:21.569570+0000","last_peered":"2024-09-17T15:29:21.569570+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:29:21.569570+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":9801,"log_dups_size":0,"ondisk_log_size":9801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40112128,"num_objects":9793,"num_object_clones":0,"num_object_copies":19586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6397,"num_write_kb":25588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5783},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9813","reported_seq":9933,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:22.340194+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:29:22.340194+0000","last_peered":"2024-09-17T15:29:22.340194+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:29:22.340194+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":9813,"log_dups_size":0,"ondisk_log_size":9813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40174051,"num_objects":9809,"num_object_clones":0,"num_object_copies":19618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9809,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7708,"num_write_kb":30832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":5732}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9903","reported_seq":10020,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:22.307847+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:29:22.307847+0000","last_peered":"2024-09-17T15:29:22.307847+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:29:22.307847+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":9903,"log_dups_size":0,"ondisk_log_size":9903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40538112,"num_objects":9897,"num_object_clones":0,"num_object_copies":19794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7665,"num_write_kb":30660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":5760}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'10002","reported_seq":10137,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:21.581622+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:29:21.581622+0000","last_peered":"2024-09-17T15:29:21.581622+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:29:21.581622+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40960000,"num_objects":10000,"num_object_clones":0,"num_object_copies":20000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10000,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6550,"num_write_kb":26200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5862},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9908","reported_seq":10049,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:21.547476+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:29:21.547476+0000","last_peered":"2024-09-17T15:29:21.547476+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:29:21.547476+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":9908,"log_dups_size":0,"ondisk_log_size":9908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40550400,"num_objects":9900,"num_object_clones":0,"num_object_copies":19800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6195,"num_write_kb":24780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5743},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805074+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805074+0000","last_peered":"2024-09-17T15:28:23.805074+0000","last_clean":"2024-09-17T15:28:23.805074+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805074+0000","last_undegraded":"2024-09-17T15:28:23.805074+0000","last_fullsized":"2024-09-17T15:28:23.805074+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9981","reported_seq":10110,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:21.475154+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:29:21.475154+0000","last_peered":"2024-09-17T15:29:21.475154+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:29:21.475154+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":9981,"log_dups_size":0,"ondisk_log_size":9981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40865792,"num_objects":9977,"num_object_clones":0,"num_object_copies":19954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6768,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5914},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9827","reported_seq":12372,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:29:21.605389+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:29:21.605389+0000","last_peered":"2024-09-17T15:29:21.605389+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:29:21.605389+0000","last_undegraded":"2024-09-17T15:29:21.605389+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":9827,"log_dups_size":0,"ondisk_log_size":9827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40203206,"num_objects":9817,"num_object_clones":0,"num_object_copies":19634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2908,"num_objects_unfound":0,"num_objects_dirty":9817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6269,"num_write_kb":25076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1201,"num_bytes_recovered":4919296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6909},{"shards":"0,2","objects":2908}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":650220254,"num_objects":158754,"num_object_clones":0,"num_object_copies":317508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":2908,"num_objects_unfound":0,"num_objects_dirty":158754,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":118861,"num_write_kb":475444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5273,"num_bytes_recovered":21590499,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1321287680,"data_stored":1321135934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158726,"ondisk_log_size":158726,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051636,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":712340,"kb_used_data":450180,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93659500,"statfs":{"total":96636764160,"available":95907328000,"internally_reserved":0,"allocated":460984320,"data_stored":460707746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":268431020},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724581,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523592,"kb_used_data":297528,"kb_used_omap":4,"kb_used_meta":226043,"kb_avail":93848248,"statfs":{"total":96636764160,"available":96100605952,"internally_reserved":0,"allocated":304668672,"data_stored":304344665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":231468715},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247864,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":844324,"kb_used_data":541972,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93527516,"statfs":{"total":96636764160,"available":95772176384,"internally_reserved":0,"allocated":554979328,"data_stored":554698677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":309587624},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,2,1,4,4,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":555401216,"data_stored":555365086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":305197056,"data_stored":305117570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":460689408,"data_stored":460653278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:29:26.779 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:29:26.779 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'9830", 'reported_seq': 9907, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:21.521279+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:29:21.521279+0000', 'last_peered': '2024-09-17T15:29:21.521279+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:29:21.521279+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 9830, 'log_dups_size': 0, 'ondisk_log_size': 9830, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40263680, 'num_objects': 9830, 'num_object_clones': 0, 'num_object_copies': 19660, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4070, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9830, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9830, 'num_write_kb': 39320, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5760}, {'shards': '0,2', 'objects': 4070}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:26.779 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:29:26.779 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'9982", 'reported_seq': 10107, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:22.220453+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:29:22.220453+0000', 'last_peered': '2024-09-17T15:29:22.220453+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:29:22.220453+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 9982, 'log_dups_size': 0, 'ondisk_log_size': 9982, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40869888, 'num_objects': 9978, 'num_object_clones': 0, 'num_object_copies': 19956, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4169, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9978, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8139, 'num_write_kb': 32556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4168}, {'shards': '2', 'objects': 5810}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:29:26.780 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:29:26.780 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'10123", 'reported_seq': 10257, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:21.361205+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:29:21.361205+0000', 'last_peered': '2024-09-17T15:29:21.361205+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:29:21.361205+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "34'100", 'ondisk_log_start': "34'100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41443811, 'num_objects': 10119, 'num_object_clones': 0, 'num_object_copies': 20238, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4271, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10119, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6622, 'num_write_kb': 26488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5849}, {'shards': '0,2', 'objects': 4268}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:26.780 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:29:26.780 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'9925", 'reported_seq': 10063, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:21.545831+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:29:21.545831+0000', 'last_peered': '2024-09-17T15:29:21.545831+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:29:21.545831+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 9925, 'log_dups_size': 0, 'ondisk_log_size': 9925, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40629190, 'num_objects': 9921, 'num_object_clones': 0, 'num_object_copies': 19842, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4174, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9921, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 6694, 'num_write_kb': 26776, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5747}, {'shards': '0,2', 'objects': 4174}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:26.780 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:29:26.780 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'9801", 'reported_seq': 9931, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:21.569570+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:29:21.569570+0000', 'last_peered': '2024-09-17T15:29:21.569570+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:29:21.569570+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 9801, 'log_dups_size': 0, 'ondisk_log_size': 9801, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40112128, 'num_objects': 9793, 'num_object_clones': 0, 'num_object_copies': 19586, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4012, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9793, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6397, 'num_write_kb': 25588, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5783}, {'shards': '0,2', 'objects': 4010}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:26.780 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:29:26.780 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'9813", 'reported_seq': 9933, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:22.340194+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:29:22.340194+0000', 'last_peered': '2024-09-17T15:29:22.340194+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:29:22.340194+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 9813, 'log_dups_size': 0, 'ondisk_log_size': 9813, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40174051, 'num_objects': 9809, 'num_object_clones': 0, 'num_object_copies': 19618, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4078, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9809, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7708, 'num_write_kb': 30832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4077}, {'shards': '2', 'objects': 5732}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:29:26.780 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:29:26.781 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'9903", 'reported_seq': 10020, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:22.307847+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:29:22.307847+0000', 'last_peered': '2024-09-17T15:29:22.307847+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:29:22.307847+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 9903, 'log_dups_size': 0, 'ondisk_log_size': 9903, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40538112, 'num_objects': 9897, 'num_object_clones': 0, 'num_object_copies': 19794, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9897, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7665, 'num_write_kb': 30660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4137}, {'shards': '2', 'objects': 5760}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:29:26.781 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:29:26.781 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'10002", 'reported_seq': 10137, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:21.581622+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:29:21.581622+0000', 'last_peered': '2024-09-17T15:29:21.581622+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:29:21.581622+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40960000, 'num_objects': 10000, 'num_object_clones': 0, 'num_object_copies': 20000, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10000, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6550, 'num_write_kb': 26200, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5862}, {'shards': '0,2', 'objects': 4138}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:26.781 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:29:26.781 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'9908", 'reported_seq': 10049, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:21.547476+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:29:21.547476+0000', 'last_peered': '2024-09-17T15:29:21.547476+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:29:21.547476+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 9908, 'log_dups_size': 0, 'ondisk_log_size': 9908, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40550400, 'num_objects': 9900, 'num_object_clones': 0, 'num_object_copies': 19800, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4160, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9900, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6195, 'num_write_kb': 24780, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5743}, {'shards': '0,2', 'objects': 4157}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:26.781 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:29:26.781 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'9981", 'reported_seq': 10110, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:21.475154+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:29:21.475154+0000', 'last_peered': '2024-09-17T15:29:21.475154+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:29:21.475154+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 9981, 'log_dups_size': 0, 'ondisk_log_size': 9981, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40865792, 'num_objects': 9977, 'num_object_clones': 0, 'num_object_copies': 19954, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4065, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9977, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6768, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5914}, {'shards': '0,2', 'objects': 4063}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:26.781 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:29:26.782 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'9827", 'reported_seq': 12372, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:29:21.605389+0000', 'last_change': '2024-09-17T15:28:22.509818+0000', 'last_active': '2024-09-17T15:29:21.605389+0000', 'last_peered': '2024-09-17T15:29:21.605389+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:29:21.605389+0000', 'last_undegraded': '2024-09-17T15:29:21.605389+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 9827, 'log_dups_size': 0, 'ondisk_log_size': 9827, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40203206, 'num_objects': 9817, 'num_object_clones': 0, 'num_object_copies': 19634, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2908, 'num_objects_unfound': 0, 'num_objects_dirty': 9817, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6269, 'num_write_kb': 25076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1201, 'num_bytes_recovered': 4919296, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6909}, {'shards': '0,2', 'objects': 2908}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:26.782 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:29:27.330 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 228 20 161206 161186 2.76098 1.58203 0.0713558 0.028289 2024-09-17T15:29:27.457 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:29:27.457 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41277/317512 objects degraded (13.000%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":304,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":158756,"data_bytes":650220273,"bytes_used":2130182144,"bytes_avail":287780110336,"bytes_total":289910292480,"degraded_objects":41277,"degraded_total":317512,"degraded_ratio":0.13000138577439593,"misplaced_objects":2908,"misplaced_total":317512,"misplaced_ratio":0.0091587089621809562,"recovering_objects_per_sec":16,"recovering_bytes_per_sec":68925,"recovering_keys_per_sec":0,"num_objects_recovered":202,"num_bytes_recovered":827392,"num_keys_recovered":0,"write_bytes_sec":1542299,"read_op_per_sec":0,"write_op_per_sec":376},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T15:27:19.616376+0000","services":{}},"progress_events":{}} 2024-09-17T15:29:27.489 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:29:27.489 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:29:28.032 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:29:28.034 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:29:28.061 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":188,"stamp":"2024-09-17T15:29:27.653037+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659423985,"num_objects":161003,"num_object_clones":0,"num_object_copies":322006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":2810,"num_objects_unfound":0,"num_objects_dirty":161003,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":121110,"num_write_kb":484434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5372,"num_bytes_recovered":21991926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160276,"ondisk_log_size":160276,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2110636,"kb_used_data":1307580,"kb_used_omap":13,"kb_used_meta":802994,"kb_avail":281004884,"statfs":{"total":289910292480,"available":287749001216,"internally_reserved":0,"allocated":1338961920,"data_stored":1338080688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":822266879},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,3,3,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27717632,"num_objects":6767,"num_object_clones":0,"num_object_copies":13534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-300,"num_objects_unfound":0,"num_objects_dirty":6767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6767,"num_write_kb":27068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":1228800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004062"},"pg_stats":[{"pgid":"2.f","version":"43'9961","reported_seq":10038,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:26.667443+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:29:26.667443+0000","last_peered":"2024-09-17T15:29:26.667443+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:29:26.667443+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9961,"log_dups_size":0,"ondisk_log_size":9961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40800256,"num_objects":9961,"num_object_clones":0,"num_object_copies":19922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9961,"num_write_kb":39844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5891},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'10116","reported_seq":10218,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:27.324301+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:29:27.324301+0000","last_peered":"2024-09-17T15:29:27.324301+0000","last_clean":"2024-09-17T15:29:27.324301+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:29:27.324301+0000","last_undegraded":"2024-09-17T15:29:27.324301+0000","last_fullsized":"2024-09-17T15:29:27.324301+0000","mapping_epoch":33,"log_start":"34'100","ondisk_log_start":"34'100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41411526,"num_objects":10112,"num_object_clones":0,"num_object_copies":20224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10112,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8473,"num_write_kb":33892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'10129","reported_seq":10254,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:27.285240+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:29:27.285240+0000","last_peered":"2024-09-17T15:29:27.285240+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:29:27.285240+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'100","ondisk_log_start":"34'100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41472000,"num_objects":10125,"num_object_clones":0,"num_object_copies":20250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10125,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8286,"num_write_kb":33144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":5957}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'10173","reported_seq":10242,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:27.134539+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:29:27.134539+0000","last_peered":"2024-09-17T15:29:27.134539+0000","last_clean":"2024-09-17T15:29:27.134539+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:29:27.134539+0000","last_undegraded":"2024-09-17T15:29:27.134539+0000","last_fullsized":"2024-09-17T15:29:27.134539+0000","mapping_epoch":33,"log_start":"34'100","ondisk_log_start":"34'100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41668608,"num_objects":10173,"num_object_clones":0,"num_object_copies":20346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10173,"num_write_kb":40692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9977","reported_seq":10106,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:26.580177+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:29:26.580177+0000","last_peered":"2024-09-17T15:29:26.580177+0000","last_clean":"2024-09-17T15:29:26.580177+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:29:26.580177+0000","last_undegraded":"2024-09-17T15:29:26.580177+0000","last_fullsized":"2024-09-17T15:29:26.580177+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":9977,"log_dups_size":0,"ondisk_log_size":9977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40837603,"num_objects":9971,"num_object_clones":0,"num_object_copies":19942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9971,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6959,"num_write_kb":27836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'10264","reported_seq":10397,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:26.624360+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:29:26.624360+0000","last_peered":"2024-09-17T15:29:26.624360+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:29:26.624360+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'200","ondisk_log_start":"34'200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42021347,"num_objects":10260,"num_object_clones":0,"num_object_copies":20520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10260,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6763,"num_write_kb":27052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5990},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'10057","reported_seq":10195,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:26.534111+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:29:26.534111+0000","last_peered":"2024-09-17T15:29:26.534111+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:29:26.534111+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41169862,"num_objects":10053,"num_object_clones":0,"num_object_copies":20106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10053,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6826,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5879},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'10129","reported_seq":10256,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:26.637325+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:29:26.637325+0000","last_peered":"2024-09-17T15:29:26.637325+0000","last_clean":"2024-09-17T15:29:26.637325+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:29:26.637325+0000","last_undegraded":"2024-09-17T15:29:26.637325+0000","last_fullsized":"2024-09-17T15:29:26.637325+0000","mapping_epoch":32,"log_start":"34'100","ondisk_log_start":"34'100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41480192,"num_objects":10127,"num_object_clones":0,"num_object_copies":20254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7083,"num_write_kb":28332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'10042","reported_seq":18324,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:25.860591+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:29:25.860591+0000","last_peered":"2024-09-17T15:29:25.860591+0000","last_clean":"2024-09-17T15:29:25.860591+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:29:25.860591+0000","last_undegraded":"2024-09-17T15:29:25.860591+0000","last_fullsized":"2024-09-17T15:29:25.860591+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41103843,"num_objects":10036,"num_object_clones":0,"num_object_copies":20072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10036,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8042,"num_write_kb":32168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804705+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804705+0000","last_peered":"2024-09-17T15:28:23.804705+0000","last_clean":"2024-09-17T15:28:23.804705+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804705+0000","last_undegraded":"2024-09-17T15:28:23.804705+0000","last_fullsized":"2024-09-17T15:28:23.804705+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9935","reported_seq":10065,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:26.542912+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:29:26.542912+0000","last_peered":"2024-09-17T15:29:26.542912+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:29:26.542912+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":9935,"log_dups_size":0,"ondisk_log_size":9935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40660992,"num_objects":9927,"num_object_clones":0,"num_object_copies":19854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6531,"num_write_kb":26124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5917},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9942","reported_seq":10062,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:27.268905+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:29:27.268905+0000","last_peered":"2024-09-17T15:29:27.268905+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:29:27.268905+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":9942,"log_dups_size":0,"ondisk_log_size":9942,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40702435,"num_objects":9938,"num_object_clones":0,"num_object_copies":19876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9938,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7837,"num_write_kb":31348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":5861}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'10044","reported_seq":10163,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:27.349810+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:29:27.349810+0000","last_peered":"2024-09-17T15:29:27.349810+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:29:27.349810+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41115648,"num_objects":10038,"num_object_clones":0,"num_object_copies":20076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7806,"num_write_kb":31224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":5901}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'10145","reported_seq":10279,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:26.671652+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:29:26.671652+0000","last_peered":"2024-09-17T15:29:26.671652+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:29:26.671652+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'100","ondisk_log_start":"34'100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41545728,"num_objects":10143,"num_object_clones":0,"num_object_copies":20286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10143,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6693,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6005},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'10062","reported_seq":10202,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:26.673955+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:29:26.673955+0000","last_peered":"2024-09-17T15:29:26.673955+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:29:26.673955+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41181184,"num_objects":10054,"num_object_clones":0,"num_object_copies":20108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10054,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6349,"num_write_kb":25396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5897},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805074+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805074+0000","last_peered":"2024-09-17T15:28:23.805074+0000","last_clean":"2024-09-17T15:28:23.805074+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805074+0000","last_undegraded":"2024-09-17T15:28:23.805074+0000","last_fullsized":"2024-09-17T15:28:23.805074+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'10113","reported_seq":10243,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:26.581818+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:29:26.581818+0000","last_peered":"2024-09-17T15:29:26.581818+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:29:26.581818+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'100","ondisk_log_start":"34'100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41406464,"num_objects":10109,"num_object_clones":0,"num_object_copies":20218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6900,"num_write_kb":27600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6046},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9984","reported_seq":12726,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:29:26.651073+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:29:26.651073+0000","last_peered":"2024-09-17T15:29:26.651073+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:29:26.651073+0000","last_undegraded":"2024-09-17T15:29:26.651073+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":9984,"log_dups_size":0,"ondisk_log_size":9984,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40846278,"num_objects":9974,"num_object_clones":0,"num_object_copies":19948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2810,"num_objects_unfound":0,"num_objects_dirty":9974,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6426,"num_write_kb":25704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1299,"num_bytes_recovered":5320704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7164},{"shards":"0,2","objects":2810}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":659423966,"num_objects":161001,"num_object_clones":0,"num_object_copies":322002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":2810,"num_objects_unfound":0,"num_objects_dirty":161001,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":121108,"num_write_kb":484432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5371,"num_bytes_recovered":21991907,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1340055552,"data_stored":1339903806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160273,"ondisk_log_size":160273,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051637,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":720952,"kb_used_data":454632,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93650888,"statfs":{"total":96636764160,"available":95898509312,"internally_reserved":0,"allocated":465543168,"data_stored":465266594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":272690860},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724582,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":534564,"kb_used_data":304340,"kb_used_omap":4,"kb_used_meta":230203,"kb_avail":93837276,"statfs":{"total":96636764160,"available":96089370624,"internally_reserved":0,"allocated":311644160,"data_stored":311320153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":235728555},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247865,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":855120,"kb_used_data":548608,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93516720,"statfs":{"total":96636764160,"available":95761121280,"internally_reserved":0,"allocated":561774592,"data_stored":561493941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":313847464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":562356224,"data_stored":562320094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":312467456,"data_stored":312387970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":465231872,"data_stored":465195742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:29:28.331 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 229 20 161623 161603 2.75603 1.62891 0.0300714 0.0283399 2024-09-17T15:29:29.331 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 230 20 162096 162076 2.75208 1.84766 0.0372298 0.0283803 2024-09-17T15:29:30.331 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 231 20 162565 162545 2.74809 1.83203 0.0641508 0.0284219 2024-09-17T15:29:31.062 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:29:31.331 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 232 20 163022 163002 2.74394 1.78516 0.0293061 0.0284649 2024-09-17T15:29:31.623 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:29:31.623 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:29:31.645 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":189,"stamp":"2024-09-17T15:29:29.653923+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659423985,"num_objects":161003,"num_object_clones":0,"num_object_copies":322006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":2810,"num_objects_unfound":0,"num_objects_dirty":161003,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":121110,"num_write_kb":484434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5372,"num_bytes_recovered":21991926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160276,"ondisk_log_size":160276,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2110636,"kb_used_data":1307580,"kb_used_omap":13,"kb_used_meta":802994,"kb_avail":281004884,"statfs":{"total":289910292480,"available":287749001216,"internally_reserved":0,"allocated":1338961920,"data_stored":1338080688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":822266879},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,3,3,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18423808,"num_objects":4498,"num_object_clones":0,"num_object_copies":8996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-203,"num_objects_unfound":0,"num_objects_dirty":4498,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4498,"num_write_kb":17992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":831488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004127"},"pg_stats":[{"pgid":"2.f","version":"43'9961","reported_seq":10038,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:26.667443+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:29:26.667443+0000","last_peered":"2024-09-17T15:29:26.667443+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:29:26.667443+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9961,"log_dups_size":0,"ondisk_log_size":9961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40800256,"num_objects":9961,"num_object_clones":0,"num_object_copies":19922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9961,"num_write_kb":39844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5891},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'10116","reported_seq":10218,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:27.324301+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:29:27.324301+0000","last_peered":"2024-09-17T15:29:27.324301+0000","last_clean":"2024-09-17T15:29:27.324301+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:29:27.324301+0000","last_undegraded":"2024-09-17T15:29:27.324301+0000","last_fullsized":"2024-09-17T15:29:27.324301+0000","mapping_epoch":33,"log_start":"34'100","ondisk_log_start":"34'100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41411526,"num_objects":10112,"num_object_clones":0,"num_object_copies":20224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10112,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8473,"num_write_kb":33892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'10129","reported_seq":10254,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:27.285240+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:29:27.285240+0000","last_peered":"2024-09-17T15:29:27.285240+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:29:27.285240+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'100","ondisk_log_start":"34'100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41472000,"num_objects":10125,"num_object_clones":0,"num_object_copies":20250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10125,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8286,"num_write_kb":33144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":5957}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'10173","reported_seq":10242,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:27.134539+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:29:27.134539+0000","last_peered":"2024-09-17T15:29:27.134539+0000","last_clean":"2024-09-17T15:29:27.134539+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:29:27.134539+0000","last_undegraded":"2024-09-17T15:29:27.134539+0000","last_fullsized":"2024-09-17T15:29:27.134539+0000","mapping_epoch":33,"log_start":"34'100","ondisk_log_start":"34'100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41668608,"num_objects":10173,"num_object_clones":0,"num_object_copies":20346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10173,"num_write_kb":40692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9977","reported_seq":10106,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:26.580177+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:29:26.580177+0000","last_peered":"2024-09-17T15:29:26.580177+0000","last_clean":"2024-09-17T15:29:26.580177+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:29:26.580177+0000","last_undegraded":"2024-09-17T15:29:26.580177+0000","last_fullsized":"2024-09-17T15:29:26.580177+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":9977,"log_dups_size":0,"ondisk_log_size":9977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40837603,"num_objects":9971,"num_object_clones":0,"num_object_copies":19942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9971,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6959,"num_write_kb":27836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'10264","reported_seq":10397,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:26.624360+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:29:26.624360+0000","last_peered":"2024-09-17T15:29:26.624360+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:29:26.624360+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'200","ondisk_log_start":"34'200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42021347,"num_objects":10260,"num_object_clones":0,"num_object_copies":20520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10260,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6763,"num_write_kb":27052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5990},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'10057","reported_seq":10195,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:26.534111+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:29:26.534111+0000","last_peered":"2024-09-17T15:29:26.534111+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:29:26.534111+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41169862,"num_objects":10053,"num_object_clones":0,"num_object_copies":20106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10053,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6826,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5879},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'10129","reported_seq":10256,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:26.637325+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:29:26.637325+0000","last_peered":"2024-09-17T15:29:26.637325+0000","last_clean":"2024-09-17T15:29:26.637325+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:29:26.637325+0000","last_undegraded":"2024-09-17T15:29:26.637325+0000","last_fullsized":"2024-09-17T15:29:26.637325+0000","mapping_epoch":32,"log_start":"34'100","ondisk_log_start":"34'100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41480192,"num_objects":10127,"num_object_clones":0,"num_object_copies":20254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7083,"num_write_kb":28332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'10042","reported_seq":18324,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:25.860591+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:29:25.860591+0000","last_peered":"2024-09-17T15:29:25.860591+0000","last_clean":"2024-09-17T15:29:25.860591+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:29:25.860591+0000","last_undegraded":"2024-09-17T15:29:25.860591+0000","last_fullsized":"2024-09-17T15:29:25.860591+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41103843,"num_objects":10036,"num_object_clones":0,"num_object_copies":20072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10036,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8042,"num_write_kb":32168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804705+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804705+0000","last_peered":"2024-09-17T15:28:23.804705+0000","last_clean":"2024-09-17T15:28:23.804705+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804705+0000","last_undegraded":"2024-09-17T15:28:23.804705+0000","last_fullsized":"2024-09-17T15:28:23.804705+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9935","reported_seq":10065,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:26.542912+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:29:26.542912+0000","last_peered":"2024-09-17T15:29:26.542912+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:29:26.542912+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":9935,"log_dups_size":0,"ondisk_log_size":9935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40660992,"num_objects":9927,"num_object_clones":0,"num_object_copies":19854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6531,"num_write_kb":26124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5917},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9942","reported_seq":10062,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:27.268905+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:29:27.268905+0000","last_peered":"2024-09-17T15:29:27.268905+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:29:27.268905+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":9942,"log_dups_size":0,"ondisk_log_size":9942,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40702435,"num_objects":9938,"num_object_clones":0,"num_object_copies":19876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9938,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7837,"num_write_kb":31348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":5861}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'10044","reported_seq":10163,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:27.349810+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:29:27.349810+0000","last_peered":"2024-09-17T15:29:27.349810+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:29:27.349810+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41115648,"num_objects":10038,"num_object_clones":0,"num_object_copies":20076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7806,"num_write_kb":31224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":5901}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'10145","reported_seq":10279,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:26.671652+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:29:26.671652+0000","last_peered":"2024-09-17T15:29:26.671652+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:29:26.671652+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'100","ondisk_log_start":"34'100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41545728,"num_objects":10143,"num_object_clones":0,"num_object_copies":20286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10143,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6693,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6005},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'10062","reported_seq":10202,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:26.673955+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:29:26.673955+0000","last_peered":"2024-09-17T15:29:26.673955+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:29:26.673955+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41181184,"num_objects":10054,"num_object_clones":0,"num_object_copies":20108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10054,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6349,"num_write_kb":25396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5897},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805074+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805074+0000","last_peered":"2024-09-17T15:28:23.805074+0000","last_clean":"2024-09-17T15:28:23.805074+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805074+0000","last_undegraded":"2024-09-17T15:28:23.805074+0000","last_fullsized":"2024-09-17T15:28:23.805074+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'10113","reported_seq":10243,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:26.581818+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:29:26.581818+0000","last_peered":"2024-09-17T15:29:26.581818+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:29:26.581818+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'100","ondisk_log_start":"34'100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41406464,"num_objects":10109,"num_object_clones":0,"num_object_copies":20218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6900,"num_write_kb":27600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6046},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9984","reported_seq":12726,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:29:26.651073+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:29:26.651073+0000","last_peered":"2024-09-17T15:29:26.651073+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:29:26.651073+0000","last_undegraded":"2024-09-17T15:29:26.651073+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":9984,"log_dups_size":0,"ondisk_log_size":9984,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40846278,"num_objects":9974,"num_object_clones":0,"num_object_copies":19948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2810,"num_objects_unfound":0,"num_objects_dirty":9974,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6426,"num_write_kb":25704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1299,"num_bytes_recovered":5320704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7164},{"shards":"0,2","objects":2810}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":659423966,"num_objects":161001,"num_object_clones":0,"num_object_copies":322002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":2810,"num_objects_unfound":0,"num_objects_dirty":161001,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":121108,"num_write_kb":484432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5371,"num_bytes_recovered":21991907,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1340055552,"data_stored":1339903806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160273,"ondisk_log_size":160273,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051637,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":720952,"kb_used_data":454632,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93650888,"statfs":{"total":96636764160,"available":95898509312,"internally_reserved":0,"allocated":465543168,"data_stored":465266594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":272690860},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724582,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":534564,"kb_used_data":304340,"kb_used_omap":4,"kb_used_meta":230203,"kb_avail":93837276,"statfs":{"total":96636764160,"available":96089370624,"internally_reserved":0,"allocated":311644160,"data_stored":311320153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":235728555},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247865,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":855120,"kb_used_data":548608,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93516720,"statfs":{"total":96636764160,"available":95761121280,"internally_reserved":0,"allocated":561774592,"data_stored":561493941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":313847464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":562356224,"data_stored":562320094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":312467456,"data_stored":312387970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":465231872,"data_stored":465195742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:29:31.647 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:29:32.221 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:29:32.221 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:29:32.248 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":190,"stamp":"2024-09-17T15:29:31.654576+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659944177,"num_objects":161130,"num_object_clones":0,"num_object_copies":322260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":2810,"num_objects_unfound":0,"num_objects_dirty":161130,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":121237,"num_write_kb":484942,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5372,"num_bytes_recovered":21991926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160303,"ondisk_log_size":160303,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2125744,"kb_used_data":1314368,"kb_used_omap":13,"kb_used_meta":811314,"kb_avail":280989776,"statfs":{"total":289910292480,"available":287733530624,"internally_reserved":0,"allocated":1345912832,"data_stored":1345031600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":830786559},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,7,2,5,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18944000,"num_objects":4625,"num_object_clones":0,"num_object_copies":9250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-203,"num_objects_unfound":0,"num_objects_dirty":4625,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4625,"num_write_kb":18500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":831488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004011"},"pg_stats":[{"pgid":"2.f","version":"43'9961","reported_seq":10038,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:26.667443+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:29:26.667443+0000","last_peered":"2024-09-17T15:29:26.667443+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:29:26.667443+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9961,"log_dups_size":0,"ondisk_log_size":9961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40800256,"num_objects":9961,"num_object_clones":0,"num_object_copies":19922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9961,"num_write_kb":39844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5891},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'10116","reported_seq":10218,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:27.324301+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:29:27.324301+0000","last_peered":"2024-09-17T15:29:27.324301+0000","last_clean":"2024-09-17T15:29:27.324301+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:29:27.324301+0000","last_undegraded":"2024-09-17T15:29:27.324301+0000","last_fullsized":"2024-09-17T15:29:27.324301+0000","mapping_epoch":33,"log_start":"34'100","ondisk_log_start":"34'100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41411526,"num_objects":10112,"num_object_clones":0,"num_object_copies":20224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10112,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8473,"num_write_kb":33892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'10129","reported_seq":10254,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:27.285240+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:29:27.285240+0000","last_peered":"2024-09-17T15:29:27.285240+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:29:27.285240+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'100","ondisk_log_start":"34'100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41472000,"num_objects":10125,"num_object_clones":0,"num_object_copies":20250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10125,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8286,"num_write_kb":33144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":5957}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'10173","reported_seq":10242,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:27.134539+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:29:27.134539+0000","last_peered":"2024-09-17T15:29:27.134539+0000","last_clean":"2024-09-17T15:29:27.134539+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:29:27.134539+0000","last_undegraded":"2024-09-17T15:29:27.134539+0000","last_fullsized":"2024-09-17T15:29:27.134539+0000","mapping_epoch":33,"log_start":"34'100","ondisk_log_start":"34'100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41668608,"num_objects":10173,"num_object_clones":0,"num_object_copies":20346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10173,"num_write_kb":40692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9977","reported_seq":10106,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:26.580177+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:29:26.580177+0000","last_peered":"2024-09-17T15:29:26.580177+0000","last_clean":"2024-09-17T15:29:26.580177+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:29:26.580177+0000","last_undegraded":"2024-09-17T15:29:26.580177+0000","last_fullsized":"2024-09-17T15:29:26.580177+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":9977,"log_dups_size":0,"ondisk_log_size":9977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40837603,"num_objects":9971,"num_object_clones":0,"num_object_copies":19942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9971,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6959,"num_write_kb":27836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'10264","reported_seq":10397,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:26.624360+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:29:26.624360+0000","last_peered":"2024-09-17T15:29:26.624360+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:29:26.624360+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'200","ondisk_log_start":"34'200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42021347,"num_objects":10260,"num_object_clones":0,"num_object_copies":20520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10260,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6763,"num_write_kb":27052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5990},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'10057","reported_seq":10195,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:26.534111+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:29:26.534111+0000","last_peered":"2024-09-17T15:29:26.534111+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:29:26.534111+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41169862,"num_objects":10053,"num_object_clones":0,"num_object_copies":20106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10053,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6826,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5879},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'10129","reported_seq":10256,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:26.637325+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:29:26.637325+0000","last_peered":"2024-09-17T15:29:26.637325+0000","last_clean":"2024-09-17T15:29:26.637325+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:29:26.637325+0000","last_undegraded":"2024-09-17T15:29:26.637325+0000","last_fullsized":"2024-09-17T15:29:26.637325+0000","mapping_epoch":32,"log_start":"34'100","ondisk_log_start":"34'100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41480192,"num_objects":10127,"num_object_clones":0,"num_object_copies":20254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7083,"num_write_kb":28332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'10169","reported_seq":18451,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:30.849293+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:29:30.849293+0000","last_peered":"2024-09-17T15:29:30.849293+0000","last_clean":"2024-09-17T15:29:30.849293+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:29:30.849293+0000","last_undegraded":"2024-09-17T15:29:30.849293+0000","last_fullsized":"2024-09-17T15:29:30.849293+0000","mapping_epoch":42,"log_start":"34'100","ondisk_log_start":"34'100","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41624035,"num_objects":10163,"num_object_clones":0,"num_object_copies":20326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10163,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8169,"num_write_kb":32676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804705+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804705+0000","last_peered":"2024-09-17T15:28:23.804705+0000","last_clean":"2024-09-17T15:28:23.804705+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804705+0000","last_undegraded":"2024-09-17T15:28:23.804705+0000","last_fullsized":"2024-09-17T15:28:23.804705+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9935","reported_seq":10065,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:26.542912+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:29:26.542912+0000","last_peered":"2024-09-17T15:29:26.542912+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:29:26.542912+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":9935,"log_dups_size":0,"ondisk_log_size":9935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40660992,"num_objects":9927,"num_object_clones":0,"num_object_copies":19854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6531,"num_write_kb":26124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5917},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9942","reported_seq":10062,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:27.268905+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:29:27.268905+0000","last_peered":"2024-09-17T15:29:27.268905+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:29:27.268905+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":9942,"log_dups_size":0,"ondisk_log_size":9942,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40702435,"num_objects":9938,"num_object_clones":0,"num_object_copies":19876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9938,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7837,"num_write_kb":31348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":5861}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'10044","reported_seq":10163,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:27.349810+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:29:27.349810+0000","last_peered":"2024-09-17T15:29:27.349810+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:29:27.349810+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41115648,"num_objects":10038,"num_object_clones":0,"num_object_copies":20076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7806,"num_write_kb":31224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":5901}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'10145","reported_seq":10279,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:26.671652+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:29:26.671652+0000","last_peered":"2024-09-17T15:29:26.671652+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:29:26.671652+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'100","ondisk_log_start":"34'100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41545728,"num_objects":10143,"num_object_clones":0,"num_object_copies":20286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10143,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6693,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6005},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'10062","reported_seq":10202,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:26.673955+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:29:26.673955+0000","last_peered":"2024-09-17T15:29:26.673955+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:29:26.673955+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41181184,"num_objects":10054,"num_object_clones":0,"num_object_copies":20108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10054,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6349,"num_write_kb":25396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5897},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805074+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805074+0000","last_peered":"2024-09-17T15:28:23.805074+0000","last_clean":"2024-09-17T15:28:23.805074+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805074+0000","last_undegraded":"2024-09-17T15:28:23.805074+0000","last_fullsized":"2024-09-17T15:28:23.805074+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'10113","reported_seq":10243,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:26.581818+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:29:26.581818+0000","last_peered":"2024-09-17T15:29:26.581818+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:29:26.581818+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'100","ondisk_log_start":"34'100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41406464,"num_objects":10109,"num_object_clones":0,"num_object_copies":20218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6900,"num_write_kb":27600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6046},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9984","reported_seq":12726,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:29:26.651073+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:29:26.651073+0000","last_peered":"2024-09-17T15:29:26.651073+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:29:26.651073+0000","last_undegraded":"2024-09-17T15:29:26.651073+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":9984,"log_dups_size":0,"ondisk_log_size":9984,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40846278,"num_objects":9974,"num_object_clones":0,"num_object_copies":19948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2810,"num_objects_unfound":0,"num_objects_dirty":9974,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6426,"num_write_kb":25704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1299,"num_bytes_recovered":5320704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7164},{"shards":"0,2","objects":2810}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":659944158,"num_objects":161128,"num_object_clones":0,"num_object_copies":322256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":2810,"num_objects_unfound":0,"num_objects_dirty":161128,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":121235,"num_write_kb":484940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5371,"num_bytes_recovered":21991907,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1347289088,"data_stored":1347137342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160300,"ondisk_log_size":160300,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051637,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":720952,"kb_used_data":454632,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93650888,"statfs":{"total":96636764160,"available":95898509312,"internally_reserved":0,"allocated":465543168,"data_stored":465266594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":272690860},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724583,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549672,"kb_used_data":311128,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93822168,"statfs":{"total":96636764160,"available":96073900032,"internally_reserved":0,"allocated":318595072,"data_stored":318271065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":244248235},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,0,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247865,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":855120,"kb_used_data":548608,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93516720,"statfs":{"total":96636764160,"available":95761121280,"internally_reserved":0,"allocated":561774592,"data_stored":561493941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":313847464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":562356224,"data_stored":562320094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":319700992,"data_stored":319621506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":465231872,"data_stored":465195742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:29:32.249 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:29:32.249 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'9961", 'reported_seq': 10038, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:26.667443+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:29:26.667443+0000', 'last_peered': '2024-09-17T15:29:26.667443+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:29:26.667443+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 9961, 'log_dups_size': 0, 'ondisk_log_size': 9961, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40800256, 'num_objects': 9961, 'num_object_clones': 0, 'num_object_copies': 19922, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4070, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9961, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9961, 'num_write_kb': 39844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5891}, {'shards': '0,2', 'objects': 4070}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:32.249 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:29:32.249 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'10129", 'reported_seq': 10254, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:27.285240+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:29:27.285240+0000', 'last_peered': '2024-09-17T15:29:27.285240+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:29:27.285240+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "34'100", 'ondisk_log_start': "34'100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41472000, 'num_objects': 10125, 'num_object_clones': 0, 'num_object_copies': 20250, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4169, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10125, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8286, 'num_write_kb': 33144, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4168}, {'shards': '2', 'objects': 5957}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:29:32.250 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:29:32.250 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'10264", 'reported_seq': 10397, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:26.624360+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:29:26.624360+0000', 'last_peered': '2024-09-17T15:29:26.624360+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:29:26.624360+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "34'200", 'ondisk_log_start': "34'200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42021347, 'num_objects': 10260, 'num_object_clones': 0, 'num_object_copies': 20520, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4271, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10260, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6763, 'num_write_kb': 27052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5990}, {'shards': '0,2', 'objects': 4268}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:32.250 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:29:32.250 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'10057", 'reported_seq': 10195, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:26.534111+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:29:26.534111+0000', 'last_peered': '2024-09-17T15:29:26.534111+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:29:26.534111+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41169862, 'num_objects': 10053, 'num_object_clones': 0, 'num_object_copies': 20106, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4174, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10053, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 6826, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5879}, {'shards': '0,2', 'objects': 4174}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:32.250 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:29:32.250 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'9935", 'reported_seq': 10065, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:26.542912+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:29:26.542912+0000', 'last_peered': '2024-09-17T15:29:26.542912+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:29:26.542912+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 9935, 'log_dups_size': 0, 'ondisk_log_size': 9935, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40660992, 'num_objects': 9927, 'num_object_clones': 0, 'num_object_copies': 19854, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4012, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9927, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6531, 'num_write_kb': 26124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5917}, {'shards': '0,2', 'objects': 4010}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:32.250 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:29:32.250 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'9942", 'reported_seq': 10062, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:27.268905+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:29:27.268905+0000', 'last_peered': '2024-09-17T15:29:27.268905+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:29:27.268905+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 9942, 'log_dups_size': 0, 'ondisk_log_size': 9942, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40702435, 'num_objects': 9938, 'num_object_clones': 0, 'num_object_copies': 19876, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4078, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9938, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7837, 'num_write_kb': 31348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4077}, {'shards': '2', 'objects': 5861}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:29:32.250 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:29:32.250 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'10044", 'reported_seq': 10163, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:27.349810+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:29:27.349810+0000', 'last_peered': '2024-09-17T15:29:27.349810+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:29:27.349810+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41115648, 'num_objects': 10038, 'num_object_clones': 0, 'num_object_copies': 20076, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10038, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7806, 'num_write_kb': 31224, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4137}, {'shards': '2', 'objects': 5901}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:29:32.251 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:29:32.251 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'10145", 'reported_seq': 10279, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:26.671652+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:29:26.671652+0000', 'last_peered': '2024-09-17T15:29:26.671652+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:29:26.671652+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "34'100", 'ondisk_log_start': "34'100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41545728, 'num_objects': 10143, 'num_object_clones': 0, 'num_object_copies': 20286, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10143, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6693, 'num_write_kb': 26772, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6005}, {'shards': '0,2', 'objects': 4138}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:32.251 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:29:32.251 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'10062", 'reported_seq': 10202, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:26.673955+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:29:26.673955+0000', 'last_peered': '2024-09-17T15:29:26.673955+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:29:26.673955+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41181184, 'num_objects': 10054, 'num_object_clones': 0, 'num_object_copies': 20108, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4160, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10054, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6349, 'num_write_kb': 25396, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5897}, {'shards': '0,2', 'objects': 4157}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:32.251 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:29:32.251 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'10113", 'reported_seq': 10243, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:26.581818+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:29:26.581818+0000', 'last_peered': '2024-09-17T15:29:26.581818+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:29:26.581818+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "34'100", 'ondisk_log_start': "34'100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41406464, 'num_objects': 10109, 'num_object_clones': 0, 'num_object_copies': 20218, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4065, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10109, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6900, 'num_write_kb': 27600, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6046}, {'shards': '0,2', 'objects': 4063}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:32.251 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:29:32.251 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'9984", 'reported_seq': 12726, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:29:26.651073+0000', 'last_change': '2024-09-17T15:28:22.509818+0000', 'last_active': '2024-09-17T15:29:26.651073+0000', 'last_peered': '2024-09-17T15:29:26.651073+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:29:26.651073+0000', 'last_undegraded': '2024-09-17T15:29:26.651073+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 9984, 'log_dups_size': 0, 'ondisk_log_size': 9984, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40846278, 'num_objects': 9974, 'num_object_clones': 0, 'num_object_copies': 19948, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2810, 'num_objects_unfound': 0, 'num_objects_dirty': 9974, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6426, 'num_write_kb': 25704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1299, 'num_bytes_recovered': 5320704, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7164}, {'shards': '0,2', 'objects': 2810}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:32.251 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:29:32.331 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 233 20 163426 163406 2.73894 1.57812 0.0318263 0.0285162 2024-09-17T15:29:32.951 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:29:32.951 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41277/322006 objects degraded (12.819%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":309,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":161003,"data_bytes":659423985,"bytes_used":2161291264,"bytes_avail":287749001216,"bytes_total":289910292480,"degraded_objects":41277,"degraded_total":322006,"degraded_ratio":0.12818705241517239,"misplaced_objects":2810,"misplaced_total":322006,"misplaced_ratio":0.0087265454680968673,"recovering_objects_per_sec":16,"recovering_bytes_per_sec":69266,"recovering_keys_per_sec":0,"num_objects_recovered":203,"num_bytes_recovered":831488,"num_keys_recovered":0,"write_bytes_sec":1534789,"read_op_per_sec":0,"write_op_per_sec":374},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T15:27:19.616376+0000","services":{}},"progress_events":{}} 2024-09-17T15:29:32.977 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:29:32.977 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:29:33.331 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 234 20 163832 163812 2.73401 1.58594 0.0819459 0.0285679 2024-09-17T15:29:33.561 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:29:33.562 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:29:33.587 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":190,"stamp":"2024-09-17T15:29:31.654576+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659944177,"num_objects":161130,"num_object_clones":0,"num_object_copies":322260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":2810,"num_objects_unfound":0,"num_objects_dirty":161130,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":121237,"num_write_kb":484942,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5372,"num_bytes_recovered":21991926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160303,"ondisk_log_size":160303,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2125744,"kb_used_data":1314368,"kb_used_omap":13,"kb_used_meta":811314,"kb_avail":280989776,"statfs":{"total":289910292480,"available":287733530624,"internally_reserved":0,"allocated":1345912832,"data_stored":1345031600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":830786559},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,7,2,5,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18944000,"num_objects":4625,"num_object_clones":0,"num_object_copies":9250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-203,"num_objects_unfound":0,"num_objects_dirty":4625,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4625,"num_write_kb":18500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":831488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004011"},"pg_stats":[{"pgid":"2.f","version":"43'9961","reported_seq":10038,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:26.667443+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:29:26.667443+0000","last_peered":"2024-09-17T15:29:26.667443+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:29:26.667443+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":9961,"log_dups_size":0,"ondisk_log_size":9961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40800256,"num_objects":9961,"num_object_clones":0,"num_object_copies":19922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9961,"num_write_kb":39844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5891},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'10116","reported_seq":10218,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:27.324301+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:29:27.324301+0000","last_peered":"2024-09-17T15:29:27.324301+0000","last_clean":"2024-09-17T15:29:27.324301+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:29:27.324301+0000","last_undegraded":"2024-09-17T15:29:27.324301+0000","last_fullsized":"2024-09-17T15:29:27.324301+0000","mapping_epoch":33,"log_start":"34'100","ondisk_log_start":"34'100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41411526,"num_objects":10112,"num_object_clones":0,"num_object_copies":20224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10112,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8473,"num_write_kb":33892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'10129","reported_seq":10254,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:27.285240+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:29:27.285240+0000","last_peered":"2024-09-17T15:29:27.285240+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:29:27.285240+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'100","ondisk_log_start":"34'100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41472000,"num_objects":10125,"num_object_clones":0,"num_object_copies":20250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10125,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8286,"num_write_kb":33144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":5957}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'10173","reported_seq":10242,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:27.134539+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:29:27.134539+0000","last_peered":"2024-09-17T15:29:27.134539+0000","last_clean":"2024-09-17T15:29:27.134539+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:29:27.134539+0000","last_undegraded":"2024-09-17T15:29:27.134539+0000","last_fullsized":"2024-09-17T15:29:27.134539+0000","mapping_epoch":33,"log_start":"34'100","ondisk_log_start":"34'100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41668608,"num_objects":10173,"num_object_clones":0,"num_object_copies":20346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10173,"num_write_kb":40692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9977","reported_seq":10106,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:26.580177+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:29:26.580177+0000","last_peered":"2024-09-17T15:29:26.580177+0000","last_clean":"2024-09-17T15:29:26.580177+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:29:26.580177+0000","last_undegraded":"2024-09-17T15:29:26.580177+0000","last_fullsized":"2024-09-17T15:29:26.580177+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":9977,"log_dups_size":0,"ondisk_log_size":9977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40837603,"num_objects":9971,"num_object_clones":0,"num_object_copies":19942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9971,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6959,"num_write_kb":27836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'10264","reported_seq":10397,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:26.624360+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:29:26.624360+0000","last_peered":"2024-09-17T15:29:26.624360+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:29:26.624360+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'200","ondisk_log_start":"34'200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42021347,"num_objects":10260,"num_object_clones":0,"num_object_copies":20520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10260,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6763,"num_write_kb":27052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5990},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'10057","reported_seq":10195,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:26.534111+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:29:26.534111+0000","last_peered":"2024-09-17T15:29:26.534111+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:29:26.534111+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41169862,"num_objects":10053,"num_object_clones":0,"num_object_copies":20106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10053,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6826,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5879},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'10129","reported_seq":10256,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:26.637325+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:29:26.637325+0000","last_peered":"2024-09-17T15:29:26.637325+0000","last_clean":"2024-09-17T15:29:26.637325+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:29:26.637325+0000","last_undegraded":"2024-09-17T15:29:26.637325+0000","last_fullsized":"2024-09-17T15:29:26.637325+0000","mapping_epoch":32,"log_start":"34'100","ondisk_log_start":"34'100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41480192,"num_objects":10127,"num_object_clones":0,"num_object_copies":20254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7083,"num_write_kb":28332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'10169","reported_seq":18451,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:30.849293+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:29:30.849293+0000","last_peered":"2024-09-17T15:29:30.849293+0000","last_clean":"2024-09-17T15:29:30.849293+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:29:30.849293+0000","last_undegraded":"2024-09-17T15:29:30.849293+0000","last_fullsized":"2024-09-17T15:29:30.849293+0000","mapping_epoch":42,"log_start":"34'100","ondisk_log_start":"34'100","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41624035,"num_objects":10163,"num_object_clones":0,"num_object_copies":20326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10163,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8169,"num_write_kb":32676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804705+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804705+0000","last_peered":"2024-09-17T15:28:23.804705+0000","last_clean":"2024-09-17T15:28:23.804705+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804705+0000","last_undegraded":"2024-09-17T15:28:23.804705+0000","last_fullsized":"2024-09-17T15:28:23.804705+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9935","reported_seq":10065,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:26.542912+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:29:26.542912+0000","last_peered":"2024-09-17T15:29:26.542912+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:29:26.542912+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":9935,"log_dups_size":0,"ondisk_log_size":9935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40660992,"num_objects":9927,"num_object_clones":0,"num_object_copies":19854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6531,"num_write_kb":26124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5917},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9942","reported_seq":10062,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:27.268905+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:29:27.268905+0000","last_peered":"2024-09-17T15:29:27.268905+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:29:27.268905+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":9942,"log_dups_size":0,"ondisk_log_size":9942,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40702435,"num_objects":9938,"num_object_clones":0,"num_object_copies":19876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9938,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7837,"num_write_kb":31348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":5861}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'10044","reported_seq":10163,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:27.349810+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:29:27.349810+0000","last_peered":"2024-09-17T15:29:27.349810+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:29:27.349810+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41115648,"num_objects":10038,"num_object_clones":0,"num_object_copies":20076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7806,"num_write_kb":31224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":5901}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'10145","reported_seq":10279,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:26.671652+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:29:26.671652+0000","last_peered":"2024-09-17T15:29:26.671652+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:29:26.671652+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'100","ondisk_log_start":"34'100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41545728,"num_objects":10143,"num_object_clones":0,"num_object_copies":20286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10143,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6693,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6005},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'10062","reported_seq":10202,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:26.673955+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:29:26.673955+0000","last_peered":"2024-09-17T15:29:26.673955+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:29:26.673955+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41181184,"num_objects":10054,"num_object_clones":0,"num_object_copies":20108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10054,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6349,"num_write_kb":25396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5897},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805074+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805074+0000","last_peered":"2024-09-17T15:28:23.805074+0000","last_clean":"2024-09-17T15:28:23.805074+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805074+0000","last_undegraded":"2024-09-17T15:28:23.805074+0000","last_fullsized":"2024-09-17T15:28:23.805074+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'10113","reported_seq":10243,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:26.581818+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:29:26.581818+0000","last_peered":"2024-09-17T15:29:26.581818+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:29:26.581818+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'100","ondisk_log_start":"34'100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41406464,"num_objects":10109,"num_object_clones":0,"num_object_copies":20218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6900,"num_write_kb":27600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6046},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9984","reported_seq":12726,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:29:26.651073+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:29:26.651073+0000","last_peered":"2024-09-17T15:29:26.651073+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:29:26.651073+0000","last_undegraded":"2024-09-17T15:29:26.651073+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":9984,"log_dups_size":0,"ondisk_log_size":9984,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40846278,"num_objects":9974,"num_object_clones":0,"num_object_copies":19948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2810,"num_objects_unfound":0,"num_objects_dirty":9974,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6426,"num_write_kb":25704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1299,"num_bytes_recovered":5320704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7164},{"shards":"0,2","objects":2810}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":659944158,"num_objects":161128,"num_object_clones":0,"num_object_copies":322256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":2810,"num_objects_unfound":0,"num_objects_dirty":161128,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":121235,"num_write_kb":484940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5371,"num_bytes_recovered":21991907,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1347289088,"data_stored":1347137342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160300,"ondisk_log_size":160300,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051637,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":720952,"kb_used_data":454632,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93650888,"statfs":{"total":96636764160,"available":95898509312,"internally_reserved":0,"allocated":465543168,"data_stored":465266594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":272690860},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724583,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549672,"kb_used_data":311128,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93822168,"statfs":{"total":96636764160,"available":96073900032,"internally_reserved":0,"allocated":318595072,"data_stored":318271065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":244248235},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,0,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247865,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":855120,"kb_used_data":548608,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93516720,"statfs":{"total":96636764160,"available":95761121280,"internally_reserved":0,"allocated":561774592,"data_stored":561493941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":313847464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":562356224,"data_stored":562320094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":319700992,"data_stored":319621506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":465231872,"data_stored":465195742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:29:34.331 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 235 20 164276 164256 2.72976 1.73438 0.0194203 0.028612 2024-09-17T15:29:35.332 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 236 20 164737 164717 2.72582 1.80078 0.0317914 0.0286531 2024-09-17T15:29:36.332 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 237 20 165206 165186 2.72205 1.83203 0.0230018 0.0286941 2024-09-17T15:29:36.589 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:29:37.149 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:29:37.152 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:29:37.176 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":192,"stamp":"2024-09-17T15:29:35.655777+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":668570353,"num_objects":163236,"num_object_clones":0,"num_object_copies":326472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":2713,"num_objects_unfound":0,"num_objects_dirty":163236,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":123343,"num_write_kb":493366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5469,"num_bytes_recovered":22389238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160909,"ondisk_log_size":160909,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2166748,"kb_used_data":1324844,"kb_used_omap":13,"kb_used_meta":841842,"kb_avail":280948772,"statfs":{"total":289910292480,"available":287691542528,"internally_reserved":0,"allocated":1356640256,"data_stored":1355759024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":862047231},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,10,5,8,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18350080,"num_objects":4480,"num_object_clones":0,"num_object_copies":8960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-195,"num_objects_unfound":0,"num_objects_dirty":4480,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4480,"num_write_kb":17920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":195,"num_bytes_recovered":798720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004211"},"pg_stats":[{"pgid":"2.f","version":"43'10114","reported_seq":10191,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:31.649896+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:29:31.649896+0000","last_peered":"2024-09-17T15:29:31.649896+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:29:31.649896+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'100","ondisk_log_start":"34'100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41426944,"num_objects":10114,"num_object_clones":0,"num_object_copies":20228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10114,"num_write_kb":40456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6044},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'10270","reported_seq":10369,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:32.257027+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:29:32.257027+0000","last_peered":"2024-09-17T15:29:32.257027+0000","last_clean":"2024-09-17T15:29:32.257027+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:29:32.257027+0000","last_undegraded":"2024-09-17T15:29:32.257027+0000","last_fullsized":"2024-09-17T15:29:32.257027+0000","mapping_epoch":33,"log_start":"34'200","ondisk_log_start":"34'200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42042310,"num_objects":10266,"num_object_clones":0,"num_object_copies":20532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10266,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8627,"num_write_kb":34508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'10266","reported_seq":10391,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:32.290645+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:29:32.290645+0000","last_peered":"2024-09-17T15:29:32.290645+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:29:32.290645+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'200","ondisk_log_start":"34'200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42033152,"num_objects":10262,"num_object_clones":0,"num_object_copies":20524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8423,"num_write_kb":33692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":6094}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'10300","reported_seq":10369,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:32.309528+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:29:32.309528+0000","last_peered":"2024-09-17T15:29:32.309528+0000","last_clean":"2024-09-17T15:29:32.309528+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:29:32.309528+0000","last_undegraded":"2024-09-17T15:29:32.309528+0000","last_fullsized":"2024-09-17T15:29:32.309528+0000","mapping_epoch":33,"log_start":"34'200","ondisk_log_start":"34'200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42188800,"num_objects":10300,"num_object_clones":0,"num_object_copies":20600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10300,"num_write_kb":41200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'10130","reported_seq":10259,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:31.678561+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:29:31.678561+0000","last_peered":"2024-09-17T15:29:31.678561+0000","last_clean":"2024-09-17T15:29:31.678561+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:29:31.678561+0000","last_undegraded":"2024-09-17T15:29:31.678561+0000","last_fullsized":"2024-09-17T15:29:31.678561+0000","mapping_epoch":32,"log_start":"34'100","ondisk_log_start":"34'100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41464291,"num_objects":10124,"num_object_clones":0,"num_object_copies":20248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10124,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7112,"num_write_kb":28448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'10407","reported_seq":10541,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:31.649726+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:29:31.649726+0000","last_peered":"2024-09-17T15:29:31.649726+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:29:31.649726+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'400","ondisk_log_start":"34'400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42607075,"num_objects":10403,"num_object_clones":0,"num_object_copies":20806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10403,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6906,"num_write_kb":27624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6133},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'10191","reported_seq":10328,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:31.733566+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:29:31.733566+0000","last_peered":"2024-09-17T15:29:31.733566+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:29:31.733566+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'100","ondisk_log_start":"34'100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41718726,"num_objects":10187,"num_object_clones":0,"num_object_copies":20374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10187,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6960,"num_write_kb":27840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6013},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'10267","reported_seq":10395,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:31.724551+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:29:31.724551+0000","last_peered":"2024-09-17T15:29:31.724551+0000","last_clean":"2024-09-17T15:29:31.724551+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:29:31.724551+0000","last_undegraded":"2024-09-17T15:29:31.724551+0000","last_fullsized":"2024-09-17T15:29:31.724551+0000","mapping_epoch":32,"log_start":"34'200","ondisk_log_start":"34'200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42045440,"num_objects":10265,"num_object_clones":0,"num_object_copies":20530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10265,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7221,"num_write_kb":28884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'10169","reported_seq":18451,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:30.849293+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:29:30.849293+0000","last_peered":"2024-09-17T15:29:30.849293+0000","last_clean":"2024-09-17T15:29:30.849293+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:29:30.849293+0000","last_undegraded":"2024-09-17T15:29:30.849293+0000","last_fullsized":"2024-09-17T15:29:30.849293+0000","mapping_epoch":42,"log_start":"34'100","ondisk_log_start":"34'100","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41624035,"num_objects":10163,"num_object_clones":0,"num_object_copies":20326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10163,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8169,"num_write_kb":32676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804747+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804747+0000","last_peered":"2024-09-17T15:28:23.804747+0000","last_clean":"2024-09-17T15:28:23.804747+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804747+0000","last_undegraded":"2024-09-17T15:28:23.804747+0000","last_fullsized":"2024-09-17T15:28:23.804747+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'10061","reported_seq":10190,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:31.660216+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:29:31.660216+0000","last_peered":"2024-09-17T15:29:31.660216+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:29:31.660216+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41177088,"num_objects":10053,"num_object_clones":0,"num_object_copies":20106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10053,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6657,"num_write_kb":26628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6043},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'10083","reported_seq":10204,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:32.338458+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:29:32.338458+0000","last_peered":"2024-09-17T15:29:32.338458+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:29:32.338458+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41279971,"num_objects":10079,"num_object_clones":0,"num_object_copies":20158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10079,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7978,"num_write_kb":31912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":6002}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'10180","reported_seq":10299,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:32.360401+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:29:32.360401+0000","last_peered":"2024-09-17T15:29:32.360401+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:29:32.360401+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'100","ondisk_log_start":"34'100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41672704,"num_objects":10174,"num_object_clones":0,"num_object_copies":20348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7942,"num_write_kb":31768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":6037}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'10272","reported_seq":10406,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:31.626995+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:29:31.626995+0000","last_peered":"2024-09-17T15:29:31.626995+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:29:31.626995+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'200","ondisk_log_start":"34'200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42065920,"num_objects":10270,"num_object_clones":0,"num_object_copies":20540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6820,"num_write_kb":27280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6132},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'10204","reported_seq":10346,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:31.751301+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:29:31.751301+0000","last_peered":"2024-09-17T15:29:31.751301+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:29:31.751301+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'200","ondisk_log_start":"34'200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41762816,"num_objects":10196,"num_object_clones":0,"num_object_copies":20392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10196,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6491,"num_write_kb":25964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6039},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805107+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805107+0000","last_peered":"2024-09-17T15:28:23.805107+0000","last_clean":"2024-09-17T15:28:23.805107+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805107+0000","last_undegraded":"2024-09-17T15:28:23.805107+0000","last_fullsized":"2024-09-17T15:28:23.805107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'10257","reported_seq":10389,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:31.680709+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:29:31.680709+0000","last_peered":"2024-09-17T15:29:31.680709+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:29:31.680709+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'200","ondisk_log_start":"34'200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41996288,"num_objects":10253,"num_object_clones":0,"num_object_copies":20506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7044,"num_write_kb":28176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6190},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'10135","reported_seq":13070,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:29:31.739167+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:29:31.739167+0000","last_peered":"2024-09-17T15:29:31.739167+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:29:31.739167+0000","last_undegraded":"2024-09-17T15:29:31.739167+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"34'100","ondisk_log_start":"34'100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41464774,"num_objects":10125,"num_object_clones":0,"num_object_copies":20250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2713,"num_objects_unfound":0,"num_objects_dirty":10125,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6577,"num_write_kb":26308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1396,"num_bytes_recovered":5718016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7412},{"shards":"0,2","objects":2713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":668570334,"num_objects":163234,"num_object_clones":0,"num_object_copies":326468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":2713,"num_objects_unfound":0,"num_objects_dirty":163234,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":123341,"num_write_kb":493364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5468,"num_bytes_recovered":22389219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1358807040,"data_stored":1358655294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160906,"ondisk_log_size":160906,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051638,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":728860,"kb_used_data":458380,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93642980,"statfs":{"total":96636764160,"available":95890411520,"internally_reserved":0,"allocated":469381120,"data_stored":469104546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":276950700},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724583,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549672,"kb_used_data":311128,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93822168,"statfs":{"total":96636764160,"available":96073900032,"internally_reserved":0,"allocated":318595072,"data_stored":318271065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":244248235},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,0,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247866,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":888216,"kb_used_data":555336,"kb_used_omap":4,"kb_used_meta":332859,"kb_avail":93483624,"statfs":{"total":96636764160,"available":95727230976,"internally_reserved":0,"allocated":568664064,"data_stored":568383413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":340848296},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,2,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":569303040,"data_stored":569266910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":319700992,"data_stored":319621506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":469803008,"data_stored":469766878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:29:37.177 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:29:37.332 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 238 20 165618 165598 2.71737 1.60938 0.0405627 0.0287422 2024-09-17T15:29:37.740 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:29:37.741 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:29:37.765 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":193,"stamp":"2024-09-17T15:29:37.656654+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":677671665,"num_objects":165458,"num_object_clones":0,"num_object_copies":330916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":2599,"num_objects_unfound":0,"num_objects_dirty":165458,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":125565,"num_write_kb":502254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5583,"num_bytes_recovered":22856182,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160531,"ondisk_log_size":160531,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2166084,"kb_used_data":1344980,"kb_used_omap":13,"kb_used_meta":821042,"kb_avail":280949436,"statfs":{"total":289910292480,"available":287692222464,"internally_reserved":0,"allocated":1377259520,"data_stored":1376378288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":840748031},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,5,3,7,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27451392,"num_objects":6702,"num_object_clones":0,"num_object_copies":13404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-309,"num_objects_unfound":0,"num_objects_dirty":6702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6702,"num_write_kb":26808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":1265664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004425"},"pg_stats":[{"pgid":"2.f","version":"43'10267","reported_seq":10342,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:36.769683+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:29:36.769683+0000","last_peered":"2024-09-17T15:29:36.769683+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:29:36.769683+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'200","ondisk_log_start":"34'200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42053632,"num_objects":10267,"num_object_clones":0,"num_object_copies":20534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10267,"num_write_kb":41068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6197},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'10416","reported_seq":10518,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:37.301478+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:29:37.301478+0000","last_peered":"2024-09-17T15:29:37.301478+0000","last_clean":"2024-09-17T15:29:37.301478+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:29:37.301478+0000","last_undegraded":"2024-09-17T15:29:37.301478+0000","last_fullsized":"2024-09-17T15:29:37.301478+0000","mapping_epoch":33,"log_start":"34'400","ondisk_log_start":"34'400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42640326,"num_objects":10412,"num_object_clones":0,"num_object_copies":20824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10412,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8773,"num_write_kb":35092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'10412","reported_seq":10537,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:37.370327+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:29:37.370327+0000","last_peered":"2024-09-17T15:29:37.370327+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:29:37.370327+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'400","ondisk_log_start":"34'400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42631168,"num_objects":10408,"num_object_clones":0,"num_object_copies":20816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8569,"num_write_kb":34276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":6240}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'10433","reported_seq":10502,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:37.218872+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:29:37.218872+0000","last_peered":"2024-09-17T15:29:37.218872+0000","last_clean":"2024-09-17T15:29:37.218872+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:29:37.218872+0000","last_undegraded":"2024-09-17T15:29:37.218872+0000","last_fullsized":"2024-09-17T15:29:37.218872+0000","mapping_epoch":33,"log_start":"34'400","ondisk_log_start":"34'400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42733568,"num_objects":10433,"num_object_clones":0,"num_object_copies":20866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10433,"num_write_kb":41732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'10262","reported_seq":10391,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:36.781511+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:29:36.781511+0000","last_peered":"2024-09-17T15:29:36.781511+0000","last_clean":"2024-09-17T15:29:36.781511+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:29:36.781511+0000","last_undegraded":"2024-09-17T15:29:36.781511+0000","last_fullsized":"2024-09-17T15:29:36.781511+0000","mapping_epoch":32,"log_start":"34'200","ondisk_log_start":"34'200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42004963,"num_objects":10256,"num_object_clones":0,"num_object_copies":20512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10256,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7244,"num_write_kb":28976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'10564","reported_seq":10697,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:36.729641+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:29:36.729641+0000","last_peered":"2024-09-17T15:29:36.729641+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:29:36.729641+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'500","ondisk_log_start":"34'500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43250147,"num_objects":10560,"num_object_clones":0,"num_object_copies":21120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10560,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7063,"num_write_kb":28252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6290},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'10327","reported_seq":10464,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:36.759196+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:29:36.759196+0000","last_peered":"2024-09-17T15:29:36.759196+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:29:36.759196+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'300","ondisk_log_start":"34'300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42275782,"num_objects":10323,"num_object_clones":0,"num_object_copies":20646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10323,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7096,"num_write_kb":28384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6149},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'10401","reported_seq":10529,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:36.738254+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:29:36.738254+0000","last_peered":"2024-09-17T15:29:36.738254+0000","last_clean":"2024-09-17T15:29:36.738254+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:29:36.738254+0000","last_undegraded":"2024-09-17T15:29:36.738254+0000","last_fullsized":"2024-09-17T15:29:36.738254+0000","mapping_epoch":32,"log_start":"34'400","ondisk_log_start":"34'400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42594304,"num_objects":10399,"num_object_clones":0,"num_object_copies":20798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7355,"num_write_kb":29420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'10322","reported_seq":18604,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:35.976544+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:29:35.976544+0000","last_peered":"2024-09-17T15:29:35.976544+0000","last_clean":"2024-09-17T15:29:35.976544+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:29:35.976544+0000","last_undegraded":"2024-09-17T15:29:35.976544+0000","last_fullsized":"2024-09-17T15:29:35.976544+0000","mapping_epoch":42,"log_start":"34'300","ondisk_log_start":"34'300","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42250723,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8322,"num_write_kb":33288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804747+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804747+0000","last_peered":"2024-09-17T15:28:23.804747+0000","last_clean":"2024-09-17T15:28:23.804747+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804747+0000","last_undegraded":"2024-09-17T15:28:23.804747+0000","last_fullsized":"2024-09-17T15:28:23.804747+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'10209","reported_seq":10339,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:36.711798+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:29:36.711798+0000","last_peered":"2024-09-17T15:29:36.711798+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:29:36.711798+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'200","ondisk_log_start":"34'200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41783296,"num_objects":10201,"num_object_clones":0,"num_object_copies":20402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6805,"num_write_kb":27220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6191},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'10212","reported_seq":10333,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:37.272579+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:29:37.272579+0000","last_peered":"2024-09-17T15:29:37.272579+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:29:37.272579+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'200","ondisk_log_start":"34'200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41808355,"num_objects":10208,"num_object_clones":0,"num_object_copies":20416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10208,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8107,"num_write_kb":32428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":6131}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'10331","reported_seq":10450,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:37.263491+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:29:37.263491+0000","last_peered":"2024-09-17T15:29:37.263491+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:29:37.263491+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'300","ondisk_log_start":"34'300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42291200,"num_objects":10325,"num_object_clones":0,"num_object_copies":20650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8093,"num_write_kb":32372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":6188}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'10413","reported_seq":10548,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:36.697612+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:29:36.697612+0000","last_peered":"2024-09-17T15:29:36.697612+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:29:36.697612+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'400","ondisk_log_start":"34'400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42643456,"num_objects":10411,"num_object_clones":0,"num_object_copies":20822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6961,"num_write_kb":27844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6273},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'10318","reported_seq":10460,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:36.590350+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:29:36.590350+0000","last_peered":"2024-09-17T15:29:36.590350+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:29:36.590350+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'300","ondisk_log_start":"34'300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42229760,"num_objects":10310,"num_object_clones":0,"num_object_copies":20620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6605,"num_write_kb":26420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6153},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805107+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805107+0000","last_peered":"2024-09-17T15:28:23.805107+0000","last_clean":"2024-09-17T15:28:23.805107+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805107+0000","last_undegraded":"2024-09-17T15:28:23.805107+0000","last_fullsized":"2024-09-17T15:28:23.805107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'10384","reported_seq":10516,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:36.696560+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:29:36.696560+0000","last_peered":"2024-09-17T15:29:36.696560+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:29:36.696560+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'300","ondisk_log_start":"34'300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42516480,"num_objects":10380,"num_object_clones":0,"num_object_copies":20760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7171,"num_write_kb":28684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6317},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'10257","reported_seq":13421,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:29:36.774536+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:29:36.774536+0000","last_peered":"2024-09-17T15:29:36.774536+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:29:36.774536+0000","last_undegraded":"2024-09-17T15:29:36.774536+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"34'200","ondisk_log_start":"34'200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41964486,"num_objects":10247,"num_object_clones":0,"num_object_copies":20494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2599,"num_objects_unfound":0,"num_objects_dirty":10247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6699,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1510,"num_bytes_recovered":6184960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7648},{"shards":"0,2","objects":2599}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":677671646,"num_objects":165456,"num_object_clones":0,"num_object_copies":330912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":2599,"num_objects_unfound":0,"num_objects_dirty":165456,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":125563,"num_write_kb":502252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5582,"num_bytes_recovered":22856163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1377484800,"data_stored":1377333054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160528,"ondisk_log_size":160528,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051639,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":738200,"kb_used_data":463560,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93633640,"statfs":{"total":96636764160,"available":95880847360,"internally_reserved":0,"allocated":474685440,"data_stored":474408866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":281210540},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,1,0,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724584,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545712,"kb_used_data":319648,"kb_used_omap":4,"kb_used_meta":226043,"kb_avail":93826128,"statfs":{"total":96636764160,"available":96077955072,"internally_reserved":0,"allocated":327319552,"data_stored":326995545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":231468715},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,1,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247867,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":882172,"kb_used_data":561772,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93489668,"statfs":{"total":96636764160,"available":95733420032,"internally_reserved":0,"allocated":575254528,"data_stored":574973877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":328068776},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":576057344,"data_stored":576021214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":327069696,"data_stored":326990210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":474357760,"data_stored":474321630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:29:37.766 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:29:37.766 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'10267", 'reported_seq': 10342, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:36.769683+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:29:36.769683+0000', 'last_peered': '2024-09-17T15:29:36.769683+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:29:36.769683+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "34'200", 'ondisk_log_start': "34'200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42053632, 'num_objects': 10267, 'num_object_clones': 0, 'num_object_copies': 20534, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4070, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10267, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10267, 'num_write_kb': 41068, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6197}, {'shards': '0,2', 'objects': 4070}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:37.767 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:29:37.767 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'10412", 'reported_seq': 10537, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:37.370327+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:29:37.370327+0000', 'last_peered': '2024-09-17T15:29:37.370327+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:29:37.370327+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "34'400", 'ondisk_log_start': "34'400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42631168, 'num_objects': 10408, 'num_object_clones': 0, 'num_object_copies': 20816, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4169, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10408, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8569, 'num_write_kb': 34276, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4168}, {'shards': '2', 'objects': 6240}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:29:37.767 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:29:37.767 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'10564", 'reported_seq': 10697, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:36.729641+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:29:36.729641+0000', 'last_peered': '2024-09-17T15:29:36.729641+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:29:36.729641+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "34'500", 'ondisk_log_start': "34'500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 43250147, 'num_objects': 10560, 'num_object_clones': 0, 'num_object_copies': 21120, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4271, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10560, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7063, 'num_write_kb': 28252, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6290}, {'shards': '0,2', 'objects': 4268}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:37.767 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:29:37.767 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'10327", 'reported_seq': 10464, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:36.759196+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:29:36.759196+0000', 'last_peered': '2024-09-17T15:29:36.759196+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:29:36.759196+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "34'300", 'ondisk_log_start': "34'300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42275782, 'num_objects': 10323, 'num_object_clones': 0, 'num_object_copies': 20646, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4174, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10323, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 7096, 'num_write_kb': 28384, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6149}, {'shards': '0,2', 'objects': 4174}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:37.767 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:29:37.767 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'10209", 'reported_seq': 10339, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:36.711798+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:29:36.711798+0000', 'last_peered': '2024-09-17T15:29:36.711798+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:29:36.711798+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "34'200", 'ondisk_log_start': "34'200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41783296, 'num_objects': 10201, 'num_object_clones': 0, 'num_object_copies': 20402, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4012, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10201, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6805, 'num_write_kb': 27220, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6191}, {'shards': '0,2', 'objects': 4010}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:37.768 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:29:37.768 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'10212", 'reported_seq': 10333, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:37.272579+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:29:37.272579+0000', 'last_peered': '2024-09-17T15:29:37.272579+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:29:37.272579+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "34'200", 'ondisk_log_start': "34'200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41808355, 'num_objects': 10208, 'num_object_clones': 0, 'num_object_copies': 20416, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4078, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10208, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8107, 'num_write_kb': 32428, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4077}, {'shards': '2', 'objects': 6131}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:29:37.768 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:29:37.768 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'10331", 'reported_seq': 10450, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:37.263491+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:29:37.263491+0000', 'last_peered': '2024-09-17T15:29:37.263491+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:29:37.263491+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "34'300", 'ondisk_log_start': "34'300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42291200, 'num_objects': 10325, 'num_object_clones': 0, 'num_object_copies': 20650, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10325, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8093, 'num_write_kb': 32372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4137}, {'shards': '2', 'objects': 6188}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:29:37.768 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:29:37.768 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'10413", 'reported_seq': 10548, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:36.697612+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:29:36.697612+0000', 'last_peered': '2024-09-17T15:29:36.697612+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:29:36.697612+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "34'400", 'ondisk_log_start': "34'400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42643456, 'num_objects': 10411, 'num_object_clones': 0, 'num_object_copies': 20822, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10411, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6961, 'num_write_kb': 27844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6273}, {'shards': '0,2', 'objects': 4138}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:37.768 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:29:37.768 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'10318", 'reported_seq': 10460, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:36.590350+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:29:36.590350+0000', 'last_peered': '2024-09-17T15:29:36.590350+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:29:36.590350+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "34'300", 'ondisk_log_start': "34'300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42229760, 'num_objects': 10310, 'num_object_clones': 0, 'num_object_copies': 20620, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4160, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10310, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6605, 'num_write_kb': 26420, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6153}, {'shards': '0,2', 'objects': 4157}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:37.768 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:29:37.768 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'10384", 'reported_seq': 10516, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:36.696560+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:29:36.696560+0000', 'last_peered': '2024-09-17T15:29:36.696560+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:29:36.696560+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "34'300", 'ondisk_log_start': "34'300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42516480, 'num_objects': 10380, 'num_object_clones': 0, 'num_object_copies': 20760, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4065, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10380, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7171, 'num_write_kb': 28684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6317}, {'shards': '0,2', 'objects': 4063}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:37.769 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:29:37.769 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'10257", 'reported_seq': 13421, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:29:36.774536+0000', 'last_change': '2024-09-17T15:28:22.509818+0000', 'last_active': '2024-09-17T15:29:36.774536+0000', 'last_peered': '2024-09-17T15:29:36.774536+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:29:36.774536+0000', 'last_undegraded': '2024-09-17T15:29:36.774536+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "34'200", 'ondisk_log_start': "34'200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41964486, 'num_objects': 10247, 'num_object_clones': 0, 'num_object_copies': 20494, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2599, 'num_objects_unfound': 0, 'num_objects_dirty': 10247, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6699, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1510, 'num_bytes_recovered': 6184960, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7648}, {'shards': '0,2', 'objects': 2599}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:37.769 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:29:38.332 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout: 239 20 166022 166002 2.71261 1.57812 0.066205 0.0287926 2024-09-17T15:29:38.424 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:29:38.424 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41277/330916 objects degraded (12.474%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":315,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":165458,"data_bytes":677671665,"bytes_used":2218070016,"bytes_avail":287692222464,"bytes_total":289910292480,"degraded_objects":41277,"degraded_total":330916,"degraded_ratio":0.12473558244388304,"misplaced_objects":2599,"misplaced_total":330916,"misplaced_ratio":0.0078539568954054811,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":105433,"recovering_keys_per_sec":0,"num_objects_recovered":309,"num_bytes_recovered":1265664,"num_keys_recovered":0,"write_bytes_sec":2286772,"read_op_per_sec":0,"write_op_per_sec":558},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T15:27:19.616376+0000","services":{}},"progress_events":{}} 2024-09-17T15:29:38.452 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:29:38.452 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:29:39.009 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:29:39.012 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:29:39.030 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":193,"stamp":"2024-09-17T15:29:37.656654+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":677671665,"num_objects":165458,"num_object_clones":0,"num_object_copies":330916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":2599,"num_objects_unfound":0,"num_objects_dirty":165458,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":125565,"num_write_kb":502254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5583,"num_bytes_recovered":22856182,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160531,"ondisk_log_size":160531,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2166084,"kb_used_data":1344980,"kb_used_omap":13,"kb_used_meta":821042,"kb_avail":280949436,"statfs":{"total":289910292480,"available":287692222464,"internally_reserved":0,"allocated":1377259520,"data_stored":1376378288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":840748031},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,5,3,7,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27451392,"num_objects":6702,"num_object_clones":0,"num_object_copies":13404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-309,"num_objects_unfound":0,"num_objects_dirty":6702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6702,"num_write_kb":26808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":1265664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004425"},"pg_stats":[{"pgid":"2.f","version":"43'10267","reported_seq":10342,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:36.769683+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:29:36.769683+0000","last_peered":"2024-09-17T15:29:36.769683+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:29:36.769683+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'200","ondisk_log_start":"34'200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42053632,"num_objects":10267,"num_object_clones":0,"num_object_copies":20534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10267,"num_write_kb":41068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6197},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'10416","reported_seq":10518,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:37.301478+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:29:37.301478+0000","last_peered":"2024-09-17T15:29:37.301478+0000","last_clean":"2024-09-17T15:29:37.301478+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:29:37.301478+0000","last_undegraded":"2024-09-17T15:29:37.301478+0000","last_fullsized":"2024-09-17T15:29:37.301478+0000","mapping_epoch":33,"log_start":"34'400","ondisk_log_start":"34'400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42640326,"num_objects":10412,"num_object_clones":0,"num_object_copies":20824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10412,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8773,"num_write_kb":35092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'10412","reported_seq":10537,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:37.370327+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:29:37.370327+0000","last_peered":"2024-09-17T15:29:37.370327+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:29:37.370327+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'400","ondisk_log_start":"34'400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42631168,"num_objects":10408,"num_object_clones":0,"num_object_copies":20816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8569,"num_write_kb":34276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":6240}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'10433","reported_seq":10502,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:37.218872+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:29:37.218872+0000","last_peered":"2024-09-17T15:29:37.218872+0000","last_clean":"2024-09-17T15:29:37.218872+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:29:37.218872+0000","last_undegraded":"2024-09-17T15:29:37.218872+0000","last_fullsized":"2024-09-17T15:29:37.218872+0000","mapping_epoch":33,"log_start":"34'400","ondisk_log_start":"34'400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42733568,"num_objects":10433,"num_object_clones":0,"num_object_copies":20866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10433,"num_write_kb":41732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'10262","reported_seq":10391,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:36.781511+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:29:36.781511+0000","last_peered":"2024-09-17T15:29:36.781511+0000","last_clean":"2024-09-17T15:29:36.781511+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:29:36.781511+0000","last_undegraded":"2024-09-17T15:29:36.781511+0000","last_fullsized":"2024-09-17T15:29:36.781511+0000","mapping_epoch":32,"log_start":"34'200","ondisk_log_start":"34'200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42004963,"num_objects":10256,"num_object_clones":0,"num_object_copies":20512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10256,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7244,"num_write_kb":28976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'10564","reported_seq":10697,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:36.729641+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:29:36.729641+0000","last_peered":"2024-09-17T15:29:36.729641+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:29:36.729641+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'500","ondisk_log_start":"34'500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43250147,"num_objects":10560,"num_object_clones":0,"num_object_copies":21120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10560,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7063,"num_write_kb":28252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6290},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'10327","reported_seq":10464,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:36.759196+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:29:36.759196+0000","last_peered":"2024-09-17T15:29:36.759196+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:29:36.759196+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'300","ondisk_log_start":"34'300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42275782,"num_objects":10323,"num_object_clones":0,"num_object_copies":20646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10323,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7096,"num_write_kb":28384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6149},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'10401","reported_seq":10529,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:36.738254+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:29:36.738254+0000","last_peered":"2024-09-17T15:29:36.738254+0000","last_clean":"2024-09-17T15:29:36.738254+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:29:36.738254+0000","last_undegraded":"2024-09-17T15:29:36.738254+0000","last_fullsized":"2024-09-17T15:29:36.738254+0000","mapping_epoch":32,"log_start":"34'400","ondisk_log_start":"34'400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42594304,"num_objects":10399,"num_object_clones":0,"num_object_copies":20798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7355,"num_write_kb":29420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'10322","reported_seq":18604,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:35.976544+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:29:35.976544+0000","last_peered":"2024-09-17T15:29:35.976544+0000","last_clean":"2024-09-17T15:29:35.976544+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:29:35.976544+0000","last_undegraded":"2024-09-17T15:29:35.976544+0000","last_fullsized":"2024-09-17T15:29:35.976544+0000","mapping_epoch":42,"log_start":"34'300","ondisk_log_start":"34'300","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42250723,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8322,"num_write_kb":33288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804747+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804747+0000","last_peered":"2024-09-17T15:28:23.804747+0000","last_clean":"2024-09-17T15:28:23.804747+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804747+0000","last_undegraded":"2024-09-17T15:28:23.804747+0000","last_fullsized":"2024-09-17T15:28:23.804747+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'10209","reported_seq":10339,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:36.711798+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:29:36.711798+0000","last_peered":"2024-09-17T15:29:36.711798+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:29:36.711798+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'200","ondisk_log_start":"34'200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41783296,"num_objects":10201,"num_object_clones":0,"num_object_copies":20402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6805,"num_write_kb":27220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6191},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'10212","reported_seq":10333,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:37.272579+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:29:37.272579+0000","last_peered":"2024-09-17T15:29:37.272579+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:29:37.272579+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'200","ondisk_log_start":"34'200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41808355,"num_objects":10208,"num_object_clones":0,"num_object_copies":20416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10208,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8107,"num_write_kb":32428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":6131}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'10331","reported_seq":10450,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:37.263491+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:29:37.263491+0000","last_peered":"2024-09-17T15:29:37.263491+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:29:37.263491+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'300","ondisk_log_start":"34'300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42291200,"num_objects":10325,"num_object_clones":0,"num_object_copies":20650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8093,"num_write_kb":32372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":6188}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'10413","reported_seq":10548,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:36.697612+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:29:36.697612+0000","last_peered":"2024-09-17T15:29:36.697612+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:29:36.697612+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'400","ondisk_log_start":"34'400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42643456,"num_objects":10411,"num_object_clones":0,"num_object_copies":20822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6961,"num_write_kb":27844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6273},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'10318","reported_seq":10460,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:36.590350+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:29:36.590350+0000","last_peered":"2024-09-17T15:29:36.590350+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:29:36.590350+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'300","ondisk_log_start":"34'300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42229760,"num_objects":10310,"num_object_clones":0,"num_object_copies":20620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6605,"num_write_kb":26420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6153},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805107+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805107+0000","last_peered":"2024-09-17T15:28:23.805107+0000","last_clean":"2024-09-17T15:28:23.805107+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805107+0000","last_undegraded":"2024-09-17T15:28:23.805107+0000","last_fullsized":"2024-09-17T15:28:23.805107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'10384","reported_seq":10516,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:36.696560+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:29:36.696560+0000","last_peered":"2024-09-17T15:29:36.696560+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:29:36.696560+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'300","ondisk_log_start":"34'300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42516480,"num_objects":10380,"num_object_clones":0,"num_object_copies":20760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7171,"num_write_kb":28684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6317},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'10257","reported_seq":13421,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:29:36.774536+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:29:36.774536+0000","last_peered":"2024-09-17T15:29:36.774536+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:29:36.774536+0000","last_undegraded":"2024-09-17T15:29:36.774536+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"34'200","ondisk_log_start":"34'200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41964486,"num_objects":10247,"num_object_clones":0,"num_object_copies":20494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2599,"num_objects_unfound":0,"num_objects_dirty":10247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6699,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1510,"num_bytes_recovered":6184960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7648},{"shards":"0,2","objects":2599}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":677671646,"num_objects":165456,"num_object_clones":0,"num_object_copies":330912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":2599,"num_objects_unfound":0,"num_objects_dirty":165456,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":125563,"num_write_kb":502252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5582,"num_bytes_recovered":22856163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1377484800,"data_stored":1377333054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160528,"ondisk_log_size":160528,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051639,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":738200,"kb_used_data":463560,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93633640,"statfs":{"total":96636764160,"available":95880847360,"internally_reserved":0,"allocated":474685440,"data_stored":474408866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":281210540},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,1,0,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724584,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545712,"kb_used_data":319648,"kb_used_omap":4,"kb_used_meta":226043,"kb_avail":93826128,"statfs":{"total":96636764160,"available":96077955072,"internally_reserved":0,"allocated":327319552,"data_stored":326995545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":231468715},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,1,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247867,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":882172,"kb_used_data":561772,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93489668,"statfs":{"total":96636764160,"available":95733420032,"internally_reserved":0,"allocated":575254528,"data_stored":574973877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":328068776},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":576057344,"data_stored":576021214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":327069696,"data_stored":326990210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":474357760,"data_stored":474321630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:29:39.332 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout:Total time run: 240.036 2024-09-17T15:29:39.332 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout:Total writes made: 166429 2024-09-17T15:29:39.332 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout:Write size: 4096 2024-09-17T15:29:39.332 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout:Object size: 4096 2024-09-17T15:29:39.332 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout:Bandwidth (MB/sec): 2.7084 2024-09-17T15:29:39.332 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout:Stddev Bandwidth: 1.93228 2024-09-17T15:29:39.332 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout:Max bandwidth (MB/sec): 11.9102 2024-09-17T15:29:39.333 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout:Min bandwidth (MB/sec): 0 2024-09-17T15:29:39.333 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout:Average IOPS: 693 2024-09-17T15:29:39.333 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout:Stddev IOPS: 494.665 2024-09-17T15:29:39.333 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout:Max IOPS: 3049 2024-09-17T15:29:39.333 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout:Min IOPS: 0 2024-09-17T15:29:39.333 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout:Average Latency(s): 0.0288393 2024-09-17T15:29:39.333 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout:Stddev Latency(s): 0.042613 2024-09-17T15:29:39.333 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout:Max latency(s): 2.64156 2024-09-17T15:29:39.333 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout:Min latency(s): 0.00187689 2024-09-17T15:29:39.341 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout:Cleaning up (deleting benchmark objects) 2024-09-17T15:29:42.032 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:29:42.574 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:29:42.576 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:29:42.607 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":195,"stamp":"2024-09-17T15:29:41.658092+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":677884657,"num_objects":165510,"num_object_clones":0,"num_object_copies":331020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":2599,"num_objects_unfound":0,"num_objects_dirty":165510,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":125693,"num_write_kb":502614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5583,"num_bytes_recovered":22856182,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160559,"ondisk_log_size":160559,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2174476,"kb_used_data":1349212,"kb_used_omap":13,"kb_used_meta":825202,"kb_avail":280941044,"statfs":{"total":289910292480,"available":287683629056,"internally_reserved":0,"allocated":1381593088,"data_stored":1380707784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":845007871},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,8,5,10,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18460672,"num_objects":4507,"num_object_clones":0,"num_object_copies":9014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-211,"num_objects_unfound":0,"num_objects_dirty":4507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4583,"num_write_kb":18180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":864256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004168"},"pg_stats":[{"pgid":"2.f","version":"43'10267","reported_seq":10342,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:36.769683+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:29:36.769683+0000","last_peered":"2024-09-17T15:29:36.769683+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:29:36.769683+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'200","ondisk_log_start":"34'200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42053632,"num_objects":10267,"num_object_clones":0,"num_object_copies":20534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10267,"num_write_kb":41068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6197},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'10416","reported_seq":10518,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:37.301478+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:29:37.301478+0000","last_peered":"2024-09-17T15:29:37.301478+0000","last_clean":"2024-09-17T15:29:37.301478+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:29:37.301478+0000","last_undegraded":"2024-09-17T15:29:37.301478+0000","last_fullsized":"2024-09-17T15:29:37.301478+0000","mapping_epoch":33,"log_start":"34'400","ondisk_log_start":"34'400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42640326,"num_objects":10412,"num_object_clones":0,"num_object_copies":20824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10412,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8773,"num_write_kb":35092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'10412","reported_seq":10537,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:37.370327+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:29:37.370327+0000","last_peered":"2024-09-17T15:29:37.370327+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:29:37.370327+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'400","ondisk_log_start":"34'400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42631168,"num_objects":10408,"num_object_clones":0,"num_object_copies":20816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8569,"num_write_kb":34276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":6240}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'10433","reported_seq":10502,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:37.218872+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:29:37.218872+0000","last_peered":"2024-09-17T15:29:37.218872+0000","last_clean":"2024-09-17T15:29:37.218872+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:29:37.218872+0000","last_undegraded":"2024-09-17T15:29:37.218872+0000","last_fullsized":"2024-09-17T15:29:37.218872+0000","mapping_epoch":33,"log_start":"34'400","ondisk_log_start":"34'400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42733568,"num_objects":10433,"num_object_clones":0,"num_object_copies":20866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10433,"num_write_kb":41732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'10262","reported_seq":10391,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:36.781511+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:29:36.781511+0000","last_peered":"2024-09-17T15:29:36.781511+0000","last_clean":"2024-09-17T15:29:36.781511+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:29:36.781511+0000","last_undegraded":"2024-09-17T15:29:36.781511+0000","last_fullsized":"2024-09-17T15:29:36.781511+0000","mapping_epoch":32,"log_start":"34'200","ondisk_log_start":"34'200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42004963,"num_objects":10256,"num_object_clones":0,"num_object_copies":20512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10256,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7244,"num_write_kb":28976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'10564","reported_seq":10697,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:36.729641+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:29:36.729641+0000","last_peered":"2024-09-17T15:29:36.729641+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:29:36.729641+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'500","ondisk_log_start":"34'500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43250147,"num_objects":10560,"num_object_clones":0,"num_object_copies":21120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10560,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7063,"num_write_kb":28252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6290},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'10327","reported_seq":10464,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:36.759196+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:29:36.759196+0000","last_peered":"2024-09-17T15:29:36.759196+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:29:36.759196+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'300","ondisk_log_start":"34'300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42275782,"num_objects":10323,"num_object_clones":0,"num_object_copies":20646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10323,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7096,"num_write_kb":28384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6149},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'10401","reported_seq":10529,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:36.738254+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:29:36.738254+0000","last_peered":"2024-09-17T15:29:36.738254+0000","last_clean":"2024-09-17T15:29:36.738254+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:29:36.738254+0000","last_undegraded":"2024-09-17T15:29:36.738254+0000","last_fullsized":"2024-09-17T15:29:36.738254+0000","mapping_epoch":32,"log_start":"34'400","ondisk_log_start":"34'400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42594304,"num_objects":10399,"num_object_clones":0,"num_object_copies":20798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7355,"num_write_kb":29420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'10450","reported_seq":18732,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:41.002891+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:29:41.002891+0000","last_peered":"2024-09-17T15:29:41.002891+0000","last_clean":"2024-09-17T15:29:41.002891+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:29:41.002891+0000","last_undegraded":"2024-09-17T15:29:41.002891+0000","last_fullsized":"2024-09-17T15:29:41.002891+0000","mapping_epoch":42,"log_start":"34'400","ondisk_log_start":"34'400","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42463715,"num_objects":10368,"num_object_clones":0,"num_object_copies":20736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10368,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8450,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804747+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804747+0000","last_peered":"2024-09-17T15:28:23.804747+0000","last_clean":"2024-09-17T15:28:23.804747+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804747+0000","last_undegraded":"2024-09-17T15:28:23.804747+0000","last_fullsized":"2024-09-17T15:28:23.804747+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'10209","reported_seq":10339,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:36.711798+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:29:36.711798+0000","last_peered":"2024-09-17T15:29:36.711798+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:29:36.711798+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'200","ondisk_log_start":"34'200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41783296,"num_objects":10201,"num_object_clones":0,"num_object_copies":20402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6805,"num_write_kb":27220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6191},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'10212","reported_seq":10333,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:37.272579+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:29:37.272579+0000","last_peered":"2024-09-17T15:29:37.272579+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:29:37.272579+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'200","ondisk_log_start":"34'200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41808355,"num_objects":10208,"num_object_clones":0,"num_object_copies":20416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10208,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8107,"num_write_kb":32428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":6131}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'10331","reported_seq":10450,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:37.263491+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:29:37.263491+0000","last_peered":"2024-09-17T15:29:37.263491+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:29:37.263491+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'300","ondisk_log_start":"34'300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42291200,"num_objects":10325,"num_object_clones":0,"num_object_copies":20650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8093,"num_write_kb":32372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":6188}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'10413","reported_seq":10548,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:36.697612+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:29:36.697612+0000","last_peered":"2024-09-17T15:29:36.697612+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:29:36.697612+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'400","ondisk_log_start":"34'400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42643456,"num_objects":10411,"num_object_clones":0,"num_object_copies":20822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6961,"num_write_kb":27844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6273},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'10318","reported_seq":10460,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:36.590350+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:29:36.590350+0000","last_peered":"2024-09-17T15:29:36.590350+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:29:36.590350+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'300","ondisk_log_start":"34'300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42229760,"num_objects":10310,"num_object_clones":0,"num_object_copies":20620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6605,"num_write_kb":26420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6153},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805107+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805107+0000","last_peered":"2024-09-17T15:28:23.805107+0000","last_clean":"2024-09-17T15:28:23.805107+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805107+0000","last_undegraded":"2024-09-17T15:28:23.805107+0000","last_fullsized":"2024-09-17T15:28:23.805107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'10384","reported_seq":10516,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:36.696560+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:29:36.696560+0000","last_peered":"2024-09-17T15:29:36.696560+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:29:36.696560+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'300","ondisk_log_start":"34'300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42516480,"num_objects":10380,"num_object_clones":0,"num_object_copies":20760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7171,"num_write_kb":28684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6317},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'10257","reported_seq":13421,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:29:36.774536+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:29:36.774536+0000","last_peered":"2024-09-17T15:29:36.774536+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:29:36.774536+0000","last_undegraded":"2024-09-17T15:29:36.774536+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"34'200","ondisk_log_start":"34'200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41964486,"num_objects":10247,"num_object_clones":0,"num_object_copies":20494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2599,"num_objects_unfound":0,"num_objects_dirty":10247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6699,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1510,"num_bytes_recovered":6184960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7648},{"shards":"0,2","objects":2599}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":677884638,"num_objects":165508,"num_object_clones":0,"num_object_copies":331016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":2599,"num_objects_unfound":0,"num_objects_dirty":165508,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":125691,"num_write_kb":502612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5582,"num_bytes_recovered":22856163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1381707776,"data_stored":1381551958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160556,"ondisk_log_size":160556,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051639,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":738200,"kb_used_data":463560,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93633640,"statfs":{"total":96636764160,"available":95880847360,"internally_reserved":0,"allocated":474685440,"data_stored":474408866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":281210540},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,1,0,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724585,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":554104,"kb_used_data":323880,"kb_used_omap":4,"kb_used_meta":230203,"kb_avail":93817736,"statfs":{"total":96636764160,"available":96069361664,"internally_reserved":0,"allocated":331653120,"data_stored":331325041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":235728555},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,3,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247867,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":882172,"kb_used_data":561772,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93489668,"statfs":{"total":96636764160,"available":95733420032,"internally_reserved":0,"allocated":575254528,"data_stored":574973877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":328068776},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":576057344,"data_stored":576021214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":331292672,"data_stored":331209114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":474357760,"data_stored":474321630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:29:42.608 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:29:43.171 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:29:43.171 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:29:43.201 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":195,"stamp":"2024-09-17T15:29:41.658092+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":677884657,"num_objects":165510,"num_object_clones":0,"num_object_copies":331020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":2599,"num_objects_unfound":0,"num_objects_dirty":165510,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":125693,"num_write_kb":502614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5583,"num_bytes_recovered":22856182,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160559,"ondisk_log_size":160559,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2174476,"kb_used_data":1349212,"kb_used_omap":13,"kb_used_meta":825202,"kb_avail":280941044,"statfs":{"total":289910292480,"available":287683629056,"internally_reserved":0,"allocated":1381593088,"data_stored":1380707784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":845007871},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,8,5,10,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18460672,"num_objects":4507,"num_object_clones":0,"num_object_copies":9014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-211,"num_objects_unfound":0,"num_objects_dirty":4507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4583,"num_write_kb":18180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":864256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004168"},"pg_stats":[{"pgid":"2.f","version":"43'10267","reported_seq":10342,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:36.769683+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:29:36.769683+0000","last_peered":"2024-09-17T15:29:36.769683+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:29:36.769683+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'200","ondisk_log_start":"34'200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42053632,"num_objects":10267,"num_object_clones":0,"num_object_copies":20534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10267,"num_write_kb":41068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6197},{"shards":"0,2","objects":4070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'10416","reported_seq":10518,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:37.301478+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:29:37.301478+0000","last_peered":"2024-09-17T15:29:37.301478+0000","last_clean":"2024-09-17T15:29:37.301478+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:29:37.301478+0000","last_undegraded":"2024-09-17T15:29:37.301478+0000","last_fullsized":"2024-09-17T15:29:37.301478+0000","mapping_epoch":33,"log_start":"34'400","ondisk_log_start":"34'400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42640326,"num_objects":10412,"num_object_clones":0,"num_object_copies":20824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10412,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8773,"num_write_kb":35092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'10412","reported_seq":10537,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:37.370327+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:29:37.370327+0000","last_peered":"2024-09-17T15:29:37.370327+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:29:37.370327+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'400","ondisk_log_start":"34'400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42631168,"num_objects":10408,"num_object_clones":0,"num_object_copies":20816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8569,"num_write_kb":34276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4168},{"shards":"2","objects":6240}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'10433","reported_seq":10502,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:37.218872+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:29:37.218872+0000","last_peered":"2024-09-17T15:29:37.218872+0000","last_clean":"2024-09-17T15:29:37.218872+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:29:37.218872+0000","last_undegraded":"2024-09-17T15:29:37.218872+0000","last_fullsized":"2024-09-17T15:29:37.218872+0000","mapping_epoch":33,"log_start":"34'400","ondisk_log_start":"34'400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42733568,"num_objects":10433,"num_object_clones":0,"num_object_copies":20866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10433,"num_write_kb":41732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'10262","reported_seq":10391,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:36.781511+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:29:36.781511+0000","last_peered":"2024-09-17T15:29:36.781511+0000","last_clean":"2024-09-17T15:29:36.781511+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:29:36.781511+0000","last_undegraded":"2024-09-17T15:29:36.781511+0000","last_fullsized":"2024-09-17T15:29:36.781511+0000","mapping_epoch":32,"log_start":"34'200","ondisk_log_start":"34'200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42004963,"num_objects":10256,"num_object_clones":0,"num_object_copies":20512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10256,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7244,"num_write_kb":28976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'10564","reported_seq":10697,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:36.729641+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:29:36.729641+0000","last_peered":"2024-09-17T15:29:36.729641+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:29:36.729641+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'500","ondisk_log_start":"34'500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43250147,"num_objects":10560,"num_object_clones":0,"num_object_copies":21120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10560,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7063,"num_write_kb":28252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6290},{"shards":"0,2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'10327","reported_seq":10464,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:36.759196+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:29:36.759196+0000","last_peered":"2024-09-17T15:29:36.759196+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:29:36.759196+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'300","ondisk_log_start":"34'300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42275782,"num_objects":10323,"num_object_clones":0,"num_object_copies":20646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10323,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7096,"num_write_kb":28384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6149},{"shards":"0,2","objects":4174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'10401","reported_seq":10529,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:36.738254+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:29:36.738254+0000","last_peered":"2024-09-17T15:29:36.738254+0000","last_clean":"2024-09-17T15:29:36.738254+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:29:36.738254+0000","last_undegraded":"2024-09-17T15:29:36.738254+0000","last_fullsized":"2024-09-17T15:29:36.738254+0000","mapping_epoch":32,"log_start":"34'400","ondisk_log_start":"34'400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42594304,"num_objects":10399,"num_object_clones":0,"num_object_copies":20798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7355,"num_write_kb":29420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'10450","reported_seq":18732,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:41.002891+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:29:41.002891+0000","last_peered":"2024-09-17T15:29:41.002891+0000","last_clean":"2024-09-17T15:29:41.002891+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:29:41.002891+0000","last_undegraded":"2024-09-17T15:29:41.002891+0000","last_fullsized":"2024-09-17T15:29:41.002891+0000","mapping_epoch":42,"log_start":"34'400","ondisk_log_start":"34'400","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42463715,"num_objects":10368,"num_object_clones":0,"num_object_copies":20736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10368,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8450,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804747+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804747+0000","last_peered":"2024-09-17T15:28:23.804747+0000","last_clean":"2024-09-17T15:28:23.804747+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804747+0000","last_undegraded":"2024-09-17T15:28:23.804747+0000","last_fullsized":"2024-09-17T15:28:23.804747+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'10209","reported_seq":10339,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:36.711798+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:29:36.711798+0000","last_peered":"2024-09-17T15:29:36.711798+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:29:36.711798+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'200","ondisk_log_start":"34'200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41783296,"num_objects":10201,"num_object_clones":0,"num_object_copies":20402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6805,"num_write_kb":27220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6191},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'10212","reported_seq":10333,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:37.272579+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:29:37.272579+0000","last_peered":"2024-09-17T15:29:37.272579+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:29:37.272579+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'200","ondisk_log_start":"34'200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41808355,"num_objects":10208,"num_object_clones":0,"num_object_copies":20416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10208,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8107,"num_write_kb":32428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4077},{"shards":"2","objects":6131}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'10331","reported_seq":10450,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:37.263491+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:29:37.263491+0000","last_peered":"2024-09-17T15:29:37.263491+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:29:37.263491+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'300","ondisk_log_start":"34'300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42291200,"num_objects":10325,"num_object_clones":0,"num_object_copies":20650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8093,"num_write_kb":32372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4137},{"shards":"2","objects":6188}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'10413","reported_seq":10548,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:36.697612+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:29:36.697612+0000","last_peered":"2024-09-17T15:29:36.697612+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:29:36.697612+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'400","ondisk_log_start":"34'400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42643456,"num_objects":10411,"num_object_clones":0,"num_object_copies":20822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6961,"num_write_kb":27844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6273},{"shards":"0,2","objects":4138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'10318","reported_seq":10460,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:36.590350+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:29:36.590350+0000","last_peered":"2024-09-17T15:29:36.590350+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:29:36.590350+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'300","ondisk_log_start":"34'300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42229760,"num_objects":10310,"num_object_clones":0,"num_object_copies":20620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6605,"num_write_kb":26420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6153},{"shards":"0,2","objects":4157}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805107+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805107+0000","last_peered":"2024-09-17T15:28:23.805107+0000","last_clean":"2024-09-17T15:28:23.805107+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805107+0000","last_undegraded":"2024-09-17T15:28:23.805107+0000","last_fullsized":"2024-09-17T15:28:23.805107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'10384","reported_seq":10516,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:36.696560+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:29:36.696560+0000","last_peered":"2024-09-17T15:29:36.696560+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:29:36.696560+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'300","ondisk_log_start":"34'300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42516480,"num_objects":10380,"num_object_clones":0,"num_object_copies":20760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7171,"num_write_kb":28684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6317},{"shards":"0,2","objects":4063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'10257","reported_seq":13421,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:29:36.774536+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:29:36.774536+0000","last_peered":"2024-09-17T15:29:36.774536+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:29:36.774536+0000","last_undegraded":"2024-09-17T15:29:36.774536+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"34'200","ondisk_log_start":"34'200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41964486,"num_objects":10247,"num_object_clones":0,"num_object_copies":20494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2599,"num_objects_unfound":0,"num_objects_dirty":10247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6699,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1510,"num_bytes_recovered":6184960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7648},{"shards":"0,2","objects":2599}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":677884638,"num_objects":165508,"num_object_clones":0,"num_object_copies":331016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":2599,"num_objects_unfound":0,"num_objects_dirty":165508,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":125691,"num_write_kb":502612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5582,"num_bytes_recovered":22856163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1381707776,"data_stored":1381551958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160556,"ondisk_log_size":160556,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051639,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":738200,"kb_used_data":463560,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93633640,"statfs":{"total":96636764160,"available":95880847360,"internally_reserved":0,"allocated":474685440,"data_stored":474408866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":281210540},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,1,0,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724585,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":554104,"kb_used_data":323880,"kb_used_omap":4,"kb_used_meta":230203,"kb_avail":93817736,"statfs":{"total":96636764160,"available":96069361664,"internally_reserved":0,"allocated":331653120,"data_stored":331325041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":235728555},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,3,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247867,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":882172,"kb_used_data":561772,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93489668,"statfs":{"total":96636764160,"available":95733420032,"internally_reserved":0,"allocated":575254528,"data_stored":574973877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":328068776},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":576057344,"data_stored":576021214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":331292672,"data_stored":331209114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":474357760,"data_stored":474321630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:29:43.202 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:29:43.202 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'10267", 'reported_seq': 10342, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:36.769683+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:29:36.769683+0000', 'last_peered': '2024-09-17T15:29:36.769683+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:29:36.769683+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "34'200", 'ondisk_log_start': "34'200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42053632, 'num_objects': 10267, 'num_object_clones': 0, 'num_object_copies': 20534, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4070, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10267, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10267, 'num_write_kb': 41068, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6197}, {'shards': '0,2', 'objects': 4070}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:43.203 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:29:43.203 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'10412", 'reported_seq': 10537, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:37.370327+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:29:37.370327+0000', 'last_peered': '2024-09-17T15:29:37.370327+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:29:37.370327+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "34'400", 'ondisk_log_start': "34'400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42631168, 'num_objects': 10408, 'num_object_clones': 0, 'num_object_copies': 20816, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4169, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10408, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8569, 'num_write_kb': 34276, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4168}, {'shards': '2', 'objects': 6240}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:29:43.203 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:29:43.203 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'10564", 'reported_seq': 10697, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:36.729641+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:29:36.729641+0000', 'last_peered': '2024-09-17T15:29:36.729641+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:29:36.729641+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "34'500", 'ondisk_log_start': "34'500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 43250147, 'num_objects': 10560, 'num_object_clones': 0, 'num_object_copies': 21120, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4271, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10560, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7063, 'num_write_kb': 28252, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6290}, {'shards': '0,2', 'objects': 4268}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:43.203 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:29:43.203 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'10327", 'reported_seq': 10464, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:36.759196+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:29:36.759196+0000', 'last_peered': '2024-09-17T15:29:36.759196+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:29:36.759196+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "34'300", 'ondisk_log_start': "34'300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42275782, 'num_objects': 10323, 'num_object_clones': 0, 'num_object_copies': 20646, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4174, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10323, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 7096, 'num_write_kb': 28384, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6149}, {'shards': '0,2', 'objects': 4174}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:43.203 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:29:43.203 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'10209", 'reported_seq': 10339, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:36.711798+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:29:36.711798+0000', 'last_peered': '2024-09-17T15:29:36.711798+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:29:36.711798+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "34'200", 'ondisk_log_start': "34'200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41783296, 'num_objects': 10201, 'num_object_clones': 0, 'num_object_copies': 20402, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4012, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10201, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6805, 'num_write_kb': 27220, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6191}, {'shards': '0,2', 'objects': 4010}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:43.203 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:29:43.203 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'10212", 'reported_seq': 10333, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:37.272579+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:29:37.272579+0000', 'last_peered': '2024-09-17T15:29:37.272579+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:29:37.272579+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "34'200", 'ondisk_log_start': "34'200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41808355, 'num_objects': 10208, 'num_object_clones': 0, 'num_object_copies': 20416, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4078, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10208, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8107, 'num_write_kb': 32428, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4077}, {'shards': '2', 'objects': 6131}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:29:43.203 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:29:43.204 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'10331", 'reported_seq': 10450, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:37.263491+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:29:37.263491+0000', 'last_peered': '2024-09-17T15:29:37.263491+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:29:37.263491+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "34'300", 'ondisk_log_start': "34'300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42291200, 'num_objects': 10325, 'num_object_clones': 0, 'num_object_copies': 20650, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10325, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8093, 'num_write_kb': 32372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4137}, {'shards': '2', 'objects': 6188}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:29:43.204 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:29:43.204 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'10413", 'reported_seq': 10548, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:36.697612+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:29:36.697612+0000', 'last_peered': '2024-09-17T15:29:36.697612+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:29:36.697612+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "34'400", 'ondisk_log_start': "34'400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42643456, 'num_objects': 10411, 'num_object_clones': 0, 'num_object_copies': 20822, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10411, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6961, 'num_write_kb': 27844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6273}, {'shards': '0,2', 'objects': 4138}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:43.204 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:29:43.204 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'10318", 'reported_seq': 10460, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:36.590350+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:29:36.590350+0000', 'last_peered': '2024-09-17T15:29:36.590350+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:29:36.590350+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "34'300", 'ondisk_log_start': "34'300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42229760, 'num_objects': 10310, 'num_object_clones': 0, 'num_object_copies': 20620, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4160, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10310, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6605, 'num_write_kb': 26420, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6153}, {'shards': '0,2', 'objects': 4157}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:43.204 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:29:43.204 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'10384", 'reported_seq': 10516, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:36.696560+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:29:36.696560+0000', 'last_peered': '2024-09-17T15:29:36.696560+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:29:36.696560+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "34'300", 'ondisk_log_start': "34'300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42516480, 'num_objects': 10380, 'num_object_clones': 0, 'num_object_copies': 20760, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4065, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10380, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7171, 'num_write_kb': 28684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6317}, {'shards': '0,2', 'objects': 4063}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:43.204 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:29:43.204 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'10257", 'reported_seq': 13421, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:29:36.774536+0000', 'last_change': '2024-09-17T15:28:22.509818+0000', 'last_active': '2024-09-17T15:29:36.774536+0000', 'last_peered': '2024-09-17T15:29:36.774536+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:29:36.774536+0000', 'last_undegraded': '2024-09-17T15:29:36.774536+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "34'200", 'ondisk_log_start': "34'200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41964486, 'num_objects': 10247, 'num_object_clones': 0, 'num_object_copies': 20494, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2599, 'num_objects_unfound': 0, 'num_objects_dirty': 10247, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6699, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1510, 'num_bytes_recovered': 6184960, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7648}, {'shards': '0,2', 'objects': 2599}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:43.205 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:29:43.896 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:29:43.896 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41277/331020 objects degraded (12.470%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":320,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":165510,"data_bytes":677884657,"bytes_used":2226663424,"bytes_avail":287683629056,"bytes_total":289910292480,"degraded_objects":41277,"degraded_total":331020,"degraded_ratio":0.12469639296719232,"misplaced_objects":2599,"misplaced_total":331020,"misplaced_ratio":0.0078514893359917827,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":71996,"recovering_keys_per_sec":0,"num_objects_recovered":211,"num_bytes_recovered":864256,"num_keys_recovered":0,"write_bytes_sec":1550821,"read_op_per_sec":0,"write_op_per_sec":381},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T15:27:19.616376+0000","services":{}},"progress_events":{}} 2024-09-17T15:29:43.928 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:29:43.928 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:29:44.468 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:29:44.470 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:29:44.493 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":196,"stamp":"2024-09-17T15:29:43.658795+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":676754185,"num_objects":165235,"num_object_clones":0,"num_object_copies":330470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41254,"num_objects_misplaced":2481,"num_objects_unfound":0,"num_objects_dirty":165235,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":127756,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5724,"num_bytes_recovered":23339510,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160822,"ondisk_log_size":160822,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2183524,"kb_used_data":1349932,"kb_used_omap":13,"kb_used_meta":833522,"kb_avail":280931996,"statfs":{"total":289910292480,"available":287674363904,"internally_reserved":0,"allocated":1382330368,"data_stored":1381440992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":853527551},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2,4,7,14,12,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16810008,"num_objects":4105,"num_object_clones":0,"num_object_copies":8210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-23,"num_objects_misplaced":-329,"num_objects_unfound":0,"num_objects_dirty":4105,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6519,"num_write_kb":21245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":352,"num_bytes_recovered":1347584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004219"},"pg_stats":[{"pgid":"2.f","version":"43'10397","reported_seq":10473,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:41.846119+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:29:41.846119+0000","last_peered":"2024-09-17T15:29:41.846119+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:29:41.846119+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'300","ondisk_log_start":"34'300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41988096,"num_objects":10251,"num_object_clones":0,"num_object_copies":20502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10251,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10397,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6181},{"shards":"0,2","objects":3997}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'10564","reported_seq":10666,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:42.323033+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:29:42.323033+0000","last_peered":"2024-09-17T15:29:42.323033+0000","last_clean":"2024-09-17T15:29:42.323033+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:29:42.323033+0000","last_undegraded":"2024-09-17T15:29:42.323033+0000","last_fullsized":"2024-09-17T15:29:42.323033+0000","mapping_epoch":33,"log_start":"34'500","ondisk_log_start":"34'500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42468294,"num_objects":10370,"num_object_clones":0,"num_object_copies":20740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10370,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8921,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'10565","reported_seq":10701,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:42.341427+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:29:42.341427+0000","last_peered":"2024-09-17T15:29:42.341427+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:29:42.341427+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'500","ondisk_log_start":"34'500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42467352,"num_objects":10369,"num_object_clones":0,"num_object_copies":20738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4164,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10369,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8722,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4072},{"shards":"2","objects":6206}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'10566","reported_seq":10635,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:42.412985+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:29:42.412985+0000","last_peered":"2024-09-17T15:29:42.412985+0000","last_clean":"2024-09-17T15:29:42.412985+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:29:42.412985+0000","last_undegraded":"2024-09-17T15:29:42.412985+0000","last_fullsized":"2024-09-17T15:29:42.412985+0000","mapping_epoch":33,"log_start":"34'500","ondisk_log_start":"34'500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42614784,"num_objects":10404,"num_object_clones":0,"num_object_copies":20808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10566,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'10406","reported_seq":10535,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:41.708873+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:29:41.708873+0000","last_peered":"2024-09-17T15:29:41.708873+0000","last_clean":"2024-09-17T15:29:41.708873+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:29:41.708873+0000","last_undegraded":"2024-09-17T15:29:41.708873+0000","last_fullsized":"2024-09-17T15:29:41.708873+0000","mapping_epoch":32,"log_start":"34'400","ondisk_log_start":"34'400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41972195,"num_objects":10248,"num_object_clones":0,"num_object_copies":20496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10248,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7388,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'10708","reported_seq":10847,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:41.657866+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:29:41.657866+0000","last_peered":"2024-09-17T15:29:41.657866+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:29:41.657866+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'700","ondisk_log_start":"34'700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43184611,"num_objects":10544,"num_object_clones":0,"num_object_copies":21088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4269,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10544,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7207,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6276},{"shards":"0,2","objects":4188}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'10469","reported_seq":10614,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:41.771244+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:29:41.771244+0000","last_peered":"2024-09-17T15:29:41.771244+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:29:41.771244+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'400","ondisk_log_start":"34'400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42185670,"num_objects":10301,"num_object_clones":0,"num_object_copies":20602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4171,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10301,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7238,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6130},{"shards":"0,2","objects":4092}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'10531","reported_seq":10659,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:41.660681+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:29:41.660681+0000","last_peered":"2024-09-17T15:29:41.660681+0000","last_clean":"2024-09-17T15:29:41.660681+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:29:41.660681+0000","last_undegraded":"2024-09-17T15:29:41.660681+0000","last_fullsized":"2024-09-17T15:29:41.660681+0000","mapping_epoch":32,"log_start":"34'500","ondisk_log_start":"34'500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42536960,"num_objects":10385,"num_object_clones":0,"num_object_copies":20770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7485,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'10450","reported_seq":18732,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:41.002891+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:29:41.002891+0000","last_peered":"2024-09-17T15:29:41.002891+0000","last_clean":"2024-09-17T15:29:41.002891+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:29:41.002891+0000","last_undegraded":"2024-09-17T15:29:41.002891+0000","last_fullsized":"2024-09-17T15:29:41.002891+0000","mapping_epoch":42,"log_start":"34'400","ondisk_log_start":"34'400","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42463715,"num_objects":10368,"num_object_clones":0,"num_object_copies":20736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10368,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8450,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804747+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804747+0000","last_peered":"2024-09-17T15:28:23.804747+0000","last_clean":"2024-09-17T15:28:23.804747+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804747+0000","last_undegraded":"2024-09-17T15:28:23.804747+0000","last_fullsized":"2024-09-17T15:28:23.804747+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'10346","reported_seq":10477,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:41.796583+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:29:41.796583+0000","last_peered":"2024-09-17T15:29:41.796583+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:29:41.796583+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'300","ondisk_log_start":"34'300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41762816,"num_objects":10196,"num_object_clones":0,"num_object_copies":20392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4010,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10196,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6942,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6188},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'10344","reported_seq":10469,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:42.344778+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:29:42.344778+0000","last_peered":"2024-09-17T15:29:42.344778+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:29:42.344778+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'300","ondisk_log_start":"34'300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41677283,"num_objects":10176,"num_object_clones":0,"num_object_copies":20352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4076,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10176,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8239,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3995},{"shards":"2","objects":6101}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'10468","reported_seq":10595,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:42.387396+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:29:42.387396+0000","last_peered":"2024-09-17T15:29:42.387396+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:29:42.387396+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'400","ondisk_log_start":"34'400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42057728,"num_objects":10268,"num_object_clones":0,"num_object_copies":20536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4135,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8230,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4040},{"shards":"2","objects":6135}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'10547","reported_seq":10682,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:41.749046+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:29:41.749046+0000","last_peered":"2024-09-17T15:29:41.749046+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:29:41.749046+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'500","ondisk_log_start":"34'500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42676224,"num_objects":10419,"num_object_clones":0,"num_object_copies":20838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10419,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7095,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6281},{"shards":"0,2","objects":4075}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'10462","reported_seq":10606,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:41.736244+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:29:41.736244+0000","last_peered":"2024-09-17T15:29:41.736244+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:29:41.736244+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'400","ondisk_log_start":"34'400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42229760,"num_objects":10310,"num_object_clones":0,"num_object_copies":20620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4159,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6749,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6154},{"shards":"0,2","objects":4085}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805107+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805107+0000","last_peered":"2024-09-17T15:28:23.805107+0000","last_clean":"2024-09-17T15:28:23.805107+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805107+0000","last_undegraded":"2024-09-17T15:28:23.805107+0000","last_fullsized":"2024-09-17T15:28:23.805107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'10511","reported_seq":10651,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:41.680548+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:29:41.680548+0000","last_peered":"2024-09-17T15:29:41.680548+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:29:41.680548+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'500","ondisk_log_start":"34'500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42496000,"num_objects":10375,"num_object_clones":0,"num_object_copies":20750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4061,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7298,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6316},{"shards":"0,2","objects":3997}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'10385","reported_seq":13785,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:29:41.814267+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:29:41.814267+0000","last_peered":"2024-09-17T15:29:41.814267+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:29:41.814267+0000","last_undegraded":"2024-09-17T15:29:41.814267+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"34'300","ondisk_log_start":"34'300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41972678,"num_objects":10249,"num_object_clones":0,"num_object_copies":20498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2481,"num_objects_unfound":0,"num_objects_dirty":10249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6827,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1628,"num_bytes_recovered":6668288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7768},{"shards":"0,2","objects":2481}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":676754166,"num_objects":165233,"num_object_clones":0,"num_object_copies":330466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41254,"num_objects_misplaced":2481,"num_objects_unfound":0,"num_objects_dirty":165233,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":127754,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5723,"num_bytes_recovered":23339491,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1379999744,"data_stored":1379839854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160819,"ondisk_log_size":160819,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051640,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":741212,"kb_used_data":462408,"kb_used_omap":4,"kb_used_meta":278779,"kb_avail":93630628,"statfs":{"total":96636764160,"available":95877763072,"internally_reserved":0,"allocated":473505792,"data_stored":473225146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":285470380},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,0,1,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724585,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":554104,"kb_used_data":323880,"kb_used_omap":4,"kb_used_meta":230203,"kb_avail":93817736,"statfs":{"total":96636764160,"available":96069361664,"internally_reserved":0,"allocated":331653120,"data_stored":331325041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":235728555},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,3,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247868,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":888208,"kb_used_data":563644,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93483632,"statfs":{"total":96636764160,"available":95727239168,"internally_reserved":0,"allocated":577171456,"data_stored":576890805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":332328616},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,8,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":575578112,"data_stored":575541982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":331292672,"data_stored":331209114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":473128960,"data_stored":473088758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:29:47.495 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:29:48.051 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:29:48.051 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:29:48.074 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":198,"stamp":"2024-09-17T15:29:47.660267+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":667665161,"num_objects":163016,"num_object_clones":0,"num_object_copies":326032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41230,"num_objects_misplaced":2346,"num_objects_unfound":0,"num_objects_dirty":163016,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":129975,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5883,"num_bytes_recovered":23892470,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160641,"ondisk_log_size":160641,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2186776,"kb_used_data":1339292,"kb_used_omap":13,"kb_used_meta":847410,"kb_avail":280928744,"statfs":{"total":289910292480,"available":287671033856,"internally_reserved":0,"allocated":1371435008,"data_stored":1370545632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":867748863},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,2,1,9,12,2,5,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-905192,"num_objects":-220,"num_object_clones":0,"num_object_copies":-440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-47,"num_objects_misplaced":-367,"num_objects_unfound":0,"num_objects_dirty":-220,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6632,"num_write_kb":12821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":414,"num_bytes_recovered":1503232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004489"},"pg_stats":[{"pgid":"2.f","version":"43'10537","reported_seq":10621,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:46.839921+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:29:46.839921+0000","last_peered":"2024-09-17T15:29:46.839921+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:29:46.839921+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'500","ondisk_log_start":"34'500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41414656,"num_objects":10111,"num_object_clones":0,"num_object_copies":20222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10537,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6044},{"shards":"0,2","objects":3857}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'10708","reported_seq":10810,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:47.390054+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:29:47.390054+0000","last_peered":"2024-09-17T15:29:47.390054+0000","last_clean":"2024-09-17T15:29:47.390054+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:29:47.390054+0000","last_undegraded":"2024-09-17T15:29:47.390054+0000","last_fullsized":"2024-09-17T15:29:47.390054+0000","mapping_epoch":33,"log_start":"34'700","ondisk_log_start":"34'700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41878470,"num_objects":10226,"num_object_clones":0,"num_object_copies":20452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10226,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9065,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'10705","reported_seq":10840,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:47.419348+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:29:47.419348+0000","last_peered":"2024-09-17T15:29:47.419348+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:29:47.419348+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'700","ondisk_log_start":"34'700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41893912,"num_objects":10229,"num_object_clones":0,"num_object_copies":20458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4164,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10229,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8862,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":6066}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'10703","reported_seq":10772,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:47.381660+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:29:47.381660+0000","last_peered":"2024-09-17T15:29:47.381660+0000","last_clean":"2024-09-17T15:29:47.381660+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:29:47.381660+0000","last_undegraded":"2024-09-17T15:29:47.381660+0000","last_fullsized":"2024-09-17T15:29:47.381660+0000","mapping_epoch":33,"log_start":"34'700","ondisk_log_start":"34'700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42053632,"num_objects":10267,"num_object_clones":0,"num_object_copies":20534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10703,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'10518","reported_seq":10647,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:46.852247+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:29:46.852247+0000","last_peered":"2024-09-17T15:29:46.852247+0000","last_clean":"2024-09-17T15:29:46.852247+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:29:46.852247+0000","last_undegraded":"2024-09-17T15:29:46.852247+0000","last_fullsized":"2024-09-17T15:29:46.852247+0000","mapping_epoch":32,"log_start":"34'500","ondisk_log_start":"34'500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41513443,"num_objects":10136,"num_object_clones":0,"num_object_copies":20272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10136,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7500,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'10855","reported_seq":10996,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:46.844198+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:29:46.844198+0000","last_peered":"2024-09-17T15:29:46.844198+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:29:46.844198+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'800","ondisk_log_start":"34'800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42582499,"num_objects":10397,"num_object_clones":0,"num_object_copies":20794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10397,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7354,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6130},{"shards":"0,2","objects":4041}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'10612","reported_seq":10763,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:46.812802+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:29:46.812802+0000","last_peered":"2024-09-17T15:29:46.812802+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:29:46.812802+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'600","ondisk_log_start":"34'600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41599942,"num_objects":10158,"num_object_clones":0,"num_object_copies":20316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4168,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10158,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7381,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5990},{"shards":"0,2","objects":3949}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'10667","reported_seq":10795,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:46.826876+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:29:46.826876+0000","last_peered":"2024-09-17T15:29:46.826876+0000","last_clean":"2024-09-17T15:29:46.826876+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:29:46.826876+0000","last_undegraded":"2024-09-17T15:29:46.826876+0000","last_fullsized":"2024-09-17T15:29:46.826876+0000","mapping_epoch":32,"log_start":"34'600","ondisk_log_start":"34'600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41979904,"num_objects":10249,"num_object_clones":0,"num_object_copies":20498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7621,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'10592","reported_seq":18874,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:46.096328+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:29:46.096328+0000","last_peered":"2024-09-17T15:29:46.096328+0000","last_clean":"2024-09-17T15:29:46.096328+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:29:46.096328+0000","last_undegraded":"2024-09-17T15:29:46.096328+0000","last_fullsized":"2024-09-17T15:29:46.096328+0000","mapping_epoch":42,"log_start":"34'500","ondisk_log_start":"34'500","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41882083,"num_objects":10226,"num_object_clones":0,"num_object_copies":20452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10226,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8592,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804747+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804747+0000","last_peered":"2024-09-17T15:28:23.804747+0000","last_clean":"2024-09-17T15:28:23.804747+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804747+0000","last_undegraded":"2024-09-17T15:28:23.804747+0000","last_fullsized":"2024-09-17T15:28:23.804747+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'10516","reported_seq":10654,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:46.829041+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:29:46.829041+0000","last_peered":"2024-09-17T15:29:46.829041+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:29:46.829041+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'500","ondisk_log_start":"34'500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41066496,"num_objects":10026,"num_object_clones":0,"num_object_copies":20052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4008,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7112,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6020},{"shards":"0,2","objects":3769}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'10480","reported_seq":10607,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:47.354934+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:29:47.354934+0000","last_peered":"2024-09-17T15:29:47.354934+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:29:47.354934+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'400","ondisk_log_start":"34'400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41120227,"num_objects":10040,"num_object_clones":0,"num_object_copies":20080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4075,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10040,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8375,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3859},{"shards":"2","objects":5966}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'10610","reported_seq":10741,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:47.276163+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:29:47.276163+0000","last_peered":"2024-09-17T15:29:47.276163+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:29:47.276163+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'600","ondisk_log_start":"34'600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41476096,"num_objects":10126,"num_object_clones":0,"num_object_copies":20252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4133,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8372,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3898},{"shards":"2","objects":5995}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'10696","reported_seq":10843,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:46.838043+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:29:46.838043+0000","last_peered":"2024-09-17T15:29:46.838043+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:29:46.838043+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'600","ondisk_log_start":"34'600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42065920,"num_objects":10270,"num_object_clones":0,"num_object_copies":20540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4133,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7244,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6138},{"shards":"0,2","objects":3926}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'10594","reported_seq":10746,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:46.763129+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:29:46.763129+0000","last_peered":"2024-09-17T15:29:46.763129+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:29:46.763129+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'500","ondisk_log_start":"34'500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41689088,"num_objects":10178,"num_object_clones":0,"num_object_copies":20356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4155,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6881,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6026},{"shards":"0,2","objects":3953}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805107+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805107+0000","last_peered":"2024-09-17T15:28:23.805107+0000","last_clean":"2024-09-17T15:28:23.805107+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805107+0000","last_undegraded":"2024-09-17T15:28:23.805107+0000","last_fullsized":"2024-09-17T15:28:23.805107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'10629","reported_seq":10773,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:46.819856+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:29:46.819856+0000","last_peered":"2024-09-17T15:29:46.819856+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:29:46.819856+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'600","ondisk_log_start":"34'600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42012672,"num_objects":10257,"num_object_clones":0,"num_object_copies":20514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4059,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10257,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7416,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6200},{"shards":"0,2","objects":3879}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'10516","reported_seq":14186,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:29:46.849151+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:29:46.849151+0000","last_peered":"2024-09-17T15:29:46.849151+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:29:46.849151+0000","last_undegraded":"2024-09-17T15:29:46.849151+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"34'500","ondisk_log_start":"34'500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41436102,"num_objects":10118,"num_object_clones":0,"num_object_copies":20236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2346,"num_objects_unfound":0,"num_objects_dirty":10118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6958,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1763,"num_bytes_recovered":7221248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7772},{"shards":"0,2","objects":2346}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":667665142,"num_objects":163014,"num_object_clones":0,"num_object_copies":326028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41230,"num_objects_misplaced":2346,"num_objects_unfound":0,"num_objects_dirty":163014,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":129973,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5882,"num_bytes_recovered":23892451,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1368178688,"data_stored":1368018798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160638,"ondisk_log_size":160638,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051641,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":739124,"kb_used_data":458916,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93632716,"statfs":{"total":96636764160,"available":95879901184,"internally_reserved":0,"allocated":469929984,"data_stored":469649338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":286912172},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724586,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":561872,"kb_used_data":323328,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93809968,"statfs":{"total":96636764160,"available":96061407232,"internally_reserved":0,"allocated":331087872,"data_stored":330759793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":244248235},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,5,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247869,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":885780,"kb_used_data":557048,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93486060,"statfs":{"total":96636764160,"available":95729725440,"internally_reserved":0,"allocated":570417152,"data_stored":570136501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":336588456},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,3,6,0,5,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":568811520,"data_stored":568775390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":330711040,"data_stored":330627482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":468656128,"data_stored":468615926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:29:48.076 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:29:48.643 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:29:48.644 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:29:48.671 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":198,"stamp":"2024-09-17T15:29:47.660267+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":667665161,"num_objects":163016,"num_object_clones":0,"num_object_copies":326032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41230,"num_objects_misplaced":2346,"num_objects_unfound":0,"num_objects_dirty":163016,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":129975,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5883,"num_bytes_recovered":23892470,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160641,"ondisk_log_size":160641,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2186776,"kb_used_data":1339292,"kb_used_omap":13,"kb_used_meta":847410,"kb_avail":280928744,"statfs":{"total":289910292480,"available":287671033856,"internally_reserved":0,"allocated":1371435008,"data_stored":1370545632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":867748863},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,2,1,9,12,2,5,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-905192,"num_objects":-220,"num_object_clones":0,"num_object_copies":-440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-47,"num_objects_misplaced":-367,"num_objects_unfound":0,"num_objects_dirty":-220,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6632,"num_write_kb":12821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":414,"num_bytes_recovered":1503232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004489"},"pg_stats":[{"pgid":"2.f","version":"43'10537","reported_seq":10621,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:46.839921+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:29:46.839921+0000","last_peered":"2024-09-17T15:29:46.839921+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:29:46.839921+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'500","ondisk_log_start":"34'500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41414656,"num_objects":10111,"num_object_clones":0,"num_object_copies":20222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10537,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6044},{"shards":"0,2","objects":3857}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'10708","reported_seq":10810,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:47.390054+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:29:47.390054+0000","last_peered":"2024-09-17T15:29:47.390054+0000","last_clean":"2024-09-17T15:29:47.390054+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:29:47.390054+0000","last_undegraded":"2024-09-17T15:29:47.390054+0000","last_fullsized":"2024-09-17T15:29:47.390054+0000","mapping_epoch":33,"log_start":"34'700","ondisk_log_start":"34'700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41878470,"num_objects":10226,"num_object_clones":0,"num_object_copies":20452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10226,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9065,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'10705","reported_seq":10840,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:47.419348+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:29:47.419348+0000","last_peered":"2024-09-17T15:29:47.419348+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:29:47.419348+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'700","ondisk_log_start":"34'700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41893912,"num_objects":10229,"num_object_clones":0,"num_object_copies":20458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4164,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10229,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8862,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":6066}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'10703","reported_seq":10772,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:47.381660+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:29:47.381660+0000","last_peered":"2024-09-17T15:29:47.381660+0000","last_clean":"2024-09-17T15:29:47.381660+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:29:47.381660+0000","last_undegraded":"2024-09-17T15:29:47.381660+0000","last_fullsized":"2024-09-17T15:29:47.381660+0000","mapping_epoch":33,"log_start":"34'700","ondisk_log_start":"34'700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42053632,"num_objects":10267,"num_object_clones":0,"num_object_copies":20534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10703,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'10518","reported_seq":10647,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:46.852247+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:29:46.852247+0000","last_peered":"2024-09-17T15:29:46.852247+0000","last_clean":"2024-09-17T15:29:46.852247+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:29:46.852247+0000","last_undegraded":"2024-09-17T15:29:46.852247+0000","last_fullsized":"2024-09-17T15:29:46.852247+0000","mapping_epoch":32,"log_start":"34'500","ondisk_log_start":"34'500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41513443,"num_objects":10136,"num_object_clones":0,"num_object_copies":20272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10136,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7500,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'10855","reported_seq":10996,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:46.844198+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:29:46.844198+0000","last_peered":"2024-09-17T15:29:46.844198+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:29:46.844198+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'800","ondisk_log_start":"34'800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42582499,"num_objects":10397,"num_object_clones":0,"num_object_copies":20794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10397,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7354,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6130},{"shards":"0,2","objects":4041}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'10612","reported_seq":10763,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:46.812802+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:29:46.812802+0000","last_peered":"2024-09-17T15:29:46.812802+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:29:46.812802+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'600","ondisk_log_start":"34'600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41599942,"num_objects":10158,"num_object_clones":0,"num_object_copies":20316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4168,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10158,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7381,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5990},{"shards":"0,2","objects":3949}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'10667","reported_seq":10795,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:46.826876+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:29:46.826876+0000","last_peered":"2024-09-17T15:29:46.826876+0000","last_clean":"2024-09-17T15:29:46.826876+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:29:46.826876+0000","last_undegraded":"2024-09-17T15:29:46.826876+0000","last_fullsized":"2024-09-17T15:29:46.826876+0000","mapping_epoch":32,"log_start":"34'600","ondisk_log_start":"34'600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41979904,"num_objects":10249,"num_object_clones":0,"num_object_copies":20498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7621,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'10592","reported_seq":18874,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:46.096328+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:29:46.096328+0000","last_peered":"2024-09-17T15:29:46.096328+0000","last_clean":"2024-09-17T15:29:46.096328+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:29:46.096328+0000","last_undegraded":"2024-09-17T15:29:46.096328+0000","last_fullsized":"2024-09-17T15:29:46.096328+0000","mapping_epoch":42,"log_start":"34'500","ondisk_log_start":"34'500","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41882083,"num_objects":10226,"num_object_clones":0,"num_object_copies":20452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10226,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8592,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804747+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804747+0000","last_peered":"2024-09-17T15:28:23.804747+0000","last_clean":"2024-09-17T15:28:23.804747+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804747+0000","last_undegraded":"2024-09-17T15:28:23.804747+0000","last_fullsized":"2024-09-17T15:28:23.804747+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'10516","reported_seq":10654,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:46.829041+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:29:46.829041+0000","last_peered":"2024-09-17T15:29:46.829041+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:29:46.829041+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'500","ondisk_log_start":"34'500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41066496,"num_objects":10026,"num_object_clones":0,"num_object_copies":20052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4008,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7112,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6020},{"shards":"0,2","objects":3769}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'10480","reported_seq":10607,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:47.354934+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:29:47.354934+0000","last_peered":"2024-09-17T15:29:47.354934+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:29:47.354934+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'400","ondisk_log_start":"34'400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41120227,"num_objects":10040,"num_object_clones":0,"num_object_copies":20080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4075,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10040,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8375,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3859},{"shards":"2","objects":5966}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'10610","reported_seq":10741,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:47.276163+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:29:47.276163+0000","last_peered":"2024-09-17T15:29:47.276163+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:29:47.276163+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'600","ondisk_log_start":"34'600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41476096,"num_objects":10126,"num_object_clones":0,"num_object_copies":20252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4133,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8372,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3898},{"shards":"2","objects":5995}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'10696","reported_seq":10843,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:46.838043+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:29:46.838043+0000","last_peered":"2024-09-17T15:29:46.838043+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:29:46.838043+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'600","ondisk_log_start":"34'600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42065920,"num_objects":10270,"num_object_clones":0,"num_object_copies":20540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4133,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7244,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6138},{"shards":"0,2","objects":3926}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'10594","reported_seq":10746,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:46.763129+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:29:46.763129+0000","last_peered":"2024-09-17T15:29:46.763129+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:29:46.763129+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'500","ondisk_log_start":"34'500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41689088,"num_objects":10178,"num_object_clones":0,"num_object_copies":20356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4155,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6881,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6026},{"shards":"0,2","objects":3953}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805107+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805107+0000","last_peered":"2024-09-17T15:28:23.805107+0000","last_clean":"2024-09-17T15:28:23.805107+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805107+0000","last_undegraded":"2024-09-17T15:28:23.805107+0000","last_fullsized":"2024-09-17T15:28:23.805107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'10629","reported_seq":10773,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:46.819856+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:29:46.819856+0000","last_peered":"2024-09-17T15:29:46.819856+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:29:46.819856+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'600","ondisk_log_start":"34'600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42012672,"num_objects":10257,"num_object_clones":0,"num_object_copies":20514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4059,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10257,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7416,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6200},{"shards":"0,2","objects":3879}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'10516","reported_seq":14186,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:29:46.849151+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:29:46.849151+0000","last_peered":"2024-09-17T15:29:46.849151+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:29:46.849151+0000","last_undegraded":"2024-09-17T15:29:46.849151+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"34'500","ondisk_log_start":"34'500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41436102,"num_objects":10118,"num_object_clones":0,"num_object_copies":20236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2346,"num_objects_unfound":0,"num_objects_dirty":10118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6958,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1763,"num_bytes_recovered":7221248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7772},{"shards":"0,2","objects":2346}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":667665142,"num_objects":163014,"num_object_clones":0,"num_object_copies":326028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41230,"num_objects_misplaced":2346,"num_objects_unfound":0,"num_objects_dirty":163014,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":129973,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5882,"num_bytes_recovered":23892451,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1368178688,"data_stored":1368018798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160638,"ondisk_log_size":160638,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051641,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":739124,"kb_used_data":458916,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93632716,"statfs":{"total":96636764160,"available":95879901184,"internally_reserved":0,"allocated":469929984,"data_stored":469649338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":286912172},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724586,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":561872,"kb_used_data":323328,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93809968,"statfs":{"total":96636764160,"available":96061407232,"internally_reserved":0,"allocated":331087872,"data_stored":330759793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":244248235},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,5,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247869,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":885780,"kb_used_data":557048,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93486060,"statfs":{"total":96636764160,"available":95729725440,"internally_reserved":0,"allocated":570417152,"data_stored":570136501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":336588456},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,3,6,0,5,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":568811520,"data_stored":568775390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":330711040,"data_stored":330627482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":468656128,"data_stored":468615926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:29:48.672 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:29:48.672 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'10537", 'reported_seq': 10621, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:46.839921+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:29:46.839921+0000', 'last_peered': '2024-09-17T15:29:46.839921+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:29:46.839921+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "34'500", 'ondisk_log_start': "34'500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41414656, 'num_objects': 10111, 'num_object_clones': 0, 'num_object_copies': 20222, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10111, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10537, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6044}, {'shards': '0,2', 'objects': 3857}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:48.672 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:29:48.672 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'10705", 'reported_seq': 10840, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:47.419348+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:29:47.419348+0000', 'last_peered': '2024-09-17T15:29:47.419348+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:29:47.419348+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "34'700", 'ondisk_log_start': "34'700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41893912, 'num_objects': 10229, 'num_object_clones': 0, 'num_object_copies': 20458, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4164, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10229, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8862, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 5, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3932}, {'shards': '2', 'objects': 6066}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:29:48.673 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:29:48.673 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'10855", 'reported_seq': 10996, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:46.844198+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:29:46.844198+0000', 'last_peered': '2024-09-17T15:29:46.844198+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:29:46.844198+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "34'800", 'ondisk_log_start': "34'800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42582499, 'num_objects': 10397, 'num_object_clones': 0, 'num_object_copies': 20794, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10397, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7354, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6130}, {'shards': '0,2', 'objects': 4041}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:48.673 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:29:48.673 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'10612", 'reported_seq': 10763, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:46.812802+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:29:46.812802+0000', 'last_peered': '2024-09-17T15:29:46.812802+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:29:46.812802+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "34'600", 'ondisk_log_start': "34'600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41599942, 'num_objects': 10158, 'num_object_clones': 0, 'num_object_copies': 20316, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4168, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10158, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 7381, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 6, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5990}, {'shards': '0,2', 'objects': 3949}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:48.673 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:29:48.673 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'10516", 'reported_seq': 10654, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:46.829041+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:29:46.829041+0000', 'last_peered': '2024-09-17T15:29:46.829041+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:29:46.829041+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "34'500", 'ondisk_log_start': "34'500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41066496, 'num_objects': 10026, 'num_object_clones': 0, 'num_object_copies': 20052, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4008, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10026, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7112, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6020}, {'shards': '0,2', 'objects': 3769}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:48.673 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:29:48.673 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'10480", 'reported_seq': 10607, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:47.354934+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:29:47.354934+0000', 'last_peered': '2024-09-17T15:29:47.354934+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:29:47.354934+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "34'400", 'ondisk_log_start': "34'400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41120227, 'num_objects': 10040, 'num_object_clones': 0, 'num_object_copies': 20080, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4075, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10040, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8375, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3859}, {'shards': '2', 'objects': 5966}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:29:48.673 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:29:48.673 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'10610", 'reported_seq': 10741, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:47.276163+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:29:47.276163+0000', 'last_peered': '2024-09-17T15:29:47.276163+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:29:47.276163+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "34'600", 'ondisk_log_start': "34'600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41476096, 'num_objects': 10126, 'num_object_clones': 0, 'num_object_copies': 20252, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4133, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10126, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8372, 'num_write_kb': 32532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 6, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3898}, {'shards': '2', 'objects': 5995}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:29:48.673 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:29:48.674 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'10696", 'reported_seq': 10843, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:46.838043+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:29:46.838043+0000', 'last_peered': '2024-09-17T15:29:46.838043+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:29:46.838043+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "34'600", 'ondisk_log_start': "34'600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42065920, 'num_objects': 10270, 'num_object_clones': 0, 'num_object_copies': 20540, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4133, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10270, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7244, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 6, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6138}, {'shards': '0,2', 'objects': 3926}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:48.674 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:29:48.674 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'10594", 'reported_seq': 10746, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:46.763129+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:29:46.763129+0000', 'last_peered': '2024-09-17T15:29:46.763129+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:29:46.763129+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "34'500", 'ondisk_log_start': "34'500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41689088, 'num_objects': 10178, 'num_object_clones': 0, 'num_object_copies': 20356, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4155, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10178, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6881, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 5, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6026}, {'shards': '0,2', 'objects': 3953}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:48.674 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:29:48.674 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'10629", 'reported_seq': 10773, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:46.819856+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:29:46.819856+0000', 'last_peered': '2024-09-17T15:29:46.819856+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:29:46.819856+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "34'600", 'ondisk_log_start': "34'600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42012672, 'num_objects': 10257, 'num_object_clones': 0, 'num_object_copies': 20514, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4059, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10257, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7416, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 6, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6200}, {'shards': '0,2', 'objects': 3879}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:48.674 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:29:48.674 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'10516", 'reported_seq': 14186, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:29:46.849151+0000', 'last_change': '2024-09-17T15:28:22.509818+0000', 'last_active': '2024-09-17T15:29:46.849151+0000', 'last_peered': '2024-09-17T15:29:46.849151+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:29:46.849151+0000', 'last_undegraded': '2024-09-17T15:29:46.849151+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "34'500", 'ondisk_log_start': "34'500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41436102, 'num_objects': 10118, 'num_object_clones': 0, 'num_object_copies': 20236, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2346, 'num_objects_unfound': 0, 'num_objects_dirty': 10118, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6958, 'num_write_kb': 27056, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1763, 'num_bytes_recovered': 7221248, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7772}, {'shards': '0,2', 'objects': 2346}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:48.674 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:29:49.370 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:29:49.370 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41230/326032 objects degraded (12.646%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":326,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":163016,"data_bytes":667665161,"bytes_used":2239258624,"bytes_avail":287671033856,"bytes_total":289910292480,"degraded_objects":41230,"degraded_total":326032,"degraded_ratio":0.12645997938852627,"misplaced_objects":2346,"misplaced_total":326032,"misplaced_ratio":0.0071956127005938066,"recovering_objects_per_sec":34,"recovering_bytes_per_sec":125222,"recovering_keys_per_sec":0,"num_objects_recovered":414,"num_bytes_recovered":1503232,"num_keys_recovered":0,"read_bytes_sec":85,"write_bytes_sec":1093649,"read_op_per_sec":0,"write_op_per_sec":552},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T15:27:19.616376+0000","services":{}},"progress_events":{}} 2024-09-17T15:29:49.399 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:29:49.400 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:29:49.970 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:29:49.973 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:29:49.992 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":199,"stamp":"2024-09-17T15:29:49.661131+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":667665161,"num_objects":163016,"num_object_clones":0,"num_object_copies":326032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41230,"num_objects_misplaced":2346,"num_objects_unfound":0,"num_objects_dirty":163016,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":129975,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5883,"num_bytes_recovered":23892470,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160641,"ondisk_log_size":160641,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2186776,"kb_used_data":1339292,"kb_used_omap":13,"kb_used_meta":847410,"kb_avail":280928744,"statfs":{"total":289910292480,"available":287671033856,"internally_reserved":0,"allocated":1371435008,"data_stored":1370545632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":867748863},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,2,1,9,12,2,5,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10006504,"num_objects":-2442,"num_object_clones":0,"num_object_copies":-4884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-47,"num_objects_misplaced":-253,"num_objects_unfound":0,"num_objects_dirty":-2442,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4410,"num_write_kb":3933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":1036288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004476"},"pg_stats":[{"pgid":"2.f","version":"43'10537","reported_seq":10621,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:46.839921+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:29:46.839921+0000","last_peered":"2024-09-17T15:29:46.839921+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:29:46.839921+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'500","ondisk_log_start":"34'500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41414656,"num_objects":10111,"num_object_clones":0,"num_object_copies":20222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10537,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6044},{"shards":"0,2","objects":3857}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'10708","reported_seq":10810,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:47.390054+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:29:47.390054+0000","last_peered":"2024-09-17T15:29:47.390054+0000","last_clean":"2024-09-17T15:29:47.390054+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:29:47.390054+0000","last_undegraded":"2024-09-17T15:29:47.390054+0000","last_fullsized":"2024-09-17T15:29:47.390054+0000","mapping_epoch":33,"log_start":"34'700","ondisk_log_start":"34'700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41878470,"num_objects":10226,"num_object_clones":0,"num_object_copies":20452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10226,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9065,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'10705","reported_seq":10840,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:47.419348+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:29:47.419348+0000","last_peered":"2024-09-17T15:29:47.419348+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:29:47.419348+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'700","ondisk_log_start":"34'700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41893912,"num_objects":10229,"num_object_clones":0,"num_object_copies":20458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4164,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10229,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8862,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":6066}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'10703","reported_seq":10772,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:47.381660+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:29:47.381660+0000","last_peered":"2024-09-17T15:29:47.381660+0000","last_clean":"2024-09-17T15:29:47.381660+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:29:47.381660+0000","last_undegraded":"2024-09-17T15:29:47.381660+0000","last_fullsized":"2024-09-17T15:29:47.381660+0000","mapping_epoch":33,"log_start":"34'700","ondisk_log_start":"34'700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42053632,"num_objects":10267,"num_object_clones":0,"num_object_copies":20534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10703,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'10518","reported_seq":10647,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:46.852247+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:29:46.852247+0000","last_peered":"2024-09-17T15:29:46.852247+0000","last_clean":"2024-09-17T15:29:46.852247+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:29:46.852247+0000","last_undegraded":"2024-09-17T15:29:46.852247+0000","last_fullsized":"2024-09-17T15:29:46.852247+0000","mapping_epoch":32,"log_start":"34'500","ondisk_log_start":"34'500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41513443,"num_objects":10136,"num_object_clones":0,"num_object_copies":20272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10136,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7500,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'10855","reported_seq":10996,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:46.844198+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:29:46.844198+0000","last_peered":"2024-09-17T15:29:46.844198+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:29:46.844198+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'800","ondisk_log_start":"34'800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42582499,"num_objects":10397,"num_object_clones":0,"num_object_copies":20794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10397,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7354,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6130},{"shards":"0,2","objects":4041}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'10612","reported_seq":10763,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:46.812802+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:29:46.812802+0000","last_peered":"2024-09-17T15:29:46.812802+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:29:46.812802+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'600","ondisk_log_start":"34'600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41599942,"num_objects":10158,"num_object_clones":0,"num_object_copies":20316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4168,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10158,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7381,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5990},{"shards":"0,2","objects":3949}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'10667","reported_seq":10795,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:46.826876+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:29:46.826876+0000","last_peered":"2024-09-17T15:29:46.826876+0000","last_clean":"2024-09-17T15:29:46.826876+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:29:46.826876+0000","last_undegraded":"2024-09-17T15:29:46.826876+0000","last_fullsized":"2024-09-17T15:29:46.826876+0000","mapping_epoch":32,"log_start":"34'600","ondisk_log_start":"34'600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41979904,"num_objects":10249,"num_object_clones":0,"num_object_copies":20498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7621,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'10592","reported_seq":18874,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:46.096328+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:29:46.096328+0000","last_peered":"2024-09-17T15:29:46.096328+0000","last_clean":"2024-09-17T15:29:46.096328+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:29:46.096328+0000","last_undegraded":"2024-09-17T15:29:46.096328+0000","last_fullsized":"2024-09-17T15:29:46.096328+0000","mapping_epoch":42,"log_start":"34'500","ondisk_log_start":"34'500","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41882083,"num_objects":10226,"num_object_clones":0,"num_object_copies":20452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10226,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8592,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804747+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804747+0000","last_peered":"2024-09-17T15:28:23.804747+0000","last_clean":"2024-09-17T15:28:23.804747+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804747+0000","last_undegraded":"2024-09-17T15:28:23.804747+0000","last_fullsized":"2024-09-17T15:28:23.804747+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'10516","reported_seq":10654,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:46.829041+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:29:46.829041+0000","last_peered":"2024-09-17T15:29:46.829041+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:29:46.829041+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'500","ondisk_log_start":"34'500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41066496,"num_objects":10026,"num_object_clones":0,"num_object_copies":20052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4008,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7112,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6020},{"shards":"0,2","objects":3769}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'10480","reported_seq":10607,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:47.354934+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:29:47.354934+0000","last_peered":"2024-09-17T15:29:47.354934+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:29:47.354934+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'400","ondisk_log_start":"34'400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41120227,"num_objects":10040,"num_object_clones":0,"num_object_copies":20080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4075,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10040,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8375,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3859},{"shards":"2","objects":5966}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'10610","reported_seq":10741,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:47.276163+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:29:47.276163+0000","last_peered":"2024-09-17T15:29:47.276163+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:29:47.276163+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'600","ondisk_log_start":"34'600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41476096,"num_objects":10126,"num_object_clones":0,"num_object_copies":20252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4133,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8372,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3898},{"shards":"2","objects":5995}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'10696","reported_seq":10843,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:46.838043+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:29:46.838043+0000","last_peered":"2024-09-17T15:29:46.838043+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:29:46.838043+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'600","ondisk_log_start":"34'600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42065920,"num_objects":10270,"num_object_clones":0,"num_object_copies":20540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4133,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7244,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6138},{"shards":"0,2","objects":3926}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'10594","reported_seq":10746,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:46.763129+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:29:46.763129+0000","last_peered":"2024-09-17T15:29:46.763129+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:29:46.763129+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'500","ondisk_log_start":"34'500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41689088,"num_objects":10178,"num_object_clones":0,"num_object_copies":20356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4155,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6881,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6026},{"shards":"0,2","objects":3953}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805107+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805107+0000","last_peered":"2024-09-17T15:28:23.805107+0000","last_clean":"2024-09-17T15:28:23.805107+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805107+0000","last_undegraded":"2024-09-17T15:28:23.805107+0000","last_fullsized":"2024-09-17T15:28:23.805107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'10629","reported_seq":10773,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:46.819856+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:29:46.819856+0000","last_peered":"2024-09-17T15:29:46.819856+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:29:46.819856+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'600","ondisk_log_start":"34'600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42012672,"num_objects":10257,"num_object_clones":0,"num_object_copies":20514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4059,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10257,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7416,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6200},{"shards":"0,2","objects":3879}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'10516","reported_seq":14186,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:29:46.849151+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:29:46.849151+0000","last_peered":"2024-09-17T15:29:46.849151+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:29:46.849151+0000","last_undegraded":"2024-09-17T15:29:46.849151+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"34'500","ondisk_log_start":"34'500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41436102,"num_objects":10118,"num_object_clones":0,"num_object_copies":20236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2346,"num_objects_unfound":0,"num_objects_dirty":10118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6958,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1763,"num_bytes_recovered":7221248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7772},{"shards":"0,2","objects":2346}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":667665142,"num_objects":163014,"num_object_clones":0,"num_object_copies":326028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41230,"num_objects_misplaced":2346,"num_objects_unfound":0,"num_objects_dirty":163014,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":129973,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5882,"num_bytes_recovered":23892451,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1368178688,"data_stored":1368018798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160638,"ondisk_log_size":160638,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051641,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":739124,"kb_used_data":458916,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93632716,"statfs":{"total":96636764160,"available":95879901184,"internally_reserved":0,"allocated":469929984,"data_stored":469649338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":286912172},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724586,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":561872,"kb_used_data":323328,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93809968,"statfs":{"total":96636764160,"available":96061407232,"internally_reserved":0,"allocated":331087872,"data_stored":330759793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":244248235},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,5,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247869,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":885780,"kb_used_data":557048,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93486060,"statfs":{"total":96636764160,"available":95729725440,"internally_reserved":0,"allocated":570417152,"data_stored":570136501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":336588456},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,3,6,0,5,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":568811520,"data_stored":568775390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":330711040,"data_stored":330627482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":468656128,"data_stored":468615926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:29:52.995 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:29:53.525 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:29:53.526 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:29:53.551 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":200,"stamp":"2024-09-17T15:29:51.661778+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":667124489,"num_objects":162884,"num_object_clones":0,"num_object_copies":325768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41230,"num_objects_misplaced":2346,"num_objects_unfound":0,"num_objects_dirty":162884,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":130107,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5883,"num_bytes_recovered":23892470,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160573,"ondisk_log_size":160573,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2191808,"kb_used_data":1338756,"kb_used_omap":13,"kb_used_meta":852978,"kb_avail":280923712,"statfs":{"total":289910292480,"available":287665881088,"internally_reserved":0,"allocated":1370886144,"data_stored":1369996768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":873450495},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,2,6,8,2,5,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10547176,"num_objects":-2574,"num_object_clones":0,"num_object_copies":-5148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-47,"num_objects_misplaced":-253,"num_objects_unfound":0,"num_objects_dirty":-2574,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4542,"num_write_kb":3933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":1036288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004239"},"pg_stats":[{"pgid":"2.f","version":"43'10537","reported_seq":10621,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:46.839921+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:29:46.839921+0000","last_peered":"2024-09-17T15:29:46.839921+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:29:46.839921+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'500","ondisk_log_start":"34'500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41414656,"num_objects":10111,"num_object_clones":0,"num_object_copies":20222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10537,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6044},{"shards":"0,2","objects":3857}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'10708","reported_seq":10810,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:47.390054+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:29:47.390054+0000","last_peered":"2024-09-17T15:29:47.390054+0000","last_clean":"2024-09-17T15:29:47.390054+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:29:47.390054+0000","last_undegraded":"2024-09-17T15:29:47.390054+0000","last_fullsized":"2024-09-17T15:29:47.390054+0000","mapping_epoch":33,"log_start":"34'700","ondisk_log_start":"34'700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41878470,"num_objects":10226,"num_object_clones":0,"num_object_copies":20452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10226,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9065,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'10705","reported_seq":10840,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:47.419348+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:29:47.419348+0000","last_peered":"2024-09-17T15:29:47.419348+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:29:47.419348+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'700","ondisk_log_start":"34'700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41893912,"num_objects":10229,"num_object_clones":0,"num_object_copies":20458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4164,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10229,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8862,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":6066}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'10703","reported_seq":10772,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:47.381660+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:29:47.381660+0000","last_peered":"2024-09-17T15:29:47.381660+0000","last_clean":"2024-09-17T15:29:47.381660+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:29:47.381660+0000","last_undegraded":"2024-09-17T15:29:47.381660+0000","last_fullsized":"2024-09-17T15:29:47.381660+0000","mapping_epoch":33,"log_start":"34'700","ondisk_log_start":"34'700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42053632,"num_objects":10267,"num_object_clones":0,"num_object_copies":20534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10703,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'10518","reported_seq":10647,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:46.852247+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:29:46.852247+0000","last_peered":"2024-09-17T15:29:46.852247+0000","last_clean":"2024-09-17T15:29:46.852247+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:29:46.852247+0000","last_undegraded":"2024-09-17T15:29:46.852247+0000","last_fullsized":"2024-09-17T15:29:46.852247+0000","mapping_epoch":32,"log_start":"34'500","ondisk_log_start":"34'500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41513443,"num_objects":10136,"num_object_clones":0,"num_object_copies":20272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10136,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7500,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'10855","reported_seq":10996,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:46.844198+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:29:46.844198+0000","last_peered":"2024-09-17T15:29:46.844198+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:29:46.844198+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'800","ondisk_log_start":"34'800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42582499,"num_objects":10397,"num_object_clones":0,"num_object_copies":20794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10397,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7354,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6130},{"shards":"0,2","objects":4041}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'10612","reported_seq":10763,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:46.812802+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:29:46.812802+0000","last_peered":"2024-09-17T15:29:46.812802+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:29:46.812802+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'600","ondisk_log_start":"34'600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41599942,"num_objects":10158,"num_object_clones":0,"num_object_copies":20316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4168,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10158,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7381,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5990},{"shards":"0,2","objects":3949}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'10667","reported_seq":10795,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:46.826876+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:29:46.826876+0000","last_peered":"2024-09-17T15:29:46.826876+0000","last_clean":"2024-09-17T15:29:46.826876+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:29:46.826876+0000","last_undegraded":"2024-09-17T15:29:46.826876+0000","last_fullsized":"2024-09-17T15:29:46.826876+0000","mapping_epoch":32,"log_start":"34'600","ondisk_log_start":"34'600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41979904,"num_objects":10249,"num_object_clones":0,"num_object_copies":20498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7621,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'10724","reported_seq":19004,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:51.095443+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:29:51.095443+0000","last_peered":"2024-09-17T15:29:51.095443+0000","last_clean":"2024-09-17T15:29:51.095443+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:29:51.095443+0000","last_undegraded":"2024-09-17T15:29:51.095443+0000","last_fullsized":"2024-09-17T15:29:51.095443+0000","mapping_epoch":42,"log_start":"34'700","ondisk_log_start":"34'700","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41341411,"num_objects":10094,"num_object_clones":0,"num_object_copies":20188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10094,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8724,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804747+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804747+0000","last_peered":"2024-09-17T15:28:23.804747+0000","last_clean":"2024-09-17T15:28:23.804747+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804747+0000","last_undegraded":"2024-09-17T15:28:23.804747+0000","last_fullsized":"2024-09-17T15:28:23.804747+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'10516","reported_seq":10654,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:46.829041+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:29:46.829041+0000","last_peered":"2024-09-17T15:29:46.829041+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:29:46.829041+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'500","ondisk_log_start":"34'500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41066496,"num_objects":10026,"num_object_clones":0,"num_object_copies":20052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4008,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7112,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6020},{"shards":"0,2","objects":3769}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'10480","reported_seq":10607,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:47.354934+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:29:47.354934+0000","last_peered":"2024-09-17T15:29:47.354934+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:29:47.354934+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'400","ondisk_log_start":"34'400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41120227,"num_objects":10040,"num_object_clones":0,"num_object_copies":20080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4075,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10040,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8375,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3859},{"shards":"2","objects":5966}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'10610","reported_seq":10741,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:47.276163+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:29:47.276163+0000","last_peered":"2024-09-17T15:29:47.276163+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:29:47.276163+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'600","ondisk_log_start":"34'600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41476096,"num_objects":10126,"num_object_clones":0,"num_object_copies":20252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4133,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8372,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3898},{"shards":"2","objects":5995}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'10696","reported_seq":10843,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:46.838043+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:29:46.838043+0000","last_peered":"2024-09-17T15:29:46.838043+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:29:46.838043+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'600","ondisk_log_start":"34'600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42065920,"num_objects":10270,"num_object_clones":0,"num_object_copies":20540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4133,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7244,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6138},{"shards":"0,2","objects":3926}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'10594","reported_seq":10746,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:46.763129+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:29:46.763129+0000","last_peered":"2024-09-17T15:29:46.763129+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:29:46.763129+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'500","ondisk_log_start":"34'500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41689088,"num_objects":10178,"num_object_clones":0,"num_object_copies":20356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4155,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6881,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6026},{"shards":"0,2","objects":3953}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805107+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805107+0000","last_peered":"2024-09-17T15:28:23.805107+0000","last_clean":"2024-09-17T15:28:23.805107+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805107+0000","last_undegraded":"2024-09-17T15:28:23.805107+0000","last_fullsized":"2024-09-17T15:28:23.805107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'10629","reported_seq":10773,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:46.819856+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:29:46.819856+0000","last_peered":"2024-09-17T15:29:46.819856+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:29:46.819856+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'600","ondisk_log_start":"34'600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42012672,"num_objects":10257,"num_object_clones":0,"num_object_copies":20514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4059,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10257,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7416,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6200},{"shards":"0,2","objects":3879}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'10516","reported_seq":14186,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:29:46.849151+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:29:46.849151+0000","last_peered":"2024-09-17T15:29:46.849151+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:29:46.849151+0000","last_undegraded":"2024-09-17T15:29:46.849151+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"34'500","ondisk_log_start":"34'500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41436102,"num_objects":10118,"num_object_clones":0,"num_object_copies":20236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2346,"num_objects_unfound":0,"num_objects_dirty":10118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6958,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1763,"num_bytes_recovered":7221248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7772},{"shards":"0,2","objects":2346}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":667124470,"num_objects":162882,"num_object_clones":0,"num_object_copies":325764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41230,"num_objects_misplaced":2346,"num_objects_unfound":0,"num_objects_dirty":162882,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":130105,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5882,"num_bytes_recovered":23892451,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1367638016,"data_stored":1367478126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160570,"ondisk_log_size":160570,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051641,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":739124,"kb_used_data":458916,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93632716,"statfs":{"total":96636764160,"available":95879901184,"internally_reserved":0,"allocated":469929984,"data_stored":469649338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":286912172},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724587,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566904,"kb_used_data":322792,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93804936,"statfs":{"total":96636764160,"available":96056254464,"internally_reserved":0,"allocated":330539008,"data_stored":330210929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":249949867},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247869,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":885780,"kb_used_data":557048,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93486060,"statfs":{"total":96636764160,"available":95729725440,"internally_reserved":0,"allocated":570417152,"data_stored":570136501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":336588456},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,3,6,0,5,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":568811520,"data_stored":568775390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":330170368,"data_stored":330086810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":468656128,"data_stored":468615926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:29:53.552 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:29:54.100 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:29:54.102 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:29:54.133 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":201,"stamp":"2024-09-17T15:29:53.662419+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658629385,"num_objects":160810,"num_object_clones":0,"num_object_copies":321620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41200,"num_objects_misplaced":2213,"num_objects_unfound":0,"num_objects_dirty":160810,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":132181,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6046,"num_bytes_recovered":24437238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160647,"ondisk_log_size":160647,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2176508,"kb_used_data":1327612,"kb_used_omap":13,"kb_used_meta":848818,"kb_avail":280939012,"statfs":{"total":289910292480,"available":287681548288,"internally_reserved":0,"allocated":1359474688,"data_stored":1358585312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":869190655},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,8,5,5,2,0,3],"upper_bound":1024},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19255272,"num_objects":-4700,"num_object_clones":0,"num_object_copies":-9400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-77,"num_objects_misplaced":-386,"num_objects_unfound":0,"num_objects_dirty":-4700,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6488,"num_write_kb":3573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":463,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004327"},"pg_stats":[{"pgid":"2.f","version":"43'10688","reported_seq":10785,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:51.891904+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:29:51.891904+0000","last_peered":"2024-09-17T15:29:51.891904+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:29:51.891904+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'600","ondisk_log_start":"34'600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40796160,"num_objects":9960,"num_object_clones":0,"num_object_copies":19920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4060,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10688,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5900},{"shards":"0,2","objects":3706}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'10842","reported_seq":10945,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:52.352625+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:29:52.352625+0000","last_peered":"2024-09-17T15:29:52.352625+0000","last_clean":"2024-09-17T15:29:52.352625+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:29:52.352625+0000","last_undegraded":"2024-09-17T15:29:52.352625+0000","last_fullsized":"2024-09-17T15:29:52.352625+0000","mapping_epoch":33,"log_start":"34'800","ondisk_log_start":"34'800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41329606,"num_objects":10092,"num_object_clones":0,"num_object_copies":20184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10092,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9199,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'10820","reported_seq":10956,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:52.386302+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:29:52.386302+0000","last_peered":"2024-09-17T15:29:52.386302+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:29:52.386302+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'800","ondisk_log_start":"34'800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41422872,"num_objects":10114,"num_object_clones":0,"num_object_copies":20228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4164,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10114,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8977,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3817},{"shards":"2","objects":5951}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'10838","reported_seq":10905,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:52.413558+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:29:52.413558+0000","last_peered":"2024-09-17T15:29:52.413558+0000","last_clean":"2024-09-17T15:29:52.413558+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:29:52.413558+0000","last_undegraded":"2024-09-17T15:29:52.413558+0000","last_fullsized":"2024-09-17T15:29:52.413558+0000","mapping_epoch":33,"log_start":"34'800","ondisk_log_start":"34'800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41500672,"num_objects":10132,"num_object_clones":0,"num_object_copies":20264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10838,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'10661","reported_seq":10790,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:51.796536+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:29:51.796536+0000","last_peered":"2024-09-17T15:29:51.796536+0000","last_clean":"2024-09-17T15:29:51.796536+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:29:51.796536+0000","last_undegraded":"2024-09-17T15:29:51.796536+0000","last_fullsized":"2024-09-17T15:29:51.796536+0000","mapping_epoch":32,"log_start":"34'600","ondisk_log_start":"34'600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40927715,"num_objects":9993,"num_object_clones":0,"num_object_copies":19986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9993,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7643,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'11002","reported_seq":11157,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:51.776395+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:29:51.776395+0000","last_peered":"2024-09-17T15:29:51.776395+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:29:51.776395+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'1000","ondisk_log_start":"34'1000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41980387,"num_objects":10250,"num_object_clones":0,"num_object_copies":20500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4261,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10250,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7501,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5990},{"shards":"0,2","objects":3894}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'10742","reported_seq":10897,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:51.887165+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:29:51.887165+0000","last_peered":"2024-09-17T15:29:51.887165+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:29:51.887165+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'700","ondisk_log_start":"34'700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41067462,"num_objects":10028,"num_object_clones":0,"num_object_copies":20056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10028,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7511,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5862},{"shards":"0,2","objects":3819}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'10806","reported_seq":10934,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:51.874608+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:29:51.874608+0000","last_peered":"2024-09-17T15:29:51.874608+0000","last_clean":"2024-09-17T15:29:51.874608+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:29:51.874608+0000","last_undegraded":"2024-09-17T15:29:51.874608+0000","last_fullsized":"2024-09-17T15:29:51.874608+0000","mapping_epoch":32,"log_start":"34'800","ondisk_log_start":"34'800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41410560,"num_objects":10110,"num_object_clones":0,"num_object_copies":20220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7760,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'10724","reported_seq":19004,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:51.095443+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:29:51.095443+0000","last_peered":"2024-09-17T15:29:51.095443+0000","last_clean":"2024-09-17T15:29:51.095443+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:29:51.095443+0000","last_undegraded":"2024-09-17T15:29:51.095443+0000","last_fullsized":"2024-09-17T15:29:51.095443+0000","mapping_epoch":42,"log_start":"34'700","ondisk_log_start":"34'700","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41341411,"num_objects":10094,"num_object_clones":0,"num_object_copies":20188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10094,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8724,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804747+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804747+0000","last_peered":"2024-09-17T15:28:23.804747+0000","last_clean":"2024-09-17T15:28:23.804747+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804747+0000","last_undegraded":"2024-09-17T15:28:23.804747+0000","last_fullsized":"2024-09-17T15:28:23.804747+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'10658","reported_seq":10804,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:51.833263+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:29:51.833263+0000","last_peered":"2024-09-17T15:29:51.833263+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:29:51.833263+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'600","ondisk_log_start":"34'600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40484864,"num_objects":9884,"num_object_clones":0,"num_object_copies":19768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4004,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9884,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7254,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5882},{"shards":"0,2","objects":3627}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'10623","reported_seq":10750,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:52.333762+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:29:52.333762+0000","last_peered":"2024-09-17T15:29:52.333762+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:29:52.333762+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'600","ondisk_log_start":"34'600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40534499,"num_objects":9897,"num_object_clones":0,"num_object_copies":19794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4075,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9897,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8518,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3716},{"shards":"2","objects":5823}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'10746","reported_seq":10877,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:52.332163+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:29:52.332163+0000","last_peered":"2024-09-17T15:29:52.332163+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:29:52.332163+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'700","ondisk_log_start":"34'700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40919040,"num_objects":9990,"num_object_clones":0,"num_object_copies":19980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4133,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8508,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3762},{"shards":"2","objects":5859}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'10847","reported_seq":11000,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:51.758867+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:29:51.758867+0000","last_peered":"2024-09-17T15:29:51.758867+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:29:51.758867+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'800","ondisk_log_start":"34'800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41447424,"num_objects":10119,"num_object_clones":0,"num_object_copies":20238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4130,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7395,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5990},{"shards":"0,2","objects":3775}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'10738","reported_seq":10892,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:51.864295+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:29:51.864295+0000","last_peered":"2024-09-17T15:29:51.864295+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:29:51.864295+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'700","ondisk_log_start":"34'700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41099264,"num_objects":10034,"num_object_clones":0,"num_object_copies":20068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10034,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7025,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5883},{"shards":"0,2","objects":3809}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805107+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805107+0000","last_peered":"2024-09-17T15:28:23.805107+0000","last_clean":"2024-09-17T15:28:23.805107+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805107+0000","last_undegraded":"2024-09-17T15:28:23.805107+0000","last_fullsized":"2024-09-17T15:28:23.805107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'10760","reported_seq":10916,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:51.689034+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:29:51.689034+0000","last_peered":"2024-09-17T15:29:51.689034+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:29:51.689034+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'700","ondisk_log_start":"34'700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41476096,"num_objects":10126,"num_object_clones":0,"num_object_copies":20252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4053,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7547,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6075},{"shards":"0,2","objects":3748}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'10649","reported_seq":14585,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:29:51.875752+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:29:51.875752+0000","last_peered":"2024-09-17T15:29:51.875752+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:29:51.875752+0000","last_undegraded":"2024-09-17T15:29:51.875752+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"34'600","ondisk_log_start":"34'600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40891334,"num_objects":9985,"num_object_clones":0,"num_object_copies":19970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2213,"num_objects_unfound":0,"num_objects_dirty":9985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7091,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1896,"num_bytes_recovered":7766016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7772},{"shards":"0,2","objects":2213}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":658629366,"num_objects":160808,"num_object_clones":0,"num_object_copies":321616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41200,"num_objects_misplaced":2213,"num_objects_unfound":0,"num_objects_dirty":160808,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":132179,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6045,"num_bytes_recovered":24437219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1356369920,"data_stored":1356210030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160644,"ondisk_log_size":160644,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051642,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":739040,"kb_used_data":454660,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93632800,"statfs":{"total":96636764160,"available":95879987200,"internally_reserved":0,"allocated":465571840,"data_stored":465291194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":291172012},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724587,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566904,"kb_used_data":322792,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93804936,"statfs":{"total":96636764160,"available":96056254464,"internally_reserved":0,"allocated":330539008,"data_stored":330210929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":249949867},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247870,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":870564,"kb_used_data":550160,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93501276,"statfs":{"total":96636764160,"available":95745306624,"internally_reserved":0,"allocated":563363840,"data_stored":563083189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":328068776},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,2,5,2,0,3],"upper_bound":1024},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":561926144,"data_stored":561890014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":330170368,"data_stored":330086810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":464273408,"data_stored":464233206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:29:54.138 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:29:54.139 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'10688", 'reported_seq': 10785, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:51.891904+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:29:51.891904+0000', 'last_peered': '2024-09-17T15:29:51.891904+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:29:51.891904+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "34'600", 'ondisk_log_start': "34'600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40796160, 'num_objects': 9960, 'num_object_clones': 0, 'num_object_copies': 19920, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4060, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9960, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10688, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 10, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5900}, {'shards': '0,2', 'objects': 3706}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:54.139 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:29:54.139 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'10820", 'reported_seq': 10956, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:52.386302+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:29:52.386302+0000', 'last_peered': '2024-09-17T15:29:52.386302+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:29:52.386302+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "34'800", 'ondisk_log_start': "34'800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41422872, 'num_objects': 10114, 'num_object_clones': 0, 'num_object_copies': 20228, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4164, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10114, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8977, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 5, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3817}, {'shards': '2', 'objects': 5951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:29:54.139 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:29:54.139 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'11002", 'reported_seq': 11157, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:51.776395+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:29:51.776395+0000', 'last_peered': '2024-09-17T15:29:51.776395+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:29:51.776395+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "34'1000", 'ondisk_log_start': "34'1000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41980387, 'num_objects': 10250, 'num_object_clones': 0, 'num_object_copies': 20500, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4261, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10250, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7501, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 10, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5990}, {'shards': '0,2', 'objects': 3894}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:54.139 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:29:54.139 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'10742", 'reported_seq': 10897, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:51.887165+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:29:51.887165+0000', 'last_peered': '2024-09-17T15:29:51.887165+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:29:51.887165+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "34'700", 'ondisk_log_start': "34'700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41067462, 'num_objects': 10028, 'num_object_clones': 0, 'num_object_copies': 20056, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10028, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 7511, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 8, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5862}, {'shards': '0,2', 'objects': 3819}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:54.139 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:29:54.139 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'10658", 'reported_seq': 10804, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:51.833263+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:29:51.833263+0000', 'last_peered': '2024-09-17T15:29:51.833263+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:29:51.833263+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "34'600", 'ondisk_log_start': "34'600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40484864, 'num_objects': 9884, 'num_object_clones': 0, 'num_object_copies': 19768, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4004, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9884, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7254, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 8, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5882}, {'shards': '0,2', 'objects': 3627}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:54.139 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:29:54.140 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'10623", 'reported_seq': 10750, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:52.333762+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:29:52.333762+0000', 'last_peered': '2024-09-17T15:29:52.333762+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:29:52.333762+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "34'600", 'ondisk_log_start': "34'600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40534499, 'num_objects': 9897, 'num_object_clones': 0, 'num_object_copies': 19794, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4075, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9897, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8518, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3716}, {'shards': '2', 'objects': 5823}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:29:54.140 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:29:54.140 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'10746", 'reported_seq': 10877, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:52.332163+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:29:52.332163+0000', 'last_peered': '2024-09-17T15:29:52.332163+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:29:52.332163+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "34'700", 'ondisk_log_start': "34'700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40919040, 'num_objects': 9990, 'num_object_clones': 0, 'num_object_copies': 19980, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4133, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9990, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8508, 'num_write_kb': 32532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 6, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3762}, {'shards': '2', 'objects': 5859}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:29:54.140 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:29:54.140 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'10847", 'reported_seq': 11000, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:51.758867+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:29:51.758867+0000', 'last_peered': '2024-09-17T15:29:51.758867+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:29:51.758867+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "34'800", 'ondisk_log_start': "34'800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41447424, 'num_objects': 10119, 'num_object_clones': 0, 'num_object_copies': 20238, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4130, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10119, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7395, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 9, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5990}, {'shards': '0,2', 'objects': 3775}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:54.140 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:29:54.140 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'10738", 'reported_seq': 10892, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:51.864295+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:29:51.864295+0000', 'last_peered': '2024-09-17T15:29:51.864295+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:29:51.864295+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "34'700", 'ondisk_log_start': "34'700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41099264, 'num_objects': 10034, 'num_object_clones': 0, 'num_object_copies': 20068, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4154, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10034, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7025, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 6, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5883}, {'shards': '0,2', 'objects': 3809}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:54.140 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:29:54.140 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'10760", 'reported_seq': 10916, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:51.689034+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:29:51.689034+0000', 'last_peered': '2024-09-17T15:29:51.689034+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:29:51.689034+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "34'700", 'ondisk_log_start': "34'700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41476096, 'num_objects': 10126, 'num_object_clones': 0, 'num_object_copies': 20252, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4053, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10126, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7547, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 12, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6075}, {'shards': '0,2', 'objects': 3748}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:54.140 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:29:54.141 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'10649", 'reported_seq': 14585, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:29:51.875752+0000', 'last_change': '2024-09-17T15:28:22.509818+0000', 'last_active': '2024-09-17T15:29:51.875752+0000', 'last_peered': '2024-09-17T15:29:51.875752+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:29:51.875752+0000', 'last_undegraded': '2024-09-17T15:29:51.875752+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "34'600", 'ondisk_log_start': "34'600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40891334, 'num_objects': 9985, 'num_object_clones': 0, 'num_object_copies': 19970, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2213, 'num_objects_unfound': 0, 'num_objects_dirty': 9985, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7091, 'num_write_kb': 27056, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1896, 'num_bytes_recovered': 7766016, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7772}, {'shards': '0,2', 'objects': 2213}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:54.141 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:29:54.830 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:29:54.830 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41200/321620 objects degraded (12.810%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":331,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":160810,"data_bytes":658629385,"bytes_used":2228744192,"bytes_avail":287681548288,"bytes_total":289910292480,"degraded_objects":41200,"degraded_total":321620,"degraded_ratio":0.12810148622598097,"misplaced_objects":2213,"misplaced_total":321620,"misplaced_ratio":0.0068807909955848519,"recovering_objects_per_sec":38,"recovering_bytes_per_sec":131707,"recovering_keys_per_sec":0,"num_objects_recovered":463,"num_bytes_recovered":1581056,"num_keys_recovered":0,"read_bytes_sec":85,"write_bytes_sec":304786,"read_op_per_sec":0,"write_op_per_sec":540},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T15:27:19.616376+0000","services":{}},"progress_events":{}} 2024-09-17T15:29:54.862 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:29:54.863 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:29:55.442 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:29:55.443 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:29:55.477 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":201,"stamp":"2024-09-17T15:29:53.662419+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658629385,"num_objects":160810,"num_object_clones":0,"num_object_copies":321620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41200,"num_objects_misplaced":2213,"num_objects_unfound":0,"num_objects_dirty":160810,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":132181,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6046,"num_bytes_recovered":24437238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160647,"ondisk_log_size":160647,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2176508,"kb_used_data":1327612,"kb_used_omap":13,"kb_used_meta":848818,"kb_avail":280939012,"statfs":{"total":289910292480,"available":287681548288,"internally_reserved":0,"allocated":1359474688,"data_stored":1358585312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":869190655},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,8,5,5,2,0,3],"upper_bound":1024},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19255272,"num_objects":-4700,"num_object_clones":0,"num_object_copies":-9400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-77,"num_objects_misplaced":-386,"num_objects_unfound":0,"num_objects_dirty":-4700,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6488,"num_write_kb":3573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":463,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004327"},"pg_stats":[{"pgid":"2.f","version":"43'10688","reported_seq":10785,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:51.891904+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:29:51.891904+0000","last_peered":"2024-09-17T15:29:51.891904+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:29:51.891904+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'600","ondisk_log_start":"34'600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40796160,"num_objects":9960,"num_object_clones":0,"num_object_copies":19920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4060,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10688,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5900},{"shards":"0,2","objects":3706}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'10842","reported_seq":10945,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:52.352625+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:29:52.352625+0000","last_peered":"2024-09-17T15:29:52.352625+0000","last_clean":"2024-09-17T15:29:52.352625+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:29:52.352625+0000","last_undegraded":"2024-09-17T15:29:52.352625+0000","last_fullsized":"2024-09-17T15:29:52.352625+0000","mapping_epoch":33,"log_start":"34'800","ondisk_log_start":"34'800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41329606,"num_objects":10092,"num_object_clones":0,"num_object_copies":20184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10092,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9199,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'10820","reported_seq":10956,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:52.386302+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:29:52.386302+0000","last_peered":"2024-09-17T15:29:52.386302+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:29:52.386302+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'800","ondisk_log_start":"34'800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41422872,"num_objects":10114,"num_object_clones":0,"num_object_copies":20228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4164,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10114,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8977,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3817},{"shards":"2","objects":5951}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'10838","reported_seq":10905,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:52.413558+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:29:52.413558+0000","last_peered":"2024-09-17T15:29:52.413558+0000","last_clean":"2024-09-17T15:29:52.413558+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:29:52.413558+0000","last_undegraded":"2024-09-17T15:29:52.413558+0000","last_fullsized":"2024-09-17T15:29:52.413558+0000","mapping_epoch":33,"log_start":"34'800","ondisk_log_start":"34'800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41500672,"num_objects":10132,"num_object_clones":0,"num_object_copies":20264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10838,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'10661","reported_seq":10790,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:51.796536+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:29:51.796536+0000","last_peered":"2024-09-17T15:29:51.796536+0000","last_clean":"2024-09-17T15:29:51.796536+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:29:51.796536+0000","last_undegraded":"2024-09-17T15:29:51.796536+0000","last_fullsized":"2024-09-17T15:29:51.796536+0000","mapping_epoch":32,"log_start":"34'600","ondisk_log_start":"34'600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40927715,"num_objects":9993,"num_object_clones":0,"num_object_copies":19986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9993,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7643,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'11002","reported_seq":11157,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:51.776395+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:29:51.776395+0000","last_peered":"2024-09-17T15:29:51.776395+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:29:51.776395+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'1000","ondisk_log_start":"34'1000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41980387,"num_objects":10250,"num_object_clones":0,"num_object_copies":20500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4261,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10250,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7501,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5990},{"shards":"0,2","objects":3894}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'10742","reported_seq":10897,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:51.887165+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:29:51.887165+0000","last_peered":"2024-09-17T15:29:51.887165+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:29:51.887165+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'700","ondisk_log_start":"34'700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41067462,"num_objects":10028,"num_object_clones":0,"num_object_copies":20056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10028,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7511,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5862},{"shards":"0,2","objects":3819}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'10806","reported_seq":10934,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:51.874608+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:29:51.874608+0000","last_peered":"2024-09-17T15:29:51.874608+0000","last_clean":"2024-09-17T15:29:51.874608+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:29:51.874608+0000","last_undegraded":"2024-09-17T15:29:51.874608+0000","last_fullsized":"2024-09-17T15:29:51.874608+0000","mapping_epoch":32,"log_start":"34'800","ondisk_log_start":"34'800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41410560,"num_objects":10110,"num_object_clones":0,"num_object_copies":20220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7760,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'10724","reported_seq":19004,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:51.095443+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:29:51.095443+0000","last_peered":"2024-09-17T15:29:51.095443+0000","last_clean":"2024-09-17T15:29:51.095443+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:29:51.095443+0000","last_undegraded":"2024-09-17T15:29:51.095443+0000","last_fullsized":"2024-09-17T15:29:51.095443+0000","mapping_epoch":42,"log_start":"34'700","ondisk_log_start":"34'700","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41341411,"num_objects":10094,"num_object_clones":0,"num_object_copies":20188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10094,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8724,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804747+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804747+0000","last_peered":"2024-09-17T15:28:23.804747+0000","last_clean":"2024-09-17T15:28:23.804747+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804747+0000","last_undegraded":"2024-09-17T15:28:23.804747+0000","last_fullsized":"2024-09-17T15:28:23.804747+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'10658","reported_seq":10804,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:51.833263+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:29:51.833263+0000","last_peered":"2024-09-17T15:29:51.833263+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:29:51.833263+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'600","ondisk_log_start":"34'600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40484864,"num_objects":9884,"num_object_clones":0,"num_object_copies":19768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4004,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9884,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7254,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5882},{"shards":"0,2","objects":3627}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'10623","reported_seq":10750,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:52.333762+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:29:52.333762+0000","last_peered":"2024-09-17T15:29:52.333762+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:29:52.333762+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'600","ondisk_log_start":"34'600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40534499,"num_objects":9897,"num_object_clones":0,"num_object_copies":19794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4075,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9897,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8518,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3716},{"shards":"2","objects":5823}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'10746","reported_seq":10877,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:52.332163+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:29:52.332163+0000","last_peered":"2024-09-17T15:29:52.332163+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:29:52.332163+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'700","ondisk_log_start":"34'700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40919040,"num_objects":9990,"num_object_clones":0,"num_object_copies":19980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4133,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8508,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3762},{"shards":"2","objects":5859}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'10847","reported_seq":11000,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:51.758867+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:29:51.758867+0000","last_peered":"2024-09-17T15:29:51.758867+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:29:51.758867+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'800","ondisk_log_start":"34'800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41447424,"num_objects":10119,"num_object_clones":0,"num_object_copies":20238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4130,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7395,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5990},{"shards":"0,2","objects":3775}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'10738","reported_seq":10892,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:51.864295+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:29:51.864295+0000","last_peered":"2024-09-17T15:29:51.864295+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:29:51.864295+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'700","ondisk_log_start":"34'700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41099264,"num_objects":10034,"num_object_clones":0,"num_object_copies":20068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10034,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7025,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5883},{"shards":"0,2","objects":3809}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805107+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805107+0000","last_peered":"2024-09-17T15:28:23.805107+0000","last_clean":"2024-09-17T15:28:23.805107+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805107+0000","last_undegraded":"2024-09-17T15:28:23.805107+0000","last_fullsized":"2024-09-17T15:28:23.805107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'10760","reported_seq":10916,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:51.689034+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:29:51.689034+0000","last_peered":"2024-09-17T15:29:51.689034+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:29:51.689034+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'700","ondisk_log_start":"34'700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41476096,"num_objects":10126,"num_object_clones":0,"num_object_copies":20252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4053,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7547,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6075},{"shards":"0,2","objects":3748}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'10649","reported_seq":14585,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:29:51.875752+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:29:51.875752+0000","last_peered":"2024-09-17T15:29:51.875752+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:29:51.875752+0000","last_undegraded":"2024-09-17T15:29:51.875752+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"34'600","ondisk_log_start":"34'600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40891334,"num_objects":9985,"num_object_clones":0,"num_object_copies":19970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2213,"num_objects_unfound":0,"num_objects_dirty":9985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7091,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1896,"num_bytes_recovered":7766016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7772},{"shards":"0,2","objects":2213}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":658629366,"num_objects":160808,"num_object_clones":0,"num_object_copies":321616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41200,"num_objects_misplaced":2213,"num_objects_unfound":0,"num_objects_dirty":160808,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":132179,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6045,"num_bytes_recovered":24437219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1356369920,"data_stored":1356210030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160644,"ondisk_log_size":160644,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051642,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":739040,"kb_used_data":454660,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93632800,"statfs":{"total":96636764160,"available":95879987200,"internally_reserved":0,"allocated":465571840,"data_stored":465291194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":291172012},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724587,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566904,"kb_used_data":322792,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93804936,"statfs":{"total":96636764160,"available":96056254464,"internally_reserved":0,"allocated":330539008,"data_stored":330210929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":249949867},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247870,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":870564,"kb_used_data":550160,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93501276,"statfs":{"total":96636764160,"available":95745306624,"internally_reserved":0,"allocated":563363840,"data_stored":563083189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":328068776},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,2,5,2,0,3],"upper_bound":1024},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":561926144,"data_stored":561890014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":330170368,"data_stored":330086810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":464273408,"data_stored":464233206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:29:58.480 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:29:59.009 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:29:59.012 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:29:59.036 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":203,"stamp":"2024-09-17T15:29:57.663900+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649593609,"num_objects":158604,"num_object_clones":0,"num_object_copies":317208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41182,"num_objects_misplaced":2073,"num_objects_unfound":0,"num_objects_dirty":158604,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":134387,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6204,"num_bytes_recovered":25010678,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160953,"ondisk_log_size":160953,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2195340,"kb_used_data":1315924,"kb_used_omap":13,"kb_used_meta":879346,"kb_avail":280920180,"statfs":{"total":289910292480,"available":287662264320,"internally_reserved":0,"allocated":1347506176,"data_stored":1346616800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":900451327},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,6,10,5,2,0,0,0,0,0,3,2],"upper_bound":16384},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27160576,"num_objects":-6631,"num_object_clones":0,"num_object_copies":-13262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-72,"num_objects_misplaced":-408,"num_objects_unfound":0,"num_objects_dirty":-6631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6631,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":480,"num_bytes_recovered":1671168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004359"},"pg_stats":[{"pgid":"2.f","version":"43'10813","reported_seq":10915,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:56.672393+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:29:56.672393+0000","last_peered":"2024-09-17T15:29:56.672393+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:29:56.672393+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'800","ondisk_log_start":"34'800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40284160,"num_objects":9835,"num_object_clones":0,"num_object_copies":19670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9835,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10813,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5777},{"shards":"0,2","objects":3581}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'10972","reported_seq":11075,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:57.428820+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:29:57.428820+0000","last_peered":"2024-09-17T15:29:57.428820+0000","last_clean":"2024-09-17T15:29:57.428820+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:29:57.428820+0000","last_undegraded":"2024-09-17T15:29:57.428820+0000","last_fullsized":"2024-09-17T15:29:57.428820+0000","mapping_epoch":33,"log_start":"34'900","ondisk_log_start":"34'900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40797126,"num_objects":9962,"num_object_clones":0,"num_object_copies":19924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9962,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9329,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'10969","reported_seq":11107,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:57.401952+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:29:57.401952+0000","last_peered":"2024-09-17T15:29:57.401952+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:29:57.401952+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'900","ondisk_log_start":"34'900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40812568,"num_objects":9965,"num_object_clones":0,"num_object_copies":19930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4163,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9965,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9126,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3668},{"shards":"2","objects":5803}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'10958","reported_seq":11028,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:57.354671+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:29:57.354671+0000","last_peered":"2024-09-17T15:29:57.354671+0000","last_clean":"2024-09-17T15:29:57.354671+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:29:57.354671+0000","last_undegraded":"2024-09-17T15:29:57.354671+0000","last_fullsized":"2024-09-17T15:29:57.354671+0000","mapping_epoch":33,"log_start":"34'900","ondisk_log_start":"34'900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41009152,"num_objects":10012,"num_object_clones":0,"num_object_copies":20024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10958,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'10810","reported_seq":10939,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:56.919423+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:29:56.919423+0000","last_peered":"2024-09-17T15:29:56.919423+0000","last_clean":"2024-09-17T15:29:56.919423+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:29:56.919423+0000","last_undegraded":"2024-09-17T15:29:56.919423+0000","last_fullsized":"2024-09-17T15:29:56.919423+0000","mapping_epoch":32,"log_start":"34'800","ondisk_log_start":"34'800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40317411,"num_objects":9844,"num_object_clones":0,"num_object_copies":19688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9844,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7792,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'11156","reported_seq":11311,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:56.868101+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:29:56.868101+0000","last_peered":"2024-09-17T15:29:56.868101+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:29:56.868101+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'1100","ondisk_log_start":"34'1100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41349603,"num_objects":10096,"num_object_clones":0,"num_object_copies":20192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4261,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10096,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7655,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5836},{"shards":"0,2","objects":3740}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'10883","reported_seq":11043,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:56.876428+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:29:56.876428+0000","last_peered":"2024-09-17T15:29:56.876428+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:29:56.876428+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'800","ondisk_log_start":"34'800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40489926,"num_objects":9887,"num_object_clones":0,"num_object_copies":19774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4163,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9887,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7652,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5724},{"shards":"0,2","objects":3678}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'10948","reported_seq":11076,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:56.816646+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:29:56.816646+0000","last_peered":"2024-09-17T15:29:56.816646+0000","last_clean":"2024-09-17T15:29:56.816646+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:29:56.816646+0000","last_undegraded":"2024-09-17T15:29:56.816646+0000","last_fullsized":"2024-09-17T15:29:56.816646+0000","mapping_epoch":32,"log_start":"34'900","ondisk_log_start":"34'900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40828928,"num_objects":9968,"num_object_clones":0,"num_object_copies":19936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9968,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7902,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'10872","reported_seq":19154,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:56.183288+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:29:56.183288+0000","last_peered":"2024-09-17T15:29:56.183288+0000","last_clean":"2024-09-17T15:29:56.183288+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:29:56.183288+0000","last_undegraded":"2024-09-17T15:29:56.183288+0000","last_fullsized":"2024-09-17T15:29:56.183288+0000","mapping_epoch":42,"log_start":"34'800","ondisk_log_start":"34'800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40735203,"num_objects":9946,"num_object_clones":0,"num_object_copies":19892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9946,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8872,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804747+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804747+0000","last_peered":"2024-09-17T15:28:23.804747+0000","last_clean":"2024-09-17T15:28:23.804747+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804747+0000","last_undegraded":"2024-09-17T15:28:23.804747+0000","last_fullsized":"2024-09-17T15:28:23.804747+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'10787","reported_seq":10935,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:56.878727+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:29:56.878727+0000","last_peered":"2024-09-17T15:29:56.878727+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:29:56.878727+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'700","ondisk_log_start":"34'700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39956480,"num_objects":9755,"num_object_clones":0,"num_object_copies":19510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4003,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7383,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5754},{"shards":"0,2","objects":3498}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'10741","reported_seq":10867,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:57.329860+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:29:57.329860+0000","last_peered":"2024-09-17T15:29:57.329860+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:29:57.329860+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'700","ondisk_log_start":"34'700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40051171,"num_objects":9779,"num_object_clones":0,"num_object_copies":19558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4075,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9779,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8636,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3598},{"shards":"2","objects":5705}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'10881","reported_seq":11016,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:57.401671+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:29:57.401671+0000","last_peered":"2024-09-17T15:29:57.401671+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:29:57.401671+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'800","ondisk_log_start":"34'800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40366080,"num_objects":9855,"num_object_clones":0,"num_object_copies":19710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8643,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3627},{"shards":"2","objects":5726}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'10974","reported_seq":11127,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:56.848988+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:29:56.848988+0000","last_peered":"2024-09-17T15:29:56.848988+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:29:56.848988+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'900","ondisk_log_start":"34'900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40927232,"num_objects":9992,"num_object_clones":0,"num_object_copies":19984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4130,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9992,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7522,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5863},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'10888","reported_seq":11045,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:56.903467+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:29:56.903467+0000","last_peered":"2024-09-17T15:29:56.903467+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:29:56.903467+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'800","ondisk_log_start":"34'800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40484864,"num_objects":9884,"num_object_clones":0,"num_object_copies":19768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4152,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9884,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7175,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5735},{"shards":"0,2","objects":3659}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805107+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805107+0000","last_peered":"2024-09-17T15:28:23.805107+0000","last_clean":"2024-09-17T15:28:23.805107+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805107+0000","last_undegraded":"2024-09-17T15:28:23.805107+0000","last_fullsized":"2024-09-17T15:28:23.805107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'10911","reported_seq":11081,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:56.699651+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:29:56.699651+0000","last_peered":"2024-09-17T15:29:56.699651+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:29:56.699651+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'900","ondisk_log_start":"34'900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40857600,"num_objects":9975,"num_object_clones":0,"num_object_copies":19950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4046,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9975,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7698,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5931},{"shards":"0,2","objects":3597}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'10787","reported_seq":15003,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:29:56.902093+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:29:56.902093+0000","last_peered":"2024-09-17T15:29:56.902093+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:29:56.902093+0000","last_undegraded":"2024-09-17T15:29:56.902093+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"34'700","ondisk_log_start":"34'700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40326086,"num_objects":9847,"num_object_clones":0,"num_object_copies":19694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2073,"num_objects_unfound":0,"num_objects_dirty":9847,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2036,"num_bytes_recovered":8339456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7774},{"shards":"0,2","objects":2073}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":649593590,"num_objects":158602,"num_object_clones":0,"num_object_copies":317204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41182,"num_objects_misplaced":2073,"num_objects_unfound":0,"num_objects_dirty":158602,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":134385,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6203,"num_bytes_recovered":25010659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1344446464,"data_stored":1344286574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160950,"ondisk_log_size":160950,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051643,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":738796,"kb_used_data":450260,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93633044,"statfs":{"total":96636764160,"available":95880237056,"internally_reserved":0,"allocated":461066240,"data_stored":460785594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":295431852},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,3,1,0,0,0,0,0,0,0,2],"upper_bound":16384},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724588,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":588512,"kb_used_data":322192,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93783328,"statfs":{"total":96636764160,"available":96034127872,"internally_reserved":0,"allocated":329924608,"data_stored":329596529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":272690859},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247871,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":868032,"kb_used_data":543472,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93503808,"statfs":{"total":96636764160,"available":95747899392,"internally_reserved":0,"allocated":556515328,"data_stored":556234677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":332328616},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,5,2,2,0,0,0,0,0,3],"upper_bound":8192},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":555114496,"data_stored":555078366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":329551872,"data_stored":329468314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":459780096,"data_stored":459739894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:29:59.037 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:29:59.575 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:29:59.575 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:29:59.608 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":203,"stamp":"2024-09-17T15:29:57.663900+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649593609,"num_objects":158604,"num_object_clones":0,"num_object_copies":317208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41182,"num_objects_misplaced":2073,"num_objects_unfound":0,"num_objects_dirty":158604,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":134387,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6204,"num_bytes_recovered":25010678,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160953,"ondisk_log_size":160953,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2195340,"kb_used_data":1315924,"kb_used_omap":13,"kb_used_meta":879346,"kb_avail":280920180,"statfs":{"total":289910292480,"available":287662264320,"internally_reserved":0,"allocated":1347506176,"data_stored":1346616800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":900451327},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,6,10,5,2,0,0,0,0,0,3,2],"upper_bound":16384},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27160576,"num_objects":-6631,"num_object_clones":0,"num_object_copies":-13262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-72,"num_objects_misplaced":-408,"num_objects_unfound":0,"num_objects_dirty":-6631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6631,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":480,"num_bytes_recovered":1671168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004359"},"pg_stats":[{"pgid":"2.f","version":"43'10813","reported_seq":10915,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:56.672393+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:29:56.672393+0000","last_peered":"2024-09-17T15:29:56.672393+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:29:56.672393+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'800","ondisk_log_start":"34'800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40284160,"num_objects":9835,"num_object_clones":0,"num_object_copies":19670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9835,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10813,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5777},{"shards":"0,2","objects":3581}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'10972","reported_seq":11075,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:57.428820+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:29:57.428820+0000","last_peered":"2024-09-17T15:29:57.428820+0000","last_clean":"2024-09-17T15:29:57.428820+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:29:57.428820+0000","last_undegraded":"2024-09-17T15:29:57.428820+0000","last_fullsized":"2024-09-17T15:29:57.428820+0000","mapping_epoch":33,"log_start":"34'900","ondisk_log_start":"34'900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40797126,"num_objects":9962,"num_object_clones":0,"num_object_copies":19924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9962,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9329,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'10969","reported_seq":11107,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:57.401952+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:29:57.401952+0000","last_peered":"2024-09-17T15:29:57.401952+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:29:57.401952+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'900","ondisk_log_start":"34'900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40812568,"num_objects":9965,"num_object_clones":0,"num_object_copies":19930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4163,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9965,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9126,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3668},{"shards":"2","objects":5803}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'10958","reported_seq":11028,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:57.354671+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:29:57.354671+0000","last_peered":"2024-09-17T15:29:57.354671+0000","last_clean":"2024-09-17T15:29:57.354671+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:29:57.354671+0000","last_undegraded":"2024-09-17T15:29:57.354671+0000","last_fullsized":"2024-09-17T15:29:57.354671+0000","mapping_epoch":33,"log_start":"34'900","ondisk_log_start":"34'900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41009152,"num_objects":10012,"num_object_clones":0,"num_object_copies":20024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10958,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'10810","reported_seq":10939,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:56.919423+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:29:56.919423+0000","last_peered":"2024-09-17T15:29:56.919423+0000","last_clean":"2024-09-17T15:29:56.919423+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:29:56.919423+0000","last_undegraded":"2024-09-17T15:29:56.919423+0000","last_fullsized":"2024-09-17T15:29:56.919423+0000","mapping_epoch":32,"log_start":"34'800","ondisk_log_start":"34'800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40317411,"num_objects":9844,"num_object_clones":0,"num_object_copies":19688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9844,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7792,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'11156","reported_seq":11311,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:56.868101+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:29:56.868101+0000","last_peered":"2024-09-17T15:29:56.868101+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:29:56.868101+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'1100","ondisk_log_start":"34'1100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41349603,"num_objects":10096,"num_object_clones":0,"num_object_copies":20192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4261,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10096,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7655,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5836},{"shards":"0,2","objects":3740}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'10883","reported_seq":11043,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:56.876428+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:29:56.876428+0000","last_peered":"2024-09-17T15:29:56.876428+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:29:56.876428+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'800","ondisk_log_start":"34'800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40489926,"num_objects":9887,"num_object_clones":0,"num_object_copies":19774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4163,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9887,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7652,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5724},{"shards":"0,2","objects":3678}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'10948","reported_seq":11076,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:56.816646+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:29:56.816646+0000","last_peered":"2024-09-17T15:29:56.816646+0000","last_clean":"2024-09-17T15:29:56.816646+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:29:56.816646+0000","last_undegraded":"2024-09-17T15:29:56.816646+0000","last_fullsized":"2024-09-17T15:29:56.816646+0000","mapping_epoch":32,"log_start":"34'900","ondisk_log_start":"34'900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40828928,"num_objects":9968,"num_object_clones":0,"num_object_copies":19936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9968,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7902,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'10872","reported_seq":19154,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:56.183288+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:29:56.183288+0000","last_peered":"2024-09-17T15:29:56.183288+0000","last_clean":"2024-09-17T15:29:56.183288+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:29:56.183288+0000","last_undegraded":"2024-09-17T15:29:56.183288+0000","last_fullsized":"2024-09-17T15:29:56.183288+0000","mapping_epoch":42,"log_start":"34'800","ondisk_log_start":"34'800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40735203,"num_objects":9946,"num_object_clones":0,"num_object_copies":19892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9946,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8872,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804747+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804747+0000","last_peered":"2024-09-17T15:28:23.804747+0000","last_clean":"2024-09-17T15:28:23.804747+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804747+0000","last_undegraded":"2024-09-17T15:28:23.804747+0000","last_fullsized":"2024-09-17T15:28:23.804747+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'10787","reported_seq":10935,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:56.878727+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:29:56.878727+0000","last_peered":"2024-09-17T15:29:56.878727+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:29:56.878727+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'700","ondisk_log_start":"34'700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39956480,"num_objects":9755,"num_object_clones":0,"num_object_copies":19510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4003,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7383,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5754},{"shards":"0,2","objects":3498}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'10741","reported_seq":10867,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:57.329860+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:29:57.329860+0000","last_peered":"2024-09-17T15:29:57.329860+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:29:57.329860+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'700","ondisk_log_start":"34'700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40051171,"num_objects":9779,"num_object_clones":0,"num_object_copies":19558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4075,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9779,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8636,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3598},{"shards":"2","objects":5705}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'10881","reported_seq":11016,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:57.401671+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:29:57.401671+0000","last_peered":"2024-09-17T15:29:57.401671+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:29:57.401671+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'800","ondisk_log_start":"34'800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40366080,"num_objects":9855,"num_object_clones":0,"num_object_copies":19710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8643,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3627},{"shards":"2","objects":5726}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'10974","reported_seq":11127,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:56.848988+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:29:56.848988+0000","last_peered":"2024-09-17T15:29:56.848988+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:29:56.848988+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'900","ondisk_log_start":"34'900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40927232,"num_objects":9992,"num_object_clones":0,"num_object_copies":19984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4130,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9992,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7522,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5863},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'10888","reported_seq":11045,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:56.903467+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:29:56.903467+0000","last_peered":"2024-09-17T15:29:56.903467+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:29:56.903467+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'800","ondisk_log_start":"34'800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40484864,"num_objects":9884,"num_object_clones":0,"num_object_copies":19768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4152,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9884,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7175,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5735},{"shards":"0,2","objects":3659}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805107+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805107+0000","last_peered":"2024-09-17T15:28:23.805107+0000","last_clean":"2024-09-17T15:28:23.805107+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805107+0000","last_undegraded":"2024-09-17T15:28:23.805107+0000","last_fullsized":"2024-09-17T15:28:23.805107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'10911","reported_seq":11081,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:56.699651+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:29:56.699651+0000","last_peered":"2024-09-17T15:29:56.699651+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:29:56.699651+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'900","ondisk_log_start":"34'900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40857600,"num_objects":9975,"num_object_clones":0,"num_object_copies":19950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4046,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9975,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7698,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5931},{"shards":"0,2","objects":3597}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'10787","reported_seq":15003,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:29:56.902093+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:29:56.902093+0000","last_peered":"2024-09-17T15:29:56.902093+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:29:56.902093+0000","last_undegraded":"2024-09-17T15:29:56.902093+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"34'700","ondisk_log_start":"34'700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40326086,"num_objects":9847,"num_object_clones":0,"num_object_copies":19694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2073,"num_objects_unfound":0,"num_objects_dirty":9847,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2036,"num_bytes_recovered":8339456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7774},{"shards":"0,2","objects":2073}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":649593590,"num_objects":158602,"num_object_clones":0,"num_object_copies":317204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41182,"num_objects_misplaced":2073,"num_objects_unfound":0,"num_objects_dirty":158602,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":134385,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6203,"num_bytes_recovered":25010659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1344446464,"data_stored":1344286574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160950,"ondisk_log_size":160950,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051643,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":738796,"kb_used_data":450260,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93633044,"statfs":{"total":96636764160,"available":95880237056,"internally_reserved":0,"allocated":461066240,"data_stored":460785594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":295431852},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,3,1,0,0,0,0,0,0,0,2],"upper_bound":16384},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724588,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":588512,"kb_used_data":322192,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93783328,"statfs":{"total":96636764160,"available":96034127872,"internally_reserved":0,"allocated":329924608,"data_stored":329596529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":272690859},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247871,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":868032,"kb_used_data":543472,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93503808,"statfs":{"total":96636764160,"available":95747899392,"internally_reserved":0,"allocated":556515328,"data_stored":556234677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":332328616},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,5,2,2,0,0,0,0,0,3],"upper_bound":8192},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":555114496,"data_stored":555078366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":329551872,"data_stored":329468314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":459780096,"data_stored":459739894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:29:59.609 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:29:59.609 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'10813", 'reported_seq': 10915, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:56.672393+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:29:56.672393+0000', 'last_peered': '2024-09-17T15:29:56.672393+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:29:56.672393+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "34'800", 'ondisk_log_start': "34'800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40284160, 'num_objects': 9835, 'num_object_clones': 0, 'num_object_copies': 19670, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9835, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10813, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 12, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5777}, {'shards': '0,2', 'objects': 3581}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:59.610 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:29:59.610 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'10969", 'reported_seq': 11107, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:57.401952+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:29:57.401952+0000', 'last_peered': '2024-09-17T15:29:57.401952+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:29:57.401952+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "34'900", 'ondisk_log_start': "34'900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40812568, 'num_objects': 9965, 'num_object_clones': 0, 'num_object_copies': 19930, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4163, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9965, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9126, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 6, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3668}, {'shards': '2', 'objects': 5803}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:29:59.610 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:29:59.610 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'11156", 'reported_seq': 11311, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:56.868101+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:29:56.868101+0000', 'last_peered': '2024-09-17T15:29:56.868101+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:29:56.868101+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "34'1100", 'ondisk_log_start': "34'1100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41349603, 'num_objects': 10096, 'num_object_clones': 0, 'num_object_copies': 20192, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4261, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10096, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7655, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 10, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5836}, {'shards': '0,2', 'objects': 3740}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:59.610 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:29:59.610 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'10883", 'reported_seq': 11043, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:56.876428+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:29:56.876428+0000', 'last_peered': '2024-09-17T15:29:56.876428+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:29:56.876428+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "34'800", 'ondisk_log_start': "34'800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40489926, 'num_objects': 9887, 'num_object_clones': 0, 'num_object_copies': 19774, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4163, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9887, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 7652, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 11, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5724}, {'shards': '0,2', 'objects': 3678}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:59.610 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:29:59.610 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'10787", 'reported_seq': 10935, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:56.878727+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:29:56.878727+0000', 'last_peered': '2024-09-17T15:29:56.878727+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:29:56.878727+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "34'700", 'ondisk_log_start': "34'700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39956480, 'num_objects': 9755, 'num_object_clones': 0, 'num_object_copies': 19510, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4003, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9755, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7383, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 9, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5754}, {'shards': '0,2', 'objects': 3498}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:59.610 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:29:59.610 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'10741", 'reported_seq': 10867, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:57.329860+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:29:57.329860+0000', 'last_peered': '2024-09-17T15:29:57.329860+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:29:57.329860+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "34'700", 'ondisk_log_start': "34'700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40051171, 'num_objects': 9779, 'num_object_clones': 0, 'num_object_copies': 19558, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4075, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9779, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8636, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3598}, {'shards': '2', 'objects': 5705}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:29:59.611 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:29:59.611 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'10881", 'reported_seq': 11016, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:57.401671+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:29:57.401671+0000', 'last_peered': '2024-09-17T15:29:57.401671+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:29:57.401671+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "34'800", 'ondisk_log_start': "34'800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40366080, 'num_objects': 9855, 'num_object_clones': 0, 'num_object_copies': 19710, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4131, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9855, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8643, 'num_write_kb': 32532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 8, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3627}, {'shards': '2', 'objects': 5726}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:29:59.611 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:29:59.611 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'10974", 'reported_seq': 11127, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:56.848988+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:29:56.848988+0000', 'last_peered': '2024-09-17T15:29:56.848988+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:29:56.848988+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "34'900", 'ondisk_log_start': "34'900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40927232, 'num_objects': 9992, 'num_object_clones': 0, 'num_object_copies': 19984, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4130, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9992, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7522, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 9, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5863}, {'shards': '0,2', 'objects': 3648}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:59.611 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:29:59.611 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'10888", 'reported_seq': 11045, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:56.903467+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:29:56.903467+0000', 'last_peered': '2024-09-17T15:29:56.903467+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:29:56.903467+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "34'800", 'ondisk_log_start': "34'800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40484864, 'num_objects': 9884, 'num_object_clones': 0, 'num_object_copies': 19768, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4152, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9884, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7175, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 8, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5735}, {'shards': '0,2', 'objects': 3659}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:59.611 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:29:59.611 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'10911", 'reported_seq': 11081, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:29:56.699651+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:29:56.699651+0000', 'last_peered': '2024-09-17T15:29:56.699651+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:29:56.699651+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "34'900", 'ondisk_log_start': "34'900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40857600, 'num_objects': 9975, 'num_object_clones': 0, 'num_object_copies': 19950, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4046, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9975, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7698, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 19, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5931}, {'shards': '0,2', 'objects': 3597}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:59.611 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:29:59.611 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'10787", 'reported_seq': 15003, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:29:56.902093+0000', 'last_change': '2024-09-17T15:28:22.509818+0000', 'last_active': '2024-09-17T15:29:56.902093+0000', 'last_peered': '2024-09-17T15:29:56.902093+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:29:56.902093+0000', 'last_undegraded': '2024-09-17T15:29:56.902093+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "34'700", 'ondisk_log_start': "34'700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40326086, 'num_objects': 9847, 'num_object_clones': 0, 'num_object_copies': 19694, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2073, 'num_objects_unfound': 0, 'num_objects_dirty': 9847, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7229, 'num_write_kb': 27056, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2036, 'num_bytes_recovered': 8339456, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7774}, {'shards': '0,2', 'objects': 2073}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:29:59.612 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:30:00.286 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:30:00.286 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41182/317208 objects degraded (12.983%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":337,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":158604,"data_bytes":649593609,"bytes_used":2248028160,"bytes_avail":287662264320,"bytes_total":289910292480,"degraded_objects":41182,"degraded_total":317208,"degraded_ratio":0.12982648609114525,"misplaced_objects":2073,"misplaced_total":317208,"misplaced_ratio":0.0065351441325565566,"recovering_objects_per_sec":39,"recovering_bytes_per_sec":139213,"recovering_keys_per_sec":0,"num_objects_recovered":480,"num_bytes_recovered":1671168,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":552},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T15:27:19.616376+0000","services":{}},"progress_events":{}} 2024-09-17T15:30:00.312 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:30:00.313 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:30:00.865 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:30:00.866 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:30:00.892 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":204,"stamp":"2024-09-17T15:29:59.666913+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649593609,"num_objects":158604,"num_object_clones":0,"num_object_copies":317208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41182,"num_objects_misplaced":2073,"num_objects_unfound":0,"num_objects_dirty":158604,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":134387,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6204,"num_bytes_recovered":25010678,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160953,"ondisk_log_size":160953,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2195340,"kb_used_data":1315924,"kb_used_omap":13,"kb_used_meta":879346,"kb_avail":280920180,"statfs":{"total":289910292480,"available":287662264320,"internally_reserved":0,"allocated":1347506176,"data_stored":1346616800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":900451327},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,6,10,5,2,0,0,0,0,0,3,2],"upper_bound":16384},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18071552,"num_objects":-4412,"num_object_clones":0,"num_object_copies":-8824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-48,"num_objects_misplaced":-273,"num_objects_unfound":0,"num_objects_dirty":-4412,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4412,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":321,"num_bytes_recovered":1118208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006646"},"pg_stats":[{"pgid":"2.f","version":"43'10813","reported_seq":10915,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:56.672393+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:29:56.672393+0000","last_peered":"2024-09-17T15:29:56.672393+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:29:56.672393+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'800","ondisk_log_start":"34'800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40284160,"num_objects":9835,"num_object_clones":0,"num_object_copies":19670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9835,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10813,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5777},{"shards":"0,2","objects":3581}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'10972","reported_seq":11075,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:57.428820+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:29:57.428820+0000","last_peered":"2024-09-17T15:29:57.428820+0000","last_clean":"2024-09-17T15:29:57.428820+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:29:57.428820+0000","last_undegraded":"2024-09-17T15:29:57.428820+0000","last_fullsized":"2024-09-17T15:29:57.428820+0000","mapping_epoch":33,"log_start":"34'900","ondisk_log_start":"34'900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40797126,"num_objects":9962,"num_object_clones":0,"num_object_copies":19924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9962,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9329,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'10969","reported_seq":11107,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:57.401952+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:29:57.401952+0000","last_peered":"2024-09-17T15:29:57.401952+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:29:57.401952+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'900","ondisk_log_start":"34'900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40812568,"num_objects":9965,"num_object_clones":0,"num_object_copies":19930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4163,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9965,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9126,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3668},{"shards":"2","objects":5803}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'10958","reported_seq":11028,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:57.354671+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:29:57.354671+0000","last_peered":"2024-09-17T15:29:57.354671+0000","last_clean":"2024-09-17T15:29:57.354671+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:29:57.354671+0000","last_undegraded":"2024-09-17T15:29:57.354671+0000","last_fullsized":"2024-09-17T15:29:57.354671+0000","mapping_epoch":33,"log_start":"34'900","ondisk_log_start":"34'900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41009152,"num_objects":10012,"num_object_clones":0,"num_object_copies":20024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10958,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'10810","reported_seq":10939,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:56.919423+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:29:56.919423+0000","last_peered":"2024-09-17T15:29:56.919423+0000","last_clean":"2024-09-17T15:29:56.919423+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:29:56.919423+0000","last_undegraded":"2024-09-17T15:29:56.919423+0000","last_fullsized":"2024-09-17T15:29:56.919423+0000","mapping_epoch":32,"log_start":"34'800","ondisk_log_start":"34'800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40317411,"num_objects":9844,"num_object_clones":0,"num_object_copies":19688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9844,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7792,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'11156","reported_seq":11311,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:56.868101+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:29:56.868101+0000","last_peered":"2024-09-17T15:29:56.868101+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:29:56.868101+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'1100","ondisk_log_start":"34'1100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41349603,"num_objects":10096,"num_object_clones":0,"num_object_copies":20192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4261,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10096,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7655,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5836},{"shards":"0,2","objects":3740}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'10883","reported_seq":11043,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:56.876428+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:29:56.876428+0000","last_peered":"2024-09-17T15:29:56.876428+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:29:56.876428+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'800","ondisk_log_start":"34'800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40489926,"num_objects":9887,"num_object_clones":0,"num_object_copies":19774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4163,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9887,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7652,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5724},{"shards":"0,2","objects":3678}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'10948","reported_seq":11076,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:56.816646+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:29:56.816646+0000","last_peered":"2024-09-17T15:29:56.816646+0000","last_clean":"2024-09-17T15:29:56.816646+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:29:56.816646+0000","last_undegraded":"2024-09-17T15:29:56.816646+0000","last_fullsized":"2024-09-17T15:29:56.816646+0000","mapping_epoch":32,"log_start":"34'900","ondisk_log_start":"34'900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40828928,"num_objects":9968,"num_object_clones":0,"num_object_copies":19936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9968,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7902,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'10872","reported_seq":19154,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:29:56.183288+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:29:56.183288+0000","last_peered":"2024-09-17T15:29:56.183288+0000","last_clean":"2024-09-17T15:29:56.183288+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:29:56.183288+0000","last_undegraded":"2024-09-17T15:29:56.183288+0000","last_fullsized":"2024-09-17T15:29:56.183288+0000","mapping_epoch":42,"log_start":"34'800","ondisk_log_start":"34'800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40735203,"num_objects":9946,"num_object_clones":0,"num_object_copies":19892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9946,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8872,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804747+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804747+0000","last_peered":"2024-09-17T15:28:23.804747+0000","last_clean":"2024-09-17T15:28:23.804747+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804747+0000","last_undegraded":"2024-09-17T15:28:23.804747+0000","last_fullsized":"2024-09-17T15:28:23.804747+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'10787","reported_seq":10935,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:56.878727+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:29:56.878727+0000","last_peered":"2024-09-17T15:29:56.878727+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:29:56.878727+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'700","ondisk_log_start":"34'700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39956480,"num_objects":9755,"num_object_clones":0,"num_object_copies":19510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4003,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7383,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5754},{"shards":"0,2","objects":3498}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'10741","reported_seq":10867,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:57.329860+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:29:57.329860+0000","last_peered":"2024-09-17T15:29:57.329860+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:29:57.329860+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'700","ondisk_log_start":"34'700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40051171,"num_objects":9779,"num_object_clones":0,"num_object_copies":19558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4075,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9779,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8636,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3598},{"shards":"2","objects":5705}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'10881","reported_seq":11016,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:57.401671+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:29:57.401671+0000","last_peered":"2024-09-17T15:29:57.401671+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:29:57.401671+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'800","ondisk_log_start":"34'800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40366080,"num_objects":9855,"num_object_clones":0,"num_object_copies":19710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8643,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3627},{"shards":"2","objects":5726}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'10974","reported_seq":11127,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:56.848988+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:29:56.848988+0000","last_peered":"2024-09-17T15:29:56.848988+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:29:56.848988+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'900","ondisk_log_start":"34'900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40927232,"num_objects":9992,"num_object_clones":0,"num_object_copies":19984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4130,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9992,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7522,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5863},{"shards":"0,2","objects":3648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'10888","reported_seq":11045,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:56.903467+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:29:56.903467+0000","last_peered":"2024-09-17T15:29:56.903467+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:29:56.903467+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'800","ondisk_log_start":"34'800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40484864,"num_objects":9884,"num_object_clones":0,"num_object_copies":19768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4152,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9884,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7175,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5735},{"shards":"0,2","objects":3659}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805107+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805107+0000","last_peered":"2024-09-17T15:28:23.805107+0000","last_clean":"2024-09-17T15:28:23.805107+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805107+0000","last_undegraded":"2024-09-17T15:28:23.805107+0000","last_fullsized":"2024-09-17T15:28:23.805107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'10911","reported_seq":11081,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:29:56.699651+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:29:56.699651+0000","last_peered":"2024-09-17T15:29:56.699651+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:29:56.699651+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'900","ondisk_log_start":"34'900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40857600,"num_objects":9975,"num_object_clones":0,"num_object_copies":19950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4046,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9975,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7698,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5931},{"shards":"0,2","objects":3597}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'10787","reported_seq":15003,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:29:56.902093+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:29:56.902093+0000","last_peered":"2024-09-17T15:29:56.902093+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:29:56.902093+0000","last_undegraded":"2024-09-17T15:29:56.902093+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"34'700","ondisk_log_start":"34'700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40326086,"num_objects":9847,"num_object_clones":0,"num_object_copies":19694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2073,"num_objects_unfound":0,"num_objects_dirty":9847,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2036,"num_bytes_recovered":8339456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7774},{"shards":"0,2","objects":2073}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":649593590,"num_objects":158602,"num_object_clones":0,"num_object_copies":317204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41182,"num_objects_misplaced":2073,"num_objects_unfound":0,"num_objects_dirty":158602,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":134385,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6203,"num_bytes_recovered":25010659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1344446464,"data_stored":1344286574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160950,"ondisk_log_size":160950,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051643,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":738796,"kb_used_data":450260,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93633044,"statfs":{"total":96636764160,"available":95880237056,"internally_reserved":0,"allocated":461066240,"data_stored":460785594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":295431852},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,3,1,0,0,0,0,0,0,0,2],"upper_bound":16384},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724588,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":588512,"kb_used_data":322192,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93783328,"statfs":{"total":96636764160,"available":96034127872,"internally_reserved":0,"allocated":329924608,"data_stored":329596529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":272690859},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247871,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":868032,"kb_used_data":543472,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93503808,"statfs":{"total":96636764160,"available":95747899392,"internally_reserved":0,"allocated":556515328,"data_stored":556234677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":332328616},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,5,2,2,0,0,0,0,0,3],"upper_bound":8192},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":555114496,"data_stored":555078366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":329551872,"data_stored":329468314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":459780096,"data_stored":459739894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:30:03.895 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:30:04.457 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:30:04.458 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:30:04.492 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":206,"stamp":"2024-09-17T15:30:03.668347+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":640668425,"num_objects":156425,"num_object_clones":0,"num_object_copies":312850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41180,"num_objects_misplaced":1906,"num_objects_unfound":0,"num_objects_dirty":156425,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":136566,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6373,"num_bytes_recovered":25694710,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160632,"ondisk_log_size":160632,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2199880,"kb_used_data":1303828,"kb_used_omap":13,"kb_used_meta":895986,"kb_avail":280915640,"statfs":{"total":289910292480,"available":287657615360,"internally_reserved":0,"allocated":1335119872,"data_stored":1334230496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":917490687},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,6,7,5,1,0,0,0,0,0,0,7],"upper_bound":16384},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26456064,"num_objects":-6459,"num_object_clones":0,"num_object_copies":-12918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-50,"num_objects_misplaced":-440,"num_objects_unfound":0,"num_objects_dirty":-6459,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6459,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":490,"num_bytes_recovered":1802240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006568"},"pg_stats":[{"pgid":"2.f","version":"43'10947","reported_seq":11048,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:01.924528+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:30:01.924528+0000","last_peered":"2024-09-17T15:30:01.924528+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:30:01.924528+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'900","ondisk_log_start":"34'900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39735296,"num_objects":9701,"num_object_clones":0,"num_object_copies":19402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9701,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10947,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5643},{"shards":"0,2","objects":3447}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'11109","reported_seq":11212,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:02.430961+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:30:02.430961+0000","last_peered":"2024-09-17T15:30:02.430961+0000","last_clean":"2024-09-17T15:30:02.430961+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:30:02.430961+0000","last_undegraded":"2024-09-17T15:30:02.430961+0000","last_fullsized":"2024-09-17T15:30:02.430961+0000","mapping_epoch":33,"log_start":"34'1100","ondisk_log_start":"34'1100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40235974,"num_objects":9825,"num_object_clones":0,"num_object_copies":19650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9825,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9466,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'11121","reported_seq":11259,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:02.420079+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:30:02.420079+0000","last_peered":"2024-09-17T15:30:02.420079+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:30:02.420079+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'1100","ondisk_log_start":"34'1100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40189976,"num_objects":9813,"num_object_clones":0,"num_object_copies":19626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4163,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9813,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9278,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3516},{"shards":"2","objects":5651}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'11082","reported_seq":11152,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:02.345005+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:30:02.345005+0000","last_peered":"2024-09-17T15:30:02.345005+0000","last_clean":"2024-09-17T15:30:02.345005+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:30:02.345005+0000","last_undegraded":"2024-09-17T15:30:02.345005+0000","last_fullsized":"2024-09-17T15:30:02.345005+0000","mapping_epoch":33,"log_start":"34'1000","ondisk_log_start":"34'1000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40501248,"num_objects":9888,"num_object_clones":0,"num_object_copies":19776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11082,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'10953","reported_seq":11082,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:01.920721+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:30:01.920721+0000","last_peered":"2024-09-17T15:30:01.920721+0000","last_clean":"2024-09-17T15:30:01.920721+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:30:01.920721+0000","last_undegraded":"2024-09-17T15:30:01.920721+0000","last_fullsized":"2024-09-17T15:30:01.920721+0000","mapping_epoch":32,"log_start":"34'900","ondisk_log_start":"34'900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39731683,"num_objects":9701,"num_object_clones":0,"num_object_copies":19402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9701,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7935,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'11302","reported_seq":11457,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:01.775750+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:30:01.775750+0000","last_peered":"2024-09-17T15:30:01.775750+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:30:01.775750+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'1300","ondisk_log_start":"34'1300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40751587,"num_objects":9950,"num_object_clones":0,"num_object_copies":19900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4261,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9950,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7801,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5690},{"shards":"0,2","objects":3594}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'11028","reported_seq":11189,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:01.819418+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:30:01.819418+0000","last_peered":"2024-09-17T15:30:01.819418+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:30:01.819418+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'1000","ondisk_log_start":"34'1000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39896006,"num_objects":9742,"num_object_clones":0,"num_object_copies":19484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4163,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9742,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7797,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5579},{"shards":"0,2","objects":3533}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'11088","reported_seq":11216,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:01.806997+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:30:01.806997+0000","last_peered":"2024-09-17T15:30:01.806997+0000","last_clean":"2024-09-17T15:30:01.806997+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:30:01.806997+0000","last_undegraded":"2024-09-17T15:30:01.806997+0000","last_fullsized":"2024-09-17T15:30:01.806997+0000","mapping_epoch":32,"log_start":"34'1000","ondisk_log_start":"34'1000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40255488,"num_objects":9828,"num_object_clones":0,"num_object_copies":19656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8042,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'11007","reported_seq":19289,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:01.118070+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:30:01.118070+0000","last_peered":"2024-09-17T15:30:01.118070+0000","last_clean":"2024-09-17T15:30:01.118070+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:30:01.118070+0000","last_undegraded":"2024-09-17T15:30:01.118070+0000","last_fullsized":"2024-09-17T15:30:01.118070+0000","mapping_epoch":42,"log_start":"34'1000","ondisk_log_start":"34'1000","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40182243,"num_objects":9811,"num_object_clones":0,"num_object_copies":19622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9811,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9007,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804747+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804747+0000","last_peered":"2024-09-17T15:28:23.804747+0000","last_clean":"2024-09-17T15:28:23.804747+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804747+0000","last_undegraded":"2024-09-17T15:28:23.804747+0000","last_fullsized":"2024-09-17T15:28:23.804747+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'10903","reported_seq":11051,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:01.859578+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:30:01.859578+0000","last_peered":"2024-09-17T15:30:01.859578+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:30:01.859578+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'900","ondisk_log_start":"34'900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39481344,"num_objects":9639,"num_object_clones":0,"num_object_copies":19278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4003,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7499,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5638},{"shards":"0,2","objects":3382}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'10870","reported_seq":11001,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:02.439090+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:30:02.439090+0000","last_peered":"2024-09-17T15:30:02.439090+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:30:02.439090+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'800","ondisk_log_start":"34'800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39522787,"num_objects":9650,"num_object_clones":0,"num_object_copies":19300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4073,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9650,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8765,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3469},{"shards":"2","objects":5578}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'10983","reported_seq":11118,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:02.333190+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:30:02.333190+0000","last_peered":"2024-09-17T15:30:02.333190+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:30:02.333190+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'900","ondisk_log_start":"34'900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39948288,"num_objects":9753,"num_object_clones":0,"num_object_copies":19506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8745,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":5624}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'11123","reported_seq":11276,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:01.926642+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:30:01.926642+0000","last_peered":"2024-09-17T15:30:01.926642+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:30:01.926642+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'1100","ondisk_log_start":"34'1100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40316928,"num_objects":9843,"num_object_clones":0,"num_object_copies":19686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4130,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7671,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5714},{"shards":"0,2","objects":3499}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'11044","reported_seq":11199,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:01.895729+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:30:01.895729+0000","last_peered":"2024-09-17T15:30:01.895729+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:30:01.895729+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'1000","ondisk_log_start":"34'1000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39845888,"num_objects":9728,"num_object_clones":0,"num_object_copies":19456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4152,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9728,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7331,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5579},{"shards":"0,2","objects":3503}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805107+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805107+0000","last_peered":"2024-09-17T15:28:23.805107+0000","last_clean":"2024-09-17T15:28:23.805107+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805107+0000","last_undegraded":"2024-09-17T15:28:23.805107+0000","last_fullsized":"2024-09-17T15:28:23.805107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'11041","reported_seq":11211,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:01.898000+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:30:01.898000+0000","last_peered":"2024-09-17T15:30:01.898000+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:30:01.898000+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'1000","ondisk_log_start":"34'1000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40325120,"num_objects":9845,"num_object_clones":0,"num_object_copies":19690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4046,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7828,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5801},{"shards":"0,2","objects":3467}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'10928","reported_seq":15478,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:30:01.947264+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:30:01.947264+0000","last_peered":"2024-09-17T15:30:01.947264+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:30:01.947264+0000","last_undegraded":"2024-09-17T15:30:01.947264+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"34'900","ondisk_log_start":"34'900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39748550,"num_objects":9706,"num_object_clones":0,"num_object_copies":19412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1906,"num_objects_unfound":0,"num_objects_dirty":9706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7370,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2203,"num_bytes_recovered":9023488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7800},{"shards":"0,2","objects":1906}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":640668406,"num_objects":156423,"num_object_clones":0,"num_object_copies":312846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41180,"num_objects_misplaced":1906,"num_objects_unfound":0,"num_objects_dirty":156423,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":136564,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6372,"num_bytes_recovered":25694691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1332903936,"data_stored":1332744046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160629,"ondisk_log_size":160629,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051644,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":737856,"kb_used_data":445168,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93633984,"statfs":{"total":96636764160,"available":95881199616,"internally_reserved":0,"allocated":455852032,"data_stored":455571386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":299691692},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724589,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":596428,"kb_used_data":321788,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93775412,"statfs":{"total":96636764160,"available":96026021888,"internally_reserved":0,"allocated":329510912,"data_stored":329182833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":281210539},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247872,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":865596,"kb_used_data":536872,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93506244,"statfs":{"total":96636764160,"available":95750393856,"internally_reserved":0,"allocated":549756928,"data_stored":549476277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":336588456},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,4,1,1,0,0,0,0,0,0,7],"upper_bound":16384},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":548327424,"data_stored":548291294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":329146368,"data_stored":329062810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":455430144,"data_stored":455389942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:30:04.493 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:30:05.032 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:30:05.033 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:30:05.055 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":206,"stamp":"2024-09-17T15:30:03.668347+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":640668425,"num_objects":156425,"num_object_clones":0,"num_object_copies":312850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41180,"num_objects_misplaced":1906,"num_objects_unfound":0,"num_objects_dirty":156425,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":136566,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6373,"num_bytes_recovered":25694710,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160632,"ondisk_log_size":160632,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2199880,"kb_used_data":1303828,"kb_used_omap":13,"kb_used_meta":895986,"kb_avail":280915640,"statfs":{"total":289910292480,"available":287657615360,"internally_reserved":0,"allocated":1335119872,"data_stored":1334230496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":917490687},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,6,7,5,1,0,0,0,0,0,0,7],"upper_bound":16384},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26456064,"num_objects":-6459,"num_object_clones":0,"num_object_copies":-12918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-50,"num_objects_misplaced":-440,"num_objects_unfound":0,"num_objects_dirty":-6459,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6459,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":490,"num_bytes_recovered":1802240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006568"},"pg_stats":[{"pgid":"2.f","version":"43'10947","reported_seq":11048,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:01.924528+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:30:01.924528+0000","last_peered":"2024-09-17T15:30:01.924528+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:30:01.924528+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'900","ondisk_log_start":"34'900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39735296,"num_objects":9701,"num_object_clones":0,"num_object_copies":19402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9701,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10947,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5643},{"shards":"0,2","objects":3447}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'11109","reported_seq":11212,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:02.430961+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:30:02.430961+0000","last_peered":"2024-09-17T15:30:02.430961+0000","last_clean":"2024-09-17T15:30:02.430961+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:30:02.430961+0000","last_undegraded":"2024-09-17T15:30:02.430961+0000","last_fullsized":"2024-09-17T15:30:02.430961+0000","mapping_epoch":33,"log_start":"34'1100","ondisk_log_start":"34'1100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40235974,"num_objects":9825,"num_object_clones":0,"num_object_copies":19650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9825,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9466,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'11121","reported_seq":11259,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:02.420079+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:30:02.420079+0000","last_peered":"2024-09-17T15:30:02.420079+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:30:02.420079+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'1100","ondisk_log_start":"34'1100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40189976,"num_objects":9813,"num_object_clones":0,"num_object_copies":19626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4163,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9813,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9278,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3516},{"shards":"2","objects":5651}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'11082","reported_seq":11152,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:02.345005+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:30:02.345005+0000","last_peered":"2024-09-17T15:30:02.345005+0000","last_clean":"2024-09-17T15:30:02.345005+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:30:02.345005+0000","last_undegraded":"2024-09-17T15:30:02.345005+0000","last_fullsized":"2024-09-17T15:30:02.345005+0000","mapping_epoch":33,"log_start":"34'1000","ondisk_log_start":"34'1000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40501248,"num_objects":9888,"num_object_clones":0,"num_object_copies":19776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11082,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'10953","reported_seq":11082,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:01.920721+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:30:01.920721+0000","last_peered":"2024-09-17T15:30:01.920721+0000","last_clean":"2024-09-17T15:30:01.920721+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:30:01.920721+0000","last_undegraded":"2024-09-17T15:30:01.920721+0000","last_fullsized":"2024-09-17T15:30:01.920721+0000","mapping_epoch":32,"log_start":"34'900","ondisk_log_start":"34'900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39731683,"num_objects":9701,"num_object_clones":0,"num_object_copies":19402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9701,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7935,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'11302","reported_seq":11457,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:01.775750+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:30:01.775750+0000","last_peered":"2024-09-17T15:30:01.775750+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:30:01.775750+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'1300","ondisk_log_start":"34'1300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40751587,"num_objects":9950,"num_object_clones":0,"num_object_copies":19900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4261,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9950,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7801,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5690},{"shards":"0,2","objects":3594}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'11028","reported_seq":11189,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:01.819418+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:30:01.819418+0000","last_peered":"2024-09-17T15:30:01.819418+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:30:01.819418+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'1000","ondisk_log_start":"34'1000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39896006,"num_objects":9742,"num_object_clones":0,"num_object_copies":19484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4163,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9742,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7797,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5579},{"shards":"0,2","objects":3533}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'11088","reported_seq":11216,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:01.806997+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:30:01.806997+0000","last_peered":"2024-09-17T15:30:01.806997+0000","last_clean":"2024-09-17T15:30:01.806997+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:30:01.806997+0000","last_undegraded":"2024-09-17T15:30:01.806997+0000","last_fullsized":"2024-09-17T15:30:01.806997+0000","mapping_epoch":32,"log_start":"34'1000","ondisk_log_start":"34'1000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40255488,"num_objects":9828,"num_object_clones":0,"num_object_copies":19656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8042,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'11007","reported_seq":19289,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:01.118070+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:30:01.118070+0000","last_peered":"2024-09-17T15:30:01.118070+0000","last_clean":"2024-09-17T15:30:01.118070+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:30:01.118070+0000","last_undegraded":"2024-09-17T15:30:01.118070+0000","last_fullsized":"2024-09-17T15:30:01.118070+0000","mapping_epoch":42,"log_start":"34'1000","ondisk_log_start":"34'1000","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40182243,"num_objects":9811,"num_object_clones":0,"num_object_copies":19622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9811,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9007,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804747+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804747+0000","last_peered":"2024-09-17T15:28:23.804747+0000","last_clean":"2024-09-17T15:28:23.804747+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804747+0000","last_undegraded":"2024-09-17T15:28:23.804747+0000","last_fullsized":"2024-09-17T15:28:23.804747+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'10903","reported_seq":11051,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:01.859578+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:30:01.859578+0000","last_peered":"2024-09-17T15:30:01.859578+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:30:01.859578+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'900","ondisk_log_start":"34'900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39481344,"num_objects":9639,"num_object_clones":0,"num_object_copies":19278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4003,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7499,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5638},{"shards":"0,2","objects":3382}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'10870","reported_seq":11001,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:02.439090+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:30:02.439090+0000","last_peered":"2024-09-17T15:30:02.439090+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:30:02.439090+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'800","ondisk_log_start":"34'800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39522787,"num_objects":9650,"num_object_clones":0,"num_object_copies":19300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4073,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9650,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8765,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3469},{"shards":"2","objects":5578}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'10983","reported_seq":11118,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:02.333190+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:30:02.333190+0000","last_peered":"2024-09-17T15:30:02.333190+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:30:02.333190+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'900","ondisk_log_start":"34'900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39948288,"num_objects":9753,"num_object_clones":0,"num_object_copies":19506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8745,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":5624}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'11123","reported_seq":11276,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:01.926642+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:30:01.926642+0000","last_peered":"2024-09-17T15:30:01.926642+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:30:01.926642+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'1100","ondisk_log_start":"34'1100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40316928,"num_objects":9843,"num_object_clones":0,"num_object_copies":19686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4130,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7671,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5714},{"shards":"0,2","objects":3499}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'11044","reported_seq":11199,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:01.895729+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:30:01.895729+0000","last_peered":"2024-09-17T15:30:01.895729+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:30:01.895729+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'1000","ondisk_log_start":"34'1000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39845888,"num_objects":9728,"num_object_clones":0,"num_object_copies":19456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4152,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9728,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7331,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5579},{"shards":"0,2","objects":3503}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805107+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805107+0000","last_peered":"2024-09-17T15:28:23.805107+0000","last_clean":"2024-09-17T15:28:23.805107+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805107+0000","last_undegraded":"2024-09-17T15:28:23.805107+0000","last_fullsized":"2024-09-17T15:28:23.805107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'11041","reported_seq":11211,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:01.898000+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:30:01.898000+0000","last_peered":"2024-09-17T15:30:01.898000+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:30:01.898000+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'1000","ondisk_log_start":"34'1000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40325120,"num_objects":9845,"num_object_clones":0,"num_object_copies":19690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4046,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7828,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5801},{"shards":"0,2","objects":3467}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'10928","reported_seq":15478,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:30:01.947264+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:30:01.947264+0000","last_peered":"2024-09-17T15:30:01.947264+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:30:01.947264+0000","last_undegraded":"2024-09-17T15:30:01.947264+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"34'900","ondisk_log_start":"34'900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39748550,"num_objects":9706,"num_object_clones":0,"num_object_copies":19412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1906,"num_objects_unfound":0,"num_objects_dirty":9706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7370,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2203,"num_bytes_recovered":9023488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7800},{"shards":"0,2","objects":1906}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":640668406,"num_objects":156423,"num_object_clones":0,"num_object_copies":312846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41180,"num_objects_misplaced":1906,"num_objects_unfound":0,"num_objects_dirty":156423,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":136564,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6372,"num_bytes_recovered":25694691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1332903936,"data_stored":1332744046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160629,"ondisk_log_size":160629,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051644,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":737856,"kb_used_data":445168,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93633984,"statfs":{"total":96636764160,"available":95881199616,"internally_reserved":0,"allocated":455852032,"data_stored":455571386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":299691692},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724589,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":596428,"kb_used_data":321788,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93775412,"statfs":{"total":96636764160,"available":96026021888,"internally_reserved":0,"allocated":329510912,"data_stored":329182833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":281210539},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247872,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":865596,"kb_used_data":536872,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93506244,"statfs":{"total":96636764160,"available":95750393856,"internally_reserved":0,"allocated":549756928,"data_stored":549476277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":336588456},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,4,1,1,0,0,0,0,0,0,7],"upper_bound":16384},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":548327424,"data_stored":548291294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":329146368,"data_stored":329062810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":455430144,"data_stored":455389942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:30:05.057 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:30:05.057 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'10947", 'reported_seq': 11048, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:01.924528+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:30:01.924528+0000', 'last_peered': '2024-09-17T15:30:01.924528+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:30:01.924528+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "34'900", 'ondisk_log_start': "34'900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39735296, 'num_objects': 9701, 'num_object_clones': 0, 'num_object_copies': 19402, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9701, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10947, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 12, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5643}, {'shards': '0,2', 'objects': 3447}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:05.057 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:30:05.057 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'11121", 'reported_seq': 11259, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:02.420079+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:30:02.420079+0000', 'last_peered': '2024-09-17T15:30:02.420079+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:30:02.420079+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "34'1100", 'ondisk_log_start': "34'1100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40189976, 'num_objects': 9813, 'num_object_clones': 0, 'num_object_copies': 19626, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4163, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9813, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9278, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 6, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3516}, {'shards': '2', 'objects': 5651}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:30:05.057 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:30:05.057 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'11302", 'reported_seq': 11457, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:01.775750+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:30:01.775750+0000', 'last_peered': '2024-09-17T15:30:01.775750+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:30:01.775750+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "34'1300", 'ondisk_log_start': "34'1300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40751587, 'num_objects': 9950, 'num_object_clones': 0, 'num_object_copies': 19900, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4261, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9950, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7801, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 10, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5690}, {'shards': '0,2', 'objects': 3594}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:05.057 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:30:05.057 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'11028", 'reported_seq': 11189, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:01.819418+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:30:01.819418+0000', 'last_peered': '2024-09-17T15:30:01.819418+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:30:01.819418+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "34'1000", 'ondisk_log_start': "34'1000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39896006, 'num_objects': 9742, 'num_object_clones': 0, 'num_object_copies': 19484, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4163, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9742, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 7797, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 11, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5579}, {'shards': '0,2', 'objects': 3533}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:05.057 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:30:05.057 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'10903", 'reported_seq': 11051, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:01.859578+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:30:01.859578+0000', 'last_peered': '2024-09-17T15:30:01.859578+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:30:01.859578+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "34'900", 'ondisk_log_start': "34'900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39481344, 'num_objects': 9639, 'num_object_clones': 0, 'num_object_copies': 19278, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4003, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9639, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7499, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 9, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5638}, {'shards': '0,2', 'objects': 3382}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:05.058 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:30:05.058 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'10870", 'reported_seq': 11001, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:02.439090+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:30:02.439090+0000', 'last_peered': '2024-09-17T15:30:02.439090+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:30:02.439090+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "34'800", 'ondisk_log_start': "34'800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39522787, 'num_objects': 9650, 'num_object_clones': 0, 'num_object_copies': 19300, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4073, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9650, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8765, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 5, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3469}, {'shards': '2', 'objects': 5578}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:30:05.058 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:30:05.058 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'10983", 'reported_seq': 11118, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:02.333190+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:30:02.333190+0000', 'last_peered': '2024-09-17T15:30:02.333190+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:30:02.333190+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "34'900", 'ondisk_log_start': "34'900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39948288, 'num_objects': 9753, 'num_object_clones': 0, 'num_object_copies': 19506, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4131, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9753, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8745, 'num_write_kb': 32532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 8, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 5624}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:30:05.058 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:30:05.058 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'11123", 'reported_seq': 11276, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:01.926642+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:30:01.926642+0000', 'last_peered': '2024-09-17T15:30:01.926642+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:30:01.926642+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "34'1100", 'ondisk_log_start': "34'1100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40316928, 'num_objects': 9843, 'num_object_clones': 0, 'num_object_copies': 19686, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4130, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9843, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7671, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 9, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5714}, {'shards': '0,2', 'objects': 3499}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:05.058 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:30:05.058 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'11044", 'reported_seq': 11199, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:01.895729+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:30:01.895729+0000', 'last_peered': '2024-09-17T15:30:01.895729+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:30:01.895729+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "34'1000", 'ondisk_log_start': "34'1000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39845888, 'num_objects': 9728, 'num_object_clones': 0, 'num_object_copies': 19456, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4152, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9728, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7331, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 8, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5579}, {'shards': '0,2', 'objects': 3503}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:05.058 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:30:05.058 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'11041", 'reported_seq': 11211, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:01.898000+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:30:01.898000+0000', 'last_peered': '2024-09-17T15:30:01.898000+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:30:01.898000+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "34'1000", 'ondisk_log_start': "34'1000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40325120, 'num_objects': 9845, 'num_object_clones': 0, 'num_object_copies': 19690, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4046, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9845, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7828, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 19, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5801}, {'shards': '0,2', 'objects': 3467}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:05.059 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:30:05.059 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'10928", 'reported_seq': 15478, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:30:01.947264+0000', 'last_change': '2024-09-17T15:28:22.509818+0000', 'last_active': '2024-09-17T15:30:01.947264+0000', 'last_peered': '2024-09-17T15:30:01.947264+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:30:01.947264+0000', 'last_undegraded': '2024-09-17T15:30:01.947264+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "34'900", 'ondisk_log_start': "34'900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39748550, 'num_objects': 9706, 'num_object_clones': 0, 'num_object_copies': 19412, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1906, 'num_objects_unfound': 0, 'num_objects_dirty': 9706, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7370, 'num_write_kb': 27056, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2203, 'num_bytes_recovered': 9023488, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7800}, {'shards': '0,2', 'objects': 1906}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:05.059 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:30:05.737 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:30:05.737 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41180/312850 objects degraded (13.163%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":342,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":156425,"data_bytes":640668425,"bytes_used":2252677120,"bytes_avail":287657615360,"bytes_total":289910292480,"degraded_objects":41180,"degraded_total":312850,"degraded_ratio":0.13162857599488573,"misplaced_objects":1906,"misplaced_total":312850,"misplaced_ratio":0.0060923765382771304,"recovering_objects_per_sec":40,"recovering_bytes_per_sec":150104,"recovering_keys_per_sec":0,"num_objects_recovered":490,"num_bytes_recovered":1802240,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":537},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T15:27:19.616376+0000","services":{}},"progress_events":{}} 2024-09-17T15:30:05.760 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:30:05.760 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:30:06.323 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:30:06.324 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:30:06.358 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":207,"stamp":"2024-09-17T15:30:05.668851+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":640668425,"num_objects":156425,"num_object_clones":0,"num_object_copies":312850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41180,"num_objects_misplaced":1906,"num_objects_unfound":0,"num_objects_dirty":156425,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":136566,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6373,"num_bytes_recovered":25694710,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160632,"ondisk_log_size":160632,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2199880,"kb_used_data":1303828,"kb_used_omap":13,"kb_used_meta":895986,"kb_avail":280915640,"statfs":{"total":289910292480,"available":287657615360,"internally_reserved":0,"allocated":1335119872,"data_stored":1334230496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":917490687},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,6,7,5,1,0,0,0,0,0,0,7],"upper_bound":16384},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17960960,"num_objects":-4385,"num_object_clones":0,"num_object_copies":-8770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-20,"num_objects_misplaced":-307,"num_objects_unfound":0,"num_objects_dirty":-4385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4385,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":1257472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006431"},"pg_stats":[{"pgid":"2.f","version":"43'10947","reported_seq":11048,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:01.924528+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:30:01.924528+0000","last_peered":"2024-09-17T15:30:01.924528+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:30:01.924528+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'900","ondisk_log_start":"34'900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39735296,"num_objects":9701,"num_object_clones":0,"num_object_copies":19402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9701,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10947,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5643},{"shards":"0,2","objects":3447}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'11109","reported_seq":11212,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:02.430961+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:30:02.430961+0000","last_peered":"2024-09-17T15:30:02.430961+0000","last_clean":"2024-09-17T15:30:02.430961+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:30:02.430961+0000","last_undegraded":"2024-09-17T15:30:02.430961+0000","last_fullsized":"2024-09-17T15:30:02.430961+0000","mapping_epoch":33,"log_start":"34'1100","ondisk_log_start":"34'1100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40235974,"num_objects":9825,"num_object_clones":0,"num_object_copies":19650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9825,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9466,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'11121","reported_seq":11259,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:02.420079+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:30:02.420079+0000","last_peered":"2024-09-17T15:30:02.420079+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:30:02.420079+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'1100","ondisk_log_start":"34'1100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40189976,"num_objects":9813,"num_object_clones":0,"num_object_copies":19626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4163,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9813,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9278,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3516},{"shards":"2","objects":5651}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'11082","reported_seq":11152,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:02.345005+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:30:02.345005+0000","last_peered":"2024-09-17T15:30:02.345005+0000","last_clean":"2024-09-17T15:30:02.345005+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:30:02.345005+0000","last_undegraded":"2024-09-17T15:30:02.345005+0000","last_fullsized":"2024-09-17T15:30:02.345005+0000","mapping_epoch":33,"log_start":"34'1000","ondisk_log_start":"34'1000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40501248,"num_objects":9888,"num_object_clones":0,"num_object_copies":19776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11082,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'10953","reported_seq":11082,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:01.920721+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:30:01.920721+0000","last_peered":"2024-09-17T15:30:01.920721+0000","last_clean":"2024-09-17T15:30:01.920721+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:30:01.920721+0000","last_undegraded":"2024-09-17T15:30:01.920721+0000","last_fullsized":"2024-09-17T15:30:01.920721+0000","mapping_epoch":32,"log_start":"34'900","ondisk_log_start":"34'900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39731683,"num_objects":9701,"num_object_clones":0,"num_object_copies":19402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9701,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7935,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'11302","reported_seq":11457,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:01.775750+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:30:01.775750+0000","last_peered":"2024-09-17T15:30:01.775750+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:30:01.775750+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'1300","ondisk_log_start":"34'1300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40751587,"num_objects":9950,"num_object_clones":0,"num_object_copies":19900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4261,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9950,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7801,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5690},{"shards":"0,2","objects":3594}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'11028","reported_seq":11189,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:01.819418+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:30:01.819418+0000","last_peered":"2024-09-17T15:30:01.819418+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:30:01.819418+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'1000","ondisk_log_start":"34'1000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39896006,"num_objects":9742,"num_object_clones":0,"num_object_copies":19484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4163,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9742,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7797,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5579},{"shards":"0,2","objects":3533}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'11088","reported_seq":11216,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:01.806997+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:30:01.806997+0000","last_peered":"2024-09-17T15:30:01.806997+0000","last_clean":"2024-09-17T15:30:01.806997+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:30:01.806997+0000","last_undegraded":"2024-09-17T15:30:01.806997+0000","last_fullsized":"2024-09-17T15:30:01.806997+0000","mapping_epoch":32,"log_start":"34'1000","ondisk_log_start":"34'1000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40255488,"num_objects":9828,"num_object_clones":0,"num_object_copies":19656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8042,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'11007","reported_seq":19289,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:01.118070+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:30:01.118070+0000","last_peered":"2024-09-17T15:30:01.118070+0000","last_clean":"2024-09-17T15:30:01.118070+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:30:01.118070+0000","last_undegraded":"2024-09-17T15:30:01.118070+0000","last_fullsized":"2024-09-17T15:30:01.118070+0000","mapping_epoch":42,"log_start":"34'1000","ondisk_log_start":"34'1000","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40182243,"num_objects":9811,"num_object_clones":0,"num_object_copies":19622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9811,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9007,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804747+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804747+0000","last_peered":"2024-09-17T15:28:23.804747+0000","last_clean":"2024-09-17T15:28:23.804747+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804747+0000","last_undegraded":"2024-09-17T15:28:23.804747+0000","last_fullsized":"2024-09-17T15:28:23.804747+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'10903","reported_seq":11051,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:01.859578+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:30:01.859578+0000","last_peered":"2024-09-17T15:30:01.859578+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:30:01.859578+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'900","ondisk_log_start":"34'900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39481344,"num_objects":9639,"num_object_clones":0,"num_object_copies":19278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4003,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7499,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5638},{"shards":"0,2","objects":3382}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'10870","reported_seq":11001,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:02.439090+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:30:02.439090+0000","last_peered":"2024-09-17T15:30:02.439090+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:30:02.439090+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'800","ondisk_log_start":"34'800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39522787,"num_objects":9650,"num_object_clones":0,"num_object_copies":19300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4073,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9650,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8765,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3469},{"shards":"2","objects":5578}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'10983","reported_seq":11118,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:02.333190+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:30:02.333190+0000","last_peered":"2024-09-17T15:30:02.333190+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:30:02.333190+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'900","ondisk_log_start":"34'900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39948288,"num_objects":9753,"num_object_clones":0,"num_object_copies":19506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8745,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":5624}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'11123","reported_seq":11276,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:01.926642+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:30:01.926642+0000","last_peered":"2024-09-17T15:30:01.926642+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:30:01.926642+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'1100","ondisk_log_start":"34'1100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40316928,"num_objects":9843,"num_object_clones":0,"num_object_copies":19686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4130,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7671,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5714},{"shards":"0,2","objects":3499}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'11044","reported_seq":11199,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:01.895729+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:30:01.895729+0000","last_peered":"2024-09-17T15:30:01.895729+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:30:01.895729+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'1000","ondisk_log_start":"34'1000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39845888,"num_objects":9728,"num_object_clones":0,"num_object_copies":19456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4152,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9728,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7331,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5579},{"shards":"0,2","objects":3503}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805107+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805107+0000","last_peered":"2024-09-17T15:28:23.805107+0000","last_clean":"2024-09-17T15:28:23.805107+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805107+0000","last_undegraded":"2024-09-17T15:28:23.805107+0000","last_fullsized":"2024-09-17T15:28:23.805107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'11041","reported_seq":11211,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:01.898000+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:30:01.898000+0000","last_peered":"2024-09-17T15:30:01.898000+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:30:01.898000+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'1000","ondisk_log_start":"34'1000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40325120,"num_objects":9845,"num_object_clones":0,"num_object_copies":19690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4046,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7828,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5801},{"shards":"0,2","objects":3467}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'10928","reported_seq":15478,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:30:01.947264+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:30:01.947264+0000","last_peered":"2024-09-17T15:30:01.947264+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:30:01.947264+0000","last_undegraded":"2024-09-17T15:30:01.947264+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"34'900","ondisk_log_start":"34'900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39748550,"num_objects":9706,"num_object_clones":0,"num_object_copies":19412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1906,"num_objects_unfound":0,"num_objects_dirty":9706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7370,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2203,"num_bytes_recovered":9023488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7800},{"shards":"0,2","objects":1906}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":640668406,"num_objects":156423,"num_object_clones":0,"num_object_copies":312846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41180,"num_objects_misplaced":1906,"num_objects_unfound":0,"num_objects_dirty":156423,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":136564,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6372,"num_bytes_recovered":25694691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1332903936,"data_stored":1332744046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160629,"ondisk_log_size":160629,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051644,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":737856,"kb_used_data":445168,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93633984,"statfs":{"total":96636764160,"available":95881199616,"internally_reserved":0,"allocated":455852032,"data_stored":455571386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":299691692},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724589,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":596428,"kb_used_data":321788,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93775412,"statfs":{"total":96636764160,"available":96026021888,"internally_reserved":0,"allocated":329510912,"data_stored":329182833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":281210539},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247872,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":865596,"kb_used_data":536872,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93506244,"statfs":{"total":96636764160,"available":95750393856,"internally_reserved":0,"allocated":549756928,"data_stored":549476277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":336588456},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,4,1,1,0,0,0,0,0,0,7],"upper_bound":16384},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":548327424,"data_stored":548291294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":329146368,"data_stored":329062810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":455430144,"data_stored":455389942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:30:09.360 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:30:09.895 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:30:09.895 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:30:09.925 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":209,"stamp":"2024-09-17T15:30:09.669812+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":631776009,"num_objects":154254,"num_object_clones":0,"num_object_copies":308508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41143,"num_objects_misplaced":1736,"num_objects_unfound":0,"num_objects_dirty":154254,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":138737,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6582,"num_bytes_recovered":26387417,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160803,"ondisk_log_size":160803,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2159324,"kb_used_data":1292384,"kb_used_omap":13,"kb_used_meta":866866,"kb_avail":280956196,"statfs":{"total":289910292480,"available":287699144704,"internally_reserved":0,"allocated":1323401216,"data_stored":1322519066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":887671807},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,6,5,0,5,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17817600,"num_objects":-4350,"num_object_clones":0,"num_object_copies":-8700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-39,"num_objects_misplaced":-337,"num_objects_unfound":0,"num_objects_dirty":-4350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4350,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":378,"num_bytes_recovered":1376739,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005911"},"pg_stats":[{"pgid":"2.f","version":"43'11081","reported_seq":11193,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:06.958213+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:30:06.958213+0000","last_peered":"2024-09-17T15:30:06.958213+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:30:06.958213+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'1000","ondisk_log_start":"34'1000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39186432,"num_objects":9567,"num_object_clones":0,"num_object_copies":19134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4053,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11081,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5514},{"shards":"0,2","objects":3313}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'11233","reported_seq":11336,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:07.308335+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:30:07.308335+0000","last_peered":"2024-09-17T15:30:07.308335+0000","last_clean":"2024-09-17T15:30:07.308335+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:30:07.308335+0000","last_undegraded":"2024-09-17T15:30:07.308335+0000","last_fullsized":"2024-09-17T15:30:07.308335+0000","mapping_epoch":33,"log_start":"34'1200","ondisk_log_start":"34'1200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39728070,"num_objects":9701,"num_object_clones":0,"num_object_copies":19402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9701,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9590,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'11272","reported_seq":11414,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:07.471165+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:30:07.471165+0000","last_peered":"2024-09-17T15:30:07.471165+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:30:07.471165+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'1200","ondisk_log_start":"34'1200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39571480,"num_objects":9662,"num_object_clones":0,"num_object_copies":19324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9662,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9429,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3365},{"shards":"2","objects":5502}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'11201","reported_seq":11271,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:07.412031+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:30:07.412031+0000","last_peered":"2024-09-17T15:30:07.412031+0000","last_clean":"2024-09-17T15:30:07.412031+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:30:07.412031+0000","last_undegraded":"2024-09-17T15:30:07.412031+0000","last_fullsized":"2024-09-17T15:30:07.412031+0000","mapping_epoch":33,"log_start":"34'1200","ondisk_log_start":"34'1200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40013824,"num_objects":9769,"num_object_clones":0,"num_object_copies":19538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11201,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'11079","reported_seq":11208,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:06.678179+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:30:06.678179+0000","last_peered":"2024-09-17T15:30:06.678179+0000","last_clean":"2024-09-17T15:30:06.678179+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:30:06.678179+0000","last_undegraded":"2024-09-17T15:30:06.678179+0000","last_fullsized":"2024-09-17T15:30:06.678179+0000","mapping_epoch":32,"log_start":"34'1000","ondisk_log_start":"34'1000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39215587,"num_objects":9575,"num_object_clones":0,"num_object_copies":19150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9575,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8061,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'11445","reported_seq":11610,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:06.902332+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:30:06.902332+0000","last_peered":"2024-09-17T15:30:06.902332+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:30:06.902332+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'1400","ondisk_log_start":"34'1400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40165859,"num_objects":9807,"num_object_clones":0,"num_object_copies":19614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4256,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9807,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7944,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5552},{"shards":"0,2","objects":3451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'11167","reported_seq":11346,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:06.956403+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:30:06.956403+0000","last_peered":"2024-09-17T15:30:06.956403+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:30:06.956403+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'1100","ondisk_log_start":"34'1100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39326662,"num_objects":9603,"num_object_clones":0,"num_object_copies":19206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9603,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7936,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5449},{"shards":"0,2","objects":3394}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'11225","reported_seq":11353,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:06.953571+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:30:06.953571+0000","last_peered":"2024-09-17T15:30:06.953571+0000","last_clean":"2024-09-17T15:30:06.953571+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:30:06.953571+0000","last_undegraded":"2024-09-17T15:30:06.953571+0000","last_fullsized":"2024-09-17T15:30:06.953571+0000","mapping_epoch":32,"log_start":"34'1200","ondisk_log_start":"34'1200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39694336,"num_objects":9691,"num_object_clones":0,"num_object_copies":19382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8179,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'11157","reported_seq":19439,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:06.173635+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:30:06.173635+0000","last_peered":"2024-09-17T15:30:06.173635+0000","last_clean":"2024-09-17T15:30:06.173635+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:30:06.173635+0000","last_undegraded":"2024-09-17T15:30:06.173635+0000","last_fullsized":"2024-09-17T15:30:06.173635+0000","mapping_epoch":42,"log_start":"34'1100","ondisk_log_start":"34'1100","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39567843,"num_objects":9661,"num_object_clones":0,"num_object_copies":19322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9661,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9157,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804747+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804747+0000","last_peered":"2024-09-17T15:28:23.804747+0000","last_clean":"2024-09-17T15:28:23.804747+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804747+0000","last_undegraded":"2024-09-17T15:28:23.804747+0000","last_fullsized":"2024-09-17T15:28:23.804747+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'11016","reported_seq":11166,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:06.845290+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:30:06.845290+0000","last_peered":"2024-09-17T15:30:06.845290+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:30:06.845290+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'1000","ondisk_log_start":"34'1000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39018496,"num_objects":9526,"num_object_clones":0,"num_object_copies":19052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4002,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9526,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7612,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5526},{"shards":"0,2","objects":3269}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'11010","reported_seq":11145,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:07.395265+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:30:07.395265+0000","last_peered":"2024-09-17T15:30:07.395265+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:30:07.395265+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'1000","ondisk_log_start":"34'1000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38949347,"num_objects":9510,"num_object_clones":0,"num_object_copies":19020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4071,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9510,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8905,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3329},{"shards":"2","objects":5440}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'11132","reported_seq":11275,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:07.400595+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:30:07.400595+0000","last_peered":"2024-09-17T15:30:07.400595+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:30:07.400595+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'1100","ondisk_log_start":"34'1100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39337984,"num_objects":9604,"num_object_clones":0,"num_object_copies":19208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4127,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8894,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3376},{"shards":"2","objects":5479}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'11269","reported_seq":11430,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:06.958095+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:30:06.958095+0000","last_peered":"2024-09-17T15:30:06.958095+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:30:06.958095+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'1200","ondisk_log_start":"34'1200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39718912,"num_objects":9697,"num_object_clones":0,"num_object_copies":19394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7817,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5572},{"shards":"0,2","objects":3353}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'11171","reported_seq":11337,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:06.964669+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:30:06.964669+0000","last_peered":"2024-09-17T15:30:06.964669+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:30:06.964669+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'1100","ondisk_log_start":"34'1100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39325696,"num_objects":9601,"num_object_clones":0,"num_object_copies":19202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4148,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7458,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5456},{"shards":"0,2","objects":3376}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805107+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805107+0000","last_peered":"2024-09-17T15:28:23.805107+0000","last_clean":"2024-09-17T15:28:23.805107+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805107+0000","last_undegraded":"2024-09-17T15:28:23.805107+0000","last_fullsized":"2024-09-17T15:28:23.805107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'11190","reported_seq":11362,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:06.965269+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:30:06.965269+0000","last_peered":"2024-09-17T15:30:06.965269+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:30:06.965269+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'1100","ondisk_log_start":"34'1100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39714816,"num_objects":9696,"num_object_clones":0,"num_object_copies":19392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4045,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9696,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7977,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5653},{"shards":"0,2","objects":3318}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'11052","reported_seq":15946,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:30:06.956332+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:30:06.956332+0000","last_peered":"2024-09-17T15:30:06.956332+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:30:06.956332+0000","last_undegraded":"2024-09-17T15:30:06.956332+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"34'1000","ondisk_log_start":"34'1000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39240646,"num_objects":9582,"num_object_clones":0,"num_object_copies":19164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1736,"num_objects_unfound":0,"num_objects_dirty":9582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7494,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2375,"num_bytes_recovered":9716195,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7846},{"shards":"0,2","objects":1736}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":631775990,"num_objects":154252,"num_object_clones":0,"num_object_copies":308504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41143,"num_objects_misplaced":1736,"num_objects_unfound":0,"num_objects_dirty":154252,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":138735,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6581,"num_bytes_recovered":26387398,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1321361408,"data_stored":1321208744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160800,"ondisk_log_size":160800,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051645,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":725060,"kb_used_data":440692,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93646780,"statfs":{"total":96636764160,"available":95894302720,"internally_reserved":0,"allocated":451268608,"data_stored":450987962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":291172012},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,1,0,0,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724590,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":583484,"kb_used_data":321320,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93788356,"statfs":{"total":96636764160,"available":96039276544,"internally_reserved":0,"allocated":329031680,"data_stored":328710827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":268431019},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247873,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":850780,"kb_used_data":530372,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93521060,"statfs":{"total":96636764160,"available":95765565440,"internally_reserved":0,"allocated":543100928,"data_stored":542820277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":328068776},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,3,0,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":541814784,"data_stored":541778654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":328654848,"data_stored":328578516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":450891776,"data_stored":450851574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:30:09.927 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:30:10.455 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:30:10.456 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:30:10.476 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":209,"stamp":"2024-09-17T15:30:09.669812+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":631776009,"num_objects":154254,"num_object_clones":0,"num_object_copies":308508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41143,"num_objects_misplaced":1736,"num_objects_unfound":0,"num_objects_dirty":154254,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":138737,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6582,"num_bytes_recovered":26387417,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160803,"ondisk_log_size":160803,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2159324,"kb_used_data":1292384,"kb_used_omap":13,"kb_used_meta":866866,"kb_avail":280956196,"statfs":{"total":289910292480,"available":287699144704,"internally_reserved":0,"allocated":1323401216,"data_stored":1322519066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":887671807},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,6,5,0,5,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17817600,"num_objects":-4350,"num_object_clones":0,"num_object_copies":-8700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-39,"num_objects_misplaced":-337,"num_objects_unfound":0,"num_objects_dirty":-4350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4350,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":378,"num_bytes_recovered":1376739,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005911"},"pg_stats":[{"pgid":"2.f","version":"43'11081","reported_seq":11193,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:06.958213+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:30:06.958213+0000","last_peered":"2024-09-17T15:30:06.958213+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:30:06.958213+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'1000","ondisk_log_start":"34'1000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39186432,"num_objects":9567,"num_object_clones":0,"num_object_copies":19134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4053,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11081,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5514},{"shards":"0,2","objects":3313}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'11233","reported_seq":11336,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:07.308335+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:30:07.308335+0000","last_peered":"2024-09-17T15:30:07.308335+0000","last_clean":"2024-09-17T15:30:07.308335+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:30:07.308335+0000","last_undegraded":"2024-09-17T15:30:07.308335+0000","last_fullsized":"2024-09-17T15:30:07.308335+0000","mapping_epoch":33,"log_start":"34'1200","ondisk_log_start":"34'1200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39728070,"num_objects":9701,"num_object_clones":0,"num_object_copies":19402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9701,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9590,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'11272","reported_seq":11414,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:07.471165+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:30:07.471165+0000","last_peered":"2024-09-17T15:30:07.471165+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:30:07.471165+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'1200","ondisk_log_start":"34'1200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39571480,"num_objects":9662,"num_object_clones":0,"num_object_copies":19324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9662,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9429,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3365},{"shards":"2","objects":5502}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'11201","reported_seq":11271,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:07.412031+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:30:07.412031+0000","last_peered":"2024-09-17T15:30:07.412031+0000","last_clean":"2024-09-17T15:30:07.412031+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:30:07.412031+0000","last_undegraded":"2024-09-17T15:30:07.412031+0000","last_fullsized":"2024-09-17T15:30:07.412031+0000","mapping_epoch":33,"log_start":"34'1200","ondisk_log_start":"34'1200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40013824,"num_objects":9769,"num_object_clones":0,"num_object_copies":19538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11201,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'11079","reported_seq":11208,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:06.678179+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:30:06.678179+0000","last_peered":"2024-09-17T15:30:06.678179+0000","last_clean":"2024-09-17T15:30:06.678179+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:30:06.678179+0000","last_undegraded":"2024-09-17T15:30:06.678179+0000","last_fullsized":"2024-09-17T15:30:06.678179+0000","mapping_epoch":32,"log_start":"34'1000","ondisk_log_start":"34'1000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39215587,"num_objects":9575,"num_object_clones":0,"num_object_copies":19150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9575,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8061,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'11445","reported_seq":11610,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:06.902332+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:30:06.902332+0000","last_peered":"2024-09-17T15:30:06.902332+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:30:06.902332+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'1400","ondisk_log_start":"34'1400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40165859,"num_objects":9807,"num_object_clones":0,"num_object_copies":19614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4256,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9807,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7944,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5552},{"shards":"0,2","objects":3451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'11167","reported_seq":11346,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:06.956403+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:30:06.956403+0000","last_peered":"2024-09-17T15:30:06.956403+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:30:06.956403+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'1100","ondisk_log_start":"34'1100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39326662,"num_objects":9603,"num_object_clones":0,"num_object_copies":19206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9603,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7936,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5449},{"shards":"0,2","objects":3394}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'11225","reported_seq":11353,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:06.953571+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:30:06.953571+0000","last_peered":"2024-09-17T15:30:06.953571+0000","last_clean":"2024-09-17T15:30:06.953571+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:30:06.953571+0000","last_undegraded":"2024-09-17T15:30:06.953571+0000","last_fullsized":"2024-09-17T15:30:06.953571+0000","mapping_epoch":32,"log_start":"34'1200","ondisk_log_start":"34'1200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39694336,"num_objects":9691,"num_object_clones":0,"num_object_copies":19382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8179,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'11157","reported_seq":19439,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:06.173635+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:30:06.173635+0000","last_peered":"2024-09-17T15:30:06.173635+0000","last_clean":"2024-09-17T15:30:06.173635+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:30:06.173635+0000","last_undegraded":"2024-09-17T15:30:06.173635+0000","last_fullsized":"2024-09-17T15:30:06.173635+0000","mapping_epoch":42,"log_start":"34'1100","ondisk_log_start":"34'1100","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39567843,"num_objects":9661,"num_object_clones":0,"num_object_copies":19322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9661,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9157,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804747+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804747+0000","last_peered":"2024-09-17T15:28:23.804747+0000","last_clean":"2024-09-17T15:28:23.804747+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804747+0000","last_undegraded":"2024-09-17T15:28:23.804747+0000","last_fullsized":"2024-09-17T15:28:23.804747+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'11016","reported_seq":11166,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:06.845290+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:30:06.845290+0000","last_peered":"2024-09-17T15:30:06.845290+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:30:06.845290+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'1000","ondisk_log_start":"34'1000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39018496,"num_objects":9526,"num_object_clones":0,"num_object_copies":19052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4002,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9526,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7612,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5526},{"shards":"0,2","objects":3269}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'11010","reported_seq":11145,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:07.395265+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:30:07.395265+0000","last_peered":"2024-09-17T15:30:07.395265+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:30:07.395265+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'1000","ondisk_log_start":"34'1000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38949347,"num_objects":9510,"num_object_clones":0,"num_object_copies":19020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4071,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9510,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8905,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3329},{"shards":"2","objects":5440}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'11132","reported_seq":11275,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:07.400595+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:30:07.400595+0000","last_peered":"2024-09-17T15:30:07.400595+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:30:07.400595+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'1100","ondisk_log_start":"34'1100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39337984,"num_objects":9604,"num_object_clones":0,"num_object_copies":19208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4127,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8894,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3376},{"shards":"2","objects":5479}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'11269","reported_seq":11430,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:06.958095+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:30:06.958095+0000","last_peered":"2024-09-17T15:30:06.958095+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:30:06.958095+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'1200","ondisk_log_start":"34'1200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39718912,"num_objects":9697,"num_object_clones":0,"num_object_copies":19394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7817,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5572},{"shards":"0,2","objects":3353}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'11171","reported_seq":11337,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:06.964669+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:30:06.964669+0000","last_peered":"2024-09-17T15:30:06.964669+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:30:06.964669+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'1100","ondisk_log_start":"34'1100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39325696,"num_objects":9601,"num_object_clones":0,"num_object_copies":19202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4148,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7458,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5456},{"shards":"0,2","objects":3376}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805107+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805107+0000","last_peered":"2024-09-17T15:28:23.805107+0000","last_clean":"2024-09-17T15:28:23.805107+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805107+0000","last_undegraded":"2024-09-17T15:28:23.805107+0000","last_fullsized":"2024-09-17T15:28:23.805107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'11190","reported_seq":11362,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:06.965269+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:30:06.965269+0000","last_peered":"2024-09-17T15:30:06.965269+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:30:06.965269+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'1100","ondisk_log_start":"34'1100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39714816,"num_objects":9696,"num_object_clones":0,"num_object_copies":19392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4045,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9696,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7977,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5653},{"shards":"0,2","objects":3318}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'11052","reported_seq":15946,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:30:06.956332+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:30:06.956332+0000","last_peered":"2024-09-17T15:30:06.956332+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:30:06.956332+0000","last_undegraded":"2024-09-17T15:30:06.956332+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"34'1000","ondisk_log_start":"34'1000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39240646,"num_objects":9582,"num_object_clones":0,"num_object_copies":19164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1736,"num_objects_unfound":0,"num_objects_dirty":9582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7494,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2375,"num_bytes_recovered":9716195,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7846},{"shards":"0,2","objects":1736}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":631775990,"num_objects":154252,"num_object_clones":0,"num_object_copies":308504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41143,"num_objects_misplaced":1736,"num_objects_unfound":0,"num_objects_dirty":154252,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":138735,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6581,"num_bytes_recovered":26387398,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1321361408,"data_stored":1321208744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160800,"ondisk_log_size":160800,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051645,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":725060,"kb_used_data":440692,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93646780,"statfs":{"total":96636764160,"available":95894302720,"internally_reserved":0,"allocated":451268608,"data_stored":450987962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":291172012},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,1,0,0,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724590,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":583484,"kb_used_data":321320,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93788356,"statfs":{"total":96636764160,"available":96039276544,"internally_reserved":0,"allocated":329031680,"data_stored":328710827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":268431019},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247873,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":850780,"kb_used_data":530372,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93521060,"statfs":{"total":96636764160,"available":95765565440,"internally_reserved":0,"allocated":543100928,"data_stored":542820277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":328068776},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,3,0,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":541814784,"data_stored":541778654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":328654848,"data_stored":328578516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":450891776,"data_stored":450851574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:30:10.478 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:30:10.478 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'11081", 'reported_seq': 11193, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:06.958213+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:30:06.958213+0000', 'last_peered': '2024-09-17T15:30:06.958213+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:30:06.958213+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "34'1000", 'ondisk_log_start': "34'1000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39186432, 'num_objects': 9567, 'num_object_clones': 0, 'num_object_copies': 19134, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4053, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9567, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11081, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 17, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5514}, {'shards': '0,2', 'objects': 3313}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:10.478 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:30:10.478 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'11272", 'reported_seq': 11414, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:07.471165+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:30:07.471165+0000', 'last_peered': '2024-09-17T15:30:07.471165+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:30:07.471165+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "34'1200", 'ondisk_log_start': "34'1200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39571480, 'num_objects': 9662, 'num_object_clones': 0, 'num_object_copies': 19324, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9662, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9429, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 8, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3365}, {'shards': '2', 'objects': 5502}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:30:10.479 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:30:10.479 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'11445", 'reported_seq': 11610, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:06.902332+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:30:06.902332+0000', 'last_peered': '2024-09-17T15:30:06.902332+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:30:06.902332+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "34'1400", 'ondisk_log_start': "34'1400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40165859, 'num_objects': 9807, 'num_object_clones': 0, 'num_object_copies': 19614, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4256, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9807, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7944, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 15, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5552}, {'shards': '0,2', 'objects': 3451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:10.479 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:30:10.479 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'11167", 'reported_seq': 11346, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:06.956403+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:30:06.956403+0000', 'last_peered': '2024-09-17T15:30:06.956403+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:30:06.956403+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "34'1100", 'ondisk_log_start': "34'1100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39326662, 'num_objects': 9603, 'num_object_clones': 0, 'num_object_copies': 19206, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4154, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9603, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 7936, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 20, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5449}, {'shards': '0,2', 'objects': 3394}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:10.480 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:30:10.480 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'11016", 'reported_seq': 11166, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:06.845290+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:30:06.845290+0000', 'last_peered': '2024-09-17T15:30:06.845290+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:30:06.845290+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "34'1000", 'ondisk_log_start': "34'1000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39018496, 'num_objects': 9526, 'num_object_clones': 0, 'num_object_copies': 19052, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4002, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9526, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7612, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 10, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5526}, {'shards': '0,2', 'objects': 3269}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:10.480 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:30:10.480 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'11010", 'reported_seq': 11145, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:07.395265+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:30:07.395265+0000', 'last_peered': '2024-09-17T15:30:07.395265+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:30:07.395265+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "34'1000", 'ondisk_log_start': "34'1000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38949347, 'num_objects': 9510, 'num_object_clones': 0, 'num_object_copies': 19020, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4071, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9510, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8905, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 7, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3329}, {'shards': '2', 'objects': 5440}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:30:10.481 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:30:10.481 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'11132", 'reported_seq': 11275, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:07.400595+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:30:07.400595+0000', 'last_peered': '2024-09-17T15:30:07.400595+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:30:07.400595+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "34'1100", 'ondisk_log_start': "34'1100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39337984, 'num_objects': 9604, 'num_object_clones': 0, 'num_object_copies': 19208, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4127, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9604, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8894, 'num_write_kb': 32532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 12, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3376}, {'shards': '2', 'objects': 5479}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:30:10.481 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:30:10.481 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'11269", 'reported_seq': 11430, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:06.958095+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:30:06.958095+0000', 'last_peered': '2024-09-17T15:30:06.958095+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:30:06.958095+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "34'1200", 'ondisk_log_start': "34'1200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39718912, 'num_objects': 9697, 'num_object_clones': 0, 'num_object_copies': 19394, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9697, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7817, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 13, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5572}, {'shards': '0,2', 'objects': 3353}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:10.482 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:30:10.482 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'11171", 'reported_seq': 11337, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:06.964669+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:30:06.964669+0000', 'last_peered': '2024-09-17T15:30:06.964669+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:30:06.964669+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "34'1100", 'ondisk_log_start': "34'1100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39325696, 'num_objects': 9601, 'num_object_clones': 0, 'num_object_copies': 19202, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4148, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9601, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7458, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 12, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5456}, {'shards': '0,2', 'objects': 3376}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:10.482 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:30:10.482 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'11190", 'reported_seq': 11362, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:06.965269+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:30:06.965269+0000', 'last_peered': '2024-09-17T15:30:06.965269+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:30:06.965269+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "34'1100", 'ondisk_log_start': "34'1100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39714816, 'num_objects': 9696, 'num_object_clones': 0, 'num_object_copies': 19392, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4045, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9696, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7977, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 20, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5653}, {'shards': '0,2', 'objects': 3318}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:10.482 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:30:10.483 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'11052", 'reported_seq': 15946, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:30:06.956332+0000', 'last_change': '2024-09-17T15:28:22.509818+0000', 'last_active': '2024-09-17T15:30:06.956332+0000', 'last_peered': '2024-09-17T15:30:06.956332+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:30:06.956332+0000', 'last_undegraded': '2024-09-17T15:30:06.956332+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "34'1000", 'ondisk_log_start': "34'1000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39240646, 'num_objects': 9582, 'num_object_clones': 0, 'num_object_copies': 19164, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1736, 'num_objects_unfound': 0, 'num_objects_dirty': 9582, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7494, 'num_write_kb': 27056, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2375, 'num_bytes_recovered': 9716195, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7846}, {'shards': '0,2', 'objects': 1736}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:10.483 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:30:11.193 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:30:11.193 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41143/308508 objects degraded (13.336%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":348,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":154254,"data_bytes":631776009,"bytes_used":2211147776,"bytes_avail":287699144704,"bytes_total":289910292480,"degraded_objects":41143,"degraded_total":308508,"degraded_ratio":0.13336120943379101,"misplaced_objects":1736,"misplaced_total":308508,"misplaced_ratio":0.0056270826040167518,"recovering_objects_per_sec":31,"recovering_bytes_per_sec":114671,"recovering_keys_per_sec":0,"num_objects_recovered":378,"num_bytes_recovered":1376739,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":362},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T15:27:19.616376+0000","services":{}},"progress_events":{}} 2024-09-17T15:30:11.221 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:30:11.221 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:30:11.799 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:30:11.800 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:30:11.824 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":210,"stamp":"2024-09-17T15:30:11.670435+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":631177993,"num_objects":154108,"num_object_clones":0,"num_object_copies":308216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41143,"num_objects_misplaced":1736,"num_objects_unfound":0,"num_objects_dirty":154108,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":138883,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6582,"num_bytes_recovered":26387417,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160749,"ondisk_log_size":160749,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2167264,"kb_used_data":1292008,"kb_used_omap":13,"kb_used_meta":875186,"kb_avail":280948256,"statfs":{"total":289910292480,"available":287691014144,"internally_reserved":0,"allocated":1323016192,"data_stored":1322134042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":896191487},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,9,6,0,5,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18415616,"num_objects":-4496,"num_object_clones":0,"num_object_copies":-8992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-39,"num_objects_misplaced":-337,"num_objects_unfound":0,"num_objects_dirty":-4496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4496,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":378,"num_bytes_recovered":1376739,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003521"},"pg_stats":[{"pgid":"2.f","version":"43'11081","reported_seq":11193,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:06.958213+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:30:06.958213+0000","last_peered":"2024-09-17T15:30:06.958213+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:30:06.958213+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'1000","ondisk_log_start":"34'1000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39186432,"num_objects":9567,"num_object_clones":0,"num_object_copies":19134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4053,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11081,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5514},{"shards":"0,2","objects":3313}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'11233","reported_seq":11336,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:07.308335+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:30:07.308335+0000","last_peered":"2024-09-17T15:30:07.308335+0000","last_clean":"2024-09-17T15:30:07.308335+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:30:07.308335+0000","last_undegraded":"2024-09-17T15:30:07.308335+0000","last_fullsized":"2024-09-17T15:30:07.308335+0000","mapping_epoch":33,"log_start":"34'1200","ondisk_log_start":"34'1200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39728070,"num_objects":9701,"num_object_clones":0,"num_object_copies":19402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9701,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9590,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'11272","reported_seq":11414,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:07.471165+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:30:07.471165+0000","last_peered":"2024-09-17T15:30:07.471165+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:30:07.471165+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'1200","ondisk_log_start":"34'1200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39571480,"num_objects":9662,"num_object_clones":0,"num_object_copies":19324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9662,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9429,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3365},{"shards":"2","objects":5502}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'11201","reported_seq":11271,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:07.412031+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:30:07.412031+0000","last_peered":"2024-09-17T15:30:07.412031+0000","last_clean":"2024-09-17T15:30:07.412031+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:30:07.412031+0000","last_undegraded":"2024-09-17T15:30:07.412031+0000","last_fullsized":"2024-09-17T15:30:07.412031+0000","mapping_epoch":33,"log_start":"34'1200","ondisk_log_start":"34'1200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40013824,"num_objects":9769,"num_object_clones":0,"num_object_copies":19538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11201,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'11079","reported_seq":11208,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:06.678179+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:30:06.678179+0000","last_peered":"2024-09-17T15:30:06.678179+0000","last_clean":"2024-09-17T15:30:06.678179+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:30:06.678179+0000","last_undegraded":"2024-09-17T15:30:06.678179+0000","last_fullsized":"2024-09-17T15:30:06.678179+0000","mapping_epoch":32,"log_start":"34'1000","ondisk_log_start":"34'1000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39215587,"num_objects":9575,"num_object_clones":0,"num_object_copies":19150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9575,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8061,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'11445","reported_seq":11610,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:06.902332+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:30:06.902332+0000","last_peered":"2024-09-17T15:30:06.902332+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:30:06.902332+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'1400","ondisk_log_start":"34'1400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40165859,"num_objects":9807,"num_object_clones":0,"num_object_copies":19614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4256,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9807,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7944,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5552},{"shards":"0,2","objects":3451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'11167","reported_seq":11346,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:06.956403+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:30:06.956403+0000","last_peered":"2024-09-17T15:30:06.956403+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:30:06.956403+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'1100","ondisk_log_start":"34'1100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39326662,"num_objects":9603,"num_object_clones":0,"num_object_copies":19206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9603,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7936,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5449},{"shards":"0,2","objects":3394}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'11225","reported_seq":11353,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:06.953571+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:30:06.953571+0000","last_peered":"2024-09-17T15:30:06.953571+0000","last_clean":"2024-09-17T15:30:06.953571+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:30:06.953571+0000","last_undegraded":"2024-09-17T15:30:06.953571+0000","last_fullsized":"2024-09-17T15:30:06.953571+0000","mapping_epoch":32,"log_start":"34'1200","ondisk_log_start":"34'1200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39694336,"num_objects":9691,"num_object_clones":0,"num_object_copies":19382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8179,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'11303","reported_seq":19585,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:11.229816+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:30:11.229816+0000","last_peered":"2024-09-17T15:30:11.229816+0000","last_clean":"2024-09-17T15:30:11.229816+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:30:11.229816+0000","last_undegraded":"2024-09-17T15:30:11.229816+0000","last_fullsized":"2024-09-17T15:30:11.229816+0000","mapping_epoch":42,"log_start":"34'1300","ondisk_log_start":"34'1300","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38969827,"num_objects":9515,"num_object_clones":0,"num_object_copies":19030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9515,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9303,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804747+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804747+0000","last_peered":"2024-09-17T15:28:23.804747+0000","last_clean":"2024-09-17T15:28:23.804747+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804747+0000","last_undegraded":"2024-09-17T15:28:23.804747+0000","last_fullsized":"2024-09-17T15:28:23.804747+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'11016","reported_seq":11166,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:06.845290+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:30:06.845290+0000","last_peered":"2024-09-17T15:30:06.845290+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:30:06.845290+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'1000","ondisk_log_start":"34'1000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39018496,"num_objects":9526,"num_object_clones":0,"num_object_copies":19052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4002,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9526,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7612,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5526},{"shards":"0,2","objects":3269}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'11010","reported_seq":11145,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:07.395265+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:30:07.395265+0000","last_peered":"2024-09-17T15:30:07.395265+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:30:07.395265+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'1000","ondisk_log_start":"34'1000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38949347,"num_objects":9510,"num_object_clones":0,"num_object_copies":19020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4071,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9510,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8905,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3329},{"shards":"2","objects":5440}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'11132","reported_seq":11275,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:07.400595+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:30:07.400595+0000","last_peered":"2024-09-17T15:30:07.400595+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:30:07.400595+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'1100","ondisk_log_start":"34'1100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39337984,"num_objects":9604,"num_object_clones":0,"num_object_copies":19208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4127,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8894,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3376},{"shards":"2","objects":5479}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'11269","reported_seq":11430,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:06.958095+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:30:06.958095+0000","last_peered":"2024-09-17T15:30:06.958095+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:30:06.958095+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'1200","ondisk_log_start":"34'1200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39718912,"num_objects":9697,"num_object_clones":0,"num_object_copies":19394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7817,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5572},{"shards":"0,2","objects":3353}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'11171","reported_seq":11337,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:06.964669+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:30:06.964669+0000","last_peered":"2024-09-17T15:30:06.964669+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:30:06.964669+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'1100","ondisk_log_start":"34'1100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39325696,"num_objects":9601,"num_object_clones":0,"num_object_copies":19202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4148,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7458,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5456},{"shards":"0,2","objects":3376}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805107+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805107+0000","last_peered":"2024-09-17T15:28:23.805107+0000","last_clean":"2024-09-17T15:28:23.805107+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805107+0000","last_undegraded":"2024-09-17T15:28:23.805107+0000","last_fullsized":"2024-09-17T15:28:23.805107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'11190","reported_seq":11362,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:06.965269+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:30:06.965269+0000","last_peered":"2024-09-17T15:30:06.965269+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:30:06.965269+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'1100","ondisk_log_start":"34'1100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39714816,"num_objects":9696,"num_object_clones":0,"num_object_copies":19392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4045,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9696,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7977,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5653},{"shards":"0,2","objects":3318}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'11052","reported_seq":15946,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:30:06.956332+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:30:06.956332+0000","last_peered":"2024-09-17T15:30:06.956332+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:30:06.956332+0000","last_undegraded":"2024-09-17T15:30:06.956332+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"34'1000","ondisk_log_start":"34'1000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39240646,"num_objects":9582,"num_object_clones":0,"num_object_copies":19164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1736,"num_objects_unfound":0,"num_objects_dirty":9582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7494,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2375,"num_bytes_recovered":9716195,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7846},{"shards":"0,2","objects":1736}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":631177974,"num_objects":154106,"num_object_clones":0,"num_object_copies":308212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41143,"num_objects_misplaced":1736,"num_objects_unfound":0,"num_objects_dirty":154106,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":138881,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6581,"num_bytes_recovered":26387398,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1320984576,"data_stored":1320831912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160746,"ondisk_log_size":160746,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051645,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":725060,"kb_used_data":440692,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93646780,"statfs":{"total":96636764160,"available":95894302720,"internally_reserved":0,"allocated":451268608,"data_stored":450987962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":291172012},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,1,0,0,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724591,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591424,"kb_used_data":320944,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93780416,"statfs":{"total":96636764160,"available":96031145984,"internally_reserved":0,"allocated":328646656,"data_stored":328325803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":276950699},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247873,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":850780,"kb_used_data":530372,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93521060,"statfs":{"total":96636764160,"available":95765565440,"internally_reserved":0,"allocated":543100928,"data_stored":542820277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":328068776},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,3,0,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":541814784,"data_stored":541778654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":328278016,"data_stored":328201684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":450891776,"data_stored":450851574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:30:14.827 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:30:15.381 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:30:15.381 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:30:15.406 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":211,"stamp":"2024-09-17T15:30:13.671192+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":622686985,"num_objects":152035,"num_object_clones":0,"num_object_copies":304070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41126,"num_objects_misplaced":1569,"num_objects_unfound":0,"num_objects_dirty":152035,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":140956,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6766,"num_bytes_recovered":27071449,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160622,"ondisk_log_size":160622,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2161380,"kb_used_data":1281960,"kb_used_omap":13,"kb_used_meta":879346,"kb_avail":280954140,"statfs":{"total":289910292480,"available":287697039360,"internally_reserved":0,"allocated":1312727040,"data_stored":1311844890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":900451327},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,1,11,3,0,0,0,0,0,8],"upper_bound":4096},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26353664,"num_objects":-6434,"num_object_clones":0,"num_object_copies":-12868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-56,"num_objects_misplaced":-504,"num_objects_unfound":0,"num_objects_dirty":-6434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6434,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":562,"num_bytes_recovered":2060771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003525"},"pg_stats":[{"pgid":"2.f","version":"43'11216","reported_seq":11338,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:11.983415+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:30:11.983415+0000","last_peered":"2024-09-17T15:30:11.983415+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:30:11.983415+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'1200","ondisk_log_start":"34'1200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38633472,"num_objects":9432,"num_object_clones":0,"num_object_copies":18864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11216,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5384},{"shards":"0,2","objects":3178}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'11365","reported_seq":11468,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:12.484432+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:30:12.484432+0000","last_peered":"2024-09-17T15:30:12.484432+0000","last_clean":"2024-09-17T15:30:12.484432+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:30:12.484432+0000","last_undegraded":"2024-09-17T15:30:12.484432+0000","last_fullsized":"2024-09-17T15:30:12.484432+0000","mapping_epoch":33,"log_start":"34'1300","ondisk_log_start":"34'1300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39187398,"num_objects":9569,"num_object_clones":0,"num_object_copies":19138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9569,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9722,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'11436","reported_seq":11578,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:12.397075+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:30:12.397075+0000","last_peered":"2024-09-17T15:30:12.397075+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:30:12.397075+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'1400","ondisk_log_start":"34'1400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38899736,"num_objects":9498,"num_object_clones":0,"num_object_copies":18996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9498,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9593,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3201},{"shards":"2","objects":5339}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'11334","reported_seq":11404,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:12.443148+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:30:12.443148+0000","last_peered":"2024-09-17T15:30:12.443148+0000","last_clean":"2024-09-17T15:30:12.443148+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:30:12.443148+0000","last_undegraded":"2024-09-17T15:30:12.443148+0000","last_fullsized":"2024-09-17T15:30:12.443148+0000","mapping_epoch":33,"log_start":"34'1300","ondisk_log_start":"34'1300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39469056,"num_objects":9636,"num_object_clones":0,"num_object_copies":19272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11334,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'11207","reported_seq":11336,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:12.000555+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:30:12.000555+0000","last_peered":"2024-09-17T15:30:12.000555+0000","last_clean":"2024-09-17T15:30:12.000555+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:30:12.000555+0000","last_undegraded":"2024-09-17T15:30:12.000555+0000","last_fullsized":"2024-09-17T15:30:12.000555+0000","mapping_epoch":32,"log_start":"34'1200","ondisk_log_start":"34'1200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38691299,"num_objects":9447,"num_object_clones":0,"num_object_copies":18894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9447,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8189,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'11583","reported_seq":11748,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:11.949223+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:30:11.949223+0000","last_peered":"2024-09-17T15:30:11.949223+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:30:11.949223+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'1500","ondisk_log_start":"34'1500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39600611,"num_objects":9669,"num_object_clones":0,"num_object_copies":19338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4256,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9669,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8082,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5414},{"shards":"0,2","objects":3313}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'11304","reported_seq":11485,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:11.990310+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:30:11.990310+0000","last_peered":"2024-09-17T15:30:11.990310+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:30:11.990310+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'1300","ondisk_log_start":"34'1300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38765510,"num_objects":9466,"num_object_clones":0,"num_object_copies":18932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4153,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9466,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8073,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5313},{"shards":"0,2","objects":3257}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'11383","reported_seq":11511,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:11.936518+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:30:11.936518+0000","last_peered":"2024-09-17T15:30:11.936518+0000","last_clean":"2024-09-17T15:30:11.936518+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:30:11.936518+0000","last_undegraded":"2024-09-17T15:30:11.936518+0000","last_fullsized":"2024-09-17T15:30:11.936518+0000","mapping_epoch":32,"log_start":"34'1300","ondisk_log_start":"34'1300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39047168,"num_objects":9533,"num_object_clones":0,"num_object_copies":19066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8337,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'11303","reported_seq":19585,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:11.229816+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:30:11.229816+0000","last_peered":"2024-09-17T15:30:11.229816+0000","last_clean":"2024-09-17T15:30:11.229816+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:30:11.229816+0000","last_undegraded":"2024-09-17T15:30:11.229816+0000","last_fullsized":"2024-09-17T15:30:11.229816+0000","mapping_epoch":42,"log_start":"34'1300","ondisk_log_start":"34'1300","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38969827,"num_objects":9515,"num_object_clones":0,"num_object_copies":19030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9515,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9303,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804747+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804747+0000","last_peered":"2024-09-17T15:28:23.804747+0000","last_clean":"2024-09-17T15:28:23.804747+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804747+0000","last_undegraded":"2024-09-17T15:28:23.804747+0000","last_fullsized":"2024-09-17T15:28:23.804747+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'11152","reported_seq":11304,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:12.023226+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:30:12.023226+0000","last_peered":"2024-09-17T15:30:12.023226+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:30:12.023226+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'1100","ondisk_log_start":"34'1100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38461440,"num_objects":9390,"num_object_clones":0,"num_object_copies":18780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4000,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7748,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5392},{"shards":"0,2","objects":3133}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'11147","reported_seq":11282,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:12.421277+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:30:12.421277+0000","last_peered":"2024-09-17T15:30:12.421277+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:30:12.421277+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'1100","ondisk_log_start":"34'1100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38388195,"num_objects":9373,"num_object_clones":0,"num_object_copies":18746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4071,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9373,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9042,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3192},{"shards":"2","objects":5303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'11268","reported_seq":11415,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:12.410663+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:30:12.410663+0000","last_peered":"2024-09-17T15:30:12.410663+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:30:12.410663+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'1200","ondisk_log_start":"34'1200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38780928,"num_objects":9468,"num_object_clones":0,"num_object_copies":18936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4125,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9468,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9030,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3240},{"shards":"2","objects":5345}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'11401","reported_seq":11570,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:12.008796+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:30:12.008796+0000","last_peered":"2024-09-17T15:30:12.008796+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:30:12.008796+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'1400","ondisk_log_start":"34'1400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39178240,"num_objects":9565,"num_object_clones":0,"num_object_copies":19130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4122,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7949,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5444},{"shards":"0,2","objects":3221}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'11311","reported_seq":11477,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:11.975741+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:30:11.975741+0000","last_peered":"2024-09-17T15:30:11.975741+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:30:11.975741+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'1300","ondisk_log_start":"34'1300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38752256,"num_objects":9461,"num_object_clones":0,"num_object_copies":18922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4148,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9461,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7598,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5316},{"shards":"0,2","objects":3236}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805107+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805107+0000","last_peered":"2024-09-17T15:28:23.805107+0000","last_clean":"2024-09-17T15:28:23.805107+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805107+0000","last_undegraded":"2024-09-17T15:28:23.805107+0000","last_fullsized":"2024-09-17T15:28:23.805107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'11323","reported_seq":11497,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:11.900498+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:30:11.900498+0000","last_peered":"2024-09-17T15:30:11.900498+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:30:11.900498+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'1300","ondisk_log_start":"34'1300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39170048,"num_objects":9563,"num_object_clones":0,"num_object_copies":19126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4043,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8110,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5522},{"shards":"0,2","objects":3185}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'11186","reported_seq":16413,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:30:11.972529+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:30:11.972529+0000","last_peered":"2024-09-17T15:30:11.972529+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:30:11.972529+0000","last_undegraded":"2024-09-17T15:30:11.972529+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"34'1100","ondisk_log_start":"34'1100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38691782,"num_objects":9448,"num_object_clones":0,"num_object_copies":18896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1569,"num_objects_unfound":0,"num_objects_dirty":9448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7628,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2542,"num_bytes_recovered":10400227,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7879},{"shards":"0,2","objects":1569}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":622686966,"num_objects":152033,"num_object_clones":0,"num_object_copies":304066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41126,"num_objects_misplaced":1569,"num_objects_unfound":0,"num_objects_dirty":152033,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":140954,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6765,"num_bytes_recovered":27071430,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1309732864,"data_stored":1309580200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160619,"ondisk_log_size":160619,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051646,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":721408,"kb_used_data":437036,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93650432,"statfs":{"total":96636764160,"available":95898042368,"internally_reserved":0,"allocated":447524864,"data_stored":447244218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":291172012},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,0,0,0,0,0,0,3],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724591,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591424,"kb_used_data":320944,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93780416,"statfs":{"total":96636764160,"available":96031145984,"internally_reserved":0,"allocated":328646656,"data_stored":328325803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":276950699},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247874,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":848548,"kb_used_data":523980,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93523292,"statfs":{"total":96636764160,"available":95767851008,"internally_reserved":0,"allocated":536555520,"data_stored":536274869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":332328616},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,0,4,1,0,0,0,0,0,5],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":535150592,"data_stored":535114462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":328278016,"data_stored":328201684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":446304256,"data_stored":446264054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:30:15.407 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:30:15.984 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:30:15.987 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:30:16.011 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2024-09-17T15:30:15.671922+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":622686985,"num_objects":152035,"num_object_clones":0,"num_object_copies":304070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41126,"num_objects_misplaced":1569,"num_objects_unfound":0,"num_objects_dirty":152035,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":140956,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6766,"num_bytes_recovered":27071449,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160622,"ondisk_log_size":160622,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2161380,"kb_used_data":1281960,"kb_used_omap":13,"kb_used_meta":879346,"kb_avail":280954140,"statfs":{"total":289910292480,"available":287697039360,"internally_reserved":0,"allocated":1312727040,"data_stored":1311844890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":900451327},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,1,11,3,0,0,0,0,0,8],"upper_bound":4096},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17981440,"num_objects":-4390,"num_object_clones":0,"num_object_copies":-8780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-54,"num_objects_misplaced":-337,"num_objects_unfound":0,"num_objects_dirty":-4390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4390,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":393,"num_bytes_recovered":1376739,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003575"},"pg_stats":[{"pgid":"2.f","version":"43'11216","reported_seq":11338,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:11.983415+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:30:11.983415+0000","last_peered":"2024-09-17T15:30:11.983415+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:30:11.983415+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'1200","ondisk_log_start":"34'1200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38633472,"num_objects":9432,"num_object_clones":0,"num_object_copies":18864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11216,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5384},{"shards":"0,2","objects":3178}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'11365","reported_seq":11468,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:12.484432+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:30:12.484432+0000","last_peered":"2024-09-17T15:30:12.484432+0000","last_clean":"2024-09-17T15:30:12.484432+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:30:12.484432+0000","last_undegraded":"2024-09-17T15:30:12.484432+0000","last_fullsized":"2024-09-17T15:30:12.484432+0000","mapping_epoch":33,"log_start":"34'1300","ondisk_log_start":"34'1300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39187398,"num_objects":9569,"num_object_clones":0,"num_object_copies":19138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9569,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9722,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'11436","reported_seq":11578,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:12.397075+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:30:12.397075+0000","last_peered":"2024-09-17T15:30:12.397075+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:30:12.397075+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'1400","ondisk_log_start":"34'1400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38899736,"num_objects":9498,"num_object_clones":0,"num_object_copies":18996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9498,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9593,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3201},{"shards":"2","objects":5339}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'11334","reported_seq":11404,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:12.443148+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:30:12.443148+0000","last_peered":"2024-09-17T15:30:12.443148+0000","last_clean":"2024-09-17T15:30:12.443148+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:30:12.443148+0000","last_undegraded":"2024-09-17T15:30:12.443148+0000","last_fullsized":"2024-09-17T15:30:12.443148+0000","mapping_epoch":33,"log_start":"34'1300","ondisk_log_start":"34'1300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39469056,"num_objects":9636,"num_object_clones":0,"num_object_copies":19272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11334,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'11207","reported_seq":11336,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:12.000555+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:30:12.000555+0000","last_peered":"2024-09-17T15:30:12.000555+0000","last_clean":"2024-09-17T15:30:12.000555+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:30:12.000555+0000","last_undegraded":"2024-09-17T15:30:12.000555+0000","last_fullsized":"2024-09-17T15:30:12.000555+0000","mapping_epoch":32,"log_start":"34'1200","ondisk_log_start":"34'1200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38691299,"num_objects":9447,"num_object_clones":0,"num_object_copies":18894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9447,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8189,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'11583","reported_seq":11748,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:11.949223+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:30:11.949223+0000","last_peered":"2024-09-17T15:30:11.949223+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:30:11.949223+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'1500","ondisk_log_start":"34'1500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39600611,"num_objects":9669,"num_object_clones":0,"num_object_copies":19338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4256,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9669,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8082,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5414},{"shards":"0,2","objects":3313}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'11304","reported_seq":11485,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:11.990310+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:30:11.990310+0000","last_peered":"2024-09-17T15:30:11.990310+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:30:11.990310+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'1300","ondisk_log_start":"34'1300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38765510,"num_objects":9466,"num_object_clones":0,"num_object_copies":18932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4153,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9466,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8073,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5313},{"shards":"0,2","objects":3257}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'11383","reported_seq":11511,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:11.936518+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:30:11.936518+0000","last_peered":"2024-09-17T15:30:11.936518+0000","last_clean":"2024-09-17T15:30:11.936518+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:30:11.936518+0000","last_undegraded":"2024-09-17T15:30:11.936518+0000","last_fullsized":"2024-09-17T15:30:11.936518+0000","mapping_epoch":32,"log_start":"34'1300","ondisk_log_start":"34'1300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39047168,"num_objects":9533,"num_object_clones":0,"num_object_copies":19066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8337,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'11303","reported_seq":19585,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:11.229816+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:30:11.229816+0000","last_peered":"2024-09-17T15:30:11.229816+0000","last_clean":"2024-09-17T15:30:11.229816+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:30:11.229816+0000","last_undegraded":"2024-09-17T15:30:11.229816+0000","last_fullsized":"2024-09-17T15:30:11.229816+0000","mapping_epoch":42,"log_start":"34'1300","ondisk_log_start":"34'1300","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38969827,"num_objects":9515,"num_object_clones":0,"num_object_copies":19030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9515,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9303,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804747+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804747+0000","last_peered":"2024-09-17T15:28:23.804747+0000","last_clean":"2024-09-17T15:28:23.804747+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804747+0000","last_undegraded":"2024-09-17T15:28:23.804747+0000","last_fullsized":"2024-09-17T15:28:23.804747+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'11152","reported_seq":11304,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:12.023226+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:30:12.023226+0000","last_peered":"2024-09-17T15:30:12.023226+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:30:12.023226+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'1100","ondisk_log_start":"34'1100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38461440,"num_objects":9390,"num_object_clones":0,"num_object_copies":18780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4000,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7748,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5392},{"shards":"0,2","objects":3133}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'11147","reported_seq":11282,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:12.421277+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:30:12.421277+0000","last_peered":"2024-09-17T15:30:12.421277+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:30:12.421277+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'1100","ondisk_log_start":"34'1100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38388195,"num_objects":9373,"num_object_clones":0,"num_object_copies":18746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4071,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9373,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9042,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3192},{"shards":"2","objects":5303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'11268","reported_seq":11415,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:12.410663+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:30:12.410663+0000","last_peered":"2024-09-17T15:30:12.410663+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:30:12.410663+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'1200","ondisk_log_start":"34'1200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38780928,"num_objects":9468,"num_object_clones":0,"num_object_copies":18936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4125,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9468,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9030,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3240},{"shards":"2","objects":5345}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'11401","reported_seq":11570,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:12.008796+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:30:12.008796+0000","last_peered":"2024-09-17T15:30:12.008796+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:30:12.008796+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'1400","ondisk_log_start":"34'1400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39178240,"num_objects":9565,"num_object_clones":0,"num_object_copies":19130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4122,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7949,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5444},{"shards":"0,2","objects":3221}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'11311","reported_seq":11477,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:11.975741+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:30:11.975741+0000","last_peered":"2024-09-17T15:30:11.975741+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:30:11.975741+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'1300","ondisk_log_start":"34'1300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38752256,"num_objects":9461,"num_object_clones":0,"num_object_copies":18922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4148,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9461,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7598,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5316},{"shards":"0,2","objects":3236}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805107+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805107+0000","last_peered":"2024-09-17T15:28:23.805107+0000","last_clean":"2024-09-17T15:28:23.805107+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805107+0000","last_undegraded":"2024-09-17T15:28:23.805107+0000","last_fullsized":"2024-09-17T15:28:23.805107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'11323","reported_seq":11497,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:11.900498+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:30:11.900498+0000","last_peered":"2024-09-17T15:30:11.900498+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:30:11.900498+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'1300","ondisk_log_start":"34'1300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39170048,"num_objects":9563,"num_object_clones":0,"num_object_copies":19126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4043,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8110,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5522},{"shards":"0,2","objects":3185}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'11186","reported_seq":16413,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:30:11.972529+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:30:11.972529+0000","last_peered":"2024-09-17T15:30:11.972529+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:30:11.972529+0000","last_undegraded":"2024-09-17T15:30:11.972529+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"34'1100","ondisk_log_start":"34'1100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38691782,"num_objects":9448,"num_object_clones":0,"num_object_copies":18896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1569,"num_objects_unfound":0,"num_objects_dirty":9448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7628,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2542,"num_bytes_recovered":10400227,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7879},{"shards":"0,2","objects":1569}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":622686966,"num_objects":152033,"num_object_clones":0,"num_object_copies":304066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41126,"num_objects_misplaced":1569,"num_objects_unfound":0,"num_objects_dirty":152033,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":140954,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6765,"num_bytes_recovered":27071430,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1309732864,"data_stored":1309580200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160619,"ondisk_log_size":160619,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051646,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":721408,"kb_used_data":437036,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93650432,"statfs":{"total":96636764160,"available":95898042368,"internally_reserved":0,"allocated":447524864,"data_stored":447244218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":291172012},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,0,0,0,0,0,0,3],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724591,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591424,"kb_used_data":320944,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93780416,"statfs":{"total":96636764160,"available":96031145984,"internally_reserved":0,"allocated":328646656,"data_stored":328325803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":276950699},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247874,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":848548,"kb_used_data":523980,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93523292,"statfs":{"total":96636764160,"available":95767851008,"internally_reserved":0,"allocated":536555520,"data_stored":536274869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":332328616},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,0,4,1,0,0,0,0,0,5],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":535150592,"data_stored":535114462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":328278016,"data_stored":328201684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":446304256,"data_stored":446264054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:30:16.012 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:30:16.012 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'11216", 'reported_seq': 11338, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:11.983415+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:30:11.983415+0000', 'last_peered': '2024-09-17T15:30:11.983415+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:30:11.983415+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "34'1200", 'ondisk_log_start': "34'1200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38633472, 'num_objects': 9432, 'num_object_clones': 0, 'num_object_copies': 18864, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4048, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9432, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11216, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 22, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5384}, {'shards': '0,2', 'objects': 3178}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:16.012 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:30:16.012 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'11436", 'reported_seq': 11578, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:12.397075+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:30:12.397075+0000', 'last_peered': '2024-09-17T15:30:12.397075+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:30:12.397075+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "34'1400", 'ondisk_log_start': "34'1400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38899736, 'num_objects': 9498, 'num_object_clones': 0, 'num_object_copies': 18996, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4160, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9498, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9593, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 9, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3201}, {'shards': '2', 'objects': 5339}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:30:16.013 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:30:16.013 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'11583", 'reported_seq': 11748, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:11.949223+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:30:11.949223+0000', 'last_peered': '2024-09-17T15:30:11.949223+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:30:11.949223+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "34'1500", 'ondisk_log_start': "34'1500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39600611, 'num_objects': 9669, 'num_object_clones': 0, 'num_object_copies': 19338, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4256, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9669, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8082, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 15, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5414}, {'shards': '0,2', 'objects': 3313}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:16.013 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:30:16.013 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'11304", 'reported_seq': 11485, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:11.990310+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:30:11.990310+0000', 'last_peered': '2024-09-17T15:30:11.990310+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:30:11.990310+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "34'1300", 'ondisk_log_start': "34'1300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38765510, 'num_objects': 9466, 'num_object_clones': 0, 'num_object_copies': 18932, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4153, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9466, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 8073, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 21, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5313}, {'shards': '0,2', 'objects': 3257}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:16.013 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:30:16.013 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'11152", 'reported_seq': 11304, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:12.023226+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:30:12.023226+0000', 'last_peered': '2024-09-17T15:30:12.023226+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:30:12.023226+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "34'1100", 'ondisk_log_start': "34'1100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38461440, 'num_objects': 9390, 'num_object_clones': 0, 'num_object_copies': 18780, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4000, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9390, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7748, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 12, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5392}, {'shards': '0,2', 'objects': 3133}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:16.013 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:30:16.013 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'11147", 'reported_seq': 11282, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:12.421277+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:30:12.421277+0000', 'last_peered': '2024-09-17T15:30:12.421277+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:30:12.421277+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "34'1100", 'ondisk_log_start': "34'1100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38388195, 'num_objects': 9373, 'num_object_clones': 0, 'num_object_copies': 18746, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4071, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9373, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9042, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 7, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3192}, {'shards': '2', 'objects': 5303}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:30:16.013 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:30:16.013 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'11268", 'reported_seq': 11415, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:12.410663+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:30:12.410663+0000', 'last_peered': '2024-09-17T15:30:12.410663+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:30:12.410663+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "34'1200", 'ondisk_log_start': "34'1200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38780928, 'num_objects': 9468, 'num_object_clones': 0, 'num_object_copies': 18936, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4125, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9468, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9030, 'num_write_kb': 32532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 14, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3240}, {'shards': '2', 'objects': 5345}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:30:16.014 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:30:16.014 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'11401", 'reported_seq': 11570, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:12.008796+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:30:12.008796+0000', 'last_peered': '2024-09-17T15:30:12.008796+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:30:12.008796+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "34'1400", 'ondisk_log_start': "34'1400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39178240, 'num_objects': 9565, 'num_object_clones': 0, 'num_object_copies': 19130, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4122, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9565, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7949, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 17, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5444}, {'shards': '0,2', 'objects': 3221}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:16.014 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:30:16.014 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'11311", 'reported_seq': 11477, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:11.975741+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:30:11.975741+0000', 'last_peered': '2024-09-17T15:30:11.975741+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:30:11.975741+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "34'1300", 'ondisk_log_start': "34'1300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38752256, 'num_objects': 9461, 'num_object_clones': 0, 'num_object_copies': 18922, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4148, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9461, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7598, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 12, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5316}, {'shards': '0,2', 'objects': 3236}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:16.014 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:30:16.014 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'11323", 'reported_seq': 11497, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:11.900498+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:30:11.900498+0000', 'last_peered': '2024-09-17T15:30:11.900498+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:30:11.900498+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "34'1300", 'ondisk_log_start': "34'1300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39170048, 'num_objects': 9563, 'num_object_clones': 0, 'num_object_copies': 19126, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4043, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9563, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8110, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 22, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5522}, {'shards': '0,2', 'objects': 3185}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:16.014 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:30:16.014 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'11186", 'reported_seq': 16413, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:30:11.972529+0000', 'last_change': '2024-09-17T15:28:22.509818+0000', 'last_active': '2024-09-17T15:30:11.972529+0000', 'last_peered': '2024-09-17T15:30:11.972529+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:30:11.972529+0000', 'last_undegraded': '2024-09-17T15:30:11.972529+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "34'1100", 'ondisk_log_start': "34'1100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38691782, 'num_objects': 9448, 'num_object_clones': 0, 'num_object_copies': 18896, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1569, 'num_objects_unfound': 0, 'num_objects_dirty': 9448, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7628, 'num_write_kb': 27056, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2542, 'num_bytes_recovered': 10400227, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7879}, {'shards': '0,2', 'objects': 1569}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:16.015 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:30:16.719 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:30:16.719 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41126/304070 objects degraded (13.525%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":353,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":152035,"data_bytes":622686985,"bytes_used":2213253120,"bytes_avail":287697039360,"bytes_total":289910292480,"degraded_objects":41126,"degraded_total":304070,"degraded_ratio":0.13525175124149044,"misplaced_objects":1569,"misplaced_total":304070,"misplaced_ratio":0.0051599960535403033,"recovering_objects_per_sec":32,"recovering_bytes_per_sec":114694,"recovering_keys_per_sec":0,"num_objects_recovered":393,"num_bytes_recovered":1376739,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":365},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T15:27:19.616376+0000","services":{}},"progress_events":{}} 2024-09-17T15:30:16.738 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:30:16.738 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:30:17.285 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:30:17.286 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:30:17.323 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2024-09-17T15:30:15.671922+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":622686985,"num_objects":152035,"num_object_clones":0,"num_object_copies":304070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41126,"num_objects_misplaced":1569,"num_objects_unfound":0,"num_objects_dirty":152035,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":140956,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6766,"num_bytes_recovered":27071449,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160622,"ondisk_log_size":160622,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2161380,"kb_used_data":1281960,"kb_used_omap":13,"kb_used_meta":879346,"kb_avail":280954140,"statfs":{"total":289910292480,"available":287697039360,"internally_reserved":0,"allocated":1312727040,"data_stored":1311844890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":900451327},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,1,11,3,0,0,0,0,0,8],"upper_bound":4096},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17981440,"num_objects":-4390,"num_object_clones":0,"num_object_copies":-8780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-54,"num_objects_misplaced":-337,"num_objects_unfound":0,"num_objects_dirty":-4390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4390,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":393,"num_bytes_recovered":1376739,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003575"},"pg_stats":[{"pgid":"2.f","version":"43'11216","reported_seq":11338,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:11.983415+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:30:11.983415+0000","last_peered":"2024-09-17T15:30:11.983415+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:30:11.983415+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'1200","ondisk_log_start":"34'1200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38633472,"num_objects":9432,"num_object_clones":0,"num_object_copies":18864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11216,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5384},{"shards":"0,2","objects":3178}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'11365","reported_seq":11468,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:12.484432+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:30:12.484432+0000","last_peered":"2024-09-17T15:30:12.484432+0000","last_clean":"2024-09-17T15:30:12.484432+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:30:12.484432+0000","last_undegraded":"2024-09-17T15:30:12.484432+0000","last_fullsized":"2024-09-17T15:30:12.484432+0000","mapping_epoch":33,"log_start":"34'1300","ondisk_log_start":"34'1300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39187398,"num_objects":9569,"num_object_clones":0,"num_object_copies":19138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9569,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9722,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'11436","reported_seq":11578,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:12.397075+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:30:12.397075+0000","last_peered":"2024-09-17T15:30:12.397075+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:30:12.397075+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'1400","ondisk_log_start":"34'1400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38899736,"num_objects":9498,"num_object_clones":0,"num_object_copies":18996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9498,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9593,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3201},{"shards":"2","objects":5339}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'11334","reported_seq":11404,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:12.443148+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:30:12.443148+0000","last_peered":"2024-09-17T15:30:12.443148+0000","last_clean":"2024-09-17T15:30:12.443148+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:30:12.443148+0000","last_undegraded":"2024-09-17T15:30:12.443148+0000","last_fullsized":"2024-09-17T15:30:12.443148+0000","mapping_epoch":33,"log_start":"34'1300","ondisk_log_start":"34'1300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39469056,"num_objects":9636,"num_object_clones":0,"num_object_copies":19272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11334,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'11207","reported_seq":11336,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:12.000555+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:30:12.000555+0000","last_peered":"2024-09-17T15:30:12.000555+0000","last_clean":"2024-09-17T15:30:12.000555+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:30:12.000555+0000","last_undegraded":"2024-09-17T15:30:12.000555+0000","last_fullsized":"2024-09-17T15:30:12.000555+0000","mapping_epoch":32,"log_start":"34'1200","ondisk_log_start":"34'1200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38691299,"num_objects":9447,"num_object_clones":0,"num_object_copies":18894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9447,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8189,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'11583","reported_seq":11748,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:11.949223+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:30:11.949223+0000","last_peered":"2024-09-17T15:30:11.949223+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:30:11.949223+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'1500","ondisk_log_start":"34'1500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39600611,"num_objects":9669,"num_object_clones":0,"num_object_copies":19338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4256,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9669,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8082,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5414},{"shards":"0,2","objects":3313}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'11304","reported_seq":11485,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:11.990310+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:30:11.990310+0000","last_peered":"2024-09-17T15:30:11.990310+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:30:11.990310+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'1300","ondisk_log_start":"34'1300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38765510,"num_objects":9466,"num_object_clones":0,"num_object_copies":18932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4153,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9466,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8073,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5313},{"shards":"0,2","objects":3257}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'11383","reported_seq":11511,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:11.936518+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:30:11.936518+0000","last_peered":"2024-09-17T15:30:11.936518+0000","last_clean":"2024-09-17T15:30:11.936518+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:30:11.936518+0000","last_undegraded":"2024-09-17T15:30:11.936518+0000","last_fullsized":"2024-09-17T15:30:11.936518+0000","mapping_epoch":32,"log_start":"34'1300","ondisk_log_start":"34'1300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39047168,"num_objects":9533,"num_object_clones":0,"num_object_copies":19066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8337,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'11303","reported_seq":19585,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:11.229816+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:30:11.229816+0000","last_peered":"2024-09-17T15:30:11.229816+0000","last_clean":"2024-09-17T15:30:11.229816+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:30:11.229816+0000","last_undegraded":"2024-09-17T15:30:11.229816+0000","last_fullsized":"2024-09-17T15:30:11.229816+0000","mapping_epoch":42,"log_start":"34'1300","ondisk_log_start":"34'1300","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38969827,"num_objects":9515,"num_object_clones":0,"num_object_copies":19030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9515,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9303,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804747+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804747+0000","last_peered":"2024-09-17T15:28:23.804747+0000","last_clean":"2024-09-17T15:28:23.804747+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804747+0000","last_undegraded":"2024-09-17T15:28:23.804747+0000","last_fullsized":"2024-09-17T15:28:23.804747+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'11152","reported_seq":11304,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:12.023226+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:30:12.023226+0000","last_peered":"2024-09-17T15:30:12.023226+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:30:12.023226+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'1100","ondisk_log_start":"34'1100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38461440,"num_objects":9390,"num_object_clones":0,"num_object_copies":18780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4000,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7748,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5392},{"shards":"0,2","objects":3133}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'11147","reported_seq":11282,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:12.421277+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:30:12.421277+0000","last_peered":"2024-09-17T15:30:12.421277+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:30:12.421277+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'1100","ondisk_log_start":"34'1100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38388195,"num_objects":9373,"num_object_clones":0,"num_object_copies":18746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4071,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9373,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9042,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3192},{"shards":"2","objects":5303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'11268","reported_seq":11415,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:12.410663+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:30:12.410663+0000","last_peered":"2024-09-17T15:30:12.410663+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:30:12.410663+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'1200","ondisk_log_start":"34'1200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38780928,"num_objects":9468,"num_object_clones":0,"num_object_copies":18936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4125,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9468,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9030,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3240},{"shards":"2","objects":5345}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'11401","reported_seq":11570,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:12.008796+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:30:12.008796+0000","last_peered":"2024-09-17T15:30:12.008796+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:30:12.008796+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'1400","ondisk_log_start":"34'1400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39178240,"num_objects":9565,"num_object_clones":0,"num_object_copies":19130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4122,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7949,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5444},{"shards":"0,2","objects":3221}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'11311","reported_seq":11477,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:11.975741+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:30:11.975741+0000","last_peered":"2024-09-17T15:30:11.975741+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:30:11.975741+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'1300","ondisk_log_start":"34'1300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38752256,"num_objects":9461,"num_object_clones":0,"num_object_copies":18922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4148,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9461,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7598,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5316},{"shards":"0,2","objects":3236}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805107+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805107+0000","last_peered":"2024-09-17T15:28:23.805107+0000","last_clean":"2024-09-17T15:28:23.805107+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805107+0000","last_undegraded":"2024-09-17T15:28:23.805107+0000","last_fullsized":"2024-09-17T15:28:23.805107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'11323","reported_seq":11497,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:11.900498+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:30:11.900498+0000","last_peered":"2024-09-17T15:30:11.900498+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:30:11.900498+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'1300","ondisk_log_start":"34'1300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39170048,"num_objects":9563,"num_object_clones":0,"num_object_copies":19126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4043,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8110,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5522},{"shards":"0,2","objects":3185}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'11186","reported_seq":16413,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:30:11.972529+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:30:11.972529+0000","last_peered":"2024-09-17T15:30:11.972529+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:30:11.972529+0000","last_undegraded":"2024-09-17T15:30:11.972529+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"34'1100","ondisk_log_start":"34'1100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38691782,"num_objects":9448,"num_object_clones":0,"num_object_copies":18896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1569,"num_objects_unfound":0,"num_objects_dirty":9448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7628,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2542,"num_bytes_recovered":10400227,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7879},{"shards":"0,2","objects":1569}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":622686966,"num_objects":152033,"num_object_clones":0,"num_object_copies":304066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41126,"num_objects_misplaced":1569,"num_objects_unfound":0,"num_objects_dirty":152033,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":140954,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6765,"num_bytes_recovered":27071430,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1309732864,"data_stored":1309580200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160619,"ondisk_log_size":160619,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051646,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":721408,"kb_used_data":437036,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93650432,"statfs":{"total":96636764160,"available":95898042368,"internally_reserved":0,"allocated":447524864,"data_stored":447244218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":291172012},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,0,0,0,0,0,0,3],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724591,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591424,"kb_used_data":320944,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93780416,"statfs":{"total":96636764160,"available":96031145984,"internally_reserved":0,"allocated":328646656,"data_stored":328325803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":276950699},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247874,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":848548,"kb_used_data":523980,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93523292,"statfs":{"total":96636764160,"available":95767851008,"internally_reserved":0,"allocated":536555520,"data_stored":536274869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":332328616},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,0,4,1,0,0,0,0,0,5],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":535150592,"data_stored":535114462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":328278016,"data_stored":328201684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":446304256,"data_stored":446264054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:30:20.325 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:30:20.859 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:30:20.861 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:30:20.881 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":214,"stamp":"2024-09-17T15:30:19.673484+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":613503753,"num_objects":149793,"num_object_clones":0,"num_object_copies":299586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41123,"num_objects_misplaced":1422,"num_objects_unfound":0,"num_objects_dirty":149793,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":143198,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6916,"num_bytes_recovered":27673561,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160764,"ondisk_log_size":160764,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2180060,"kb_used_data":1270108,"kb_used_omap":13,"kb_used_meta":909874,"kb_avail":280935460,"statfs":{"total":289910292480,"available":287677911040,"internally_reserved":0,"allocated":1300590592,"data_stored":1299708442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":931711999},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,5,9,12,7,6,2,0,0,0,0,1,3],"upper_bound":16384},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18272256,"num_objects":-4461,"num_object_clones":0,"num_object_copies":-8922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-20,"num_objects_misplaced":-314,"num_objects_unfound":0,"num_objects_dirty":-4461,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4461,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":334,"num_bytes_recovered":1286144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004190"},"pg_stats":[{"pgid":"2.f","version":"43'11357","reported_seq":11479,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:16.916797+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:30:16.916797+0000","last_peered":"2024-09-17T15:30:16.916797+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:30:16.916797+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'1300","ondisk_log_start":"34'1300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38055936,"num_objects":9291,"num_object_clones":0,"num_object_copies":18582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9291,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11357,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5243},{"shards":"0,2","objects":3037}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'11503","reported_seq":11606,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:17.409550+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:30:17.409550+0000","last_peered":"2024-09-17T15:30:17.409550+0000","last_clean":"2024-09-17T15:30:17.409550+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:30:17.409550+0000","last_undegraded":"2024-09-17T15:30:17.409550+0000","last_fullsized":"2024-09-17T15:30:17.409550+0000","mapping_epoch":33,"log_start":"34'1500","ondisk_log_start":"34'1500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38622150,"num_objects":9431,"num_object_clones":0,"num_object_copies":18862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9431,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9860,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'11567","reported_seq":11717,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:17.507400+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:30:17.507400+0000","last_peered":"2024-09-17T15:30:17.507400+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:30:17.507400+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'1500","ondisk_log_start":"34'1500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38363160,"num_objects":9367,"num_object_clones":0,"num_object_copies":18734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9367,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9724,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3070},{"shards":"2","objects":5211}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'11469","reported_seq":11539,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:17.456623+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:30:17.456623+0000","last_peered":"2024-09-17T15:30:17.456623+0000","last_clean":"2024-09-17T15:30:17.456623+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:30:17.456623+0000","last_undegraded":"2024-09-17T15:30:17.456623+0000","last_fullsized":"2024-09-17T15:30:17.456623+0000","mapping_epoch":33,"log_start":"34'1400","ondisk_log_start":"34'1400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38916096,"num_objects":9501,"num_object_clones":0,"num_object_copies":19002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11469,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'11357","reported_seq":11486,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:16.933616+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:30:16.933616+0000","last_peered":"2024-09-17T15:30:16.933616+0000","last_clean":"2024-09-17T15:30:16.933616+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:30:16.933616+0000","last_undegraded":"2024-09-17T15:30:16.933616+0000","last_fullsized":"2024-09-17T15:30:16.933616+0000","mapping_epoch":32,"log_start":"34'1300","ondisk_log_start":"34'1300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38076899,"num_objects":9297,"num_object_clones":0,"num_object_copies":18594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9297,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8339,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'11725","reported_seq":11890,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:16.884386+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:30:16.884386+0000","last_peered":"2024-09-17T15:30:16.884386+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:30:16.884386+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'1700","ondisk_log_start":"34'1700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39018979,"num_objects":9527,"num_object_clones":0,"num_object_copies":19054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4256,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9527,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8224,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5272},{"shards":"0,2","objects":3171}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'11437","reported_seq":11618,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:17.038000+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:30:17.038000+0000","last_peered":"2024-09-17T15:30:17.038000+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:30:17.038000+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'1400","ondisk_log_start":"34'1400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38220742,"num_objects":9333,"num_object_clones":0,"num_object_copies":18666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4153,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9333,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8206,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5180},{"shards":"0,2","objects":3124}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'11522","reported_seq":11650,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:17.018400+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:30:17.018400+0000","last_peered":"2024-09-17T15:30:17.018400+0000","last_clean":"2024-09-17T15:30:17.018400+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:30:17.018400+0000","last_undegraded":"2024-09-17T15:30:17.018400+0000","last_fullsized":"2024-09-17T15:30:17.018400+0000","mapping_epoch":32,"log_start":"34'1500","ondisk_log_start":"34'1500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38477824,"num_objects":9394,"num_object_clones":0,"num_object_copies":18788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9394,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8476,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'11438","reported_seq":19720,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:16.257719+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:30:16.257719+0000","last_peered":"2024-09-17T15:30:16.257719+0000","last_clean":"2024-09-17T15:30:16.257719+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:30:16.257719+0000","last_undegraded":"2024-09-17T15:30:16.257719+0000","last_fullsized":"2024-09-17T15:30:16.257719+0000","mapping_epoch":42,"log_start":"34'1400","ondisk_log_start":"34'1400","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38416867,"num_objects":9380,"num_object_clones":0,"num_object_copies":18760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9380,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9438,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804747+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804747+0000","last_peered":"2024-09-17T15:28:23.804747+0000","last_clean":"2024-09-17T15:28:23.804747+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804747+0000","last_undegraded":"2024-09-17T15:28:23.804747+0000","last_fullsized":"2024-09-17T15:28:23.804747+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'11267","reported_seq":11421,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:16.996089+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:30:16.996089+0000","last_peered":"2024-09-17T15:30:16.996089+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:30:16.996089+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'1200","ondisk_log_start":"34'1200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37990400,"num_objects":9275,"num_object_clones":0,"num_object_copies":18550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4000,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9275,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7863,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5277},{"shards":"0,2","objects":3018}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'11299","reported_seq":11434,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:17.479231+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:30:17.479231+0000","last_peered":"2024-09-17T15:30:17.479231+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:30:17.479231+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'1200","ondisk_log_start":"34'1200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37765603,"num_objects":9221,"num_object_clones":0,"num_object_copies":18442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4071,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9221,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9194,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3040},{"shards":"2","objects":5151}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'11421","reported_seq":11568,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:17.480252+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:30:17.480252+0000","last_peered":"2024-09-17T15:30:17.480252+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:30:17.480252+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'1400","ondisk_log_start":"34'1400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38154240,"num_objects":9315,"num_object_clones":0,"num_object_copies":18630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4125,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9315,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9183,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3087},{"shards":"2","objects":5192}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'11527","reported_seq":11696,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:16.982650+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:30:16.982650+0000","last_peered":"2024-09-17T15:30:16.982650+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:30:16.982650+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'1500","ondisk_log_start":"34'1500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38662144,"num_objects":9439,"num_object_clones":0,"num_object_copies":18878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4122,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9439,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8075,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5318},{"shards":"0,2","objects":3095}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'11465","reported_seq":11631,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:17.020331+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:30:17.020331+0000","last_peered":"2024-09-17T15:30:17.020331+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:30:17.020331+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'1400","ondisk_log_start":"34'1400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38121472,"num_objects":9307,"num_object_clones":0,"num_object_copies":18614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4148,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7752,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5162},{"shards":"0,2","objects":3082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805107+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805107+0000","last_peered":"2024-09-17T15:28:23.805107+0000","last_clean":"2024-09-17T15:28:23.805107+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805107+0000","last_undegraded":"2024-09-17T15:28:23.805107+0000","last_fullsized":"2024-09-17T15:28:23.805107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'11469","reported_seq":11645,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:17.045656+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:30:17.045656+0000","last_peered":"2024-09-17T15:30:17.045656+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:30:17.045656+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'1400","ondisk_log_start":"34'1400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38572032,"num_objects":9417,"num_object_clones":0,"num_object_copies":18834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4043,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9417,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8256,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5376},{"shards":"0,2","objects":3039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'11338","reported_seq":16860,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:30:17.042735+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:30:17.042735+0000","last_peered":"2024-09-17T15:30:17.042735+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:30:17.042735+0000","last_undegraded":"2024-09-17T15:30:17.042735+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"34'1300","ondisk_log_start":"34'1300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38069190,"num_objects":9296,"num_object_clones":0,"num_object_copies":18592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1422,"num_objects_unfound":0,"num_objects_dirty":9296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7780,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2689,"num_bytes_recovered":11002339,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7874},{"shards":"0,2","objects":1422}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":613503734,"num_objects":149791,"num_object_clones":0,"num_object_copies":299582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41123,"num_objects_misplaced":1422,"num_objects_unfound":0,"num_objects_dirty":149791,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":143196,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6915,"num_bytes_recovered":27673542,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1297698816,"data_stored":1297546152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160761,"ondisk_log_size":160761,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051647,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":739000,"kb_used_data":432416,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93632840,"statfs":{"total":96636764160,"available":95880028160,"internally_reserved":0,"allocated":442793984,"data_stored":442513338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":313913004},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,8,1,3,2,2,0,0,0,0,0,3],"upper_bound":16384},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724592,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":595016,"kb_used_data":320372,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93776824,"statfs":{"total":96636764160,"available":96027467776,"internally_reserved":0,"allocated":328060928,"data_stored":327740075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":281210539},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,5,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247875,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":846044,"kb_used_data":517320,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93525796,"statfs":{"total":96636764160,"available":95770415104,"internally_reserved":0,"allocated":529735680,"data_stored":529455029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":336588456},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,6,3,3,0,0,0,0,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":528326656,"data_stored":528290526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":327692288,"data_stored":327615956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":441679872,"data_stored":441639670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:30:20.883 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:30:21.440 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:30:21.441 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:30:21.465 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":214,"stamp":"2024-09-17T15:30:19.673484+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":613503753,"num_objects":149793,"num_object_clones":0,"num_object_copies":299586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41123,"num_objects_misplaced":1422,"num_objects_unfound":0,"num_objects_dirty":149793,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":143198,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6916,"num_bytes_recovered":27673561,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160764,"ondisk_log_size":160764,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2180060,"kb_used_data":1270108,"kb_used_omap":13,"kb_used_meta":909874,"kb_avail":280935460,"statfs":{"total":289910292480,"available":287677911040,"internally_reserved":0,"allocated":1300590592,"data_stored":1299708442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":931711999},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,5,9,12,7,6,2,0,0,0,0,1,3],"upper_bound":16384},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18272256,"num_objects":-4461,"num_object_clones":0,"num_object_copies":-8922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-20,"num_objects_misplaced":-314,"num_objects_unfound":0,"num_objects_dirty":-4461,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4461,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":334,"num_bytes_recovered":1286144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004190"},"pg_stats":[{"pgid":"2.f","version":"43'11357","reported_seq":11479,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:16.916797+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:30:16.916797+0000","last_peered":"2024-09-17T15:30:16.916797+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:30:16.916797+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'1300","ondisk_log_start":"34'1300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38055936,"num_objects":9291,"num_object_clones":0,"num_object_copies":18582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9291,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11357,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5243},{"shards":"0,2","objects":3037}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'11503","reported_seq":11606,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:17.409550+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:30:17.409550+0000","last_peered":"2024-09-17T15:30:17.409550+0000","last_clean":"2024-09-17T15:30:17.409550+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:30:17.409550+0000","last_undegraded":"2024-09-17T15:30:17.409550+0000","last_fullsized":"2024-09-17T15:30:17.409550+0000","mapping_epoch":33,"log_start":"34'1500","ondisk_log_start":"34'1500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38622150,"num_objects":9431,"num_object_clones":0,"num_object_copies":18862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9431,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9860,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'11567","reported_seq":11717,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:17.507400+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:30:17.507400+0000","last_peered":"2024-09-17T15:30:17.507400+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:30:17.507400+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'1500","ondisk_log_start":"34'1500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38363160,"num_objects":9367,"num_object_clones":0,"num_object_copies":18734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9367,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9724,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3070},{"shards":"2","objects":5211}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'11469","reported_seq":11539,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:17.456623+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:30:17.456623+0000","last_peered":"2024-09-17T15:30:17.456623+0000","last_clean":"2024-09-17T15:30:17.456623+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:30:17.456623+0000","last_undegraded":"2024-09-17T15:30:17.456623+0000","last_fullsized":"2024-09-17T15:30:17.456623+0000","mapping_epoch":33,"log_start":"34'1400","ondisk_log_start":"34'1400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38916096,"num_objects":9501,"num_object_clones":0,"num_object_copies":19002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11469,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'11357","reported_seq":11486,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:16.933616+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:30:16.933616+0000","last_peered":"2024-09-17T15:30:16.933616+0000","last_clean":"2024-09-17T15:30:16.933616+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:30:16.933616+0000","last_undegraded":"2024-09-17T15:30:16.933616+0000","last_fullsized":"2024-09-17T15:30:16.933616+0000","mapping_epoch":32,"log_start":"34'1300","ondisk_log_start":"34'1300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38076899,"num_objects":9297,"num_object_clones":0,"num_object_copies":18594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9297,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8339,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'11725","reported_seq":11890,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:16.884386+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:30:16.884386+0000","last_peered":"2024-09-17T15:30:16.884386+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:30:16.884386+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'1700","ondisk_log_start":"34'1700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39018979,"num_objects":9527,"num_object_clones":0,"num_object_copies":19054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4256,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9527,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8224,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5272},{"shards":"0,2","objects":3171}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'11437","reported_seq":11618,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:17.038000+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:30:17.038000+0000","last_peered":"2024-09-17T15:30:17.038000+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:30:17.038000+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'1400","ondisk_log_start":"34'1400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38220742,"num_objects":9333,"num_object_clones":0,"num_object_copies":18666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4153,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9333,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8206,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5180},{"shards":"0,2","objects":3124}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'11522","reported_seq":11650,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:17.018400+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:30:17.018400+0000","last_peered":"2024-09-17T15:30:17.018400+0000","last_clean":"2024-09-17T15:30:17.018400+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:30:17.018400+0000","last_undegraded":"2024-09-17T15:30:17.018400+0000","last_fullsized":"2024-09-17T15:30:17.018400+0000","mapping_epoch":32,"log_start":"34'1500","ondisk_log_start":"34'1500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38477824,"num_objects":9394,"num_object_clones":0,"num_object_copies":18788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9394,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8476,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804326+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804326+0000","last_peered":"2024-09-17T15:28:23.804326+0000","last_clean":"2024-09-17T15:28:23.804326+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804326+0000","last_undegraded":"2024-09-17T15:28:23.804326+0000","last_fullsized":"2024-09-17T15:28:23.804326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'11438","reported_seq":19720,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:16.257719+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:30:16.257719+0000","last_peered":"2024-09-17T15:30:16.257719+0000","last_clean":"2024-09-17T15:30:16.257719+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:30:16.257719+0000","last_undegraded":"2024-09-17T15:30:16.257719+0000","last_fullsized":"2024-09-17T15:30:16.257719+0000","mapping_epoch":42,"log_start":"34'1400","ondisk_log_start":"34'1400","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38416867,"num_objects":9380,"num_object_clones":0,"num_object_copies":18760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9380,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9438,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804747+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804747+0000","last_peered":"2024-09-17T15:28:23.804747+0000","last_clean":"2024-09-17T15:28:23.804747+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804747+0000","last_undegraded":"2024-09-17T15:28:23.804747+0000","last_fullsized":"2024-09-17T15:28:23.804747+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'11267","reported_seq":11421,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:16.996089+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:30:16.996089+0000","last_peered":"2024-09-17T15:30:16.996089+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:30:16.996089+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'1200","ondisk_log_start":"34'1200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37990400,"num_objects":9275,"num_object_clones":0,"num_object_copies":18550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4000,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9275,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7863,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5277},{"shards":"0,2","objects":3018}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'11299","reported_seq":11434,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:17.479231+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:30:17.479231+0000","last_peered":"2024-09-17T15:30:17.479231+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:30:17.479231+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'1200","ondisk_log_start":"34'1200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37765603,"num_objects":9221,"num_object_clones":0,"num_object_copies":18442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4071,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9221,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9194,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3040},{"shards":"2","objects":5151}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'11421","reported_seq":11568,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:17.480252+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:30:17.480252+0000","last_peered":"2024-09-17T15:30:17.480252+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:30:17.480252+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'1400","ondisk_log_start":"34'1400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38154240,"num_objects":9315,"num_object_clones":0,"num_object_copies":18630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4125,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9315,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9183,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3087},{"shards":"2","objects":5192}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'11527","reported_seq":11696,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:16.982650+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:30:16.982650+0000","last_peered":"2024-09-17T15:30:16.982650+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:30:16.982650+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'1500","ondisk_log_start":"34'1500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38662144,"num_objects":9439,"num_object_clones":0,"num_object_copies":18878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4122,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9439,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8075,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5318},{"shards":"0,2","objects":3095}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'11465","reported_seq":11631,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:17.020331+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:30:17.020331+0000","last_peered":"2024-09-17T15:30:17.020331+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:30:17.020331+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'1400","ondisk_log_start":"34'1400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38121472,"num_objects":9307,"num_object_clones":0,"num_object_copies":18614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4148,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7752,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5162},{"shards":"0,2","objects":3082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805107+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805107+0000","last_peered":"2024-09-17T15:28:23.805107+0000","last_clean":"2024-09-17T15:28:23.805107+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805107+0000","last_undegraded":"2024-09-17T15:28:23.805107+0000","last_fullsized":"2024-09-17T15:28:23.805107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'11469","reported_seq":11645,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:17.045656+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:30:17.045656+0000","last_peered":"2024-09-17T15:30:17.045656+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:30:17.045656+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'1400","ondisk_log_start":"34'1400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38572032,"num_objects":9417,"num_object_clones":0,"num_object_copies":18834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4043,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9417,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8256,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5376},{"shards":"0,2","objects":3039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'11338","reported_seq":16860,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:30:17.042735+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:30:17.042735+0000","last_peered":"2024-09-17T15:30:17.042735+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:30:17.042735+0000","last_undegraded":"2024-09-17T15:30:17.042735+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"34'1300","ondisk_log_start":"34'1300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38069190,"num_objects":9296,"num_object_clones":0,"num_object_copies":18592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1422,"num_objects_unfound":0,"num_objects_dirty":9296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7780,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2689,"num_bytes_recovered":11002339,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7874},{"shards":"0,2","objects":1422}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":613503734,"num_objects":149791,"num_object_clones":0,"num_object_copies":299582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41123,"num_objects_misplaced":1422,"num_objects_unfound":0,"num_objects_dirty":149791,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":143196,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6915,"num_bytes_recovered":27673542,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1297698816,"data_stored":1297546152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160761,"ondisk_log_size":160761,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051647,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":739000,"kb_used_data":432416,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93632840,"statfs":{"total":96636764160,"available":95880028160,"internally_reserved":0,"allocated":442793984,"data_stored":442513338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":313913004},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,8,1,3,2,2,0,0,0,0,0,3],"upper_bound":16384},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724592,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":595016,"kb_used_data":320372,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93776824,"statfs":{"total":96636764160,"available":96027467776,"internally_reserved":0,"allocated":328060928,"data_stored":327740075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":281210539},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,5,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247875,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":846044,"kb_used_data":517320,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93525796,"statfs":{"total":96636764160,"available":95770415104,"internally_reserved":0,"allocated":529735680,"data_stored":529455029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":336588456},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,6,3,3,0,0,0,0,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":528326656,"data_stored":528290526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":327692288,"data_stored":327615956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":441679872,"data_stored":441639670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:30:21.466 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:30:21.466 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'11357", 'reported_seq': 11479, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:16.916797+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:30:16.916797+0000', 'last_peered': '2024-09-17T15:30:16.916797+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:30:16.916797+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "34'1300", 'ondisk_log_start': "34'1300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38055936, 'num_objects': 9291, 'num_object_clones': 0, 'num_object_copies': 18582, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4048, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9291, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11357, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 22, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5243}, {'shards': '0,2', 'objects': 3037}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:21.466 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:30:21.466 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'11567", 'reported_seq': 11717, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:17.507400+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:30:17.507400+0000', 'last_peered': '2024-09-17T15:30:17.507400+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:30:17.507400+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "34'1500", 'ondisk_log_start': "34'1500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38363160, 'num_objects': 9367, 'num_object_clones': 0, 'num_object_copies': 18734, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4157, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9367, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9724, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 12, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3070}, {'shards': '2', 'objects': 5211}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:30:21.467 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:30:21.467 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'11725", 'reported_seq': 11890, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:16.884386+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:30:16.884386+0000', 'last_peered': '2024-09-17T15:30:16.884386+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:30:16.884386+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "34'1700", 'ondisk_log_start': "34'1700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39018979, 'num_objects': 9527, 'num_object_clones': 0, 'num_object_copies': 19054, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4256, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9527, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8224, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 15, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5272}, {'shards': '0,2', 'objects': 3171}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:21.467 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:30:21.467 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'11437", 'reported_seq': 11618, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:17.038000+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:30:17.038000+0000', 'last_peered': '2024-09-17T15:30:17.038000+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:30:17.038000+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "34'1400", 'ondisk_log_start': "34'1400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38220742, 'num_objects': 9333, 'num_object_clones': 0, 'num_object_copies': 18666, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4153, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9333, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 8206, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 21, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5180}, {'shards': '0,2', 'objects': 3124}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:21.467 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:30:21.467 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'11267", 'reported_seq': 11421, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:16.996089+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:30:16.996089+0000', 'last_peered': '2024-09-17T15:30:16.996089+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:30:16.996089+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "34'1200", 'ondisk_log_start': "34'1200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37990400, 'num_objects': 9275, 'num_object_clones': 0, 'num_object_copies': 18550, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4000, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9275, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7863, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 12, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5277}, {'shards': '0,2', 'objects': 3018}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:21.467 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:30:21.467 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'11299", 'reported_seq': 11434, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:17.479231+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:30:17.479231+0000', 'last_peered': '2024-09-17T15:30:17.479231+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:30:17.479231+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "34'1200", 'ondisk_log_start': "34'1200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37765603, 'num_objects': 9221, 'num_object_clones': 0, 'num_object_copies': 18442, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4071, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9221, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9194, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 7, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3040}, {'shards': '2', 'objects': 5151}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:30:21.467 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:30:21.467 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'11421", 'reported_seq': 11568, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:17.480252+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:30:17.480252+0000', 'last_peered': '2024-09-17T15:30:17.480252+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:30:17.480252+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "34'1400", 'ondisk_log_start': "34'1400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38154240, 'num_objects': 9315, 'num_object_clones': 0, 'num_object_copies': 18630, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4125, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9315, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9183, 'num_write_kb': 32532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 14, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3087}, {'shards': '2', 'objects': 5192}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:30:21.468 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:30:21.468 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'11527", 'reported_seq': 11696, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:16.982650+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:30:16.982650+0000', 'last_peered': '2024-09-17T15:30:16.982650+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:30:16.982650+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "34'1500", 'ondisk_log_start': "34'1500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38662144, 'num_objects': 9439, 'num_object_clones': 0, 'num_object_copies': 18878, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4122, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9439, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8075, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 17, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5318}, {'shards': '0,2', 'objects': 3095}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:21.468 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:30:21.468 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'11465", 'reported_seq': 11631, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:17.020331+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:30:17.020331+0000', 'last_peered': '2024-09-17T15:30:17.020331+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:30:17.020331+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "34'1400", 'ondisk_log_start': "34'1400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38121472, 'num_objects': 9307, 'num_object_clones': 0, 'num_object_copies': 18614, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4148, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9307, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7752, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 12, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5162}, {'shards': '0,2', 'objects': 3082}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:21.468 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:30:21.468 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'11469", 'reported_seq': 11645, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:17.045656+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:30:17.045656+0000', 'last_peered': '2024-09-17T15:30:17.045656+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:30:17.045656+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "34'1400", 'ondisk_log_start': "34'1400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38572032, 'num_objects': 9417, 'num_object_clones': 0, 'num_object_copies': 18834, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4043, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9417, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8256, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 22, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5376}, {'shards': '0,2', 'objects': 3039}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:21.468 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:30:21.468 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'11338", 'reported_seq': 16860, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:30:17.042735+0000', 'last_change': '2024-09-17T15:28:22.509818+0000', 'last_active': '2024-09-17T15:30:17.042735+0000', 'last_peered': '2024-09-17T15:30:17.042735+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:30:17.042735+0000', 'last_undegraded': '2024-09-17T15:30:17.042735+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "34'1300", 'ondisk_log_start': "34'1300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38069190, 'num_objects': 9296, 'num_object_clones': 0, 'num_object_copies': 18592, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1422, 'num_objects_unfound': 0, 'num_objects_dirty': 9296, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7780, 'num_write_kb': 27056, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2689, 'num_bytes_recovered': 11002339, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7874}, {'shards': '0,2', 'objects': 1422}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:21.469 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:30:22.157 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:30:22.157 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41123/299294 objects degraded (13.740%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":359,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":149647,"data_bytes":612905737,"bytes_used":2223157248,"bytes_avail":287687135232,"bytes_total":289910292480,"degraded_objects":41123,"degraded_total":299294,"degraded_ratio":0.13740001470126365,"misplaced_objects":1422,"misplaced_total":299294,"misplaced_ratio":0.0047511811128856576,"recovering_objects_per_sec":27,"recovering_bytes_per_sec":107140,"recovering_keys_per_sec":0,"num_objects_recovered":334,"num_bytes_recovered":1286144,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":383},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T15:27:19.616376+0000","services":{}},"progress_events":{}} 2024-09-17T15:30:22.184 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:30:22.184 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:30:22.740 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:30:22.742 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:30:22.767 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":215,"stamp":"2024-09-17T15:30:21.674107+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":612905737,"num_objects":149647,"num_object_clones":0,"num_object_copies":299294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41123,"num_objects_misplaced":1422,"num_objects_unfound":0,"num_objects_dirty":149647,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":143344,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6916,"num_bytes_recovered":27673561,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160810,"ondisk_log_size":160810,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2171052,"kb_used_data":1269424,"kb_used_omap":13,"kb_used_meta":901554,"kb_avail":280944468,"statfs":{"total":289910292480,"available":287687135232,"internally_reserved":0,"allocated":1299890176,"data_stored":1299008026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":923192319},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,3,14,8,13,6,2,0,0,0,0,1,3],"upper_bound":16384},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18870272,"num_objects":-4607,"num_object_clones":0,"num_object_copies":-9214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-20,"num_objects_misplaced":-314,"num_objects_unfound":0,"num_objects_dirty":-4607,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4607,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":334,"num_bytes_recovered":1286144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004295"},"pg_stats":[{"pgid":"2.f","version":"43'11357","reported_seq":11479,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:16.916797+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:30:16.916797+0000","last_peered":"2024-09-17T15:30:16.916797+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:30:16.916797+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'1300","ondisk_log_start":"34'1300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38055936,"num_objects":9291,"num_object_clones":0,"num_object_copies":18582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9291,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11357,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5243},{"shards":"0,2","objects":3037}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'11503","reported_seq":11606,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:17.409550+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:30:17.409550+0000","last_peered":"2024-09-17T15:30:17.409550+0000","last_clean":"2024-09-17T15:30:17.409550+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:30:17.409550+0000","last_undegraded":"2024-09-17T15:30:17.409550+0000","last_fullsized":"2024-09-17T15:30:17.409550+0000","mapping_epoch":33,"log_start":"34'1500","ondisk_log_start":"34'1500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38622150,"num_objects":9431,"num_object_clones":0,"num_object_copies":18862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9431,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9860,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'11567","reported_seq":11717,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:17.507400+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:30:17.507400+0000","last_peered":"2024-09-17T15:30:17.507400+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:30:17.507400+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'1500","ondisk_log_start":"34'1500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38363160,"num_objects":9367,"num_object_clones":0,"num_object_copies":18734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9367,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9724,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3070},{"shards":"2","objects":5211}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'11469","reported_seq":11539,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:17.456623+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:30:17.456623+0000","last_peered":"2024-09-17T15:30:17.456623+0000","last_clean":"2024-09-17T15:30:17.456623+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:30:17.456623+0000","last_undegraded":"2024-09-17T15:30:17.456623+0000","last_fullsized":"2024-09-17T15:30:17.456623+0000","mapping_epoch":33,"log_start":"34'1400","ondisk_log_start":"34'1400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38916096,"num_objects":9501,"num_object_clones":0,"num_object_copies":19002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11469,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'11357","reported_seq":11486,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:16.933616+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:30:16.933616+0000","last_peered":"2024-09-17T15:30:16.933616+0000","last_clean":"2024-09-17T15:30:16.933616+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:30:16.933616+0000","last_undegraded":"2024-09-17T15:30:16.933616+0000","last_fullsized":"2024-09-17T15:30:16.933616+0000","mapping_epoch":32,"log_start":"34'1300","ondisk_log_start":"34'1300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38076899,"num_objects":9297,"num_object_clones":0,"num_object_copies":18594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9297,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8339,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'11725","reported_seq":11890,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:16.884386+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:30:16.884386+0000","last_peered":"2024-09-17T15:30:16.884386+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:30:16.884386+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'1700","ondisk_log_start":"34'1700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39018979,"num_objects":9527,"num_object_clones":0,"num_object_copies":19054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4256,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9527,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8224,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5272},{"shards":"0,2","objects":3171}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'11437","reported_seq":11618,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:17.038000+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:30:17.038000+0000","last_peered":"2024-09-17T15:30:17.038000+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:30:17.038000+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'1400","ondisk_log_start":"34'1400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38220742,"num_objects":9333,"num_object_clones":0,"num_object_copies":18666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4153,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9333,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8206,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5180},{"shards":"0,2","objects":3124}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'11522","reported_seq":11650,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:17.018400+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:30:17.018400+0000","last_peered":"2024-09-17T15:30:17.018400+0000","last_clean":"2024-09-17T15:30:17.018400+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:30:17.018400+0000","last_undegraded":"2024-09-17T15:30:17.018400+0000","last_fullsized":"2024-09-17T15:30:17.018400+0000","mapping_epoch":32,"log_start":"34'1500","ondisk_log_start":"34'1500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38477824,"num_objects":9394,"num_object_clones":0,"num_object_copies":18788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9394,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8476,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804377+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804377+0000","last_peered":"2024-09-17T15:28:23.804377+0000","last_clean":"2024-09-17T15:28:23.804377+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804377+0000","last_undegraded":"2024-09-17T15:28:23.804377+0000","last_fullsized":"2024-09-17T15:28:23.804377+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'11584","reported_seq":19866,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:21.301092+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:30:21.301092+0000","last_peered":"2024-09-17T15:30:21.301092+0000","last_clean":"2024-09-17T15:30:21.301092+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:30:21.301092+0000","last_undegraded":"2024-09-17T15:30:21.301092+0000","last_fullsized":"2024-09-17T15:30:21.301092+0000","mapping_epoch":42,"log_start":"34'1500","ondisk_log_start":"34'1500","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37818851,"num_objects":9234,"num_object_clones":0,"num_object_copies":18468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9234,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9584,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804747+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804747+0000","last_peered":"2024-09-17T15:28:23.804747+0000","last_clean":"2024-09-17T15:28:23.804747+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804747+0000","last_undegraded":"2024-09-17T15:28:23.804747+0000","last_fullsized":"2024-09-17T15:28:23.804747+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'11267","reported_seq":11421,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:16.996089+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:30:16.996089+0000","last_peered":"2024-09-17T15:30:16.996089+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:30:16.996089+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'1200","ondisk_log_start":"34'1200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37990400,"num_objects":9275,"num_object_clones":0,"num_object_copies":18550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4000,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9275,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7863,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5277},{"shards":"0,2","objects":3018}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'11299","reported_seq":11434,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:17.479231+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:30:17.479231+0000","last_peered":"2024-09-17T15:30:17.479231+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:30:17.479231+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'1200","ondisk_log_start":"34'1200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37765603,"num_objects":9221,"num_object_clones":0,"num_object_copies":18442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4071,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9221,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9194,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3040},{"shards":"2","objects":5151}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'11421","reported_seq":11568,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:17.480252+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:30:17.480252+0000","last_peered":"2024-09-17T15:30:17.480252+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:30:17.480252+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'1400","ondisk_log_start":"34'1400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38154240,"num_objects":9315,"num_object_clones":0,"num_object_copies":18630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4125,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9315,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9183,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3087},{"shards":"2","objects":5192}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'11527","reported_seq":11696,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:16.982650+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:30:16.982650+0000","last_peered":"2024-09-17T15:30:16.982650+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:30:16.982650+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'1500","ondisk_log_start":"34'1500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38662144,"num_objects":9439,"num_object_clones":0,"num_object_copies":18878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4122,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9439,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8075,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5318},{"shards":"0,2","objects":3095}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'11465","reported_seq":11631,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:17.020331+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:30:17.020331+0000","last_peered":"2024-09-17T15:30:17.020331+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:30:17.020331+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'1400","ondisk_log_start":"34'1400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38121472,"num_objects":9307,"num_object_clones":0,"num_object_copies":18614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4148,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7752,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5162},{"shards":"0,2","objects":3082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805107+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805107+0000","last_peered":"2024-09-17T15:28:23.805107+0000","last_clean":"2024-09-17T15:28:23.805107+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805107+0000","last_undegraded":"2024-09-17T15:28:23.805107+0000","last_fullsized":"2024-09-17T15:28:23.805107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'11469","reported_seq":11645,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:17.045656+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:30:17.045656+0000","last_peered":"2024-09-17T15:30:17.045656+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:30:17.045656+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'1400","ondisk_log_start":"34'1400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38572032,"num_objects":9417,"num_object_clones":0,"num_object_copies":18834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4043,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9417,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8256,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5376},{"shards":"0,2","objects":3039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'11338","reported_seq":16860,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:30:17.042735+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:30:17.042735+0000","last_peered":"2024-09-17T15:30:17.042735+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:30:17.042735+0000","last_undegraded":"2024-09-17T15:30:17.042735+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"34'1300","ondisk_log_start":"34'1300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38069190,"num_objects":9296,"num_object_clones":0,"num_object_copies":18592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1422,"num_objects_unfound":0,"num_objects_dirty":9296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7780,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2689,"num_bytes_recovered":11002339,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7874},{"shards":"0,2","objects":1422}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":612905718,"num_objects":149645,"num_object_clones":0,"num_object_copies":299290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41123,"num_objects_misplaced":1422,"num_objects_unfound":0,"num_objects_dirty":149645,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":143342,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6915,"num_bytes_recovered":27673542,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1296986112,"data_stored":1296833448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160807,"ondisk_log_size":160807,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051647,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":739000,"kb_used_data":432416,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93632840,"statfs":{"total":96636764160,"available":95880028160,"internally_reserved":0,"allocated":442793984,"data_stored":442513338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":313913004},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,8,1,3,2,2,0,0,0,0,0,3],"upper_bound":16384},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724593,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":586008,"kb_used_data":319688,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93785832,"statfs":{"total":96636764160,"available":96036691968,"internally_reserved":0,"allocated":327360512,"data_stored":327039659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":272690859},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,1,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247875,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":846044,"kb_used_data":517320,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93525796,"statfs":{"total":96636764160,"available":95770415104,"internally_reserved":0,"allocated":529735680,"data_stored":529455029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":336588456},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,6,3,3,0,0,0,0,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":528326656,"data_stored":528290526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":326979584,"data_stored":326903252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":441679872,"data_stored":441639670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:30:25.769 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:30:26.320 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:30:26.321 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:30:26.347 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":217,"stamp":"2024-09-17T15:30:25.675411+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":604193545,"num_objects":147520,"num_object_clones":0,"num_object_copies":295040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41108,"num_objects_misplaced":1275,"num_objects_unfound":0,"num_objects_dirty":147520,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":145471,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7078,"num_bytes_recovered":28275673,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160737,"ondisk_log_size":160737,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2191656,"kb_used_data":1258100,"kb_used_omap":13,"kb_used_meta":933490,"kb_avail":280923864,"statfs":{"total":289910292480,"available":287666036736,"internally_reserved":0,"allocated":1288294400,"data_stored":1287412250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":955894783},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,6,6,13,2,0,0,0,0,3,0,1],"upper_bound":16384},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18493440,"num_objects":-4515,"num_object_clones":0,"num_object_copies":-9030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-18,"num_objects_misplaced":-294,"num_objects_unfound":0,"num_objects_dirty":-4515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4515,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":1204224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004219"},"pg_stats":[{"pgid":"2.f","version":"43'11486","reported_seq":11607,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:22.081937+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:30:22.081937+0000","last_peered":"2024-09-17T15:30:22.081937+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:30:22.081937+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'1400","ondisk_log_start":"34'1400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37527552,"num_objects":9162,"num_object_clones":0,"num_object_copies":18324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11486,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5114},{"shards":"0,2","objects":2908}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'11644","reported_seq":11747,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:22.518096+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:30:22.518096+0000","last_peered":"2024-09-17T15:30:22.518096+0000","last_clean":"2024-09-17T15:30:22.518096+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:30:22.518096+0000","last_undegraded":"2024-09-17T15:30:22.518096+0000","last_fullsized":"2024-09-17T15:30:22.518096+0000","mapping_epoch":33,"log_start":"34'1600","ondisk_log_start":"34'1600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38044614,"num_objects":9290,"num_object_clones":0,"num_object_copies":18580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9290,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10001,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'11721","reported_seq":11873,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:22.460320+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:30:22.460320+0000","last_peered":"2024-09-17T15:30:22.460320+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:30:22.460320+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'1700","ondisk_log_start":"34'1700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37732376,"num_objects":9213,"num_object_clones":0,"num_object_copies":18426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9213,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9878,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2916},{"shards":"2","objects":5058}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'11622","reported_seq":11692,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:22.497892+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:30:22.497892+0000","last_peered":"2024-09-17T15:30:22.497892+0000","last_clean":"2024-09-17T15:30:22.497892+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:30:22.497892+0000","last_undegraded":"2024-09-17T15:30:22.497892+0000","last_fullsized":"2024-09-17T15:30:22.497892+0000","mapping_epoch":33,"log_start":"34'1600","ondisk_log_start":"34'1600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38289408,"num_objects":9348,"num_object_clones":0,"num_object_copies":18696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9348,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11622,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'11484","reported_seq":11613,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:22.076462+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:30:22.076462+0000","last_peered":"2024-09-17T15:30:22.076462+0000","last_clean":"2024-09-17T15:30:22.076462+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:30:22.076462+0000","last_undegraded":"2024-09-17T15:30:22.076462+0000","last_fullsized":"2024-09-17T15:30:22.076462+0000","mapping_epoch":32,"log_start":"34'1400","ondisk_log_start":"34'1400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37556707,"num_objects":9170,"num_object_clones":0,"num_object_copies":18340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9170,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8466,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'11858","reported_seq":12025,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:22.061470+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:30:22.061470+0000","last_peered":"2024-09-17T15:30:22.061470+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:30:22.061470+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'1800","ondisk_log_start":"34'1800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38474211,"num_objects":9394,"num_object_clones":0,"num_object_copies":18788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4255,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9394,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8357,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5140},{"shards":"0,2","objects":3038}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'11584","reported_seq":11769,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:21.992393+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:30:21.992393+0000","last_peered":"2024-09-17T15:30:21.992393+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:30:21.992393+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'1500","ondisk_log_start":"34'1500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37618630,"num_objects":9186,"num_object_clones":0,"num_object_copies":18372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4151,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9186,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8353,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5035},{"shards":"0,2","objects":2977}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'11647","reported_seq":11775,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:22.002766+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:30:22.002766+0000","last_peered":"2024-09-17T15:30:22.002766+0000","last_clean":"2024-09-17T15:30:22.002766+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:30:22.002766+0000","last_undegraded":"2024-09-17T15:30:22.002766+0000","last_fullsized":"2024-09-17T15:30:22.002766+0000","mapping_epoch":32,"log_start":"34'1600","ondisk_log_start":"34'1600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37965824,"num_objects":9269,"num_object_clones":0,"num_object_copies":18538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9269,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8601,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804377+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804377+0000","last_peered":"2024-09-17T15:28:23.804377+0000","last_clean":"2024-09-17T15:28:23.804377+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804377+0000","last_undegraded":"2024-09-17T15:28:23.804377+0000","last_fullsized":"2024-09-17T15:28:23.804377+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'11584","reported_seq":19866,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:21.301092+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:30:21.301092+0000","last_peered":"2024-09-17T15:30:21.301092+0000","last_clean":"2024-09-17T15:30:21.301092+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:30:21.301092+0000","last_undegraded":"2024-09-17T15:30:21.301092+0000","last_fullsized":"2024-09-17T15:30:21.301092+0000","mapping_epoch":42,"log_start":"34'1500","ondisk_log_start":"34'1500","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37818851,"num_objects":9234,"num_object_clones":0,"num_object_copies":18468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9234,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9584,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804747+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804747+0000","last_peered":"2024-09-17T15:28:23.804747+0000","last_clean":"2024-09-17T15:28:23.804747+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804747+0000","last_undegraded":"2024-09-17T15:28:23.804747+0000","last_fullsized":"2024-09-17T15:28:23.804747+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'11421","reported_seq":11575,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:22.059926+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:30:22.059926+0000","last_peered":"2024-09-17T15:30:22.059926+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:30:22.059926+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'1400","ondisk_log_start":"34'1400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37359616,"num_objects":9121,"num_object_clones":0,"num_object_copies":18242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4000,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8017,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5123},{"shards":"0,2","objects":2864}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'11446","reported_seq":11595,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:22.426963+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:30:22.426963+0000","last_peered":"2024-09-17T15:30:22.426963+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:30:22.426963+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'1400","ondisk_log_start":"34'1400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37163491,"num_objects":9074,"num_object_clones":0,"num_object_copies":18148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4064,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9074,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9341,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2893},{"shards":"2","objects":5011}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'11547","reported_seq":11695,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:22.514852+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:30:22.514852+0000","last_peered":"2024-09-17T15:30:22.514852+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:30:22.514852+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'1500","ondisk_log_start":"34'1500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37638144,"num_objects":9189,"num_object_clones":0,"num_object_copies":18378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9309,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2961},{"shards":"2","objects":5067}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'11657","reported_seq":11826,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:22.006195+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:30:22.006195+0000","last_peered":"2024-09-17T15:30:22.006195+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:30:22.006195+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'1600","ondisk_log_start":"34'1600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38129664,"num_objects":9309,"num_object_clones":0,"num_object_copies":18618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4122,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8205,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5188},{"shards":"0,2","objects":2965}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'11616","reported_seq":11782,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:22.086584+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:30:22.086584+0000","last_peered":"2024-09-17T15:30:22.086584+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:30:22.086584+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'1600","ondisk_log_start":"34'1600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37502976,"num_objects":9156,"num_object_clones":0,"num_object_copies":18312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4148,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7903,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5011},{"shards":"0,2","objects":2931}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805107+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805107+0000","last_peered":"2024-09-17T15:28:23.805107+0000","last_clean":"2024-09-17T15:28:23.805107+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805107+0000","last_undegraded":"2024-09-17T15:28:23.805107+0000","last_fullsized":"2024-09-17T15:28:23.805107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'11615","reported_seq":11797,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:22.082991+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:30:22.082991+0000","last_peered":"2024-09-17T15:30:22.082991+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:30:22.082991+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'1600","ondisk_log_start":"34'1600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37974016,"num_objects":9271,"num_object_clones":0,"num_object_copies":18542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4040,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8402,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5233},{"shards":"0,2","objects":2893}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'11502","reported_seq":17318,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:30:22.093670+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:30:22.093670+0000","last_peered":"2024-09-17T15:30:22.093670+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:30:22.093670+0000","last_undegraded":"2024-09-17T15:30:22.093670+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"34'1500","ondisk_log_start":"34'1500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37397446,"num_objects":9132,"num_object_clones":0,"num_object_copies":18264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1275,"num_objects_unfound":0,"num_objects_dirty":9132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7944,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2836,"num_bytes_recovered":11604451,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7857},{"shards":"0,2","objects":1275}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":604193526,"num_objects":147518,"num_object_clones":0,"num_object_copies":295036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41108,"num_objects_misplaced":1275,"num_objects_unfound":0,"num_objects_dirty":147518,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":145469,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7077,"num_bytes_recovered":28275654,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1285451776,"data_stored":1285299112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160734,"ondisk_log_size":160734,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051648,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":738664,"kb_used_data":427928,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93633176,"statfs":{"total":96636764160,"available":95880372224,"internally_reserved":0,"allocated":438198272,"data_stored":437917626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":318172844},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,1,2,1,0,0,0,0,3],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724593,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":586008,"kb_used_data":319688,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93785832,"statfs":{"total":96636764160,"available":96036691968,"internally_reserved":0,"allocated":327360512,"data_stored":327039659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":272690859},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,1,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247876,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":866984,"kb_used_data":510484,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93504856,"statfs":{"total":96636764160,"available":95748972544,"internally_reserved":0,"allocated":522735616,"data_stored":522454965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":365031080},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,4,4,0,0,0,0,0,0,0,1],"upper_bound":16384},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":521322496,"data_stored":521286366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":326979584,"data_stored":326903252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":437149696,"data_stored":437109494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:30:26.348 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:30:26.900 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:30:26.900 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:30:26.922 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":217,"stamp":"2024-09-17T15:30:25.675411+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":604193545,"num_objects":147520,"num_object_clones":0,"num_object_copies":295040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41108,"num_objects_misplaced":1275,"num_objects_unfound":0,"num_objects_dirty":147520,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":145471,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7078,"num_bytes_recovered":28275673,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160737,"ondisk_log_size":160737,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2191656,"kb_used_data":1258100,"kb_used_omap":13,"kb_used_meta":933490,"kb_avail":280923864,"statfs":{"total":289910292480,"available":287666036736,"internally_reserved":0,"allocated":1288294400,"data_stored":1287412250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":955894783},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,6,6,13,2,0,0,0,0,3,0,1],"upper_bound":16384},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18493440,"num_objects":-4515,"num_object_clones":0,"num_object_copies":-9030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-18,"num_objects_misplaced":-294,"num_objects_unfound":0,"num_objects_dirty":-4515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4515,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":1204224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004219"},"pg_stats":[{"pgid":"2.f","version":"43'11486","reported_seq":11607,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:22.081937+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:30:22.081937+0000","last_peered":"2024-09-17T15:30:22.081937+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:30:22.081937+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'1400","ondisk_log_start":"34'1400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37527552,"num_objects":9162,"num_object_clones":0,"num_object_copies":18324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11486,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5114},{"shards":"0,2","objects":2908}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'11644","reported_seq":11747,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:22.518096+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:30:22.518096+0000","last_peered":"2024-09-17T15:30:22.518096+0000","last_clean":"2024-09-17T15:30:22.518096+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:30:22.518096+0000","last_undegraded":"2024-09-17T15:30:22.518096+0000","last_fullsized":"2024-09-17T15:30:22.518096+0000","mapping_epoch":33,"log_start":"34'1600","ondisk_log_start":"34'1600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38044614,"num_objects":9290,"num_object_clones":0,"num_object_copies":18580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9290,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10001,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'11721","reported_seq":11873,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:22.460320+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:30:22.460320+0000","last_peered":"2024-09-17T15:30:22.460320+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:30:22.460320+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'1700","ondisk_log_start":"34'1700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37732376,"num_objects":9213,"num_object_clones":0,"num_object_copies":18426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9213,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9878,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2916},{"shards":"2","objects":5058}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'11622","reported_seq":11692,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:22.497892+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:30:22.497892+0000","last_peered":"2024-09-17T15:30:22.497892+0000","last_clean":"2024-09-17T15:30:22.497892+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:30:22.497892+0000","last_undegraded":"2024-09-17T15:30:22.497892+0000","last_fullsized":"2024-09-17T15:30:22.497892+0000","mapping_epoch":33,"log_start":"34'1600","ondisk_log_start":"34'1600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38289408,"num_objects":9348,"num_object_clones":0,"num_object_copies":18696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9348,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11622,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'11484","reported_seq":11613,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:22.076462+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:30:22.076462+0000","last_peered":"2024-09-17T15:30:22.076462+0000","last_clean":"2024-09-17T15:30:22.076462+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:30:22.076462+0000","last_undegraded":"2024-09-17T15:30:22.076462+0000","last_fullsized":"2024-09-17T15:30:22.076462+0000","mapping_epoch":32,"log_start":"34'1400","ondisk_log_start":"34'1400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37556707,"num_objects":9170,"num_object_clones":0,"num_object_copies":18340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9170,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8466,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'11858","reported_seq":12025,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:22.061470+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:30:22.061470+0000","last_peered":"2024-09-17T15:30:22.061470+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:30:22.061470+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'1800","ondisk_log_start":"34'1800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38474211,"num_objects":9394,"num_object_clones":0,"num_object_copies":18788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4255,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9394,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8357,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5140},{"shards":"0,2","objects":3038}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'11584","reported_seq":11769,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:21.992393+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:30:21.992393+0000","last_peered":"2024-09-17T15:30:21.992393+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:30:21.992393+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'1500","ondisk_log_start":"34'1500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37618630,"num_objects":9186,"num_object_clones":0,"num_object_copies":18372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4151,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9186,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8353,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5035},{"shards":"0,2","objects":2977}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'11647","reported_seq":11775,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:22.002766+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:30:22.002766+0000","last_peered":"2024-09-17T15:30:22.002766+0000","last_clean":"2024-09-17T15:30:22.002766+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:30:22.002766+0000","last_undegraded":"2024-09-17T15:30:22.002766+0000","last_fullsized":"2024-09-17T15:30:22.002766+0000","mapping_epoch":32,"log_start":"34'1600","ondisk_log_start":"34'1600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37965824,"num_objects":9269,"num_object_clones":0,"num_object_copies":18538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9269,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8601,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804377+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804377+0000","last_peered":"2024-09-17T15:28:23.804377+0000","last_clean":"2024-09-17T15:28:23.804377+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804377+0000","last_undegraded":"2024-09-17T15:28:23.804377+0000","last_fullsized":"2024-09-17T15:28:23.804377+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'11584","reported_seq":19866,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:21.301092+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:30:21.301092+0000","last_peered":"2024-09-17T15:30:21.301092+0000","last_clean":"2024-09-17T15:30:21.301092+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:30:21.301092+0000","last_undegraded":"2024-09-17T15:30:21.301092+0000","last_fullsized":"2024-09-17T15:30:21.301092+0000","mapping_epoch":42,"log_start":"34'1500","ondisk_log_start":"34'1500","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37818851,"num_objects":9234,"num_object_clones":0,"num_object_copies":18468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9234,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9584,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804747+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804747+0000","last_peered":"2024-09-17T15:28:23.804747+0000","last_clean":"2024-09-17T15:28:23.804747+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804747+0000","last_undegraded":"2024-09-17T15:28:23.804747+0000","last_fullsized":"2024-09-17T15:28:23.804747+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'11421","reported_seq":11575,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:22.059926+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:30:22.059926+0000","last_peered":"2024-09-17T15:30:22.059926+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:30:22.059926+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'1400","ondisk_log_start":"34'1400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37359616,"num_objects":9121,"num_object_clones":0,"num_object_copies":18242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4000,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8017,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5123},{"shards":"0,2","objects":2864}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'11446","reported_seq":11595,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:22.426963+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:30:22.426963+0000","last_peered":"2024-09-17T15:30:22.426963+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:30:22.426963+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'1400","ondisk_log_start":"34'1400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37163491,"num_objects":9074,"num_object_clones":0,"num_object_copies":18148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4064,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9074,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9341,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2893},{"shards":"2","objects":5011}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'11547","reported_seq":11695,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:22.514852+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:30:22.514852+0000","last_peered":"2024-09-17T15:30:22.514852+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:30:22.514852+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'1500","ondisk_log_start":"34'1500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37638144,"num_objects":9189,"num_object_clones":0,"num_object_copies":18378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9309,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2961},{"shards":"2","objects":5067}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'11657","reported_seq":11826,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:22.006195+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:30:22.006195+0000","last_peered":"2024-09-17T15:30:22.006195+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:30:22.006195+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'1600","ondisk_log_start":"34'1600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38129664,"num_objects":9309,"num_object_clones":0,"num_object_copies":18618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4122,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8205,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5188},{"shards":"0,2","objects":2965}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'11616","reported_seq":11782,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:22.086584+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:30:22.086584+0000","last_peered":"2024-09-17T15:30:22.086584+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:30:22.086584+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'1600","ondisk_log_start":"34'1600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37502976,"num_objects":9156,"num_object_clones":0,"num_object_copies":18312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4148,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7903,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5011},{"shards":"0,2","objects":2931}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805107+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805107+0000","last_peered":"2024-09-17T15:28:23.805107+0000","last_clean":"2024-09-17T15:28:23.805107+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805107+0000","last_undegraded":"2024-09-17T15:28:23.805107+0000","last_fullsized":"2024-09-17T15:28:23.805107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'11615","reported_seq":11797,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:22.082991+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:30:22.082991+0000","last_peered":"2024-09-17T15:30:22.082991+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:30:22.082991+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'1600","ondisk_log_start":"34'1600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37974016,"num_objects":9271,"num_object_clones":0,"num_object_copies":18542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4040,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8402,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5233},{"shards":"0,2","objects":2893}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'11502","reported_seq":17318,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:30:22.093670+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:30:22.093670+0000","last_peered":"2024-09-17T15:30:22.093670+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:30:22.093670+0000","last_undegraded":"2024-09-17T15:30:22.093670+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"34'1500","ondisk_log_start":"34'1500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37397446,"num_objects":9132,"num_object_clones":0,"num_object_copies":18264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1275,"num_objects_unfound":0,"num_objects_dirty":9132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7944,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2836,"num_bytes_recovered":11604451,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7857},{"shards":"0,2","objects":1275}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":604193526,"num_objects":147518,"num_object_clones":0,"num_object_copies":295036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41108,"num_objects_misplaced":1275,"num_objects_unfound":0,"num_objects_dirty":147518,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":145469,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7077,"num_bytes_recovered":28275654,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1285451776,"data_stored":1285299112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160734,"ondisk_log_size":160734,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051648,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":738664,"kb_used_data":427928,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93633176,"statfs":{"total":96636764160,"available":95880372224,"internally_reserved":0,"allocated":438198272,"data_stored":437917626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":318172844},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,1,2,1,0,0,0,0,3],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724593,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":586008,"kb_used_data":319688,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93785832,"statfs":{"total":96636764160,"available":96036691968,"internally_reserved":0,"allocated":327360512,"data_stored":327039659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":272690859},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,1,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247876,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":866984,"kb_used_data":510484,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93504856,"statfs":{"total":96636764160,"available":95748972544,"internally_reserved":0,"allocated":522735616,"data_stored":522454965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":365031080},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,4,4,0,0,0,0,0,0,0,1],"upper_bound":16384},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":521322496,"data_stored":521286366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":326979584,"data_stored":326903252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":437149696,"data_stored":437109494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:30:26.923 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:30:26.923 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'11486", 'reported_seq': 11607, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:22.081937+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:30:22.081937+0000', 'last_peered': '2024-09-17T15:30:22.081937+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:30:22.081937+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "34'1400", 'ondisk_log_start': "34'1400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37527552, 'num_objects': 9162, 'num_object_clones': 0, 'num_object_copies': 18324, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4048, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9162, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11486, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 22, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5114}, {'shards': '0,2', 'objects': 2908}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:26.923 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:30:26.924 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'11721", 'reported_seq': 11873, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:22.460320+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:30:22.460320+0000', 'last_peered': '2024-09-17T15:30:22.460320+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:30:22.460320+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "34'1700", 'ondisk_log_start': "34'1700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37732376, 'num_objects': 9213, 'num_object_clones': 0, 'num_object_copies': 18426, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4156, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9213, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9878, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 13, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2916}, {'shards': '2', 'objects': 5058}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:30:26.924 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:30:26.924 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'11858", 'reported_seq': 12025, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:22.061470+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:30:22.061470+0000', 'last_peered': '2024-09-17T15:30:22.061470+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:30:22.061470+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "34'1800", 'ondisk_log_start': "34'1800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38474211, 'num_objects': 9394, 'num_object_clones': 0, 'num_object_copies': 18788, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4255, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9394, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8357, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 16, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5140}, {'shards': '0,2', 'objects': 3038}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:26.924 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:30:26.924 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'11584", 'reported_seq': 11769, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:21.992393+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:30:21.992393+0000', 'last_peered': '2024-09-17T15:30:21.992393+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:30:21.992393+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "34'1500", 'ondisk_log_start': "34'1500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37618630, 'num_objects': 9186, 'num_object_clones': 0, 'num_object_copies': 18372, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4151, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9186, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 8353, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 23, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5035}, {'shards': '0,2', 'objects': 2977}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:26.924 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:30:26.924 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'11421", 'reported_seq': 11575, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:22.059926+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:30:22.059926+0000', 'last_peered': '2024-09-17T15:30:22.059926+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:30:22.059926+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "34'1400", 'ondisk_log_start': "34'1400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37359616, 'num_objects': 9121, 'num_object_clones': 0, 'num_object_copies': 18242, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4000, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9121, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8017, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 12, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5123}, {'shards': '0,2', 'objects': 2864}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:26.924 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:30:26.924 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'11446", 'reported_seq': 11595, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:22.426963+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:30:22.426963+0000', 'last_peered': '2024-09-17T15:30:22.426963+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:30:22.426963+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "34'1400", 'ondisk_log_start': "34'1400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37163491, 'num_objects': 9074, 'num_object_clones': 0, 'num_object_copies': 18148, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4064, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9074, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9341, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 14, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2893}, {'shards': '2', 'objects': 5011}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:30:26.925 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:30:26.925 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'11547", 'reported_seq': 11695, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:22.514852+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:30:22.514852+0000', 'last_peered': '2024-09-17T15:30:22.514852+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:30:22.514852+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "34'1500", 'ondisk_log_start': "34'1500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37638144, 'num_objects': 9189, 'num_object_clones': 0, 'num_object_copies': 18378, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9189, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9309, 'num_write_kb': 32532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 15, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2961}, {'shards': '2', 'objects': 5067}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:30:26.925 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:30:26.925 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'11657", 'reported_seq': 11826, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:22.006195+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:30:22.006195+0000', 'last_peered': '2024-09-17T15:30:22.006195+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:30:22.006195+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "34'1600", 'ondisk_log_start': "34'1600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38129664, 'num_objects': 9309, 'num_object_clones': 0, 'num_object_copies': 18618, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4122, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9309, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8205, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 17, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5188}, {'shards': '0,2', 'objects': 2965}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:26.925 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:30:26.925 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'11616", 'reported_seq': 11782, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:22.086584+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:30:22.086584+0000', 'last_peered': '2024-09-17T15:30:22.086584+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:30:22.086584+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "34'1600", 'ondisk_log_start': "34'1600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37502976, 'num_objects': 9156, 'num_object_clones': 0, 'num_object_copies': 18312, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4148, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9156, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7903, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 12, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5011}, {'shards': '0,2', 'objects': 2931}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:26.925 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:30:26.925 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'11615", 'reported_seq': 11797, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:22.082991+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:30:22.082991+0000', 'last_peered': '2024-09-17T15:30:22.082991+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:30:22.082991+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "34'1600", 'ondisk_log_start': "34'1600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37974016, 'num_objects': 9271, 'num_object_clones': 0, 'num_object_copies': 18542, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4040, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9271, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8402, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 25, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5233}, {'shards': '0,2', 'objects': 2893}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:26.925 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:30:26.926 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'11502", 'reported_seq': 17318, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:30:22.093670+0000', 'last_change': '2024-09-17T15:28:22.509818+0000', 'last_active': '2024-09-17T15:30:22.093670+0000', 'last_peered': '2024-09-17T15:30:22.093670+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:30:22.093670+0000', 'last_undegraded': '2024-09-17T15:30:22.093670+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "34'1500", 'ondisk_log_start': "34'1500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37397446, 'num_objects': 9132, 'num_object_clones': 0, 'num_object_copies': 18264, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1275, 'num_objects_unfound': 0, 'num_objects_dirty': 9132, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7944, 'num_write_kb': 27056, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2836, 'num_bytes_recovered': 11604451, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7857}, {'shards': '0,2', 'objects': 1275}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:26.926 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:30:27.621 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:30:27.622 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41108/295040 objects degraded (13.933%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":364,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":147520,"data_bytes":604193545,"bytes_used":2244255744,"bytes_avail":287666036736,"bytes_total":289910292480,"degraded_objects":41108,"degraded_total":295040,"degraded_ratio":0.13933026030368764,"misplaced_objects":1275,"misplaced_total":295040,"misplaced_ratio":0.0043214479392624733,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":100316,"recovering_keys_per_sec":0,"num_objects_recovered":312,"num_bytes_recovered":1204224,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":376},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T15:27:19.616376+0000","services":{}},"progress_events":{}} 2024-09-17T15:30:27.648 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:30:27.648 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:30:28.197 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:30:28.199 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:30:28.222 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":218,"stamp":"2024-09-17T15:30:27.675859+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595055369,"num_objects":145289,"num_object_clones":0,"num_object_copies":290578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41077,"num_objects_misplaced":1104,"num_objects_unfound":0,"num_objects_dirty":145289,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":147702,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7280,"num_bytes_recovered":28976089,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2176104,"kb_used_data":1246708,"kb_used_omap":13,"kb_used_meta":929330,"kb_avail":280939416,"statfs":{"total":289910292480,"available":287681961984,"internally_reserved":0,"allocated":1276628992,"data_stored":1275746842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":951634943},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,3,8,5,5,8,2,5,0,0,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27631616,"num_objects":-6746,"num_object_clones":0,"num_object_copies":-13492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-49,"num_objects_misplaced":-465,"num_objects_unfound":0,"num_objects_dirty":-6746,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6746,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":514,"num_bytes_recovered":1904640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003936"},"pg_stats":[{"pgid":"2.f","version":"43'11637","reported_seq":11765,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:27.078828+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:30:27.078828+0000","last_peered":"2024-09-17T15:30:27.078828+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:30:27.078828+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'1600","ondisk_log_start":"34'1600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36909056,"num_objects":9011,"num_object_clones":0,"num_object_copies":18022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4045,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11637,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4966},{"shards":"0,2","objects":2757}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'11768","reported_seq":11871,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:27.504819+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:30:27.504819+0000","last_peered":"2024-09-17T15:30:27.504819+0000","last_clean":"2024-09-17T15:30:27.504819+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:30:27.504819+0000","last_undegraded":"2024-09-17T15:30:27.504819+0000","last_fullsized":"2024-09-17T15:30:27.504819+0000","mapping_epoch":33,"log_start":"34'1700","ondisk_log_start":"34'1700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37536710,"num_objects":9166,"num_object_clones":0,"num_object_copies":18332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9166,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10125,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'11853","reported_seq":12010,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:27.476781+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:30:27.476781+0000","last_peered":"2024-09-17T15:30:27.476781+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:30:27.476781+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'1800","ondisk_log_start":"34'1800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37191704,"num_objects":9081,"num_object_clones":0,"num_object_copies":18162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9081,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10010,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2784},{"shards":"2","objects":4928}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'11758","reported_seq":11828,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:27.533291+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:30:27.533291+0000","last_peered":"2024-09-17T15:30:27.533291+0000","last_clean":"2024-09-17T15:30:27.533291+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:30:27.533291+0000","last_undegraded":"2024-09-17T15:30:27.533291+0000","last_fullsized":"2024-09-17T15:30:27.533291+0000","mapping_epoch":33,"log_start":"34'1700","ondisk_log_start":"34'1700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37732352,"num_objects":9212,"num_object_clones":0,"num_object_copies":18424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11758,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'11615","reported_seq":11744,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:27.102324+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:30:27.102324+0000","last_peered":"2024-09-17T15:30:27.102324+0000","last_clean":"2024-09-17T15:30:27.102324+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:30:27.102324+0000","last_undegraded":"2024-09-17T15:30:27.102324+0000","last_fullsized":"2024-09-17T15:30:27.102324+0000","mapping_epoch":32,"log_start":"34'1600","ondisk_log_start":"34'1600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37020131,"num_objects":9039,"num_object_clones":0,"num_object_copies":18078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9039,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8597,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'12005","reported_seq":12178,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:27.025316+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:30:27.025316+0000","last_peered":"2024-09-17T15:30:27.025316+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:30:27.025316+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'2000","ondisk_log_start":"34'2000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37872099,"num_objects":9247,"num_object_clones":0,"num_object_copies":18494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4252,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9247,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8504,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4996},{"shards":"0,2","objects":2891}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'11736","reported_seq":11928,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:27.051705+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:30:27.051705+0000","last_peered":"2024-09-17T15:30:27.051705+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:30:27.051705+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'1700","ondisk_log_start":"34'1700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36996038,"num_objects":9034,"num_object_clones":0,"num_object_copies":18068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4147,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9034,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8505,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4887},{"shards":"0,2","objects":2825}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'11795","reported_seq":11923,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:27.101975+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:30:27.101975+0000","last_peered":"2024-09-17T15:30:27.101975+0000","last_clean":"2024-09-17T15:30:27.101975+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:30:27.101975+0000","last_undegraded":"2024-09-17T15:30:27.101975+0000","last_fullsized":"2024-09-17T15:30:27.101975+0000","mapping_epoch":32,"log_start":"34'1700","ondisk_log_start":"34'1700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37359616,"num_objects":9121,"num_object_clones":0,"num_object_copies":18242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8749,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804377+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804377+0000","last_peered":"2024-09-17T15:28:23.804377+0000","last_clean":"2024-09-17T15:28:23.804377+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804377+0000","last_undegraded":"2024-09-17T15:28:23.804377+0000","last_fullsized":"2024-09-17T15:28:23.804377+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'11724","reported_seq":20006,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:26.332212+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:30:26.332212+0000","last_peered":"2024-09-17T15:30:26.332212+0000","last_clean":"2024-09-17T15:30:26.332212+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:30:26.332212+0000","last_undegraded":"2024-09-17T15:30:26.332212+0000","last_fullsized":"2024-09-17T15:30:26.332212+0000","mapping_epoch":42,"log_start":"34'1700","ondisk_log_start":"34'1700","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37245411,"num_objects":9094,"num_object_clones":0,"num_object_copies":18188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9094,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9724,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804747+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804747+0000","last_peered":"2024-09-17T15:28:23.804747+0000","last_clean":"2024-09-17T15:28:23.804747+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804747+0000","last_undegraded":"2024-09-17T15:28:23.804747+0000","last_fullsized":"2024-09-17T15:28:23.804747+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'11559","reported_seq":11718,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:27.067285+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:30:27.067285+0000","last_peered":"2024-09-17T15:30:27.067285+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:30:27.067285+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'1500","ondisk_log_start":"34'1500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36794368,"num_objects":8983,"num_object_clones":0,"num_object_copies":17966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3998,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8155,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4987},{"shards":"0,2","objects":2726}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'11600","reported_seq":11751,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:27.526047+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:30:27.526047+0000","last_peered":"2024-09-17T15:30:27.526047+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:30:27.526047+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'1500","ondisk_log_start":"34'1500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36532707,"num_objects":8920,"num_object_clones":0,"num_object_copies":17840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4063,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8920,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9495,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2739},{"shards":"2","objects":4858}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":77,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804441+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:23.804441+0000","last_peered":"2024-09-17T15:28:23.804441+0000","last_clean":"2024-09-17T15:28:23.804441+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:23.804441+0000","last_undegraded":"2024-09-17T15:28:23.804441+0000","last_fullsized":"2024-09-17T15:28:23.804441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'11668","reported_seq":11819,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:27.462513+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:30:27.462513+0000","last_peered":"2024-09-17T15:30:27.462513+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:30:27.462513+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'1600","ondisk_log_start":"34'1600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37142528,"num_objects":9068,"num_object_clones":0,"num_object_copies":18136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9430,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2840},{"shards":"2","objects":4947}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'11808","reported_seq":11987,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:27.009291+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:30:27.009291+0000","last_peered":"2024-09-17T15:30:27.009291+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:30:27.009291+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'1800","ondisk_log_start":"34'1800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37511168,"num_objects":9158,"num_object_clones":0,"num_object_copies":18316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4117,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8356,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5042},{"shards":"0,2","objects":2814}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804602+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:23.804602+0000","last_peered":"2024-09-17T15:28:23.804602+0000","last_clean":"2024-09-17T15:28:23.804602+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:23.804602+0000","last_undegraded":"2024-09-17T15:28:23.804602+0000","last_fullsized":"2024-09-17T15:28:23.804602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'11759","reported_seq":11935,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:27.142351+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:30:27.142351+0000","last_peered":"2024-09-17T15:30:27.142351+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:30:27.142351+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'1700","ondisk_log_start":"34'1700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36917248,"num_objects":9013,"num_object_clones":0,"num_object_copies":18026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4143,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9013,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8046,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4873},{"shards":"0,2","objects":2788}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805107+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805107+0000","last_peered":"2024-09-17T15:28:23.805107+0000","last_clean":"2024-09-17T15:28:23.805107+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805107+0000","last_undegraded":"2024-09-17T15:28:23.805107+0000","last_fullsized":"2024-09-17T15:28:23.805107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'11759","reported_seq":11951,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:27.079184+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:30:27.079184+0000","last_peered":"2024-09-17T15:30:27.079184+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:30:27.079184+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'1700","ondisk_log_start":"34'1700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37384192,"num_objects":9127,"num_object_clones":0,"num_object_copies":18254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4035,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8546,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5094},{"shards":"0,2","objects":2749}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804819+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:23.804819+0000","last_peered":"2024-09-17T15:28:23.804819+0000","last_clean":"2024-09-17T15:28:23.804819+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:23.804819+0000","last_undegraded":"2024-09-17T15:28:23.804819+0000","last_fullsized":"2024-09-17T15:28:23.804819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'11621","reported_seq":17779,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:30:27.121236+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:30:27.121236+0000","last_peered":"2024-09-17T15:30:27.121236+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:30:27.121236+0000","last_undegraded":"2024-09-17T15:30:27.121236+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"34'1600","ondisk_log_start":"34'1600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36910022,"num_objects":9013,"num_object_clones":0,"num_object_copies":18026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1104,"num_objects_unfound":0,"num_objects_dirty":9013,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8063,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3007,"num_bytes_recovered":12304867,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7909},{"shards":"0,2","objects":1104}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":595055350,"num_objects":145287,"num_object_clones":0,"num_object_copies":290574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41077,"num_objects_misplaced":1104,"num_objects_unfound":0,"num_objects_dirty":145287,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":147700,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7279,"num_bytes_recovered":28976070,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1273745408,"data_stored":1273592744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160765,"ondisk_log_size":160765,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051649,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":721816,"kb_used_data":423560,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93650024,"statfs":{"total":96636764160,"available":95897624576,"internally_reserved":0,"allocated":433725440,"data_stored":433444794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":305393324},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,4,2,1,2,2,0,0,0,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724594,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589840,"kb_used_data":319356,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93782000,"statfs":{"total":96636764160,"available":96032768000,"internally_reserved":0,"allocated":327020544,"data_stored":326699691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":276950699},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,1,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247877,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":864448,"kb_used_data":503792,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93507392,"statfs":{"total":96636764160,"available":95751569408,"internally_reserved":0,"allocated":515883008,"data_stored":515602357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":369290920},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,2,2,1,0,0,5],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":514392064,"data_stored":514355934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":326627328,"data_stored":326550996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":432726016,"data_stored":432685814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:30:31.224 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:30:31.746 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:30:31.747 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:30:31.768 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2024-09-17T15:30:31.677014+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594473737,"num_objects":145147,"num_object_clones":0,"num_object_copies":290294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41077,"num_objects_misplaced":1104,"num_objects_unfound":0,"num_objects_dirty":145147,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":147844,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7280,"num_bytes_recovered":28976089,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160810,"ondisk_log_size":160810,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2202132,"kb_used_data":1246372,"kb_used_omap":13,"kb_used_meta":955698,"kb_avail":280913388,"statfs":{"total":289910292480,"available":287655309312,"internally_reserved":0,"allocated":1276284928,"data_stored":1275402778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":978635775},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,4,7,6,4,2,2,5,0,0,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19030016,"num_objects":-4646,"num_object_clones":0,"num_object_copies":-9292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-46,"num_objects_misplaced":-318,"num_objects_unfound":0,"num_objects_dirty":-4646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4646,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":364,"num_bytes_recovered":1302528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003529"},"pg_stats":[{"pgid":"2.f","version":"43'11637","reported_seq":11765,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:27.078828+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:30:27.078828+0000","last_peered":"2024-09-17T15:30:27.078828+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:30:27.078828+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'1600","ondisk_log_start":"34'1600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36909056,"num_objects":9011,"num_object_clones":0,"num_object_copies":18022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4045,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11637,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4966},{"shards":"0,2","objects":2757}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'11768","reported_seq":11871,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:27.504819+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:30:27.504819+0000","last_peered":"2024-09-17T15:30:27.504819+0000","last_clean":"2024-09-17T15:30:27.504819+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:30:27.504819+0000","last_undegraded":"2024-09-17T15:30:27.504819+0000","last_fullsized":"2024-09-17T15:30:27.504819+0000","mapping_epoch":33,"log_start":"34'1700","ondisk_log_start":"34'1700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37536710,"num_objects":9166,"num_object_clones":0,"num_object_copies":18332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9166,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10125,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'11853","reported_seq":12010,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:27.476781+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:30:27.476781+0000","last_peered":"2024-09-17T15:30:27.476781+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:30:27.476781+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'1800","ondisk_log_start":"34'1800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37191704,"num_objects":9081,"num_object_clones":0,"num_object_copies":18162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9081,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10010,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2784},{"shards":"2","objects":4928}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'11758","reported_seq":11828,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:27.533291+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:30:27.533291+0000","last_peered":"2024-09-17T15:30:27.533291+0000","last_clean":"2024-09-17T15:30:27.533291+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:30:27.533291+0000","last_undegraded":"2024-09-17T15:30:27.533291+0000","last_fullsized":"2024-09-17T15:30:27.533291+0000","mapping_epoch":33,"log_start":"34'1700","ondisk_log_start":"34'1700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37732352,"num_objects":9212,"num_object_clones":0,"num_object_copies":18424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11758,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'11615","reported_seq":11744,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:27.102324+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:30:27.102324+0000","last_peered":"2024-09-17T15:30:27.102324+0000","last_clean":"2024-09-17T15:30:27.102324+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:30:27.102324+0000","last_undegraded":"2024-09-17T15:30:27.102324+0000","last_fullsized":"2024-09-17T15:30:27.102324+0000","mapping_epoch":32,"log_start":"34'1600","ondisk_log_start":"34'1600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37020131,"num_objects":9039,"num_object_clones":0,"num_object_copies":18078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9039,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8597,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'12005","reported_seq":12178,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:27.025316+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:30:27.025316+0000","last_peered":"2024-09-17T15:30:27.025316+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:30:27.025316+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'2000","ondisk_log_start":"34'2000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37872099,"num_objects":9247,"num_object_clones":0,"num_object_copies":18494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4252,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9247,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8504,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4996},{"shards":"0,2","objects":2891}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'11736","reported_seq":11928,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:27.051705+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:30:27.051705+0000","last_peered":"2024-09-17T15:30:27.051705+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:30:27.051705+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'1700","ondisk_log_start":"34'1700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36996038,"num_objects":9034,"num_object_clones":0,"num_object_copies":18068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4147,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9034,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8505,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4887},{"shards":"0,2","objects":2825}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'11795","reported_seq":11923,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:27.101975+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:30:27.101975+0000","last_peered":"2024-09-17T15:30:27.101975+0000","last_clean":"2024-09-17T15:30:27.101975+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:30:27.101975+0000","last_undegraded":"2024-09-17T15:30:27.101975+0000","last_fullsized":"2024-09-17T15:30:27.101975+0000","mapping_epoch":32,"log_start":"34'1700","ondisk_log_start":"34'1700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37359616,"num_objects":9121,"num_object_clones":0,"num_object_copies":18242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8749,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804377+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804377+0000","last_peered":"2024-09-17T15:28:23.804377+0000","last_clean":"2024-09-17T15:28:23.804377+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804377+0000","last_undegraded":"2024-09-17T15:28:23.804377+0000","last_fullsized":"2024-09-17T15:28:23.804377+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'11866","reported_seq":20146,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:31.416759+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:30:31.416759+0000","last_peered":"2024-09-17T15:30:31.416759+0000","last_clean":"2024-09-17T15:30:31.416759+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:30:31.416759+0000","last_undegraded":"2024-09-17T15:30:31.416759+0000","last_fullsized":"2024-09-17T15:30:31.416759+0000","mapping_epoch":42,"log_start":"34'1800","ondisk_log_start":"34'1800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36663779,"num_objects":8952,"num_object_clones":0,"num_object_copies":17904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8952,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9866,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804747+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804747+0000","last_peered":"2024-09-17T15:28:23.804747+0000","last_clean":"2024-09-17T15:28:23.804747+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804747+0000","last_undegraded":"2024-09-17T15:28:23.804747+0000","last_fullsized":"2024-09-17T15:28:23.804747+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'11559","reported_seq":11718,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:27.067285+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:30:27.067285+0000","last_peered":"2024-09-17T15:30:27.067285+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:30:27.067285+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'1500","ondisk_log_start":"34'1500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36794368,"num_objects":8983,"num_object_clones":0,"num_object_copies":17966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3998,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8155,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4987},{"shards":"0,2","objects":2726}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'11600","reported_seq":11751,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:27.526047+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:30:27.526047+0000","last_peered":"2024-09-17T15:30:27.526047+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:30:27.526047+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'1500","ondisk_log_start":"34'1500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36532707,"num_objects":8920,"num_object_clones":0,"num_object_copies":17840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4063,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8920,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9495,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2739},{"shards":"2","objects":4858}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":78,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.244375+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:25.244375+0000","last_peered":"2024-09-17T15:28:25.244375+0000","last_clean":"2024-09-17T15:28:25.244375+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:25.244375+0000","last_undegraded":"2024-09-17T15:28:25.244375+0000","last_fullsized":"2024-09-17T15:28:25.244375+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'11668","reported_seq":11819,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:27.462513+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:30:27.462513+0000","last_peered":"2024-09-17T15:30:27.462513+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:30:27.462513+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'1600","ondisk_log_start":"34'1600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37142528,"num_objects":9068,"num_object_clones":0,"num_object_copies":18136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9430,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2840},{"shards":"2","objects":4947}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'11808","reported_seq":11987,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:27.009291+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:30:27.009291+0000","last_peered":"2024-09-17T15:30:27.009291+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:30:27.009291+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'1800","ondisk_log_start":"34'1800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37511168,"num_objects":9158,"num_object_clones":0,"num_object_copies":18316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4117,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8356,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5042},{"shards":"0,2","objects":2814}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.244410+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:25.244410+0000","last_peered":"2024-09-17T15:28:25.244410+0000","last_clean":"2024-09-17T15:28:25.244410+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:25.244410+0000","last_undegraded":"2024-09-17T15:28:25.244410+0000","last_fullsized":"2024-09-17T15:28:25.244410+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'11759","reported_seq":11935,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:27.142351+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:30:27.142351+0000","last_peered":"2024-09-17T15:30:27.142351+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:30:27.142351+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'1700","ondisk_log_start":"34'1700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36917248,"num_objects":9013,"num_object_clones":0,"num_object_copies":18026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4143,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9013,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8046,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4873},{"shards":"0,2","objects":2788}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805107+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805107+0000","last_peered":"2024-09-17T15:28:23.805107+0000","last_clean":"2024-09-17T15:28:23.805107+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805107+0000","last_undegraded":"2024-09-17T15:28:23.805107+0000","last_fullsized":"2024-09-17T15:28:23.805107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'11759","reported_seq":11951,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:27.079184+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:30:27.079184+0000","last_peered":"2024-09-17T15:30:27.079184+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:30:27.079184+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'1700","ondisk_log_start":"34'1700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37384192,"num_objects":9127,"num_object_clones":0,"num_object_copies":18254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4035,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8546,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5094},{"shards":"0,2","objects":2749}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.244422+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:25.244422+0000","last_peered":"2024-09-17T15:28:25.244422+0000","last_clean":"2024-09-17T15:28:25.244422+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:25.244422+0000","last_undegraded":"2024-09-17T15:28:25.244422+0000","last_fullsized":"2024-09-17T15:28:25.244422+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'11621","reported_seq":17779,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:30:27.121236+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:30:27.121236+0000","last_peered":"2024-09-17T15:30:27.121236+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:30:27.121236+0000","last_undegraded":"2024-09-17T15:30:27.121236+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"34'1600","ondisk_log_start":"34'1600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36910022,"num_objects":9013,"num_object_clones":0,"num_object_copies":18026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1104,"num_objects_unfound":0,"num_objects_dirty":9013,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8063,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3007,"num_bytes_recovered":12304867,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7909},{"shards":"0,2","objects":1104}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":594473718,"num_objects":145145,"num_object_clones":0,"num_object_copies":290290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41077,"num_objects_misplaced":1104,"num_objects_unfound":0,"num_objects_dirty":145145,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":147842,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7279,"num_bytes_recovered":28976070,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1273384960,"data_stored":1273232296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160807,"ondisk_log_size":160807,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051649,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":721816,"kb_used_data":423560,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93650024,"statfs":{"total":96636764160,"available":95897624576,"internally_reserved":0,"allocated":433725440,"data_stored":433444794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":305393324},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,4,2,1,2,2,0,0,0,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724595,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":615868,"kb_used_data":319020,"kb_used_omap":4,"kb_used_meta":296827,"kb_avail":93755972,"statfs":{"total":96636764160,"available":96006115328,"internally_reserved":0,"allocated":326676480,"data_stored":326355627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":303951531},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247877,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":864448,"kb_used_data":503792,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93507392,"statfs":{"total":96636764160,"available":95751569408,"internally_reserved":0,"allocated":515883008,"data_stored":515602357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":369290920},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,2,2,1,0,0,5],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":514392064,"data_stored":514355934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":326266880,"data_stored":326190548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":432726016,"data_stored":432685814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:30:31.770 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:30:32.309 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:30:32.312 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:30:32.341 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2024-09-17T15:30:31.677014+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594473737,"num_objects":145147,"num_object_clones":0,"num_object_copies":290294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41077,"num_objects_misplaced":1104,"num_objects_unfound":0,"num_objects_dirty":145147,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":147844,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7280,"num_bytes_recovered":28976089,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160810,"ondisk_log_size":160810,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2202132,"kb_used_data":1246372,"kb_used_omap":13,"kb_used_meta":955698,"kb_avail":280913388,"statfs":{"total":289910292480,"available":287655309312,"internally_reserved":0,"allocated":1276284928,"data_stored":1275402778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":978635775},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,4,7,6,4,2,2,5,0,0,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19030016,"num_objects":-4646,"num_object_clones":0,"num_object_copies":-9292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-46,"num_objects_misplaced":-318,"num_objects_unfound":0,"num_objects_dirty":-4646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4646,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":364,"num_bytes_recovered":1302528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003529"},"pg_stats":[{"pgid":"2.f","version":"43'11637","reported_seq":11765,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:27.078828+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:30:27.078828+0000","last_peered":"2024-09-17T15:30:27.078828+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:30:27.078828+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'1600","ondisk_log_start":"34'1600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36909056,"num_objects":9011,"num_object_clones":0,"num_object_copies":18022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4045,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11637,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4966},{"shards":"0,2","objects":2757}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'11768","reported_seq":11871,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:27.504819+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:30:27.504819+0000","last_peered":"2024-09-17T15:30:27.504819+0000","last_clean":"2024-09-17T15:30:27.504819+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:30:27.504819+0000","last_undegraded":"2024-09-17T15:30:27.504819+0000","last_fullsized":"2024-09-17T15:30:27.504819+0000","mapping_epoch":33,"log_start":"34'1700","ondisk_log_start":"34'1700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37536710,"num_objects":9166,"num_object_clones":0,"num_object_copies":18332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9166,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10125,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'11853","reported_seq":12010,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:27.476781+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:30:27.476781+0000","last_peered":"2024-09-17T15:30:27.476781+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:30:27.476781+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'1800","ondisk_log_start":"34'1800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37191704,"num_objects":9081,"num_object_clones":0,"num_object_copies":18162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9081,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10010,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2784},{"shards":"2","objects":4928}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'11758","reported_seq":11828,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:27.533291+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:30:27.533291+0000","last_peered":"2024-09-17T15:30:27.533291+0000","last_clean":"2024-09-17T15:30:27.533291+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:30:27.533291+0000","last_undegraded":"2024-09-17T15:30:27.533291+0000","last_fullsized":"2024-09-17T15:30:27.533291+0000","mapping_epoch":33,"log_start":"34'1700","ondisk_log_start":"34'1700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37732352,"num_objects":9212,"num_object_clones":0,"num_object_copies":18424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11758,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'11615","reported_seq":11744,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:27.102324+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:30:27.102324+0000","last_peered":"2024-09-17T15:30:27.102324+0000","last_clean":"2024-09-17T15:30:27.102324+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:30:27.102324+0000","last_undegraded":"2024-09-17T15:30:27.102324+0000","last_fullsized":"2024-09-17T15:30:27.102324+0000","mapping_epoch":32,"log_start":"34'1600","ondisk_log_start":"34'1600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37020131,"num_objects":9039,"num_object_clones":0,"num_object_copies":18078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9039,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8597,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'12005","reported_seq":12178,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:27.025316+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:30:27.025316+0000","last_peered":"2024-09-17T15:30:27.025316+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:30:27.025316+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'2000","ondisk_log_start":"34'2000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37872099,"num_objects":9247,"num_object_clones":0,"num_object_copies":18494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4252,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9247,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8504,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4996},{"shards":"0,2","objects":2891}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'11736","reported_seq":11928,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:27.051705+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:30:27.051705+0000","last_peered":"2024-09-17T15:30:27.051705+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:30:27.051705+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'1700","ondisk_log_start":"34'1700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36996038,"num_objects":9034,"num_object_clones":0,"num_object_copies":18068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4147,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9034,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8505,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4887},{"shards":"0,2","objects":2825}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'11795","reported_seq":11923,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:27.101975+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:30:27.101975+0000","last_peered":"2024-09-17T15:30:27.101975+0000","last_clean":"2024-09-17T15:30:27.101975+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:30:27.101975+0000","last_undegraded":"2024-09-17T15:30:27.101975+0000","last_fullsized":"2024-09-17T15:30:27.101975+0000","mapping_epoch":32,"log_start":"34'1700","ondisk_log_start":"34'1700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37359616,"num_objects":9121,"num_object_clones":0,"num_object_copies":18242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8749,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804377+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804377+0000","last_peered":"2024-09-17T15:28:23.804377+0000","last_clean":"2024-09-17T15:28:23.804377+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804377+0000","last_undegraded":"2024-09-17T15:28:23.804377+0000","last_fullsized":"2024-09-17T15:28:23.804377+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'11866","reported_seq":20146,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:31.416759+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:30:31.416759+0000","last_peered":"2024-09-17T15:30:31.416759+0000","last_clean":"2024-09-17T15:30:31.416759+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:30:31.416759+0000","last_undegraded":"2024-09-17T15:30:31.416759+0000","last_fullsized":"2024-09-17T15:30:31.416759+0000","mapping_epoch":42,"log_start":"34'1800","ondisk_log_start":"34'1800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36663779,"num_objects":8952,"num_object_clones":0,"num_object_copies":17904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8952,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9866,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804747+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804747+0000","last_peered":"2024-09-17T15:28:23.804747+0000","last_clean":"2024-09-17T15:28:23.804747+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804747+0000","last_undegraded":"2024-09-17T15:28:23.804747+0000","last_fullsized":"2024-09-17T15:28:23.804747+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'11559","reported_seq":11718,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:27.067285+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:30:27.067285+0000","last_peered":"2024-09-17T15:30:27.067285+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:30:27.067285+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'1500","ondisk_log_start":"34'1500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36794368,"num_objects":8983,"num_object_clones":0,"num_object_copies":17966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3998,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8155,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4987},{"shards":"0,2","objects":2726}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'11600","reported_seq":11751,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:27.526047+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:30:27.526047+0000","last_peered":"2024-09-17T15:30:27.526047+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:30:27.526047+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'1500","ondisk_log_start":"34'1500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36532707,"num_objects":8920,"num_object_clones":0,"num_object_copies":17840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4063,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8920,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9495,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2739},{"shards":"2","objects":4858}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":78,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.244375+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:25.244375+0000","last_peered":"2024-09-17T15:28:25.244375+0000","last_clean":"2024-09-17T15:28:25.244375+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:25.244375+0000","last_undegraded":"2024-09-17T15:28:25.244375+0000","last_fullsized":"2024-09-17T15:28:25.244375+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'11668","reported_seq":11819,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:27.462513+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:30:27.462513+0000","last_peered":"2024-09-17T15:30:27.462513+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:30:27.462513+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'1600","ondisk_log_start":"34'1600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37142528,"num_objects":9068,"num_object_clones":0,"num_object_copies":18136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9430,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2840},{"shards":"2","objects":4947}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'11808","reported_seq":11987,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:27.009291+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:30:27.009291+0000","last_peered":"2024-09-17T15:30:27.009291+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:30:27.009291+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'1800","ondisk_log_start":"34'1800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37511168,"num_objects":9158,"num_object_clones":0,"num_object_copies":18316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4117,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8356,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5042},{"shards":"0,2","objects":2814}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.244410+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:25.244410+0000","last_peered":"2024-09-17T15:28:25.244410+0000","last_clean":"2024-09-17T15:28:25.244410+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:25.244410+0000","last_undegraded":"2024-09-17T15:28:25.244410+0000","last_fullsized":"2024-09-17T15:28:25.244410+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'11759","reported_seq":11935,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:27.142351+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:30:27.142351+0000","last_peered":"2024-09-17T15:30:27.142351+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:30:27.142351+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'1700","ondisk_log_start":"34'1700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36917248,"num_objects":9013,"num_object_clones":0,"num_object_copies":18026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4143,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9013,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8046,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4873},{"shards":"0,2","objects":2788}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805107+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805107+0000","last_peered":"2024-09-17T15:28:23.805107+0000","last_clean":"2024-09-17T15:28:23.805107+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805107+0000","last_undegraded":"2024-09-17T15:28:23.805107+0000","last_fullsized":"2024-09-17T15:28:23.805107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'11759","reported_seq":11951,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:27.079184+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:30:27.079184+0000","last_peered":"2024-09-17T15:30:27.079184+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:30:27.079184+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'1700","ondisk_log_start":"34'1700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37384192,"num_objects":9127,"num_object_clones":0,"num_object_copies":18254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4035,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8546,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5094},{"shards":"0,2","objects":2749}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.244422+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:25.244422+0000","last_peered":"2024-09-17T15:28:25.244422+0000","last_clean":"2024-09-17T15:28:25.244422+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:25.244422+0000","last_undegraded":"2024-09-17T15:28:25.244422+0000","last_fullsized":"2024-09-17T15:28:25.244422+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'11621","reported_seq":17779,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:30:27.121236+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:30:27.121236+0000","last_peered":"2024-09-17T15:30:27.121236+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:30:27.121236+0000","last_undegraded":"2024-09-17T15:30:27.121236+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"34'1600","ondisk_log_start":"34'1600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36910022,"num_objects":9013,"num_object_clones":0,"num_object_copies":18026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1104,"num_objects_unfound":0,"num_objects_dirty":9013,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8063,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3007,"num_bytes_recovered":12304867,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7909},{"shards":"0,2","objects":1104}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":594473718,"num_objects":145145,"num_object_clones":0,"num_object_copies":290290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41077,"num_objects_misplaced":1104,"num_objects_unfound":0,"num_objects_dirty":145145,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":147842,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7279,"num_bytes_recovered":28976070,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1273384960,"data_stored":1273232296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160807,"ondisk_log_size":160807,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051649,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":721816,"kb_used_data":423560,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93650024,"statfs":{"total":96636764160,"available":95897624576,"internally_reserved":0,"allocated":433725440,"data_stored":433444794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":305393324},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,4,2,1,2,2,0,0,0,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724595,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":615868,"kb_used_data":319020,"kb_used_omap":4,"kb_used_meta":296827,"kb_avail":93755972,"statfs":{"total":96636764160,"available":96006115328,"internally_reserved":0,"allocated":326676480,"data_stored":326355627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":303951531},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247877,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":864448,"kb_used_data":503792,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93507392,"statfs":{"total":96636764160,"available":95751569408,"internally_reserved":0,"allocated":515883008,"data_stored":515602357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":369290920},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,2,2,1,0,0,5],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":514392064,"data_stored":514355934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":326266880,"data_stored":326190548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":432726016,"data_stored":432685814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:30:32.342 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:30:32.342 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'11637", 'reported_seq': 11765, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:27.078828+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:30:27.078828+0000', 'last_peered': '2024-09-17T15:30:27.078828+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:30:27.078828+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "34'1600", 'ondisk_log_start': "34'1600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36909056, 'num_objects': 9011, 'num_object_clones': 0, 'num_object_copies': 18022, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4045, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9011, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11637, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 25, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4966}, {'shards': '0,2', 'objects': 2757}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:32.342 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:30:32.342 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'11853", 'reported_seq': 12010, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:27.476781+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:30:27.476781+0000', 'last_peered': '2024-09-17T15:30:27.476781+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:30:27.476781+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "34'1800", 'ondisk_log_start': "34'1800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37191704, 'num_objects': 9081, 'num_object_clones': 0, 'num_object_copies': 18162, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4154, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9081, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10010, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 15, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2784}, {'shards': '2', 'objects': 4928}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:30:32.343 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:30:32.343 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'12005", 'reported_seq': 12178, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:27.025316+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:30:27.025316+0000', 'last_peered': '2024-09-17T15:30:27.025316+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:30:27.025316+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "34'2000", 'ondisk_log_start': "34'2000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37872099, 'num_objects': 9247, 'num_object_clones': 0, 'num_object_copies': 18494, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4252, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9247, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8504, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 19, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4996}, {'shards': '0,2', 'objects': 2891}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:32.343 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:30:32.343 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'11736", 'reported_seq': 11928, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:27.051705+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:30:27.051705+0000', 'last_peered': '2024-09-17T15:30:27.051705+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:30:27.051705+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "34'1700", 'ondisk_log_start': "34'1700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36996038, 'num_objects': 9034, 'num_object_clones': 0, 'num_object_copies': 18068, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4147, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9034, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 8505, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4887}, {'shards': '0,2', 'objects': 2825}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:32.343 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:30:32.343 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'11559", 'reported_seq': 11718, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:27.067285+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:30:27.067285+0000', 'last_peered': '2024-09-17T15:30:27.067285+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:30:27.067285+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "34'1500", 'ondisk_log_start': "34'1500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36794368, 'num_objects': 8983, 'num_object_clones': 0, 'num_object_copies': 17966, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3998, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8983, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8155, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 14, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4987}, {'shards': '0,2', 'objects': 2726}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:32.343 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:30:32.343 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'11600", 'reported_seq': 11751, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:27.526047+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:30:27.526047+0000', 'last_peered': '2024-09-17T15:30:27.526047+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:30:27.526047+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "34'1500", 'ondisk_log_start': "34'1500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36532707, 'num_objects': 8920, 'num_object_clones': 0, 'num_object_copies': 17840, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4063, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8920, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9495, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 15, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2739}, {'shards': '2', 'objects': 4858}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:30:32.343 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:30:32.343 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'11668", 'reported_seq': 11819, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:27.462513+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:30:27.462513+0000', 'last_peered': '2024-09-17T15:30:27.462513+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:30:27.462513+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "34'1600", 'ondisk_log_start': "34'1600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37142528, 'num_objects': 9068, 'num_object_clones': 0, 'num_object_copies': 18136, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4123, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9068, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9430, 'num_write_kb': 32532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 16, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2840}, {'shards': '2', 'objects': 4947}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:30:32.343 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:30:32.344 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'11808", 'reported_seq': 11987, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:27.009291+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:30:27.009291+0000', 'last_peered': '2024-09-17T15:30:27.009291+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:30:27.009291+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "34'1800", 'ondisk_log_start': "34'1800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37511168, 'num_objects': 9158, 'num_object_clones': 0, 'num_object_copies': 18316, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4117, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9158, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8356, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 22, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5042}, {'shards': '0,2', 'objects': 2814}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:32.344 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:30:32.344 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'11759", 'reported_seq': 11935, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:27.142351+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:30:27.142351+0000', 'last_peered': '2024-09-17T15:30:27.142351+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:30:27.142351+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "34'1700", 'ondisk_log_start': "34'1700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36917248, 'num_objects': 9013, 'num_object_clones': 0, 'num_object_copies': 18026, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4143, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9013, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8046, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 17, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4873}, {'shards': '0,2', 'objects': 2788}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:32.344 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:30:32.344 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'11759", 'reported_seq': 11951, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:27.079184+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:30:27.079184+0000', 'last_peered': '2024-09-17T15:30:27.079184+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:30:27.079184+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "34'1700", 'ondisk_log_start': "34'1700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37384192, 'num_objects': 9127, 'num_object_clones': 0, 'num_object_copies': 18254, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4035, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9127, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8546, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5094}, {'shards': '0,2', 'objects': 2749}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:32.344 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:30:32.344 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'11621", 'reported_seq': 17779, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:30:27.121236+0000', 'last_change': '2024-09-17T15:28:22.509818+0000', 'last_active': '2024-09-17T15:30:27.121236+0000', 'last_peered': '2024-09-17T15:30:27.121236+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:30:27.121236+0000', 'last_undegraded': '2024-09-17T15:30:27.121236+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "34'1600", 'ondisk_log_start': "34'1600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36910022, 'num_objects': 9013, 'num_object_clones': 0, 'num_object_copies': 18026, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1104, 'num_objects_unfound': 0, 'num_objects_dirty': 9013, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8063, 'num_write_kb': 27056, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3007, 'num_bytes_recovered': 12304867, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7909}, {'shards': '0,2', 'objects': 1104}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:32.344 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:30:32.988 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:30:32.989 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41077/290294 objects degraded (14.150%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":369,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":145147,"data_bytes":594473737,"bytes_used":2254983168,"bytes_avail":287655309312,"bytes_total":289910292480,"degraded_objects":41077,"degraded_total":290294,"degraded_ratio":0.14150137446864214,"misplaced_objects":1104,"misplaced_total":290294,"misplaced_ratio":0.0038030410549305181,"recovering_objects_per_sec":30,"recovering_bytes_per_sec":108512,"recovering_keys_per_sec":0,"num_objects_recovered":364,"num_bytes_recovered":1302528,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":387},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T15:27:19.616376+0000","services":{}},"progress_events":{}} 2024-09-17T15:30:33.015 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:30:33.015 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:30:33.556 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:30:33.559 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:30:33.590 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2024-09-17T15:30:31.677014+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594473737,"num_objects":145147,"num_object_clones":0,"num_object_copies":290294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41077,"num_objects_misplaced":1104,"num_objects_unfound":0,"num_objects_dirty":145147,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":147844,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7280,"num_bytes_recovered":28976089,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160810,"ondisk_log_size":160810,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2202132,"kb_used_data":1246372,"kb_used_omap":13,"kb_used_meta":955698,"kb_avail":280913388,"statfs":{"total":289910292480,"available":287655309312,"internally_reserved":0,"allocated":1276284928,"data_stored":1275402778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":978635775},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,4,7,6,4,2,2,5,0,0,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19030016,"num_objects":-4646,"num_object_clones":0,"num_object_copies":-9292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-46,"num_objects_misplaced":-318,"num_objects_unfound":0,"num_objects_dirty":-4646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4646,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":364,"num_bytes_recovered":1302528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003529"},"pg_stats":[{"pgid":"2.f","version":"43'11637","reported_seq":11765,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:27.078828+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:30:27.078828+0000","last_peered":"2024-09-17T15:30:27.078828+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:30:27.078828+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'1600","ondisk_log_start":"34'1600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36909056,"num_objects":9011,"num_object_clones":0,"num_object_copies":18022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4045,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11637,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4966},{"shards":"0,2","objects":2757}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'11768","reported_seq":11871,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:27.504819+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:30:27.504819+0000","last_peered":"2024-09-17T15:30:27.504819+0000","last_clean":"2024-09-17T15:30:27.504819+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:30:27.504819+0000","last_undegraded":"2024-09-17T15:30:27.504819+0000","last_fullsized":"2024-09-17T15:30:27.504819+0000","mapping_epoch":33,"log_start":"34'1700","ondisk_log_start":"34'1700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37536710,"num_objects":9166,"num_object_clones":0,"num_object_copies":18332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9166,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10125,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'11853","reported_seq":12010,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:27.476781+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:30:27.476781+0000","last_peered":"2024-09-17T15:30:27.476781+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:30:27.476781+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'1800","ondisk_log_start":"34'1800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37191704,"num_objects":9081,"num_object_clones":0,"num_object_copies":18162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9081,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10010,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2784},{"shards":"2","objects":4928}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'11758","reported_seq":11828,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:27.533291+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:30:27.533291+0000","last_peered":"2024-09-17T15:30:27.533291+0000","last_clean":"2024-09-17T15:30:27.533291+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:30:27.533291+0000","last_undegraded":"2024-09-17T15:30:27.533291+0000","last_fullsized":"2024-09-17T15:30:27.533291+0000","mapping_epoch":33,"log_start":"34'1700","ondisk_log_start":"34'1700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37732352,"num_objects":9212,"num_object_clones":0,"num_object_copies":18424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11758,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'11615","reported_seq":11744,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:27.102324+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:30:27.102324+0000","last_peered":"2024-09-17T15:30:27.102324+0000","last_clean":"2024-09-17T15:30:27.102324+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:30:27.102324+0000","last_undegraded":"2024-09-17T15:30:27.102324+0000","last_fullsized":"2024-09-17T15:30:27.102324+0000","mapping_epoch":32,"log_start":"34'1600","ondisk_log_start":"34'1600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37020131,"num_objects":9039,"num_object_clones":0,"num_object_copies":18078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9039,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8597,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'12005","reported_seq":12178,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:27.025316+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:30:27.025316+0000","last_peered":"2024-09-17T15:30:27.025316+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:30:27.025316+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'2000","ondisk_log_start":"34'2000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37872099,"num_objects":9247,"num_object_clones":0,"num_object_copies":18494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4252,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9247,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8504,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4996},{"shards":"0,2","objects":2891}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'11736","reported_seq":11928,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:27.051705+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:30:27.051705+0000","last_peered":"2024-09-17T15:30:27.051705+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:30:27.051705+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'1700","ondisk_log_start":"34'1700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36996038,"num_objects":9034,"num_object_clones":0,"num_object_copies":18068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4147,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9034,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8505,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4887},{"shards":"0,2","objects":2825}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'11795","reported_seq":11923,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:27.101975+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:30:27.101975+0000","last_peered":"2024-09-17T15:30:27.101975+0000","last_clean":"2024-09-17T15:30:27.101975+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:30:27.101975+0000","last_undegraded":"2024-09-17T15:30:27.101975+0000","last_fullsized":"2024-09-17T15:30:27.101975+0000","mapping_epoch":32,"log_start":"34'1700","ondisk_log_start":"34'1700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37359616,"num_objects":9121,"num_object_clones":0,"num_object_copies":18242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8749,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804377+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804377+0000","last_peered":"2024-09-17T15:28:23.804377+0000","last_clean":"2024-09-17T15:28:23.804377+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804377+0000","last_undegraded":"2024-09-17T15:28:23.804377+0000","last_fullsized":"2024-09-17T15:28:23.804377+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'11866","reported_seq":20146,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:31.416759+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:30:31.416759+0000","last_peered":"2024-09-17T15:30:31.416759+0000","last_clean":"2024-09-17T15:30:31.416759+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:30:31.416759+0000","last_undegraded":"2024-09-17T15:30:31.416759+0000","last_fullsized":"2024-09-17T15:30:31.416759+0000","mapping_epoch":42,"log_start":"34'1800","ondisk_log_start":"34'1800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36663779,"num_objects":8952,"num_object_clones":0,"num_object_copies":17904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8952,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9866,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804747+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804747+0000","last_peered":"2024-09-17T15:28:23.804747+0000","last_clean":"2024-09-17T15:28:23.804747+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804747+0000","last_undegraded":"2024-09-17T15:28:23.804747+0000","last_fullsized":"2024-09-17T15:28:23.804747+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'11559","reported_seq":11718,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:27.067285+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:30:27.067285+0000","last_peered":"2024-09-17T15:30:27.067285+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:30:27.067285+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'1500","ondisk_log_start":"34'1500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36794368,"num_objects":8983,"num_object_clones":0,"num_object_copies":17966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3998,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8155,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4987},{"shards":"0,2","objects":2726}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'11600","reported_seq":11751,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:27.526047+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:30:27.526047+0000","last_peered":"2024-09-17T15:30:27.526047+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:30:27.526047+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'1500","ondisk_log_start":"34'1500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36532707,"num_objects":8920,"num_object_clones":0,"num_object_copies":17840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4063,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8920,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9495,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2739},{"shards":"2","objects":4858}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":78,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.244375+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:25.244375+0000","last_peered":"2024-09-17T15:28:25.244375+0000","last_clean":"2024-09-17T15:28:25.244375+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:25.244375+0000","last_undegraded":"2024-09-17T15:28:25.244375+0000","last_fullsized":"2024-09-17T15:28:25.244375+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'11668","reported_seq":11819,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:27.462513+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:30:27.462513+0000","last_peered":"2024-09-17T15:30:27.462513+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:30:27.462513+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'1600","ondisk_log_start":"34'1600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37142528,"num_objects":9068,"num_object_clones":0,"num_object_copies":18136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9430,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2840},{"shards":"2","objects":4947}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'11808","reported_seq":11987,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:27.009291+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:30:27.009291+0000","last_peered":"2024-09-17T15:30:27.009291+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:30:27.009291+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'1800","ondisk_log_start":"34'1800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37511168,"num_objects":9158,"num_object_clones":0,"num_object_copies":18316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4117,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8356,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5042},{"shards":"0,2","objects":2814}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.244410+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:25.244410+0000","last_peered":"2024-09-17T15:28:25.244410+0000","last_clean":"2024-09-17T15:28:25.244410+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:25.244410+0000","last_undegraded":"2024-09-17T15:28:25.244410+0000","last_fullsized":"2024-09-17T15:28:25.244410+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'11759","reported_seq":11935,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:27.142351+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:30:27.142351+0000","last_peered":"2024-09-17T15:30:27.142351+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:30:27.142351+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'1700","ondisk_log_start":"34'1700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36917248,"num_objects":9013,"num_object_clones":0,"num_object_copies":18026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4143,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9013,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8046,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4873},{"shards":"0,2","objects":2788}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805107+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805107+0000","last_peered":"2024-09-17T15:28:23.805107+0000","last_clean":"2024-09-17T15:28:23.805107+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805107+0000","last_undegraded":"2024-09-17T15:28:23.805107+0000","last_fullsized":"2024-09-17T15:28:23.805107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'11759","reported_seq":11951,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:27.079184+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:30:27.079184+0000","last_peered":"2024-09-17T15:30:27.079184+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:30:27.079184+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'1700","ondisk_log_start":"34'1700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37384192,"num_objects":9127,"num_object_clones":0,"num_object_copies":18254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4035,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8546,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5094},{"shards":"0,2","objects":2749}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.244422+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:25.244422+0000","last_peered":"2024-09-17T15:28:25.244422+0000","last_clean":"2024-09-17T15:28:25.244422+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:25.244422+0000","last_undegraded":"2024-09-17T15:28:25.244422+0000","last_fullsized":"2024-09-17T15:28:25.244422+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'11621","reported_seq":17779,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:30:27.121236+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:30:27.121236+0000","last_peered":"2024-09-17T15:30:27.121236+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:30:27.121236+0000","last_undegraded":"2024-09-17T15:30:27.121236+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"34'1600","ondisk_log_start":"34'1600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36910022,"num_objects":9013,"num_object_clones":0,"num_object_copies":18026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1104,"num_objects_unfound":0,"num_objects_dirty":9013,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8063,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3007,"num_bytes_recovered":12304867,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7909},{"shards":"0,2","objects":1104}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":594473718,"num_objects":145145,"num_object_clones":0,"num_object_copies":290290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41077,"num_objects_misplaced":1104,"num_objects_unfound":0,"num_objects_dirty":145145,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":147842,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7279,"num_bytes_recovered":28976070,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1273384960,"data_stored":1273232296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160807,"ondisk_log_size":160807,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051649,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":721816,"kb_used_data":423560,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93650024,"statfs":{"total":96636764160,"available":95897624576,"internally_reserved":0,"allocated":433725440,"data_stored":433444794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":305393324},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,4,2,1,2,2,0,0,0,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724595,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":615868,"kb_used_data":319020,"kb_used_omap":4,"kb_used_meta":296827,"kb_avail":93755972,"statfs":{"total":96636764160,"available":96006115328,"internally_reserved":0,"allocated":326676480,"data_stored":326355627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":303951531},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247877,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":864448,"kb_used_data":503792,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93507392,"statfs":{"total":96636764160,"available":95751569408,"internally_reserved":0,"allocated":515883008,"data_stored":515602357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":369290920},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,2,2,1,0,0,5],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":514392064,"data_stored":514355934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":326266880,"data_stored":326190548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":432726016,"data_stored":432685814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:30:36.592 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:30:37.155 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:30:37.155 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:30:37.180 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":222,"stamp":"2024-09-17T15:30:35.678208+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":585880329,"num_objects":143049,"num_object_clones":0,"num_object_copies":286098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41048,"num_objects_misplaced":913,"num_objects_unfound":0,"num_objects_dirty":143049,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":149942,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7500,"num_bytes_recovered":29758425,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161008,"ondisk_log_size":161008,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2199108,"kb_used_data":1235028,"kb_used_omap":13,"kb_used_meta":964018,"kb_avail":280916412,"statfs":{"total":289910292480,"available":287658405888,"internally_reserved":0,"allocated":1264668672,"data_stored":1263786522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":987155455},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,4,7,5,1,0,0,0,8,0,0,2],"upper_bound":16384},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18313216,"num_objects":-4471,"num_object_clones":0,"num_object_copies":-8942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-60,"num_objects_misplaced":-362,"num_objects_unfound":0,"num_objects_dirty":-4471,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4471,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":422,"num_bytes_recovered":1482752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003495"},"pg_stats":[{"pgid":"2.f","version":"43'11763","reported_seq":11905,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:32.155538+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:30:32.155538+0000","last_peered":"2024-09-17T15:30:32.155538+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:30:32.155538+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'1700","ondisk_log_start":"34'1700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36392960,"num_objects":8885,"num_object_clones":0,"num_object_copies":17770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4038,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11763,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4847},{"shards":"0,2","objects":2631}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'11894","reported_seq":11997,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:32.387946+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:30:32.387946+0000","last_peered":"2024-09-17T15:30:32.387946+0000","last_clean":"2024-09-17T15:30:32.387946+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:30:32.387946+0000","last_undegraded":"2024-09-17T15:30:32.387946+0000","last_fullsized":"2024-09-17T15:30:32.387946+0000","mapping_epoch":33,"log_start":"34'1800","ondisk_log_start":"34'1800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37020614,"num_objects":9040,"num_object_clones":0,"num_object_copies":18080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9040,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10251,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'11980","reported_seq":12137,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:32.424120+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:30:32.424120+0000","last_peered":"2024-09-17T15:30:32.424120+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:30:32.424120+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'1900","ondisk_log_start":"34'1900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36671512,"num_objects":8954,"num_object_clones":0,"num_object_copies":17908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8954,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10137,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2657},{"shards":"2","objects":4801}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'11903","reported_seq":11973,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:32.410508+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:30:32.410508+0000","last_peered":"2024-09-17T15:30:32.410508+0000","last_clean":"2024-09-17T15:30:32.410508+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:30:32.410508+0000","last_undegraded":"2024-09-17T15:30:32.410508+0000","last_fullsized":"2024-09-17T15:30:32.410508+0000","mapping_epoch":33,"log_start":"34'1900","ondisk_log_start":"34'1900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37138432,"num_objects":9067,"num_object_clones":0,"num_object_copies":18134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11903,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'11761","reported_seq":11890,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:32.177160+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:30:32.177160+0000","last_peered":"2024-09-17T15:30:32.177160+0000","last_clean":"2024-09-17T15:30:32.177160+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:30:32.177160+0000","last_undegraded":"2024-09-17T15:30:32.177160+0000","last_fullsized":"2024-09-17T15:30:32.177160+0000","mapping_epoch":32,"log_start":"34'1700","ondisk_log_start":"34'1700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36422115,"num_objects":8893,"num_object_clones":0,"num_object_copies":17786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8893,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8743,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'12151","reported_seq":12330,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:32.166587+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:30:32.166587+0000","last_peered":"2024-09-17T15:30:32.166587+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:30:32.166587+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'2100","ondisk_log_start":"34'2100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37274083,"num_objects":9101,"num_object_clones":0,"num_object_copies":18202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4249,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9101,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8650,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4853},{"shards":"0,2","objects":2745}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'11893","reported_seq":12100,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:32.144325+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:30:32.144325+0000","last_peered":"2024-09-17T15:30:32.144325+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:30:32.144325+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'1800","ondisk_log_start":"34'1800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36352966,"num_objects":8877,"num_object_clones":0,"num_object_copies":17754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4140,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8877,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8662,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4737},{"shards":"0,2","objects":2668}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'11939","reported_seq":12067,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:32.128094+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:30:32.128094+0000","last_peered":"2024-09-17T15:30:32.128094+0000","last_clean":"2024-09-17T15:30:32.128094+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:30:32.128094+0000","last_undegraded":"2024-09-17T15:30:32.128094+0000","last_fullsized":"2024-09-17T15:30:32.128094+0000","mapping_epoch":32,"log_start":"34'1900","ondisk_log_start":"34'1900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36769792,"num_objects":8977,"num_object_clones":0,"num_object_copies":17954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8893,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804377+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804377+0000","last_peered":"2024-09-17T15:28:23.804377+0000","last_clean":"2024-09-17T15:28:23.804377+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804377+0000","last_undegraded":"2024-09-17T15:28:23.804377+0000","last_fullsized":"2024-09-17T15:28:23.804377+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'11866","reported_seq":20146,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:31.416759+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:30:31.416759+0000","last_peered":"2024-09-17T15:30:31.416759+0000","last_clean":"2024-09-17T15:30:31.416759+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:30:31.416759+0000","last_undegraded":"2024-09-17T15:30:31.416759+0000","last_fullsized":"2024-09-17T15:30:31.416759+0000","mapping_epoch":42,"log_start":"34'1800","ondisk_log_start":"34'1800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36663779,"num_objects":8952,"num_object_clones":0,"num_object_copies":17904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8952,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9866,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804747+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804747+0000","last_peered":"2024-09-17T15:28:23.804747+0000","last_clean":"2024-09-17T15:28:23.804747+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804747+0000","last_undegraded":"2024-09-17T15:28:23.804747+0000","last_fullsized":"2024-09-17T15:28:23.804747+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'11699","reported_seq":11868,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:32.144056+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:30:32.144056+0000","last_peered":"2024-09-17T15:30:32.144056+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:30:32.144056+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'1600","ondisk_log_start":"34'1600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36220928,"num_objects":8843,"num_object_clones":0,"num_object_copies":17686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3993,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8295,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4852},{"shards":"0,2","objects":2586}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'11721","reported_seq":11872,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:32.451221+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:30:32.451221+0000","last_peered":"2024-09-17T15:30:32.451221+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:30:32.451221+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'1700","ondisk_log_start":"34'1700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36037091,"num_objects":8799,"num_object_clones":0,"num_object_copies":17598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4063,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8799,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9616,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2618},{"shards":"2","objects":4737}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":78,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.244375+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:25.244375+0000","last_peered":"2024-09-17T15:28:25.244375+0000","last_clean":"2024-09-17T15:28:25.244375+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:25.244375+0000","last_undegraded":"2024-09-17T15:28:25.244375+0000","last_fullsized":"2024-09-17T15:28:25.244375+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'11820","reported_seq":11971,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:32.446482+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:30:32.446482+0000","last_peered":"2024-09-17T15:30:32.446482+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:30:32.446482+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'1800","ondisk_log_start":"34'1800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36519936,"num_objects":8916,"num_object_clones":0,"num_object_copies":17832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9582,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2688},{"shards":"2","objects":4795}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'11970","reported_seq":12158,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:32.195446+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:30:32.195446+0000","last_peered":"2024-09-17T15:30:32.195446+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:30:32.195446+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'1900","ondisk_log_start":"34'1900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36847616,"num_objects":8996,"num_object_clones":0,"num_object_copies":17992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4113,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8996,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8518,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4884},{"shards":"0,2","objects":2652}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.244410+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:25.244410+0000","last_peered":"2024-09-17T15:28:25.244410+0000","last_clean":"2024-09-17T15:28:25.244410+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:25.244410+0000","last_undegraded":"2024-09-17T15:28:25.244410+0000","last_fullsized":"2024-09-17T15:28:25.244410+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'11895","reported_seq":12077,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:32.035174+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:30:32.035174+0000","last_peered":"2024-09-17T15:30:32.035174+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:30:32.035174+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'1800","ondisk_log_start":"34'1800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36360192,"num_objects":8877,"num_object_clones":0,"num_object_copies":17754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4140,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8877,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8182,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4740},{"shards":"0,2","objects":2652}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805107+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805107+0000","last_peered":"2024-09-17T15:28:23.805107+0000","last_clean":"2024-09-17T15:28:23.805107+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805107+0000","last_undegraded":"2024-09-17T15:28:23.805107+0000","last_fullsized":"2024-09-17T15:28:23.805107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'11896","reported_seq":12088,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:32.169027+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:30:32.169027+0000","last_peered":"2024-09-17T15:30:32.169027+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:30:32.169027+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'1800","ondisk_log_start":"34'1800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36823040,"num_objects":8990,"num_object_clones":0,"num_object_copies":17980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4035,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8683,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4957},{"shards":"0,2","objects":2612}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.244422+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:25.244422+0000","last_peered":"2024-09-17T15:28:25.244422+0000","last_clean":"2024-09-17T15:28:25.244422+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:25.244422+0000","last_undegraded":"2024-09-17T15:28:25.244422+0000","last_fullsized":"2024-09-17T15:28:25.244422+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'11754","reported_seq":18293,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:30:32.181098+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:30:32.181098+0000","last_peered":"2024-09-17T15:30:32.181098+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:30:32.181098+0000","last_undegraded":"2024-09-17T15:30:32.181098+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"34'1700","ondisk_log_start":"34'1700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36365254,"num_objects":8880,"num_object_clones":0,"num_object_copies":17760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":913,"num_objects_unfound":0,"num_objects_dirty":8880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8196,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3198,"num_bytes_recovered":13087203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7967},{"shards":"0,2","objects":913}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":585880310,"num_objects":143047,"num_object_clones":0,"num_object_copies":286094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41048,"num_objects_misplaced":913,"num_objects_unfound":0,"num_objects_dirty":143047,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":149940,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7499,"num_bytes_recovered":29758406,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1261903872,"data_stored":1261751208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161005,"ondisk_log_size":161005,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051650,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":721496,"kb_used_data":419080,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93650344,"statfs":{"total":96636764160,"available":95897952256,"internally_reserved":0,"allocated":429137920,"data_stored":428857274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":309653164},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,0,2,0,0,0,0,0,0,0,2],"upper_bound":16384},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724595,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":615868,"kb_used_data":319020,"kb_used_omap":4,"kb_used_meta":296827,"kb_avail":93755972,"statfs":{"total":96636764160,"available":96006115328,"internally_reserved":0,"allocated":326676480,"data_stored":326355627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":303951531},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247878,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":861744,"kb_used_data":496928,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93510096,"statfs":{"total":96636764160,"available":95754338304,"internally_reserved":0,"allocated":508854272,"data_stored":508573621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":373550760},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,5,1,1,0,0,0,8],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":507441152,"data_stored":507405022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":326266880,"data_stored":326190548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":428195840,"data_stored":428155638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:30:37.181 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:30:37.704 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:30:37.705 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:30:37.735 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":223,"stamp":"2024-09-17T15:30:37.678804+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":576496393,"num_objects":140758,"num_object_clones":0,"num_object_copies":281516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41032,"num_objects_misplaced":744,"num_objects_unfound":0,"num_objects_dirty":140758,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":152233,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7685,"num_bytes_recovered":30450649,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160799,"ondisk_log_size":160799,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2170416,"kb_used_data":1222972,"kb_used_omap":13,"kb_used_meta":947378,"kb_avail":280945104,"statfs":{"total":289910292480,"available":287687786496,"internally_reserved":0,"allocated":1252323328,"data_stored":1251441178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":970116095},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,11,4,10,5,0,0,0,0,0,1,2],"upper_bound":16384},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27697152,"num_objects":-6762,"num_object_clones":0,"num_object_copies":-13524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-76,"num_objects_misplaced":-531,"num_objects_unfound":0,"num_objects_dirty":-6762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6762,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":607,"num_bytes_recovered":2174976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003392"},"pg_stats":[{"pgid":"2.f","version":"43'11878","reported_seq":12020,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:37.107753+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:30:37.107753+0000","last_peered":"2024-09-17T15:30:37.107753+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:30:37.107753+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'1800","ondisk_log_start":"34'1800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35921920,"num_objects":8770,"num_object_clones":0,"num_object_copies":17540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4038,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11878,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4732},{"shards":"0,2","objects":2516}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'12034","reported_seq":12137,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:37.505035+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:30:37.505035+0000","last_peered":"2024-09-17T15:30:37.505035+0000","last_clean":"2024-09-17T15:30:37.505035+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:30:37.505035+0000","last_undegraded":"2024-09-17T15:30:37.505035+0000","last_fullsized":"2024-09-17T15:30:37.505035+0000","mapping_epoch":33,"log_start":"34'2000","ondisk_log_start":"34'2000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36447174,"num_objects":8900,"num_object_clones":0,"num_object_copies":17800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8900,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10391,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'12136","reported_seq":12293,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:37.491559+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:30:37.491559+0000","last_peered":"2024-09-17T15:30:37.491559+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:30:37.491559+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'2100","ondisk_log_start":"34'2100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36032536,"num_objects":8798,"num_object_clones":0,"num_object_copies":17596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8798,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10293,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2501},{"shards":"2","objects":4645}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'12053","reported_seq":12122,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:37.531979+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:30:37.531979+0000","last_peered":"2024-09-17T15:30:37.531979+0000","last_clean":"2024-09-17T15:30:37.531979+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:30:37.531979+0000","last_undegraded":"2024-09-17T15:30:37.531979+0000","last_fullsized":"2024-09-17T15:30:37.531979+0000","mapping_epoch":33,"log_start":"34'2000","ondisk_log_start":"34'2000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36524032,"num_objects":8917,"num_object_clones":0,"num_object_copies":17834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12053,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'11908","reported_seq":12038,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:37.187069+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:30:37.187069+0000","last_peered":"2024-09-17T15:30:37.187069+0000","last_clean":"2024-09-17T15:30:37.187069+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:30:37.187069+0000","last_undegraded":"2024-09-17T15:30:37.187069+0000","last_fullsized":"2024-09-17T15:30:37.187069+0000","mapping_epoch":32,"log_start":"34'1900","ondisk_log_start":"34'1900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35820003,"num_objects":8746,"num_object_clones":0,"num_object_copies":17492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8746,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8890,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'12301","reported_seq":12494,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:37.165504+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:30:37.165504+0000","last_peered":"2024-09-17T15:30:37.165504+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:30:37.165504+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'2300","ondisk_log_start":"34'2300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36659683,"num_objects":8951,"num_object_clones":0,"num_object_copies":17902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4242,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8951,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8800,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4710},{"shards":"0,2","objects":2595}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'12049","reported_seq":12258,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:37.159528+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:30:37.159528+0000","last_peered":"2024-09-17T15:30:37.159528+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:30:37.159528+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'2000","ondisk_log_start":"34'2000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35713990,"num_objects":8721,"num_object_clones":0,"num_object_copies":17442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8721,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8818,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4582},{"shards":"0,2","objects":2512}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'12088","reported_seq":12216,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:37.151150+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:30:37.151150+0000","last_peered":"2024-09-17T15:30:37.151150+0000","last_clean":"2024-09-17T15:30:37.151150+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:30:37.151150+0000","last_undegraded":"2024-09-17T15:30:37.151150+0000","last_fullsized":"2024-09-17T15:30:37.151150+0000","mapping_epoch":32,"log_start":"34'2000","ondisk_log_start":"34'2000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36159488,"num_objects":8828,"num_object_clones":0,"num_object_copies":17656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9042,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804377+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804377+0000","last_peered":"2024-09-17T15:28:23.804377+0000","last_clean":"2024-09-17T15:28:23.804377+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804377+0000","last_undegraded":"2024-09-17T15:28:23.804377+0000","last_fullsized":"2024-09-17T15:28:23.804377+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'11998","reported_seq":20281,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:36.405989+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:30:36.405989+0000","last_peered":"2024-09-17T15:30:36.405989+0000","last_clean":"2024-09-17T15:30:36.405989+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:30:36.405989+0000","last_undegraded":"2024-09-17T15:30:36.405989+0000","last_fullsized":"2024-09-17T15:30:36.405989+0000","mapping_epoch":42,"log_start":"34'1900","ondisk_log_start":"34'1900","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36123107,"num_objects":8820,"num_object_clones":0,"num_object_copies":17640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8820,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9998,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804747+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804747+0000","last_peered":"2024-09-17T15:28:23.804747+0000","last_clean":"2024-09-17T15:28:23.804747+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804747+0000","last_undegraded":"2024-09-17T15:28:23.804747+0000","last_fullsized":"2024-09-17T15:28:23.804747+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'11836","reported_seq":12009,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:37.197515+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:30:37.197515+0000","last_peered":"2024-09-17T15:30:37.197515+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:30:37.197515+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'1800","ondisk_log_start":"34'1800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35659776,"num_objects":8706,"num_object_clones":0,"num_object_copies":17412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3991,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8432,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4717},{"shards":"0,2","objects":2449}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'11856","reported_seq":12007,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:37.477790+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:30:37.477790+0000","last_peered":"2024-09-17T15:30:37.477790+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:30:37.477790+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'1800","ondisk_log_start":"34'1800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35484131,"num_objects":8664,"num_object_clones":0,"num_object_copies":17328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4063,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8664,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9751,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2483},{"shards":"2","objects":4602}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":78,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.244375+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:25.244375+0000","last_peered":"2024-09-17T15:28:25.244375+0000","last_clean":"2024-09-17T15:28:25.244375+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:25.244375+0000","last_undegraded":"2024-09-17T15:28:25.244375+0000","last_fullsized":"2024-09-17T15:28:25.244375+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'11983","reported_seq":12134,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:37.473502+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:30:37.473502+0000","last_peered":"2024-09-17T15:30:37.473502+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:30:37.473502+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'1900","ondisk_log_start":"34'1900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35852288,"num_objects":8753,"num_object_clones":0,"num_object_copies":17506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9745,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2525},{"shards":"2","objects":4632}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'12093","reported_seq":12287,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:36.998506+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:30:36.998506+0000","last_peered":"2024-09-17T15:30:36.998506+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:30:36.998506+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'2000","ondisk_log_start":"34'2000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36343808,"num_objects":8873,"num_object_clones":0,"num_object_copies":17746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4110,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8641,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4764},{"shards":"0,2","objects":2529}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.244410+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:25.244410+0000","last_peered":"2024-09-17T15:28:25.244410+0000","last_clean":"2024-09-17T15:28:25.244410+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:25.244410+0000","last_undegraded":"2024-09-17T15:28:25.244410+0000","last_fullsized":"2024-09-17T15:28:25.244410+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'12035","reported_seq":12221,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:37.175243+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:30:37.175243+0000","last_peered":"2024-09-17T15:30:37.175243+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:30:37.175243+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'2000","ondisk_log_start":"34'2000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35786752,"num_objects":8737,"num_object_clones":0,"num_object_copies":17474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4138,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8322,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4602},{"shards":"0,2","objects":2512}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805107+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805107+0000","last_peered":"2024-09-17T15:28:23.805107+0000","last_clean":"2024-09-17T15:28:23.805107+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805107+0000","last_undegraded":"2024-09-17T15:28:23.805107+0000","last_fullsized":"2024-09-17T15:28:23.805107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'12046","reported_seq":12240,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:37.137356+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:30:37.137356+0000","last_peered":"2024-09-17T15:30:37.137356+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:30:37.137356+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'2000","ondisk_log_start":"34'2000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36208640,"num_objects":8840,"num_object_clones":0,"num_object_copies":17680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8833,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4808},{"shards":"0,2","objects":2462}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.244422+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:25.244422+0000","last_peered":"2024-09-17T15:28:25.244422+0000","last_clean":"2024-09-17T15:28:25.244422+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:25.244422+0000","last_undegraded":"2024-09-17T15:28:25.244422+0000","last_fullsized":"2024-09-17T15:28:25.244422+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'11902","reported_seq":18780,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:30:37.220505+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:30:37.220505+0000","last_peered":"2024-09-17T15:30:37.220505+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:30:37.220505+0000","last_undegraded":"2024-09-17T15:30:37.220505+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"34'1900","ondisk_log_start":"34'1900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35759046,"num_objects":8732,"num_object_clones":0,"num_object_copies":17464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":744,"num_objects_unfound":0,"num_objects_dirty":8732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8344,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":13779427,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7988},{"shards":"0,2","objects":744}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":576496374,"num_objects":140756,"num_object_clones":0,"num_object_copies":281512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41032,"num_objects_misplaced":744,"num_objects_unfound":0,"num_objects_dirty":140756,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":152231,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7684,"num_bytes_recovered":30450630,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1249656832,"data_stored":1249504168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160796,"ondisk_log_size":160796,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051651,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":721016,"kb_used_data":414436,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93650824,"statfs":{"total":96636764160,"available":95898443776,"internally_reserved":0,"allocated":424382464,"data_stored":424101818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":313913004},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,0,2,0,0,0,0,0,0,0,2],"upper_bound":16384},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724596,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":602880,"kb_used_data":318512,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93768960,"statfs":{"total":96636764160,"available":96019415040,"internally_reserved":0,"allocated":326156288,"data_stored":325835435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":291172011},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247879,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":846520,"kb_used_data":490024,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93525320,"statfs":{"total":96636764160,"available":95769927680,"internally_reserved":0,"allocated":501784576,"data_stored":501503925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":365031080},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,7,2,4,5,0,0,0,0,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":500461568,"data_stored":500425438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":325791744,"data_stored":325715412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":423403520,"data_stored":423363318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:30:37.736 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:30:37.736 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'11878", 'reported_seq': 12020, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:37.107753+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:30:37.107753+0000', 'last_peered': '2024-09-17T15:30:37.107753+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:30:37.107753+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "34'1800", 'ondisk_log_start': "34'1800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35921920, 'num_objects': 8770, 'num_object_clones': 0, 'num_object_copies': 17540, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4038, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8770, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11878, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 32, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4732}, {'shards': '0,2', 'objects': 2516}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:37.736 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:30:37.736 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'12136", 'reported_seq': 12293, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:37.491559+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:30:37.491559+0000', 'last_peered': '2024-09-17T15:30:37.491559+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:30:37.491559+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "34'2100", 'ondisk_log_start': "34'2100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36032536, 'num_objects': 8798, 'num_object_clones': 0, 'num_object_copies': 17596, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4154, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8798, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10293, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 15, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2501}, {'shards': '2', 'objects': 4645}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:30:37.737 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:30:37.737 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'12301", 'reported_seq': 12494, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:37.165504+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:30:37.165504+0000', 'last_peered': '2024-09-17T15:30:37.165504+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:30:37.165504+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "34'2300", 'ondisk_log_start': "34'2300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36659683, 'num_objects': 8951, 'num_object_clones': 0, 'num_object_copies': 17902, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4242, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8951, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8800, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4710}, {'shards': '0,2', 'objects': 2595}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:37.737 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:30:37.737 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'12049", 'reported_seq': 12258, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:37.159528+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:30:37.159528+0000', 'last_peered': '2024-09-17T15:30:37.159528+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:30:37.159528+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "34'2000", 'ondisk_log_start': "34'2000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35713990, 'num_objects': 8721, 'num_object_clones': 0, 'num_object_copies': 17442, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8721, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 8818, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 35, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4582}, {'shards': '0,2', 'objects': 2512}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:37.737 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:30:37.737 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'11836", 'reported_seq': 12009, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:37.197515+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:30:37.197515+0000', 'last_peered': '2024-09-17T15:30:37.197515+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:30:37.197515+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "34'1800", 'ondisk_log_start': "34'1800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35659776, 'num_objects': 8706, 'num_object_clones': 0, 'num_object_copies': 17412, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3991, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8706, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8432, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 21, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4717}, {'shards': '0,2', 'objects': 2449}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:37.737 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:30:37.737 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'11856", 'reported_seq': 12007, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:37.477790+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:30:37.477790+0000', 'last_peered': '2024-09-17T15:30:37.477790+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:30:37.477790+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "34'1800", 'ondisk_log_start': "34'1800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35484131, 'num_objects': 8664, 'num_object_clones': 0, 'num_object_copies': 17328, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4063, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8664, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9751, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 15, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2483}, {'shards': '2', 'objects': 4602}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:30:37.737 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:30:37.737 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'11983", 'reported_seq': 12134, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:37.473502+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:30:37.473502+0000', 'last_peered': '2024-09-17T15:30:37.473502+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:30:37.473502+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "34'1900", 'ondisk_log_start': "34'1900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35852288, 'num_objects': 8753, 'num_object_clones': 0, 'num_object_copies': 17506, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4123, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8753, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9745, 'num_write_kb': 32532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 16, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2525}, {'shards': '2', 'objects': 4632}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:30:37.738 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:30:37.738 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'12093", 'reported_seq': 12287, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:36.998506+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:30:36.998506+0000', 'last_peered': '2024-09-17T15:30:36.998506+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:30:36.998506+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "34'2000", 'ondisk_log_start': "34'2000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36343808, 'num_objects': 8873, 'num_object_clones': 0, 'num_object_copies': 17746, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4110, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8873, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8641, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4764}, {'shards': '0,2', 'objects': 2529}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:37.738 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:30:37.738 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'12035", 'reported_seq': 12221, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:37.175243+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:30:37.175243+0000', 'last_peered': '2024-09-17T15:30:37.175243+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:30:37.175243+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "34'2000", 'ondisk_log_start': "34'2000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35786752, 'num_objects': 8737, 'num_object_clones': 0, 'num_object_copies': 17474, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4138, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8737, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8322, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 22, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4602}, {'shards': '0,2', 'objects': 2512}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:37.738 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:30:37.738 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'12046", 'reported_seq': 12240, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:37.137356+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:30:37.137356+0000', 'last_peered': '2024-09-17T15:30:37.137356+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:30:37.137356+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "34'2000", 'ondisk_log_start': "34'2000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36208640, 'num_objects': 8840, 'num_object_clones': 0, 'num_object_copies': 17680, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4034, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8840, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8833, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 31, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4808}, {'shards': '0,2', 'objects': 2462}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:37.738 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:30:37.738 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'11902", 'reported_seq': 18780, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:30:37.220505+0000', 'last_change': '2024-09-17T15:28:22.509818+0000', 'last_active': '2024-09-17T15:30:37.220505+0000', 'last_peered': '2024-09-17T15:30:37.220505+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:30:37.220505+0000', 'last_undegraded': '2024-09-17T15:30:37.220505+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "34'1900", 'ondisk_log_start': "34'1900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35759046, 'num_objects': 8732, 'num_object_clones': 0, 'num_object_copies': 17464, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 744, 'num_objects_unfound': 0, 'num_objects_dirty': 8732, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8344, 'num_write_kb': 27056, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3367, 'num_bytes_recovered': 13779427, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7988}, {'shards': '0,2', 'objects': 744}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:37.738 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:30:38.402 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:30:38.403 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41032/281516 objects degraded (14.575%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":375,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":140758,"data_bytes":576496393,"bytes_used":2222505984,"bytes_avail":287687786496,"bytes_total":289910292480,"degraded_objects":41032,"degraded_total":281516,"degraded_ratio":0.14575370494039416,"misplaced_objects":744,"misplaced_total":281516,"misplaced_ratio":0.0026428337998550704,"recovering_objects_per_sec":50,"recovering_bytes_per_sec":181196,"recovering_keys_per_sec":0,"num_objects_recovered":607,"num_bytes_recovered":2174976,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":563},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T15:27:19.616376+0000","services":{}},"progress_events":{}} 2024-09-17T15:30:38.424 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:30:38.424 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:30:38.969 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:30:38.972 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:30:38.998 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":223,"stamp":"2024-09-17T15:30:37.678804+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":576496393,"num_objects":140758,"num_object_clones":0,"num_object_copies":281516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41032,"num_objects_misplaced":744,"num_objects_unfound":0,"num_objects_dirty":140758,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":152233,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7685,"num_bytes_recovered":30450649,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160799,"ondisk_log_size":160799,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2170416,"kb_used_data":1222972,"kb_used_omap":13,"kb_used_meta":947378,"kb_avail":280945104,"statfs":{"total":289910292480,"available":287687786496,"internally_reserved":0,"allocated":1252323328,"data_stored":1251441178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":970116095},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,11,4,10,5,0,0,0,0,0,1,2],"upper_bound":16384},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27697152,"num_objects":-6762,"num_object_clones":0,"num_object_copies":-13524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-76,"num_objects_misplaced":-531,"num_objects_unfound":0,"num_objects_dirty":-6762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6762,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":607,"num_bytes_recovered":2174976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003392"},"pg_stats":[{"pgid":"2.f","version":"43'11878","reported_seq":12020,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:37.107753+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:30:37.107753+0000","last_peered":"2024-09-17T15:30:37.107753+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:30:37.107753+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'1800","ondisk_log_start":"34'1800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35921920,"num_objects":8770,"num_object_clones":0,"num_object_copies":17540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4038,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11878,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4732},{"shards":"0,2","objects":2516}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'12034","reported_seq":12137,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:37.505035+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:30:37.505035+0000","last_peered":"2024-09-17T15:30:37.505035+0000","last_clean":"2024-09-17T15:30:37.505035+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:30:37.505035+0000","last_undegraded":"2024-09-17T15:30:37.505035+0000","last_fullsized":"2024-09-17T15:30:37.505035+0000","mapping_epoch":33,"log_start":"34'2000","ondisk_log_start":"34'2000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36447174,"num_objects":8900,"num_object_clones":0,"num_object_copies":17800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8900,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10391,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'12136","reported_seq":12293,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:37.491559+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:30:37.491559+0000","last_peered":"2024-09-17T15:30:37.491559+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:30:37.491559+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'2100","ondisk_log_start":"34'2100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36032536,"num_objects":8798,"num_object_clones":0,"num_object_copies":17596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8798,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10293,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2501},{"shards":"2","objects":4645}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'12053","reported_seq":12122,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:37.531979+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:30:37.531979+0000","last_peered":"2024-09-17T15:30:37.531979+0000","last_clean":"2024-09-17T15:30:37.531979+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:30:37.531979+0000","last_undegraded":"2024-09-17T15:30:37.531979+0000","last_fullsized":"2024-09-17T15:30:37.531979+0000","mapping_epoch":33,"log_start":"34'2000","ondisk_log_start":"34'2000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36524032,"num_objects":8917,"num_object_clones":0,"num_object_copies":17834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12053,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'11908","reported_seq":12038,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:37.187069+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:30:37.187069+0000","last_peered":"2024-09-17T15:30:37.187069+0000","last_clean":"2024-09-17T15:30:37.187069+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:30:37.187069+0000","last_undegraded":"2024-09-17T15:30:37.187069+0000","last_fullsized":"2024-09-17T15:30:37.187069+0000","mapping_epoch":32,"log_start":"34'1900","ondisk_log_start":"34'1900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35820003,"num_objects":8746,"num_object_clones":0,"num_object_copies":17492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8746,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8890,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'12301","reported_seq":12494,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:37.165504+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:30:37.165504+0000","last_peered":"2024-09-17T15:30:37.165504+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:30:37.165504+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'2300","ondisk_log_start":"34'2300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36659683,"num_objects":8951,"num_object_clones":0,"num_object_copies":17902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4242,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8951,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8800,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4710},{"shards":"0,2","objects":2595}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'12049","reported_seq":12258,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:37.159528+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:30:37.159528+0000","last_peered":"2024-09-17T15:30:37.159528+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:30:37.159528+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'2000","ondisk_log_start":"34'2000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35713990,"num_objects":8721,"num_object_clones":0,"num_object_copies":17442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8721,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8818,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4582},{"shards":"0,2","objects":2512}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'12088","reported_seq":12216,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:37.151150+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:30:37.151150+0000","last_peered":"2024-09-17T15:30:37.151150+0000","last_clean":"2024-09-17T15:30:37.151150+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:30:37.151150+0000","last_undegraded":"2024-09-17T15:30:37.151150+0000","last_fullsized":"2024-09-17T15:30:37.151150+0000","mapping_epoch":32,"log_start":"34'2000","ondisk_log_start":"34'2000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36159488,"num_objects":8828,"num_object_clones":0,"num_object_copies":17656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9042,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804377+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804377+0000","last_peered":"2024-09-17T15:28:23.804377+0000","last_clean":"2024-09-17T15:28:23.804377+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804377+0000","last_undegraded":"2024-09-17T15:28:23.804377+0000","last_fullsized":"2024-09-17T15:28:23.804377+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'11998","reported_seq":20281,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:36.405989+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:30:36.405989+0000","last_peered":"2024-09-17T15:30:36.405989+0000","last_clean":"2024-09-17T15:30:36.405989+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:30:36.405989+0000","last_undegraded":"2024-09-17T15:30:36.405989+0000","last_fullsized":"2024-09-17T15:30:36.405989+0000","mapping_epoch":42,"log_start":"34'1900","ondisk_log_start":"34'1900","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36123107,"num_objects":8820,"num_object_clones":0,"num_object_copies":17640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8820,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9998,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804747+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804747+0000","last_peered":"2024-09-17T15:28:23.804747+0000","last_clean":"2024-09-17T15:28:23.804747+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804747+0000","last_undegraded":"2024-09-17T15:28:23.804747+0000","last_fullsized":"2024-09-17T15:28:23.804747+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'11836","reported_seq":12009,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:37.197515+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:30:37.197515+0000","last_peered":"2024-09-17T15:30:37.197515+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:30:37.197515+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'1800","ondisk_log_start":"34'1800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35659776,"num_objects":8706,"num_object_clones":0,"num_object_copies":17412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3991,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8432,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4717},{"shards":"0,2","objects":2449}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'11856","reported_seq":12007,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:37.477790+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:30:37.477790+0000","last_peered":"2024-09-17T15:30:37.477790+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:30:37.477790+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'1800","ondisk_log_start":"34'1800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35484131,"num_objects":8664,"num_object_clones":0,"num_object_copies":17328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4063,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8664,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9751,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2483},{"shards":"2","objects":4602}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":78,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.244375+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:25.244375+0000","last_peered":"2024-09-17T15:28:25.244375+0000","last_clean":"2024-09-17T15:28:25.244375+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:25.244375+0000","last_undegraded":"2024-09-17T15:28:25.244375+0000","last_fullsized":"2024-09-17T15:28:25.244375+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'11983","reported_seq":12134,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:37.473502+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:30:37.473502+0000","last_peered":"2024-09-17T15:30:37.473502+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:30:37.473502+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'1900","ondisk_log_start":"34'1900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35852288,"num_objects":8753,"num_object_clones":0,"num_object_copies":17506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9745,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2525},{"shards":"2","objects":4632}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'12093","reported_seq":12287,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:36.998506+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:30:36.998506+0000","last_peered":"2024-09-17T15:30:36.998506+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:30:36.998506+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'2000","ondisk_log_start":"34'2000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36343808,"num_objects":8873,"num_object_clones":0,"num_object_copies":17746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4110,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8641,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4764},{"shards":"0,2","objects":2529}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.244410+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:25.244410+0000","last_peered":"2024-09-17T15:28:25.244410+0000","last_clean":"2024-09-17T15:28:25.244410+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:25.244410+0000","last_undegraded":"2024-09-17T15:28:25.244410+0000","last_fullsized":"2024-09-17T15:28:25.244410+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'12035","reported_seq":12221,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:37.175243+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:30:37.175243+0000","last_peered":"2024-09-17T15:30:37.175243+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:30:37.175243+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'2000","ondisk_log_start":"34'2000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35786752,"num_objects":8737,"num_object_clones":0,"num_object_copies":17474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4138,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8322,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4602},{"shards":"0,2","objects":2512}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805107+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805107+0000","last_peered":"2024-09-17T15:28:23.805107+0000","last_clean":"2024-09-17T15:28:23.805107+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805107+0000","last_undegraded":"2024-09-17T15:28:23.805107+0000","last_fullsized":"2024-09-17T15:28:23.805107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'12046","reported_seq":12240,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:37.137356+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:30:37.137356+0000","last_peered":"2024-09-17T15:30:37.137356+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:30:37.137356+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'2000","ondisk_log_start":"34'2000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36208640,"num_objects":8840,"num_object_clones":0,"num_object_copies":17680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8833,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4808},{"shards":"0,2","objects":2462}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.244422+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:25.244422+0000","last_peered":"2024-09-17T15:28:25.244422+0000","last_clean":"2024-09-17T15:28:25.244422+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:25.244422+0000","last_undegraded":"2024-09-17T15:28:25.244422+0000","last_fullsized":"2024-09-17T15:28:25.244422+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.806564+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:23.806564+0000","last_peered":"2024-09-17T15:28:23.806564+0000","last_clean":"2024-09-17T15:28:23.806564+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:23.806564+0000","last_undegraded":"2024-09-17T15:28:23.806564+0000","last_fullsized":"2024-09-17T15:28:23.806564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'11902","reported_seq":18780,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:30:37.220505+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:30:37.220505+0000","last_peered":"2024-09-17T15:30:37.220505+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:30:37.220505+0000","last_undegraded":"2024-09-17T15:30:37.220505+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"34'1900","ondisk_log_start":"34'1900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35759046,"num_objects":8732,"num_object_clones":0,"num_object_copies":17464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":744,"num_objects_unfound":0,"num_objects_dirty":8732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8344,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":13779427,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7988},{"shards":"0,2","objects":744}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":576496374,"num_objects":140756,"num_object_clones":0,"num_object_copies":281512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41032,"num_objects_misplaced":744,"num_objects_unfound":0,"num_objects_dirty":140756,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":152231,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7684,"num_bytes_recovered":30450630,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1249656832,"data_stored":1249504168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160796,"ondisk_log_size":160796,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051651,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":721016,"kb_used_data":414436,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93650824,"statfs":{"total":96636764160,"available":95898443776,"internally_reserved":0,"allocated":424382464,"data_stored":424101818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":313913004},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,0,2,0,0,0,0,0,0,0,2],"upper_bound":16384},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724596,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":602880,"kb_used_data":318512,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93768960,"statfs":{"total":96636764160,"available":96019415040,"internally_reserved":0,"allocated":326156288,"data_stored":325835435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":291172011},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247879,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":846520,"kb_used_data":490024,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93525320,"statfs":{"total":96636764160,"available":95769927680,"internally_reserved":0,"allocated":501784576,"data_stored":501503925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":365031080},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,7,2,4,5,0,0,0,0,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":500461568,"data_stored":500425438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":325791744,"data_stored":325715412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":423403520,"data_stored":423363318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:30:41.999 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:30:42.554 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:30:42.554 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:30:42.577 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":225,"stamp":"2024-09-17T15:30:41.680189+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":576000777,"num_objects":140637,"num_object_clones":0,"num_object_copies":281274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41032,"num_objects_misplaced":744,"num_objects_unfound":0,"num_objects_dirty":140637,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":152354,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7685,"num_bytes_recovered":30450649,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160720,"ondisk_log_size":160720,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2178524,"kb_used_data":1222756,"kb_used_omap":13,"kb_used_meta":955698,"kb_avail":280936996,"statfs":{"total":289910292480,"available":287679483904,"internally_reserved":0,"allocated":1252102144,"data_stored":1251219994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":978635775},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,7,7,7,8,0,0,0,0,0,1,2],"upper_bound":16384},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19054592,"num_objects":-4652,"num_object_clones":0,"num_object_copies":-9304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-45,"num_objects_misplaced":-360,"num_objects_unfound":0,"num_objects_dirty":-4652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4652,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":405,"num_bytes_recovered":1474560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003794"},"pg_stats":[{"pgid":"2.f","version":"43'11878","reported_seq":12020,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:37.107753+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:30:37.107753+0000","last_peered":"2024-09-17T15:30:37.107753+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:30:37.107753+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'1800","ondisk_log_start":"34'1800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35921920,"num_objects":8770,"num_object_clones":0,"num_object_copies":17540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4038,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11878,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4732},{"shards":"0,2","objects":2516}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'12034","reported_seq":12137,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:37.505035+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:30:37.505035+0000","last_peered":"2024-09-17T15:30:37.505035+0000","last_clean":"2024-09-17T15:30:37.505035+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:30:37.505035+0000","last_undegraded":"2024-09-17T15:30:37.505035+0000","last_fullsized":"2024-09-17T15:30:37.505035+0000","mapping_epoch":33,"log_start":"34'2000","ondisk_log_start":"34'2000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36447174,"num_objects":8900,"num_object_clones":0,"num_object_copies":17800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8900,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10391,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'12136","reported_seq":12293,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:37.491559+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:30:37.491559+0000","last_peered":"2024-09-17T15:30:37.491559+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:30:37.491559+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'2100","ondisk_log_start":"34'2100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36032536,"num_objects":8798,"num_object_clones":0,"num_object_copies":17596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8798,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10293,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2501},{"shards":"2","objects":4645}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'12053","reported_seq":12122,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:37.531979+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:30:37.531979+0000","last_peered":"2024-09-17T15:30:37.531979+0000","last_clean":"2024-09-17T15:30:37.531979+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:30:37.531979+0000","last_undegraded":"2024-09-17T15:30:37.531979+0000","last_fullsized":"2024-09-17T15:30:37.531979+0000","mapping_epoch":33,"log_start":"34'2000","ondisk_log_start":"34'2000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36524032,"num_objects":8917,"num_object_clones":0,"num_object_copies":17834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12053,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'11908","reported_seq":12038,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:37.187069+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:30:37.187069+0000","last_peered":"2024-09-17T15:30:37.187069+0000","last_clean":"2024-09-17T15:30:37.187069+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:30:37.187069+0000","last_undegraded":"2024-09-17T15:30:37.187069+0000","last_fullsized":"2024-09-17T15:30:37.187069+0000","mapping_epoch":32,"log_start":"34'1900","ondisk_log_start":"34'1900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35820003,"num_objects":8746,"num_object_clones":0,"num_object_copies":17492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8746,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8890,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'12301","reported_seq":12494,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:37.165504+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:30:37.165504+0000","last_peered":"2024-09-17T15:30:37.165504+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:30:37.165504+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'2300","ondisk_log_start":"34'2300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36659683,"num_objects":8951,"num_object_clones":0,"num_object_copies":17902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4242,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8951,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8800,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4710},{"shards":"0,2","objects":2595}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'12049","reported_seq":12258,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:37.159528+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:30:37.159528+0000","last_peered":"2024-09-17T15:30:37.159528+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:30:37.159528+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'2000","ondisk_log_start":"34'2000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35713990,"num_objects":8721,"num_object_clones":0,"num_object_copies":17442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8721,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8818,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4582},{"shards":"0,2","objects":2512}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'12088","reported_seq":12216,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:37.151150+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:30:37.151150+0000","last_peered":"2024-09-17T15:30:37.151150+0000","last_clean":"2024-09-17T15:30:37.151150+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:30:37.151150+0000","last_undegraded":"2024-09-17T15:30:37.151150+0000","last_fullsized":"2024-09-17T15:30:37.151150+0000","mapping_epoch":32,"log_start":"34'2000","ondisk_log_start":"34'2000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36159488,"num_objects":8828,"num_object_clones":0,"num_object_copies":17656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9042,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804377+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804377+0000","last_peered":"2024-09-17T15:28:23.804377+0000","last_clean":"2024-09-17T15:28:23.804377+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804377+0000","last_undegraded":"2024-09-17T15:28:23.804377+0000","last_fullsized":"2024-09-17T15:28:23.804377+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'12119","reported_seq":20402,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:41.456873+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:30:41.456873+0000","last_peered":"2024-09-17T15:30:41.456873+0000","last_clean":"2024-09-17T15:30:41.456873+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:30:41.456873+0000","last_undegraded":"2024-09-17T15:30:41.456873+0000","last_fullsized":"2024-09-17T15:30:41.456873+0000","mapping_epoch":42,"log_start":"34'2100","ondisk_log_start":"34'2100","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35627491,"num_objects":8699,"num_object_clones":0,"num_object_copies":17398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8699,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10119,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804747+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804747+0000","last_peered":"2024-09-17T15:28:23.804747+0000","last_clean":"2024-09-17T15:28:23.804747+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804747+0000","last_undegraded":"2024-09-17T15:28:23.804747+0000","last_fullsized":"2024-09-17T15:28:23.804747+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'11836","reported_seq":12009,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:37.197515+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:30:37.197515+0000","last_peered":"2024-09-17T15:30:37.197515+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:30:37.197515+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'1800","ondisk_log_start":"34'1800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35659776,"num_objects":8706,"num_object_clones":0,"num_object_copies":17412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3991,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8432,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4717},{"shards":"0,2","objects":2449}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'11856","reported_seq":12007,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:37.477790+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:30:37.477790+0000","last_peered":"2024-09-17T15:30:37.477790+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:30:37.477790+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'1800","ondisk_log_start":"34'1800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35484131,"num_objects":8664,"num_object_clones":0,"num_object_copies":17328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4063,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8664,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9751,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2483},{"shards":"2","objects":4602}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":78,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.244375+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:25.244375+0000","last_peered":"2024-09-17T15:28:25.244375+0000","last_clean":"2024-09-17T15:28:25.244375+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:25.244375+0000","last_undegraded":"2024-09-17T15:28:25.244375+0000","last_fullsized":"2024-09-17T15:28:25.244375+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'11983","reported_seq":12134,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:37.473502+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:30:37.473502+0000","last_peered":"2024-09-17T15:30:37.473502+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:30:37.473502+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'1900","ondisk_log_start":"34'1900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35852288,"num_objects":8753,"num_object_clones":0,"num_object_copies":17506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9745,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2525},{"shards":"2","objects":4632}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'12093","reported_seq":12287,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:36.998506+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:30:36.998506+0000","last_peered":"2024-09-17T15:30:36.998506+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:30:36.998506+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'2000","ondisk_log_start":"34'2000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36343808,"num_objects":8873,"num_object_clones":0,"num_object_copies":17746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4110,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8641,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4764},{"shards":"0,2","objects":2529}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.244410+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:25.244410+0000","last_peered":"2024-09-17T15:28:25.244410+0000","last_clean":"2024-09-17T15:28:25.244410+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:25.244410+0000","last_undegraded":"2024-09-17T15:28:25.244410+0000","last_fullsized":"2024-09-17T15:28:25.244410+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'12035","reported_seq":12221,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:37.175243+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:30:37.175243+0000","last_peered":"2024-09-17T15:30:37.175243+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:30:37.175243+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'2000","ondisk_log_start":"34'2000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35786752,"num_objects":8737,"num_object_clones":0,"num_object_copies":17474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4138,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8322,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4602},{"shards":"0,2","objects":2512}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805107+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805107+0000","last_peered":"2024-09-17T15:28:23.805107+0000","last_clean":"2024-09-17T15:28:23.805107+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805107+0000","last_undegraded":"2024-09-17T15:28:23.805107+0000","last_fullsized":"2024-09-17T15:28:23.805107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'12046","reported_seq":12240,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:37.137356+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:30:37.137356+0000","last_peered":"2024-09-17T15:30:37.137356+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:30:37.137356+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'2000","ondisk_log_start":"34'2000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36208640,"num_objects":8840,"num_object_clones":0,"num_object_copies":17680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8833,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4808},{"shards":"0,2","objects":2462}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.244422+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:25.244422+0000","last_peered":"2024-09-17T15:28:25.244422+0000","last_clean":"2024-09-17T15:28:25.244422+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:25.244422+0000","last_undegraded":"2024-09-17T15:28:25.244422+0000","last_fullsized":"2024-09-17T15:28:25.244422+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:30.302636+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:30.302636+0000","last_peered":"2024-09-17T15:28:30.302636+0000","last_clean":"2024-09-17T15:28:30.302636+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:30.302636+0000","last_undegraded":"2024-09-17T15:28:30.302636+0000","last_fullsized":"2024-09-17T15:28:30.302636+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'11902","reported_seq":18780,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:30:37.220505+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:30:37.220505+0000","last_peered":"2024-09-17T15:30:37.220505+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:30:37.220505+0000","last_undegraded":"2024-09-17T15:30:37.220505+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"34'1900","ondisk_log_start":"34'1900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35759046,"num_objects":8732,"num_object_clones":0,"num_object_copies":17464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":744,"num_objects_unfound":0,"num_objects_dirty":8732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8344,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":13779427,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7988},{"shards":"0,2","objects":744}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":576000758,"num_objects":140635,"num_object_clones":0,"num_object_copies":281270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41032,"num_objects_misplaced":744,"num_objects_unfound":0,"num_objects_dirty":140635,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":152352,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7684,"num_bytes_recovered":30450630,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1249427456,"data_stored":1249274792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160717,"ondisk_log_size":160717,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051651,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":721016,"kb_used_data":414436,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93650824,"statfs":{"total":96636764160,"available":95898443776,"internally_reserved":0,"allocated":424382464,"data_stored":424101818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":313913004},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,0,2,0,0,0,0,0,0,0,2],"upper_bound":16384},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724597,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610988,"kb_used_data":318296,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93760852,"statfs":{"total":96636764160,"available":96011112448,"internally_reserved":0,"allocated":325935104,"data_stored":325614251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":299691691},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,5,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247879,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":846520,"kb_used_data":490024,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93525320,"statfs":{"total":96636764160,"available":95769927680,"internally_reserved":0,"allocated":501784576,"data_stored":501503925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":365031080},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,7,2,4,5,0,0,0,0,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":500461568,"data_stored":500425438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":325562368,"data_stored":325486036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":423403520,"data_stored":423363318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:30:42.578 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:30:43.123 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:30:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:30:43.146 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":225,"stamp":"2024-09-17T15:30:41.680189+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":576000777,"num_objects":140637,"num_object_clones":0,"num_object_copies":281274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41032,"num_objects_misplaced":744,"num_objects_unfound":0,"num_objects_dirty":140637,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":152354,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7685,"num_bytes_recovered":30450649,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160720,"ondisk_log_size":160720,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2178524,"kb_used_data":1222756,"kb_used_omap":13,"kb_used_meta":955698,"kb_avail":280936996,"statfs":{"total":289910292480,"available":287679483904,"internally_reserved":0,"allocated":1252102144,"data_stored":1251219994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":978635775},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,7,7,7,8,0,0,0,0,0,1,2],"upper_bound":16384},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19054592,"num_objects":-4652,"num_object_clones":0,"num_object_copies":-9304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-45,"num_objects_misplaced":-360,"num_objects_unfound":0,"num_objects_dirty":-4652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4652,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":405,"num_bytes_recovered":1474560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003794"},"pg_stats":[{"pgid":"2.f","version":"43'11878","reported_seq":12020,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:37.107753+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:30:37.107753+0000","last_peered":"2024-09-17T15:30:37.107753+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:30:37.107753+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'1800","ondisk_log_start":"34'1800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35921920,"num_objects":8770,"num_object_clones":0,"num_object_copies":17540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4038,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11878,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4732},{"shards":"0,2","objects":2516}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'12034","reported_seq":12137,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:37.505035+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:30:37.505035+0000","last_peered":"2024-09-17T15:30:37.505035+0000","last_clean":"2024-09-17T15:30:37.505035+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:30:37.505035+0000","last_undegraded":"2024-09-17T15:30:37.505035+0000","last_fullsized":"2024-09-17T15:30:37.505035+0000","mapping_epoch":33,"log_start":"34'2000","ondisk_log_start":"34'2000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36447174,"num_objects":8900,"num_object_clones":0,"num_object_copies":17800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8900,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10391,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'12136","reported_seq":12293,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:37.491559+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:30:37.491559+0000","last_peered":"2024-09-17T15:30:37.491559+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:30:37.491559+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'2100","ondisk_log_start":"34'2100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36032536,"num_objects":8798,"num_object_clones":0,"num_object_copies":17596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8798,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10293,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2501},{"shards":"2","objects":4645}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'12053","reported_seq":12122,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:37.531979+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:30:37.531979+0000","last_peered":"2024-09-17T15:30:37.531979+0000","last_clean":"2024-09-17T15:30:37.531979+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:30:37.531979+0000","last_undegraded":"2024-09-17T15:30:37.531979+0000","last_fullsized":"2024-09-17T15:30:37.531979+0000","mapping_epoch":33,"log_start":"34'2000","ondisk_log_start":"34'2000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36524032,"num_objects":8917,"num_object_clones":0,"num_object_copies":17834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12053,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'11908","reported_seq":12038,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:37.187069+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:30:37.187069+0000","last_peered":"2024-09-17T15:30:37.187069+0000","last_clean":"2024-09-17T15:30:37.187069+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:30:37.187069+0000","last_undegraded":"2024-09-17T15:30:37.187069+0000","last_fullsized":"2024-09-17T15:30:37.187069+0000","mapping_epoch":32,"log_start":"34'1900","ondisk_log_start":"34'1900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35820003,"num_objects":8746,"num_object_clones":0,"num_object_copies":17492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8746,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8890,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'12301","reported_seq":12494,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:37.165504+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:30:37.165504+0000","last_peered":"2024-09-17T15:30:37.165504+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:30:37.165504+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'2300","ondisk_log_start":"34'2300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36659683,"num_objects":8951,"num_object_clones":0,"num_object_copies":17902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4242,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8951,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8800,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4710},{"shards":"0,2","objects":2595}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'12049","reported_seq":12258,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:37.159528+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:30:37.159528+0000","last_peered":"2024-09-17T15:30:37.159528+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:30:37.159528+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'2000","ondisk_log_start":"34'2000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35713990,"num_objects":8721,"num_object_clones":0,"num_object_copies":17442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8721,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8818,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4582},{"shards":"0,2","objects":2512}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'12088","reported_seq":12216,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:37.151150+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:30:37.151150+0000","last_peered":"2024-09-17T15:30:37.151150+0000","last_clean":"2024-09-17T15:30:37.151150+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:30:37.151150+0000","last_undegraded":"2024-09-17T15:30:37.151150+0000","last_fullsized":"2024-09-17T15:30:37.151150+0000","mapping_epoch":32,"log_start":"34'2000","ondisk_log_start":"34'2000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36159488,"num_objects":8828,"num_object_clones":0,"num_object_copies":17656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9042,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804377+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804377+0000","last_peered":"2024-09-17T15:28:23.804377+0000","last_clean":"2024-09-17T15:28:23.804377+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804377+0000","last_undegraded":"2024-09-17T15:28:23.804377+0000","last_fullsized":"2024-09-17T15:28:23.804377+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'12119","reported_seq":20402,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:41.456873+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:30:41.456873+0000","last_peered":"2024-09-17T15:30:41.456873+0000","last_clean":"2024-09-17T15:30:41.456873+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:30:41.456873+0000","last_undegraded":"2024-09-17T15:30:41.456873+0000","last_fullsized":"2024-09-17T15:30:41.456873+0000","mapping_epoch":42,"log_start":"34'2100","ondisk_log_start":"34'2100","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35627491,"num_objects":8699,"num_object_clones":0,"num_object_copies":17398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8699,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10119,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804747+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804747+0000","last_peered":"2024-09-17T15:28:23.804747+0000","last_clean":"2024-09-17T15:28:23.804747+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804747+0000","last_undegraded":"2024-09-17T15:28:23.804747+0000","last_fullsized":"2024-09-17T15:28:23.804747+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'11836","reported_seq":12009,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:37.197515+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:30:37.197515+0000","last_peered":"2024-09-17T15:30:37.197515+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:30:37.197515+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'1800","ondisk_log_start":"34'1800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35659776,"num_objects":8706,"num_object_clones":0,"num_object_copies":17412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3991,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8432,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4717},{"shards":"0,2","objects":2449}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'11856","reported_seq":12007,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:37.477790+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:30:37.477790+0000","last_peered":"2024-09-17T15:30:37.477790+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:30:37.477790+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'1800","ondisk_log_start":"34'1800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35484131,"num_objects":8664,"num_object_clones":0,"num_object_copies":17328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4063,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8664,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9751,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2483},{"shards":"2","objects":4602}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":78,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.244375+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:25.244375+0000","last_peered":"2024-09-17T15:28:25.244375+0000","last_clean":"2024-09-17T15:28:25.244375+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:25.244375+0000","last_undegraded":"2024-09-17T15:28:25.244375+0000","last_fullsized":"2024-09-17T15:28:25.244375+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'11983","reported_seq":12134,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:37.473502+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:30:37.473502+0000","last_peered":"2024-09-17T15:30:37.473502+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:30:37.473502+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'1900","ondisk_log_start":"34'1900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35852288,"num_objects":8753,"num_object_clones":0,"num_object_copies":17506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9745,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2525},{"shards":"2","objects":4632}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'12093","reported_seq":12287,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:36.998506+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:30:36.998506+0000","last_peered":"2024-09-17T15:30:36.998506+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:30:36.998506+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'2000","ondisk_log_start":"34'2000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36343808,"num_objects":8873,"num_object_clones":0,"num_object_copies":17746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4110,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8641,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4764},{"shards":"0,2","objects":2529}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.244410+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:25.244410+0000","last_peered":"2024-09-17T15:28:25.244410+0000","last_clean":"2024-09-17T15:28:25.244410+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:25.244410+0000","last_undegraded":"2024-09-17T15:28:25.244410+0000","last_fullsized":"2024-09-17T15:28:25.244410+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'12035","reported_seq":12221,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:37.175243+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:30:37.175243+0000","last_peered":"2024-09-17T15:30:37.175243+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:30:37.175243+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'2000","ondisk_log_start":"34'2000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35786752,"num_objects":8737,"num_object_clones":0,"num_object_copies":17474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4138,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8322,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4602},{"shards":"0,2","objects":2512}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805107+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805107+0000","last_peered":"2024-09-17T15:28:23.805107+0000","last_clean":"2024-09-17T15:28:23.805107+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805107+0000","last_undegraded":"2024-09-17T15:28:23.805107+0000","last_fullsized":"2024-09-17T15:28:23.805107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'12046","reported_seq":12240,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:37.137356+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:30:37.137356+0000","last_peered":"2024-09-17T15:30:37.137356+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:30:37.137356+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'2000","ondisk_log_start":"34'2000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36208640,"num_objects":8840,"num_object_clones":0,"num_object_copies":17680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8833,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4808},{"shards":"0,2","objects":2462}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.244422+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:25.244422+0000","last_peered":"2024-09-17T15:28:25.244422+0000","last_clean":"2024-09-17T15:28:25.244422+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:25.244422+0000","last_undegraded":"2024-09-17T15:28:25.244422+0000","last_fullsized":"2024-09-17T15:28:25.244422+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:30.302636+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:30.302636+0000","last_peered":"2024-09-17T15:28:30.302636+0000","last_clean":"2024-09-17T15:28:30.302636+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:30.302636+0000","last_undegraded":"2024-09-17T15:28:30.302636+0000","last_fullsized":"2024-09-17T15:28:30.302636+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'11902","reported_seq":18780,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:30:37.220505+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:30:37.220505+0000","last_peered":"2024-09-17T15:30:37.220505+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:30:37.220505+0000","last_undegraded":"2024-09-17T15:30:37.220505+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"34'1900","ondisk_log_start":"34'1900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35759046,"num_objects":8732,"num_object_clones":0,"num_object_copies":17464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":744,"num_objects_unfound":0,"num_objects_dirty":8732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8344,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":13779427,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7988},{"shards":"0,2","objects":744}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":576000758,"num_objects":140635,"num_object_clones":0,"num_object_copies":281270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41032,"num_objects_misplaced":744,"num_objects_unfound":0,"num_objects_dirty":140635,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":152352,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7684,"num_bytes_recovered":30450630,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1249427456,"data_stored":1249274792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160717,"ondisk_log_size":160717,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051651,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":721016,"kb_used_data":414436,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93650824,"statfs":{"total":96636764160,"available":95898443776,"internally_reserved":0,"allocated":424382464,"data_stored":424101818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":313913004},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,0,2,0,0,0,0,0,0,0,2],"upper_bound":16384},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724597,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610988,"kb_used_data":318296,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93760852,"statfs":{"total":96636764160,"available":96011112448,"internally_reserved":0,"allocated":325935104,"data_stored":325614251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":299691691},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,5,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247879,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":846520,"kb_used_data":490024,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93525320,"statfs":{"total":96636764160,"available":95769927680,"internally_reserved":0,"allocated":501784576,"data_stored":501503925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":365031080},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,7,2,4,5,0,0,0,0,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":500461568,"data_stored":500425438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":325562368,"data_stored":325486036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":423403520,"data_stored":423363318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:30:43.148 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:30:43.148 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'11878", 'reported_seq': 12020, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:37.107753+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:30:37.107753+0000', 'last_peered': '2024-09-17T15:30:37.107753+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:30:37.107753+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "34'1800", 'ondisk_log_start': "34'1800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35921920, 'num_objects': 8770, 'num_object_clones': 0, 'num_object_copies': 17540, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4038, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8770, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11878, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 32, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4732}, {'shards': '0,2', 'objects': 2516}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:43.148 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:30:43.148 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'12136", 'reported_seq': 12293, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:37.491559+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:30:37.491559+0000', 'last_peered': '2024-09-17T15:30:37.491559+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:30:37.491559+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "34'2100", 'ondisk_log_start': "34'2100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36032536, 'num_objects': 8798, 'num_object_clones': 0, 'num_object_copies': 17596, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4154, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8798, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10293, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 15, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2501}, {'shards': '2', 'objects': 4645}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:30:43.148 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:30:43.148 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'12301", 'reported_seq': 12494, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:37.165504+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:30:37.165504+0000', 'last_peered': '2024-09-17T15:30:37.165504+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:30:37.165504+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "34'2300", 'ondisk_log_start': "34'2300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36659683, 'num_objects': 8951, 'num_object_clones': 0, 'num_object_copies': 17902, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4242, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8951, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8800, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4710}, {'shards': '0,2', 'objects': 2595}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:43.148 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:30:43.148 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'12049", 'reported_seq': 12258, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:37.159528+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:30:37.159528+0000', 'last_peered': '2024-09-17T15:30:37.159528+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:30:37.159528+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "34'2000", 'ondisk_log_start': "34'2000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35713990, 'num_objects': 8721, 'num_object_clones': 0, 'num_object_copies': 17442, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8721, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 8818, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 35, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4582}, {'shards': '0,2', 'objects': 2512}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:43.148 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:30:43.149 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'11836", 'reported_seq': 12009, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:37.197515+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:30:37.197515+0000', 'last_peered': '2024-09-17T15:30:37.197515+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:30:37.197515+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "34'1800", 'ondisk_log_start': "34'1800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35659776, 'num_objects': 8706, 'num_object_clones': 0, 'num_object_copies': 17412, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3991, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8706, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8432, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 21, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4717}, {'shards': '0,2', 'objects': 2449}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:43.149 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:30:43.149 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'11856", 'reported_seq': 12007, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:37.477790+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:30:37.477790+0000', 'last_peered': '2024-09-17T15:30:37.477790+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:30:37.477790+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "34'1800", 'ondisk_log_start': "34'1800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35484131, 'num_objects': 8664, 'num_object_clones': 0, 'num_object_copies': 17328, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4063, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8664, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9751, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 15, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2483}, {'shards': '2', 'objects': 4602}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:30:43.149 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:30:43.149 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'11983", 'reported_seq': 12134, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:37.473502+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:30:37.473502+0000', 'last_peered': '2024-09-17T15:30:37.473502+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:30:37.473502+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "34'1900", 'ondisk_log_start': "34'1900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35852288, 'num_objects': 8753, 'num_object_clones': 0, 'num_object_copies': 17506, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4123, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8753, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9745, 'num_write_kb': 32532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 16, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2525}, {'shards': '2', 'objects': 4632}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:30:43.149 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:30:43.149 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'12093", 'reported_seq': 12287, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:36.998506+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:30:36.998506+0000', 'last_peered': '2024-09-17T15:30:36.998506+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:30:36.998506+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "34'2000", 'ondisk_log_start': "34'2000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36343808, 'num_objects': 8873, 'num_object_clones': 0, 'num_object_copies': 17746, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4110, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8873, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8641, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4764}, {'shards': '0,2', 'objects': 2529}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:43.149 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:30:43.149 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'12035", 'reported_seq': 12221, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:37.175243+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:30:37.175243+0000', 'last_peered': '2024-09-17T15:30:37.175243+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:30:37.175243+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "34'2000", 'ondisk_log_start': "34'2000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35786752, 'num_objects': 8737, 'num_object_clones': 0, 'num_object_copies': 17474, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4138, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8737, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8322, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 22, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4602}, {'shards': '0,2', 'objects': 2512}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:43.150 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:30:43.150 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'12046", 'reported_seq': 12240, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:37.137356+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:30:37.137356+0000', 'last_peered': '2024-09-17T15:30:37.137356+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:30:37.137356+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "34'2000", 'ondisk_log_start': "34'2000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36208640, 'num_objects': 8840, 'num_object_clones': 0, 'num_object_copies': 17680, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4034, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8840, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8833, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 31, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4808}, {'shards': '0,2', 'objects': 2462}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:43.150 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:30:43.150 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'11902", 'reported_seq': 18780, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:30:37.220505+0000', 'last_change': '2024-09-17T15:28:22.509818+0000', 'last_active': '2024-09-17T15:30:37.220505+0000', 'last_peered': '2024-09-17T15:30:37.220505+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:30:37.220505+0000', 'last_undegraded': '2024-09-17T15:30:37.220505+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "34'1900", 'ondisk_log_start': "34'1900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35759046, 'num_objects': 8732, 'num_object_clones': 0, 'num_object_copies': 17464, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 744, 'num_objects_unfound': 0, 'num_objects_dirty': 8732, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8344, 'num_write_kb': 27056, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3367, 'num_bytes_recovered': 13779427, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7988}, {'shards': '0,2', 'objects': 744}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:43.150 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:30:43.856 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:30:43.856 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41032/281274 objects degraded (14.588%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":380,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":140637,"data_bytes":576000777,"bytes_used":2230808576,"bytes_avail":287679483904,"bytes_total":289910292480,"degraded_objects":41032,"degraded_total":281274,"degraded_ratio":0.14587910720507405,"misplaced_objects":744,"misplaced_total":281274,"misplaced_ratio":0.0026451076174833082,"recovering_objects_per_sec":33,"recovering_bytes_per_sec":122841,"recovering_keys_per_sec":0,"num_objects_recovered":405,"num_bytes_recovered":1474560,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":387},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T15:27:19.616376+0000","services":{}},"progress_events":{}} 2024-09-17T15:30:43.882 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:30:43.882 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:30:44.415 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:30:44.415 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:30:44.444 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":226,"stamp":"2024-09-17T15:30:43.681011+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":567345929,"num_objects":138524,"num_object_clones":0,"num_object_copies":277048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41007,"num_objects_misplaced":549,"num_objects_unfound":0,"num_objects_dirty":138524,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":154467,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7905,"num_bytes_recovered":31249369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160733,"ondisk_log_size":160733,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2175784,"kb_used_data":1211700,"kb_used_omap":13,"kb_used_meta":964018,"kb_avail":280939736,"statfs":{"total":289910292480,"available":287682289664,"internally_reserved":0,"allocated":1240780800,"data_stored":1239898650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":987155455},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,3,2,2,10,5,4,0,0,4],"upper_bound":1024},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27127808,"num_objects":-6623,"num_object_clones":0,"num_object_copies":-13246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-70,"num_objects_misplaced":-555,"num_objects_unfound":0,"num_objects_dirty":-6623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6623,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":625,"num_bytes_recovered":2273280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003997"},"pg_stats":[{"pgid":"2.f","version":"43'12035","reported_seq":12179,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:42.224765+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:30:42.224765+0000","last_peered":"2024-09-17T15:30:42.224765+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:30:42.224765+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'2000","ondisk_log_start":"34'2000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35278848,"num_objects":8613,"num_object_clones":0,"num_object_copies":17226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4037,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8613,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12035,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4576},{"shards":"0,2","objects":2359}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'12163","reported_seq":12266,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:42.537025+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:30:42.537025+0000","last_peered":"2024-09-17T15:30:42.537025+0000","last_clean":"2024-09-17T15:30:42.537025+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:30:42.537025+0000","last_undegraded":"2024-09-17T15:30:42.537025+0000","last_fullsized":"2024-09-17T15:30:42.537025+0000","mapping_epoch":33,"log_start":"34'2100","ondisk_log_start":"34'2100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35918790,"num_objects":8771,"num_object_clones":0,"num_object_copies":17542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8771,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10520,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'12263","reported_seq":12429,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:42.438850+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:30:42.438850+0000","last_peered":"2024-09-17T15:30:42.438850+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:30:42.438850+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'2200","ondisk_log_start":"34'2200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35512344,"num_objects":8671,"num_object_clones":0,"num_object_copies":17342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8671,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10420,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2374},{"shards":"2","objects":4523}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'12205","reported_seq":12275,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:42.406061+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:30:42.406061+0000","last_peered":"2024-09-17T15:30:42.406061+0000","last_clean":"2024-09-17T15:30:42.406061+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:30:42.406061+0000","last_undegraded":"2024-09-17T15:30:42.406061+0000","last_fullsized":"2024-09-17T15:30:42.406061+0000","mapping_epoch":33,"log_start":"34'2200","ondisk_log_start":"34'2200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35901440,"num_objects":8765,"num_object_clones":0,"num_object_copies":17530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8765,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12205,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'12041","reported_seq":12171,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:42.156257+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:30:42.156257+0000","last_peered":"2024-09-17T15:30:42.156257+0000","last_clean":"2024-09-17T15:30:42.156257+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:30:42.156257+0000","last_undegraded":"2024-09-17T15:30:42.156257+0000","last_fullsized":"2024-09-17T15:30:42.156257+0000","mapping_epoch":32,"log_start":"34'2000","ondisk_log_start":"34'2000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35275235,"num_objects":8613,"num_object_clones":0,"num_object_copies":17226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8613,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9023,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'12444","reported_seq":12643,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:42.016107+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:30:42.016107+0000","last_peered":"2024-09-17T15:30:42.016107+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:30:42.016107+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'2400","ondisk_log_start":"34'2400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36073955,"num_objects":8808,"num_object_clones":0,"num_object_copies":17616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4239,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8808,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8943,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4570},{"shards":"0,2","objects":2452}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'12199","reported_seq":12412,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:42.236208+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:30:42.236208+0000","last_peered":"2024-09-17T15:30:42.236208+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:30:42.236208+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'2100","ondisk_log_start":"34'2100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35099590,"num_objects":8571,"num_object_clones":0,"num_object_copies":17142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4137,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8571,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8968,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4434},{"shards":"0,2","objects":2362}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'12204","reported_seq":12332,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:42.221060+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:30:42.221060+0000","last_peered":"2024-09-17T15:30:42.221060+0000","last_clean":"2024-09-17T15:30:42.221060+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:30:42.221060+0000","last_undegraded":"2024-09-17T15:30:42.221060+0000","last_fullsized":"2024-09-17T15:30:42.221060+0000","mapping_epoch":32,"log_start":"34'2200","ondisk_log_start":"34'2200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35684352,"num_objects":8712,"num_object_clones":0,"num_object_copies":17424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8712,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9158,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804377+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804377+0000","last_peered":"2024-09-17T15:28:23.804377+0000","last_clean":"2024-09-17T15:28:23.804377+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804377+0000","last_undegraded":"2024-09-17T15:28:23.804377+0000","last_fullsized":"2024-09-17T15:28:23.804377+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'12119","reported_seq":20402,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:41.456873+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:30:41.456873+0000","last_peered":"2024-09-17T15:30:41.456873+0000","last_clean":"2024-09-17T15:30:41.456873+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:30:41.456873+0000","last_undegraded":"2024-09-17T15:30:41.456873+0000","last_fullsized":"2024-09-17T15:30:41.456873+0000","mapping_epoch":42,"log_start":"34'2100","ondisk_log_start":"34'2100","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35627491,"num_objects":8699,"num_object_clones":0,"num_object_copies":17398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8699,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10119,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804747+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804747+0000","last_peered":"2024-09-17T15:28:23.804747+0000","last_clean":"2024-09-17T15:28:23.804747+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804747+0000","last_undegraded":"2024-09-17T15:28:23.804747+0000","last_fullsized":"2024-09-17T15:28:23.804747+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'11975","reported_seq":12150,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:42.168441+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:30:42.168441+0000","last_peered":"2024-09-17T15:30:42.168441+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:30:42.168441+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'1900","ondisk_log_start":"34'1900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35090432,"num_objects":8567,"num_object_clones":0,"num_object_copies":17134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3990,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8571,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4579},{"shards":"0,2","objects":2310}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'12016","reported_seq":12171,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:42.523014+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:30:42.523014+0000","last_peered":"2024-09-17T15:30:42.523014+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:30:42.523014+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'2000","ondisk_log_start":"34'2000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34828771,"num_objects":8504,"num_object_clones":0,"num_object_copies":17008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4061,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8504,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9911,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2323},{"shards":"2","objects":4444}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":78,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.244375+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:25.244375+0000","last_peered":"2024-09-17T15:28:25.244375+0000","last_clean":"2024-09-17T15:28:25.244375+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:25.244375+0000","last_undegraded":"2024-09-17T15:28:25.244375+0000","last_fullsized":"2024-09-17T15:28:25.244375+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'12136","reported_seq":12299,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:42.515260+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:30:42.515260+0000","last_peered":"2024-09-17T15:30:42.515260+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:30:42.515260+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'2100","ondisk_log_start":"34'2100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35225600,"num_objects":8600,"num_object_clones":0,"num_object_copies":17200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4117,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8600,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9898,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2372},{"shards":"2","objects":4485}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'12250","reported_seq":12443,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:42.247525+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:30:42.247525+0000","last_peered":"2024-09-17T15:30:42.247525+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:30:42.247525+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'2200","ondisk_log_start":"34'2200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35700736,"num_objects":8716,"num_object_clones":0,"num_object_copies":17432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4110,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8798,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4607},{"shards":"0,2","objects":2372}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.244410+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:25.244410+0000","last_peered":"2024-09-17T15:28:25.244410+0000","last_clean":"2024-09-17T15:28:25.244410+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:25.244410+0000","last_undegraded":"2024-09-17T15:28:25.244410+0000","last_fullsized":"2024-09-17T15:28:25.244410+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'12153","reported_seq":12345,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:42.186177+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:30:42.186177+0000","last_peered":"2024-09-17T15:30:42.186177+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:30:42.186177+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'2100","ondisk_log_start":"34'2100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35303424,"num_objects":8619,"num_object_clones":0,"num_object_copies":17238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4135,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8440,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4487},{"shards":"0,2","objects":2394}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805107+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805107+0000","last_peered":"2024-09-17T15:28:23.805107+0000","last_clean":"2024-09-17T15:28:23.805107+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805107+0000","last_undegraded":"2024-09-17T15:28:23.805107+0000","last_fullsized":"2024-09-17T15:28:23.805107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'12189","reported_seq":12387,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:42.201587+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:30:42.201587+0000","last_peered":"2024-09-17T15:30:42.201587+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:30:42.201587+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'2100","ondisk_log_start":"34'2100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35622912,"num_objects":8697,"num_object_clones":0,"num_object_copies":17394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8976,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4667},{"shards":"0,2","objects":2319}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.244422+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:25.244422+0000","last_peered":"2024-09-17T15:28:25.244422+0000","last_clean":"2024-09-17T15:28:25.244422+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:25.244422+0000","last_undegraded":"2024-09-17T15:28:25.244422+0000","last_fullsized":"2024-09-17T15:28:25.244422+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:30.302636+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:30.302636+0000","last_peered":"2024-09-17T15:28:30.302636+0000","last_clean":"2024-09-17T15:28:30.302636+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:30.302636+0000","last_undegraded":"2024-09-17T15:28:30.302636+0000","last_fullsized":"2024-09-17T15:28:30.302636+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'12038","reported_seq":19306,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:30:42.223099+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:30:42.223099+0000","last_peered":"2024-09-17T15:30:42.223099+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:30:42.223099+0000","last_undegraded":"2024-09-17T15:30:42.223099+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"34'2000","ondisk_log_start":"34'2000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35201990,"num_objects":8596,"num_object_clones":0,"num_object_copies":17192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":549,"num_objects_unfound":0,"num_objects_dirty":8596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8480,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3562,"num_bytes_recovered":14578147,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8047},{"shards":"0,2","objects":549}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":567345910,"num_objects":138522,"num_object_clones":0,"num_object_copies":277044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41007,"num_objects_misplaced":549,"num_objects_unfound":0,"num_objects_dirty":138522,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":154465,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7904,"num_bytes_recovered":31249350,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1238073344,"data_stored":1237920680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160730,"ondisk_log_size":160730,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051652,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":720796,"kb_used_data":410060,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93651044,"statfs":{"total":96636764160,"available":95898669056,"internally_reserved":0,"allocated":419901440,"data_stored":419620794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":318172844},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,2,1,0,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724597,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610988,"kb_used_data":318296,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93760852,"statfs":{"total":96636764160,"available":96011112448,"internally_reserved":0,"allocated":325935104,"data_stored":325614251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":299691691},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,5,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247880,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":844000,"kb_used_data":483344,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93527840,"statfs":{"total":96636764160,"available":95772508160,"internally_reserved":0,"allocated":494944256,"data_stored":494663605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":369290920},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,1,1,4,2,0,0,0,3],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":493592576,"data_stored":493556446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":325562368,"data_stored":325486036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":418918400,"data_stored":418878198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:30:47.446 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:30:48.002 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:30:48.003 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:30:48.033 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":228,"stamp":"2024-09-17T15:30:47.682673+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":558174985,"num_objects":136285,"num_object_clones":0,"num_object_copies":272570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40975,"num_objects_misplaced":339,"num_objects_unfound":0,"num_objects_dirty":136285,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":156706,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8147,"num_bytes_recovered":32109529,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160872,"ondisk_log_size":160872,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2143748,"kb_used_data":1200468,"kb_used_omap":13,"kb_used_meta":943218,"kb_avail":280971772,"statfs":{"total":289910292480,"available":287715094528,"internally_reserved":0,"allocated":1229279232,"data_stored":1228397082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":965856255},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,6,3,4,9,6,0,0,0,0,0,3],"upper_bound":8192},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27705344,"num_objects":-6764,"num_object_clones":0,"num_object_copies":-13528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-73,"num_objects_misplaced":-574,"num_objects_unfound":0,"num_objects_dirty":-6764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6764,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":647,"num_bytes_recovered":2351104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004464"},"pg_stats":[{"pgid":"2.f","version":"43'12163","reported_seq":12317,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:47.106710+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:30:47.106710+0000","last_peered":"2024-09-17T15:30:47.106710+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:30:47.106710+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'2100","ondisk_log_start":"34'2100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34754560,"num_objects":8485,"num_object_clones":0,"num_object_copies":16970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12163,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4453},{"shards":"0,2","objects":2231}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'12289","reported_seq":12392,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:47.534173+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:30:47.534173+0000","last_peered":"2024-09-17T15:30:47.534173+0000","last_clean":"2024-09-17T15:30:47.534173+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:30:47.534173+0000","last_undegraded":"2024-09-17T15:30:47.534173+0000","last_fullsized":"2024-09-17T15:30:47.534173+0000","mapping_epoch":33,"log_start":"34'2200","ondisk_log_start":"34'2200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35402694,"num_objects":8645,"num_object_clones":0,"num_object_copies":17290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8645,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10646,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'12408","reported_seq":12579,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:47.465179+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:30:47.465179+0000","last_peered":"2024-09-17T15:30:47.465179+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:30:47.465179+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'2400","ondisk_log_start":"34'2400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34918424,"num_objects":8526,"num_object_clones":0,"num_object_copies":17052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4147,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8526,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10565,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2229},{"shards":"2","objects":4380}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'12358","reported_seq":12428,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:47.491453+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:30:47.491453+0000","last_peered":"2024-09-17T15:30:47.491453+0000","last_clean":"2024-09-17T15:30:47.491453+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:30:47.491453+0000","last_undegraded":"2024-09-17T15:30:47.491453+0000","last_fullsized":"2024-09-17T15:30:47.491453+0000","mapping_epoch":33,"log_start":"34'2300","ondisk_log_start":"34'2300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35274752,"num_objects":8612,"num_object_clones":0,"num_object_copies":17224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12358,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'12174","reported_seq":12304,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:47.136904+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:30:47.136904+0000","last_peered":"2024-09-17T15:30:47.136904+0000","last_clean":"2024-09-17T15:30:47.136904+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:30:47.136904+0000","last_undegraded":"2024-09-17T15:30:47.136904+0000","last_fullsized":"2024-09-17T15:30:47.136904+0000","mapping_epoch":32,"log_start":"34'2100","ondisk_log_start":"34'2100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34730467,"num_objects":8480,"num_object_clones":0,"num_object_copies":16960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8480,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9156,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'12580","reported_seq":12785,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:47.249073+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:30:47.249073+0000","last_peered":"2024-09-17T15:30:47.249073+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:30:47.249073+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'2500","ondisk_log_start":"34'2500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35516899,"num_objects":8672,"num_object_clones":0,"num_object_copies":17344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4236,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8672,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9079,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4437},{"shards":"0,2","objects":2316}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'12327","reported_seq":12540,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:47.243975+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:30:47.243975+0000","last_peered":"2024-09-17T15:30:47.243975+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:30:47.243975+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'2300","ondisk_log_start":"34'2300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34575302,"num_objects":8443,"num_object_clones":0,"num_object_copies":16886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4137,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8443,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9096,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4306},{"shards":"0,2","objects":2234}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'12354","reported_seq":12482,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:47.182491+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:30:47.182491+0000","last_peered":"2024-09-17T15:30:47.182491+0000","last_clean":"2024-09-17T15:30:47.182491+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:30:47.182491+0000","last_undegraded":"2024-09-17T15:30:47.182491+0000","last_fullsized":"2024-09-17T15:30:47.182491+0000","mapping_epoch":32,"log_start":"34'2300","ondisk_log_start":"34'2300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35069952,"num_objects":8562,"num_object_clones":0,"num_object_copies":17124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8562,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9308,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804377+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804377+0000","last_peered":"2024-09-17T15:28:23.804377+0000","last_clean":"2024-09-17T15:28:23.804377+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804377+0000","last_undegraded":"2024-09-17T15:28:23.804377+0000","last_fullsized":"2024-09-17T15:28:23.804377+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'12254","reported_seq":20536,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:46.490297+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:30:46.490297+0000","last_peered":"2024-09-17T15:30:46.490297+0000","last_clean":"2024-09-17T15:30:46.490297+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:30:46.490297+0000","last_undegraded":"2024-09-17T15:30:46.490297+0000","last_fullsized":"2024-09-17T15:30:46.490297+0000","mapping_epoch":42,"log_start":"34'2200","ondisk_log_start":"34'2200","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35074531,"num_objects":8564,"num_object_clones":0,"num_object_copies":17128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8564,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10254,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804747+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804747+0000","last_peered":"2024-09-17T15:28:23.804747+0000","last_clean":"2024-09-17T15:28:23.804747+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804747+0000","last_undegraded":"2024-09-17T15:28:23.804747+0000","last_fullsized":"2024-09-17T15:28:23.804747+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'12121","reported_seq":12314,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:47.121250+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:30:47.121250+0000","last_peered":"2024-09-17T15:30:47.121250+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:30:47.121250+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'2100","ondisk_log_start":"34'2100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34492416,"num_objects":8421,"num_object_clones":0,"num_object_copies":16842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3981,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8717,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4442},{"shards":"0,2","objects":2164}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'12151","reported_seq":12308,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:47.444833+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:30:47.444833+0000","last_peered":"2024-09-17T15:30:47.444833+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:30:47.444833+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'2100","ondisk_log_start":"34'2100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34275811,"num_objects":8369,"num_object_clones":0,"num_object_copies":16738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4060,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8369,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10046,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2188},{"shards":"2","objects":4310}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":78,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.244375+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:25.244375+0000","last_peered":"2024-09-17T15:28:25.244375+0000","last_clean":"2024-09-17T15:28:25.244375+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:25.244375+0000","last_undegraded":"2024-09-17T15:28:25.244375+0000","last_fullsized":"2024-09-17T15:28:25.244375+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'12274","reported_seq":12439,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:47.564891+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:30:47.564891+0000","last_peered":"2024-09-17T15:30:47.564891+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:30:47.564891+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'2200","ondisk_log_start":"34'2200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34660352,"num_objects":8462,"num_object_clones":0,"num_object_copies":16924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10036,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2234},{"shards":"2","objects":4348}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'12428","reported_seq":12628,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:47.245703+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:30:47.245703+0000","last_peered":"2024-09-17T15:30:47.245703+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:30:47.245703+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'2400","ondisk_log_start":"34'2400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34971648,"num_objects":8538,"num_object_clones":0,"num_object_copies":17076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4106,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8538,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8976,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4433},{"shards":"0,2","objects":2194}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.244410+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:25.244410+0000","last_peered":"2024-09-17T15:28:25.244410+0000","last_clean":"2024-09-17T15:28:25.244410+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:25.244410+0000","last_undegraded":"2024-09-17T15:28:25.244410+0000","last_fullsized":"2024-09-17T15:28:25.244410+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'12294","reported_seq":12492,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:47.236321+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:30:47.236321+0000","last_peered":"2024-09-17T15:30:47.236321+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:30:47.236321+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'2200","ondisk_log_start":"34'2200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34725888,"num_objects":8478,"num_object_clones":0,"num_object_copies":16956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8581,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4349},{"shards":"0,2","objects":2253}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805107+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805107+0000","last_peered":"2024-09-17T15:28:23.805107+0000","last_clean":"2024-09-17T15:28:23.805107+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805107+0000","last_undegraded":"2024-09-17T15:28:23.805107+0000","last_fullsized":"2024-09-17T15:28:23.805107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'12325","reported_seq":12531,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:47.200308+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:30:47.200308+0000","last_peered":"2024-09-17T15:30:47.200308+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:30:47.200308+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'2300","ondisk_log_start":"34'2300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35065856,"num_objects":8561,"num_object_clones":0,"num_object_copies":17122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9112,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4535},{"shards":"0,2","objects":2183}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.244422+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:25.244422+0000","last_peered":"2024-09-17T15:28:25.244422+0000","last_clean":"2024-09-17T15:28:25.244422+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:25.244422+0000","last_undegraded":"2024-09-17T15:28:25.244422+0000","last_fullsized":"2024-09-17T15:28:25.244422+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:30.302636+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:30.302636+0000","last_peered":"2024-09-17T15:28:30.302636+0000","last_clean":"2024-09-17T15:28:30.302636+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:30.302636+0000","last_undegraded":"2024-09-17T15:28:30.302636+0000","last_fullsized":"2024-09-17T15:28:30.302636+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'12169","reported_seq":19857,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:30:47.254971+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:30:47.254971+0000","last_peered":"2024-09-17T15:30:47.254971+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:30:47.254971+0000","last_undegraded":"2024-09-17T15:30:47.254971+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"34'2100","ondisk_log_start":"34'2100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34665414,"num_objects":8465,"num_object_clones":0,"num_object_copies":16930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":339,"num_objects_unfound":0,"num_objects_dirty":8465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8611,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3772,"num_bytes_recovered":15438307,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8126},{"shards":"0,2","objects":339}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":558174966,"num_objects":136283,"num_object_clones":0,"num_object_copies":272566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40975,"num_objects_misplaced":339,"num_objects_unfound":0,"num_objects_dirty":136283,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":156704,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8146,"num_bytes_recovered":32109510,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1226330112,"data_stored":1226177448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160869,"ondisk_log_size":160869,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051653,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703872,"kb_used_data":405616,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93667968,"statfs":{"total":96636764160,"available":95915999232,"internally_reserved":0,"allocated":415350784,"data_stored":415070138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":305393324},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,0,2,2,0,0,0,0,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724598,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":598312,"kb_used_data":318104,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93773528,"statfs":{"total":96636764160,"available":96024092672,"internally_reserved":0,"allocated":325738496,"data_stored":325417643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":286912171},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,4,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247881,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":841564,"kb_used_data":476748,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93530276,"statfs":{"total":96636764160,"available":95775002624,"internally_reserved":0,"allocated":488189952,"data_stored":487909301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":373550760},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,0,0,3,4,0,0,0,0,0,2],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":486723584,"data_stored":486687454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":325337088,"data_stored":325260756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":414269440,"data_stored":414229238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:30:48.034 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:30:48.592 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:30:48.595 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:30:48.616 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":228,"stamp":"2024-09-17T15:30:47.682673+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":558174985,"num_objects":136285,"num_object_clones":0,"num_object_copies":272570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40975,"num_objects_misplaced":339,"num_objects_unfound":0,"num_objects_dirty":136285,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":156706,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8147,"num_bytes_recovered":32109529,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160872,"ondisk_log_size":160872,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2143748,"kb_used_data":1200468,"kb_used_omap":13,"kb_used_meta":943218,"kb_avail":280971772,"statfs":{"total":289910292480,"available":287715094528,"internally_reserved":0,"allocated":1229279232,"data_stored":1228397082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":965856255},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,6,3,4,9,6,0,0,0,0,0,3],"upper_bound":8192},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27705344,"num_objects":-6764,"num_object_clones":0,"num_object_copies":-13528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-73,"num_objects_misplaced":-574,"num_objects_unfound":0,"num_objects_dirty":-6764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6764,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":647,"num_bytes_recovered":2351104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004464"},"pg_stats":[{"pgid":"2.f","version":"43'12163","reported_seq":12317,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:47.106710+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:30:47.106710+0000","last_peered":"2024-09-17T15:30:47.106710+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:30:47.106710+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'2100","ondisk_log_start":"34'2100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34754560,"num_objects":8485,"num_object_clones":0,"num_object_copies":16970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12163,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4453},{"shards":"0,2","objects":2231}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'12289","reported_seq":12392,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:47.534173+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:30:47.534173+0000","last_peered":"2024-09-17T15:30:47.534173+0000","last_clean":"2024-09-17T15:30:47.534173+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:30:47.534173+0000","last_undegraded":"2024-09-17T15:30:47.534173+0000","last_fullsized":"2024-09-17T15:30:47.534173+0000","mapping_epoch":33,"log_start":"34'2200","ondisk_log_start":"34'2200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35402694,"num_objects":8645,"num_object_clones":0,"num_object_copies":17290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8645,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10646,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'12408","reported_seq":12579,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:47.465179+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:30:47.465179+0000","last_peered":"2024-09-17T15:30:47.465179+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:30:47.465179+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'2400","ondisk_log_start":"34'2400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34918424,"num_objects":8526,"num_object_clones":0,"num_object_copies":17052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4147,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8526,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10565,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2229},{"shards":"2","objects":4380}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'12358","reported_seq":12428,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:47.491453+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:30:47.491453+0000","last_peered":"2024-09-17T15:30:47.491453+0000","last_clean":"2024-09-17T15:30:47.491453+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:30:47.491453+0000","last_undegraded":"2024-09-17T15:30:47.491453+0000","last_fullsized":"2024-09-17T15:30:47.491453+0000","mapping_epoch":33,"log_start":"34'2300","ondisk_log_start":"34'2300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35274752,"num_objects":8612,"num_object_clones":0,"num_object_copies":17224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12358,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'12174","reported_seq":12304,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:47.136904+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:30:47.136904+0000","last_peered":"2024-09-17T15:30:47.136904+0000","last_clean":"2024-09-17T15:30:47.136904+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:30:47.136904+0000","last_undegraded":"2024-09-17T15:30:47.136904+0000","last_fullsized":"2024-09-17T15:30:47.136904+0000","mapping_epoch":32,"log_start":"34'2100","ondisk_log_start":"34'2100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34730467,"num_objects":8480,"num_object_clones":0,"num_object_copies":16960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8480,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9156,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'12580","reported_seq":12785,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:47.249073+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:30:47.249073+0000","last_peered":"2024-09-17T15:30:47.249073+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:30:47.249073+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'2500","ondisk_log_start":"34'2500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35516899,"num_objects":8672,"num_object_clones":0,"num_object_copies":17344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4236,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8672,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9079,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4437},{"shards":"0,2","objects":2316}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'12327","reported_seq":12540,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:47.243975+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:30:47.243975+0000","last_peered":"2024-09-17T15:30:47.243975+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:30:47.243975+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'2300","ondisk_log_start":"34'2300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34575302,"num_objects":8443,"num_object_clones":0,"num_object_copies":16886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4137,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8443,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9096,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4306},{"shards":"0,2","objects":2234}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'12354","reported_seq":12482,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:47.182491+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:30:47.182491+0000","last_peered":"2024-09-17T15:30:47.182491+0000","last_clean":"2024-09-17T15:30:47.182491+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:30:47.182491+0000","last_undegraded":"2024-09-17T15:30:47.182491+0000","last_fullsized":"2024-09-17T15:30:47.182491+0000","mapping_epoch":32,"log_start":"34'2300","ondisk_log_start":"34'2300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35069952,"num_objects":8562,"num_object_clones":0,"num_object_copies":17124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8562,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9308,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804377+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804377+0000","last_peered":"2024-09-17T15:28:23.804377+0000","last_clean":"2024-09-17T15:28:23.804377+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804377+0000","last_undegraded":"2024-09-17T15:28:23.804377+0000","last_fullsized":"2024-09-17T15:28:23.804377+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'12254","reported_seq":20536,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:46.490297+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:30:46.490297+0000","last_peered":"2024-09-17T15:30:46.490297+0000","last_clean":"2024-09-17T15:30:46.490297+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:30:46.490297+0000","last_undegraded":"2024-09-17T15:30:46.490297+0000","last_fullsized":"2024-09-17T15:30:46.490297+0000","mapping_epoch":42,"log_start":"34'2200","ondisk_log_start":"34'2200","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35074531,"num_objects":8564,"num_object_clones":0,"num_object_copies":17128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8564,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10254,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804747+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804747+0000","last_peered":"2024-09-17T15:28:23.804747+0000","last_clean":"2024-09-17T15:28:23.804747+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804747+0000","last_undegraded":"2024-09-17T15:28:23.804747+0000","last_fullsized":"2024-09-17T15:28:23.804747+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'12121","reported_seq":12314,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:47.121250+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:30:47.121250+0000","last_peered":"2024-09-17T15:30:47.121250+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:30:47.121250+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'2100","ondisk_log_start":"34'2100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34492416,"num_objects":8421,"num_object_clones":0,"num_object_copies":16842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3981,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8717,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4442},{"shards":"0,2","objects":2164}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'12151","reported_seq":12308,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:47.444833+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:30:47.444833+0000","last_peered":"2024-09-17T15:30:47.444833+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:30:47.444833+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'2100","ondisk_log_start":"34'2100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34275811,"num_objects":8369,"num_object_clones":0,"num_object_copies":16738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4060,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8369,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10046,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2188},{"shards":"2","objects":4310}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":78,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.244375+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:25.244375+0000","last_peered":"2024-09-17T15:28:25.244375+0000","last_clean":"2024-09-17T15:28:25.244375+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:25.244375+0000","last_undegraded":"2024-09-17T15:28:25.244375+0000","last_fullsized":"2024-09-17T15:28:25.244375+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'12274","reported_seq":12439,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:47.564891+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:30:47.564891+0000","last_peered":"2024-09-17T15:30:47.564891+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:30:47.564891+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'2200","ondisk_log_start":"34'2200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34660352,"num_objects":8462,"num_object_clones":0,"num_object_copies":16924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10036,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2234},{"shards":"2","objects":4348}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'12428","reported_seq":12628,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:47.245703+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:30:47.245703+0000","last_peered":"2024-09-17T15:30:47.245703+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:30:47.245703+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'2400","ondisk_log_start":"34'2400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34971648,"num_objects":8538,"num_object_clones":0,"num_object_copies":17076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4106,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8538,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8976,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4433},{"shards":"0,2","objects":2194}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.244410+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:25.244410+0000","last_peered":"2024-09-17T15:28:25.244410+0000","last_clean":"2024-09-17T15:28:25.244410+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:25.244410+0000","last_undegraded":"2024-09-17T15:28:25.244410+0000","last_fullsized":"2024-09-17T15:28:25.244410+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'12294","reported_seq":12492,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:47.236321+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:30:47.236321+0000","last_peered":"2024-09-17T15:30:47.236321+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:30:47.236321+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'2200","ondisk_log_start":"34'2200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34725888,"num_objects":8478,"num_object_clones":0,"num_object_copies":16956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8581,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4349},{"shards":"0,2","objects":2253}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805107+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805107+0000","last_peered":"2024-09-17T15:28:23.805107+0000","last_clean":"2024-09-17T15:28:23.805107+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805107+0000","last_undegraded":"2024-09-17T15:28:23.805107+0000","last_fullsized":"2024-09-17T15:28:23.805107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'12325","reported_seq":12531,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:47.200308+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:30:47.200308+0000","last_peered":"2024-09-17T15:30:47.200308+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:30:47.200308+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'2300","ondisk_log_start":"34'2300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35065856,"num_objects":8561,"num_object_clones":0,"num_object_copies":17122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9112,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4535},{"shards":"0,2","objects":2183}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.244422+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:25.244422+0000","last_peered":"2024-09-17T15:28:25.244422+0000","last_clean":"2024-09-17T15:28:25.244422+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:25.244422+0000","last_undegraded":"2024-09-17T15:28:25.244422+0000","last_fullsized":"2024-09-17T15:28:25.244422+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:30.302636+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:30.302636+0000","last_peered":"2024-09-17T15:28:30.302636+0000","last_clean":"2024-09-17T15:28:30.302636+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:30.302636+0000","last_undegraded":"2024-09-17T15:28:30.302636+0000","last_fullsized":"2024-09-17T15:28:30.302636+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'12169","reported_seq":19857,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:30:47.254971+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:30:47.254971+0000","last_peered":"2024-09-17T15:30:47.254971+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:30:47.254971+0000","last_undegraded":"2024-09-17T15:30:47.254971+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"34'2100","ondisk_log_start":"34'2100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34665414,"num_objects":8465,"num_object_clones":0,"num_object_copies":16930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":339,"num_objects_unfound":0,"num_objects_dirty":8465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8611,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3772,"num_bytes_recovered":15438307,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8126},{"shards":"0,2","objects":339}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":558174966,"num_objects":136283,"num_object_clones":0,"num_object_copies":272566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40975,"num_objects_misplaced":339,"num_objects_unfound":0,"num_objects_dirty":136283,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":156704,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8146,"num_bytes_recovered":32109510,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1226330112,"data_stored":1226177448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160869,"ondisk_log_size":160869,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051653,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703872,"kb_used_data":405616,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93667968,"statfs":{"total":96636764160,"available":95915999232,"internally_reserved":0,"allocated":415350784,"data_stored":415070138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":305393324},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,0,2,2,0,0,0,0,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724598,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":598312,"kb_used_data":318104,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93773528,"statfs":{"total":96636764160,"available":96024092672,"internally_reserved":0,"allocated":325738496,"data_stored":325417643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":286912171},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,4,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247881,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":841564,"kb_used_data":476748,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93530276,"statfs":{"total":96636764160,"available":95775002624,"internally_reserved":0,"allocated":488189952,"data_stored":487909301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":373550760},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,0,0,3,4,0,0,0,0,0,2],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":486723584,"data_stored":486687454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":325337088,"data_stored":325260756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":414269440,"data_stored":414229238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:30:48.617 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:30:48.618 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'12163", 'reported_seq': 12317, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:47.106710+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:30:47.106710+0000', 'last_peered': '2024-09-17T15:30:47.106710+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:30:47.106710+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "34'2100", 'ondisk_log_start': "34'2100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34754560, 'num_objects': 8485, 'num_object_clones': 0, 'num_object_copies': 16970, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8485, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12163, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 38, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4453}, {'shards': '0,2', 'objects': 2231}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:48.618 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:30:48.618 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'12408", 'reported_seq': 12579, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:47.465179+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:30:47.465179+0000', 'last_peered': '2024-09-17T15:30:47.465179+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:30:47.465179+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "34'2400", 'ondisk_log_start': "34'2400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34918424, 'num_objects': 8526, 'num_object_clones': 0, 'num_object_copies': 17052, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4147, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8526, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10565, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 22, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2229}, {'shards': '2', 'objects': 4380}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:30:48.618 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:30:48.618 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'12580", 'reported_seq': 12785, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:47.249073+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:30:47.249073+0000', 'last_peered': '2024-09-17T15:30:47.249073+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:30:47.249073+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "34'2500", 'ondisk_log_start': "34'2500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35516899, 'num_objects': 8672, 'num_object_clones': 0, 'num_object_copies': 17344, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4236, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8672, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9079, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 35, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4437}, {'shards': '0,2', 'objects': 2316}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:48.618 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:30:48.618 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'12327", 'reported_seq': 12540, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:47.243975+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:30:47.243975+0000', 'last_peered': '2024-09-17T15:30:47.243975+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:30:47.243975+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "34'2300", 'ondisk_log_start': "34'2300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34575302, 'num_objects': 8443, 'num_object_clones': 0, 'num_object_copies': 16886, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4137, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8443, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 9096, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 37, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4306}, {'shards': '0,2', 'objects': 2234}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:48.618 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:30:48.618 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'12121", 'reported_seq': 12314, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:47.121250+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:30:47.121250+0000', 'last_peered': '2024-09-17T15:30:47.121250+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:30:47.121250+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "34'2100", 'ondisk_log_start': "34'2100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34492416, 'num_objects': 8421, 'num_object_clones': 0, 'num_object_copies': 16842, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3981, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8421, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8717, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 31, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4442}, {'shards': '0,2', 'objects': 2164}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:48.619 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:30:48.619 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'12151", 'reported_seq': 12308, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:47.444833+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:30:47.444833+0000', 'last_peered': '2024-09-17T15:30:47.444833+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:30:47.444833+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "34'2100", 'ondisk_log_start': "34'2100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34275811, 'num_objects': 8369, 'num_object_clones': 0, 'num_object_copies': 16738, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4060, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8369, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10046, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 18, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2188}, {'shards': '2', 'objects': 4310}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:30:48.619 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:30:48.619 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'12274", 'reported_seq': 12439, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:47.564891+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:30:47.564891+0000', 'last_peered': '2024-09-17T15:30:47.564891+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:30:47.564891+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "34'2200", 'ondisk_log_start': "34'2200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34660352, 'num_objects': 8462, 'num_object_clones': 0, 'num_object_copies': 16924, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4116, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8462, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10036, 'num_write_kb': 32532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 23, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2234}, {'shards': '2', 'objects': 4348}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:30:48.619 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:30:48.619 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'12428", 'reported_seq': 12628, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:47.245703+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:30:47.245703+0000', 'last_peered': '2024-09-17T15:30:47.245703+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:30:47.245703+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "34'2400", 'ondisk_log_start': "34'2400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34971648, 'num_objects': 8538, 'num_object_clones': 0, 'num_object_copies': 17076, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4106, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8538, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8976, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 33, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4433}, {'shards': '0,2', 'objects': 2194}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:48.619 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:30:48.619 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'12294", 'reported_seq': 12492, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:47.236321+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:30:47.236321+0000', 'last_peered': '2024-09-17T15:30:47.236321+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:30:47.236321+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "34'2200", 'ondisk_log_start': "34'2200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34725888, 'num_objects': 8478, 'num_object_clones': 0, 'num_object_copies': 16956, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8478, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8581, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 28, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4349}, {'shards': '0,2', 'objects': 2253}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:48.619 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:30:48.619 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'12325", 'reported_seq': 12531, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:47.200308+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:30:47.200308+0000', 'last_peered': '2024-09-17T15:30:47.200308+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:30:47.200308+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "34'2300", 'ondisk_log_start': "34'2300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35065856, 'num_objects': 8561, 'num_object_clones': 0, 'num_object_copies': 17122, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4028, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8561, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9112, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 37, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4535}, {'shards': '0,2', 'objects': 2183}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:48.620 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:30:48.620 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'12169", 'reported_seq': 19857, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:30:47.254971+0000', 'last_change': '2024-09-17T15:28:22.509818+0000', 'last_active': '2024-09-17T15:30:47.254971+0000', 'last_peered': '2024-09-17T15:30:47.254971+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:30:47.254971+0000', 'last_undegraded': '2024-09-17T15:30:47.254971+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "34'2100", 'ondisk_log_start': "34'2100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34665414, 'num_objects': 8465, 'num_object_clones': 0, 'num_object_copies': 16930, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 339, 'num_objects_unfound': 0, 'num_objects_dirty': 8465, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8611, 'num_write_kb': 27056, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3772, 'num_bytes_recovered': 15438307, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 8126}, {'shards': '0,2', 'objects': 339}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:48.620 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:30:49.288 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:30:49.288 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 40975/272570 objects degraded (15.033%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":386,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":136285,"data_bytes":558174985,"bytes_used":2195197952,"bytes_avail":287715094528,"bytes_total":289910292480,"degraded_objects":40975,"degraded_total":272570,"degraded_ratio":0.15032835601863742,"misplaced_objects":339,"misplaced_total":272570,"misplaced_ratio":0.0012437172102579153,"recovering_objects_per_sec":53,"recovering_bytes_per_sec":195852,"recovering_keys_per_sec":0,"num_objects_recovered":647,"num_bytes_recovered":2351104,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":563},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T15:27:19.616376+0000","services":{}},"progress_events":{}} 2024-09-17T15:30:49.308 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:30:49.309 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:30:49.865 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:30:49.867 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:30:49.887 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":229,"stamp":"2024-09-17T15:30:49.683397+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":558174985,"num_objects":136285,"num_object_clones":0,"num_object_copies":272570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40975,"num_objects_misplaced":339,"num_objects_unfound":0,"num_objects_dirty":136285,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":156706,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8147,"num_bytes_recovered":32109529,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160872,"ondisk_log_size":160872,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2143748,"kb_used_data":1200468,"kb_used_omap":13,"kb_used_meta":943218,"kb_avail":280971772,"statfs":{"total":289910292480,"available":287715094528,"internally_reserved":0,"allocated":1229279232,"data_stored":1228397082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":965856255},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,6,3,4,9,6,0,0,0,0,0,3],"upper_bound":8192},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18321408,"num_objects":-4473,"num_object_clones":0,"num_object_copies":-8946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-57,"num_objects_misplaced":-405,"num_objects_unfound":0,"num_objects_dirty":-4473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4473,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":462,"num_bytes_recovered":1658880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004593"},"pg_stats":[{"pgid":"2.f","version":"43'12163","reported_seq":12317,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:47.106710+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:30:47.106710+0000","last_peered":"2024-09-17T15:30:47.106710+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:30:47.106710+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'2100","ondisk_log_start":"34'2100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34754560,"num_objects":8485,"num_object_clones":0,"num_object_copies":16970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12163,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4453},{"shards":"0,2","objects":2231}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'12289","reported_seq":12392,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:47.534173+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:30:47.534173+0000","last_peered":"2024-09-17T15:30:47.534173+0000","last_clean":"2024-09-17T15:30:47.534173+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:30:47.534173+0000","last_undegraded":"2024-09-17T15:30:47.534173+0000","last_fullsized":"2024-09-17T15:30:47.534173+0000","mapping_epoch":33,"log_start":"34'2200","ondisk_log_start":"34'2200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35402694,"num_objects":8645,"num_object_clones":0,"num_object_copies":17290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8645,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10646,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'12408","reported_seq":12579,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:47.465179+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:30:47.465179+0000","last_peered":"2024-09-17T15:30:47.465179+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:30:47.465179+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'2400","ondisk_log_start":"34'2400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34918424,"num_objects":8526,"num_object_clones":0,"num_object_copies":17052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4147,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8526,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10565,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2229},{"shards":"2","objects":4380}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'12358","reported_seq":12428,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:47.491453+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:30:47.491453+0000","last_peered":"2024-09-17T15:30:47.491453+0000","last_clean":"2024-09-17T15:30:47.491453+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:30:47.491453+0000","last_undegraded":"2024-09-17T15:30:47.491453+0000","last_fullsized":"2024-09-17T15:30:47.491453+0000","mapping_epoch":33,"log_start":"34'2300","ondisk_log_start":"34'2300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35274752,"num_objects":8612,"num_object_clones":0,"num_object_copies":17224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12358,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'12174","reported_seq":12304,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:47.136904+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:30:47.136904+0000","last_peered":"2024-09-17T15:30:47.136904+0000","last_clean":"2024-09-17T15:30:47.136904+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:30:47.136904+0000","last_undegraded":"2024-09-17T15:30:47.136904+0000","last_fullsized":"2024-09-17T15:30:47.136904+0000","mapping_epoch":32,"log_start":"34'2100","ondisk_log_start":"34'2100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34730467,"num_objects":8480,"num_object_clones":0,"num_object_copies":16960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8480,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9156,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'12580","reported_seq":12785,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:47.249073+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:30:47.249073+0000","last_peered":"2024-09-17T15:30:47.249073+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:30:47.249073+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'2500","ondisk_log_start":"34'2500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35516899,"num_objects":8672,"num_object_clones":0,"num_object_copies":17344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4236,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8672,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9079,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4437},{"shards":"0,2","objects":2316}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'12327","reported_seq":12540,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:47.243975+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:30:47.243975+0000","last_peered":"2024-09-17T15:30:47.243975+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:30:47.243975+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'2300","ondisk_log_start":"34'2300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34575302,"num_objects":8443,"num_object_clones":0,"num_object_copies":16886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4137,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8443,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9096,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4306},{"shards":"0,2","objects":2234}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'12354","reported_seq":12482,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:47.182491+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:30:47.182491+0000","last_peered":"2024-09-17T15:30:47.182491+0000","last_clean":"2024-09-17T15:30:47.182491+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:30:47.182491+0000","last_undegraded":"2024-09-17T15:30:47.182491+0000","last_fullsized":"2024-09-17T15:30:47.182491+0000","mapping_epoch":32,"log_start":"34'2300","ondisk_log_start":"34'2300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35069952,"num_objects":8562,"num_object_clones":0,"num_object_copies":17124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8562,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9308,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804377+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804377+0000","last_peered":"2024-09-17T15:28:23.804377+0000","last_clean":"2024-09-17T15:28:23.804377+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804377+0000","last_undegraded":"2024-09-17T15:28:23.804377+0000","last_fullsized":"2024-09-17T15:28:23.804377+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'12254","reported_seq":20536,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:46.490297+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:30:46.490297+0000","last_peered":"2024-09-17T15:30:46.490297+0000","last_clean":"2024-09-17T15:30:46.490297+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:30:46.490297+0000","last_undegraded":"2024-09-17T15:30:46.490297+0000","last_fullsized":"2024-09-17T15:30:46.490297+0000","mapping_epoch":42,"log_start":"34'2200","ondisk_log_start":"34'2200","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35074531,"num_objects":8564,"num_object_clones":0,"num_object_copies":17128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8564,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10254,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804747+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804747+0000","last_peered":"2024-09-17T15:28:23.804747+0000","last_clean":"2024-09-17T15:28:23.804747+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804747+0000","last_undegraded":"2024-09-17T15:28:23.804747+0000","last_fullsized":"2024-09-17T15:28:23.804747+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'12121","reported_seq":12314,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:47.121250+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:30:47.121250+0000","last_peered":"2024-09-17T15:30:47.121250+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:30:47.121250+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'2100","ondisk_log_start":"34'2100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34492416,"num_objects":8421,"num_object_clones":0,"num_object_copies":16842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3981,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8717,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4442},{"shards":"0,2","objects":2164}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'12151","reported_seq":12308,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:47.444833+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:30:47.444833+0000","last_peered":"2024-09-17T15:30:47.444833+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:30:47.444833+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'2100","ondisk_log_start":"34'2100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34275811,"num_objects":8369,"num_object_clones":0,"num_object_copies":16738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4060,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8369,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10046,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2188},{"shards":"2","objects":4310}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":78,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.244375+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:25.244375+0000","last_peered":"2024-09-17T15:28:25.244375+0000","last_clean":"2024-09-17T15:28:25.244375+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:25.244375+0000","last_undegraded":"2024-09-17T15:28:25.244375+0000","last_fullsized":"2024-09-17T15:28:25.244375+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'12274","reported_seq":12439,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:47.564891+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:30:47.564891+0000","last_peered":"2024-09-17T15:30:47.564891+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:30:47.564891+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'2200","ondisk_log_start":"34'2200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34660352,"num_objects":8462,"num_object_clones":0,"num_object_copies":16924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10036,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2234},{"shards":"2","objects":4348}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'12428","reported_seq":12628,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:47.245703+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:30:47.245703+0000","last_peered":"2024-09-17T15:30:47.245703+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:30:47.245703+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'2400","ondisk_log_start":"34'2400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34971648,"num_objects":8538,"num_object_clones":0,"num_object_copies":17076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4106,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8538,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8976,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4433},{"shards":"0,2","objects":2194}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.244410+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:25.244410+0000","last_peered":"2024-09-17T15:28:25.244410+0000","last_clean":"2024-09-17T15:28:25.244410+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:25.244410+0000","last_undegraded":"2024-09-17T15:28:25.244410+0000","last_fullsized":"2024-09-17T15:28:25.244410+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'12294","reported_seq":12492,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:47.236321+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:30:47.236321+0000","last_peered":"2024-09-17T15:30:47.236321+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:30:47.236321+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'2200","ondisk_log_start":"34'2200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34725888,"num_objects":8478,"num_object_clones":0,"num_object_copies":16956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8581,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4349},{"shards":"0,2","objects":2253}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805107+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805107+0000","last_peered":"2024-09-17T15:28:23.805107+0000","last_clean":"2024-09-17T15:28:23.805107+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805107+0000","last_undegraded":"2024-09-17T15:28:23.805107+0000","last_fullsized":"2024-09-17T15:28:23.805107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'12325","reported_seq":12531,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:47.200308+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:30:47.200308+0000","last_peered":"2024-09-17T15:30:47.200308+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:30:47.200308+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'2300","ondisk_log_start":"34'2300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35065856,"num_objects":8561,"num_object_clones":0,"num_object_copies":17122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9112,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4535},{"shards":"0,2","objects":2183}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.244422+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:25.244422+0000","last_peered":"2024-09-17T15:28:25.244422+0000","last_clean":"2024-09-17T15:28:25.244422+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:25.244422+0000","last_undegraded":"2024-09-17T15:28:25.244422+0000","last_fullsized":"2024-09-17T15:28:25.244422+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:30.302636+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:30.302636+0000","last_peered":"2024-09-17T15:28:30.302636+0000","last_clean":"2024-09-17T15:28:30.302636+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:30.302636+0000","last_undegraded":"2024-09-17T15:28:30.302636+0000","last_fullsized":"2024-09-17T15:28:30.302636+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'12169","reported_seq":19857,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:30:47.254971+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:30:47.254971+0000","last_peered":"2024-09-17T15:30:47.254971+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:30:47.254971+0000","last_undegraded":"2024-09-17T15:30:47.254971+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"34'2100","ondisk_log_start":"34'2100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34665414,"num_objects":8465,"num_object_clones":0,"num_object_copies":16930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":339,"num_objects_unfound":0,"num_objects_dirty":8465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8611,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3772,"num_bytes_recovered":15438307,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8126},{"shards":"0,2","objects":339}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":558174966,"num_objects":136283,"num_object_clones":0,"num_object_copies":272566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40975,"num_objects_misplaced":339,"num_objects_unfound":0,"num_objects_dirty":136283,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":156704,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8146,"num_bytes_recovered":32109510,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1226330112,"data_stored":1226177448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160869,"ondisk_log_size":160869,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051653,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703872,"kb_used_data":405616,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93667968,"statfs":{"total":96636764160,"available":95915999232,"internally_reserved":0,"allocated":415350784,"data_stored":415070138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":305393324},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,0,2,2,0,0,0,0,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724598,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":598312,"kb_used_data":318104,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93773528,"statfs":{"total":96636764160,"available":96024092672,"internally_reserved":0,"allocated":325738496,"data_stored":325417643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":286912171},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,4,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247881,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":841564,"kb_used_data":476748,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93530276,"statfs":{"total":96636764160,"available":95775002624,"internally_reserved":0,"allocated":488189952,"data_stored":487909301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":373550760},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,0,0,3,4,0,0,0,0,0,2],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":486723584,"data_stored":486687454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":325337088,"data_stored":325260756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":414269440,"data_stored":414229238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:30:52.890 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:30:53.361 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:30:53.363 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:30:53.388 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":230,"stamp":"2024-09-17T15:30:51.684017+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":557576969,"num_objects":136139,"num_object_clones":0,"num_object_copies":272278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40975,"num_objects_misplaced":339,"num_objects_unfound":0,"num_objects_dirty":136139,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":156852,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8147,"num_bytes_recovered":32109529,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160918,"ondisk_log_size":160918,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2151704,"kb_used_data":1200100,"kb_used_omap":13,"kb_used_meta":951538,"kb_avail":280963816,"statfs":{"total":289910292480,"available":287706947584,"internally_reserved":0,"allocated":1228902400,"data_stored":1228020250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":974375935},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,9,2,0,7,11,0,0,0,0,0,3],"upper_bound":8192},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18919424,"num_objects":-4619,"num_object_clones":0,"num_object_copies":-9238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-57,"num_objects_misplaced":-405,"num_objects_unfound":0,"num_objects_dirty":-4619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4619,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":462,"num_bytes_recovered":1658880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004502"},"pg_stats":[{"pgid":"2.f","version":"43'12163","reported_seq":12317,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:47.106710+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:30:47.106710+0000","last_peered":"2024-09-17T15:30:47.106710+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:30:47.106710+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'2100","ondisk_log_start":"34'2100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34754560,"num_objects":8485,"num_object_clones":0,"num_object_copies":16970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12163,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4453},{"shards":"0,2","objects":2231}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'12289","reported_seq":12392,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:47.534173+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:30:47.534173+0000","last_peered":"2024-09-17T15:30:47.534173+0000","last_clean":"2024-09-17T15:30:47.534173+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:30:47.534173+0000","last_undegraded":"2024-09-17T15:30:47.534173+0000","last_fullsized":"2024-09-17T15:30:47.534173+0000","mapping_epoch":33,"log_start":"34'2200","ondisk_log_start":"34'2200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35402694,"num_objects":8645,"num_object_clones":0,"num_object_copies":17290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8645,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10646,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'12408","reported_seq":12579,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:47.465179+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:30:47.465179+0000","last_peered":"2024-09-17T15:30:47.465179+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:30:47.465179+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'2400","ondisk_log_start":"34'2400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34918424,"num_objects":8526,"num_object_clones":0,"num_object_copies":17052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4147,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8526,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10565,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2229},{"shards":"2","objects":4380}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'12358","reported_seq":12428,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:47.491453+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:30:47.491453+0000","last_peered":"2024-09-17T15:30:47.491453+0000","last_clean":"2024-09-17T15:30:47.491453+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:30:47.491453+0000","last_undegraded":"2024-09-17T15:30:47.491453+0000","last_fullsized":"2024-09-17T15:30:47.491453+0000","mapping_epoch":33,"log_start":"34'2300","ondisk_log_start":"34'2300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35274752,"num_objects":8612,"num_object_clones":0,"num_object_copies":17224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12358,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'12174","reported_seq":12304,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:47.136904+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:30:47.136904+0000","last_peered":"2024-09-17T15:30:47.136904+0000","last_clean":"2024-09-17T15:30:47.136904+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:30:47.136904+0000","last_undegraded":"2024-09-17T15:30:47.136904+0000","last_fullsized":"2024-09-17T15:30:47.136904+0000","mapping_epoch":32,"log_start":"34'2100","ondisk_log_start":"34'2100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34730467,"num_objects":8480,"num_object_clones":0,"num_object_copies":16960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8480,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9156,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'12580","reported_seq":12785,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:47.249073+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:30:47.249073+0000","last_peered":"2024-09-17T15:30:47.249073+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:30:47.249073+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'2500","ondisk_log_start":"34'2500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35516899,"num_objects":8672,"num_object_clones":0,"num_object_copies":17344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4236,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8672,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9079,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4437},{"shards":"0,2","objects":2316}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'12327","reported_seq":12540,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:47.243975+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:30:47.243975+0000","last_peered":"2024-09-17T15:30:47.243975+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:30:47.243975+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'2300","ondisk_log_start":"34'2300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34575302,"num_objects":8443,"num_object_clones":0,"num_object_copies":16886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4137,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8443,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9096,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4306},{"shards":"0,2","objects":2234}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'12354","reported_seq":12482,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:47.182491+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:30:47.182491+0000","last_peered":"2024-09-17T15:30:47.182491+0000","last_clean":"2024-09-17T15:30:47.182491+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:30:47.182491+0000","last_undegraded":"2024-09-17T15:30:47.182491+0000","last_fullsized":"2024-09-17T15:30:47.182491+0000","mapping_epoch":32,"log_start":"34'2300","ondisk_log_start":"34'2300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35069952,"num_objects":8562,"num_object_clones":0,"num_object_copies":17124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8562,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9308,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804377+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804377+0000","last_peered":"2024-09-17T15:28:23.804377+0000","last_clean":"2024-09-17T15:28:23.804377+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804377+0000","last_undegraded":"2024-09-17T15:28:23.804377+0000","last_fullsized":"2024-09-17T15:28:23.804377+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'12400","reported_seq":20683,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:51.590308+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:30:51.590308+0000","last_peered":"2024-09-17T15:30:51.590308+0000","last_clean":"2024-09-17T15:30:51.590308+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:30:51.590308+0000","last_undegraded":"2024-09-17T15:30:51.590308+0000","last_fullsized":"2024-09-17T15:30:51.590308+0000","mapping_epoch":42,"log_start":"34'2300","ondisk_log_start":"34'2300","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34476515,"num_objects":8418,"num_object_clones":0,"num_object_copies":16836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8418,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10400,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804747+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804747+0000","last_peered":"2024-09-17T15:28:23.804747+0000","last_clean":"2024-09-17T15:28:23.804747+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804747+0000","last_undegraded":"2024-09-17T15:28:23.804747+0000","last_fullsized":"2024-09-17T15:28:23.804747+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'12121","reported_seq":12314,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:47.121250+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:30:47.121250+0000","last_peered":"2024-09-17T15:30:47.121250+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:30:47.121250+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'2100","ondisk_log_start":"34'2100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34492416,"num_objects":8421,"num_object_clones":0,"num_object_copies":16842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3981,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8717,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4442},{"shards":"0,2","objects":2164}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'12151","reported_seq":12308,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:47.444833+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:30:47.444833+0000","last_peered":"2024-09-17T15:30:47.444833+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:30:47.444833+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'2100","ondisk_log_start":"34'2100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34275811,"num_objects":8369,"num_object_clones":0,"num_object_copies":16738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4060,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8369,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10046,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2188},{"shards":"2","objects":4310}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":78,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.244375+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:25.244375+0000","last_peered":"2024-09-17T15:28:25.244375+0000","last_clean":"2024-09-17T15:28:25.244375+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:25.244375+0000","last_undegraded":"2024-09-17T15:28:25.244375+0000","last_fullsized":"2024-09-17T15:28:25.244375+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'12274","reported_seq":12439,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:47.564891+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:30:47.564891+0000","last_peered":"2024-09-17T15:30:47.564891+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:30:47.564891+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'2200","ondisk_log_start":"34'2200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34660352,"num_objects":8462,"num_object_clones":0,"num_object_copies":16924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10036,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2234},{"shards":"2","objects":4348}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'12428","reported_seq":12628,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:47.245703+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:30:47.245703+0000","last_peered":"2024-09-17T15:30:47.245703+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:30:47.245703+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'2400","ondisk_log_start":"34'2400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34971648,"num_objects":8538,"num_object_clones":0,"num_object_copies":17076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4106,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8538,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8976,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4433},{"shards":"0,2","objects":2194}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.244410+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:25.244410+0000","last_peered":"2024-09-17T15:28:25.244410+0000","last_clean":"2024-09-17T15:28:25.244410+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:25.244410+0000","last_undegraded":"2024-09-17T15:28:25.244410+0000","last_fullsized":"2024-09-17T15:28:25.244410+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'12294","reported_seq":12492,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:47.236321+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:30:47.236321+0000","last_peered":"2024-09-17T15:30:47.236321+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:30:47.236321+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'2200","ondisk_log_start":"34'2200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34725888,"num_objects":8478,"num_object_clones":0,"num_object_copies":16956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8581,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4349},{"shards":"0,2","objects":2253}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805107+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805107+0000","last_peered":"2024-09-17T15:28:23.805107+0000","last_clean":"2024-09-17T15:28:23.805107+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805107+0000","last_undegraded":"2024-09-17T15:28:23.805107+0000","last_fullsized":"2024-09-17T15:28:23.805107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'12325","reported_seq":12531,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:47.200308+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:30:47.200308+0000","last_peered":"2024-09-17T15:30:47.200308+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:30:47.200308+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'2300","ondisk_log_start":"34'2300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35065856,"num_objects":8561,"num_object_clones":0,"num_object_copies":17122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9112,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4535},{"shards":"0,2","objects":2183}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.244422+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:25.244422+0000","last_peered":"2024-09-17T15:28:25.244422+0000","last_clean":"2024-09-17T15:28:25.244422+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:25.244422+0000","last_undegraded":"2024-09-17T15:28:25.244422+0000","last_fullsized":"2024-09-17T15:28:25.244422+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:30.302636+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:30.302636+0000","last_peered":"2024-09-17T15:28:30.302636+0000","last_clean":"2024-09-17T15:28:30.302636+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:30.302636+0000","last_undegraded":"2024-09-17T15:28:30.302636+0000","last_fullsized":"2024-09-17T15:28:30.302636+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'12169","reported_seq":19857,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:30:47.254971+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:30:47.254971+0000","last_peered":"2024-09-17T15:30:47.254971+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:30:47.254971+0000","last_undegraded":"2024-09-17T15:30:47.254971+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"34'2100","ondisk_log_start":"34'2100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34665414,"num_objects":8465,"num_object_clones":0,"num_object_copies":16930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":339,"num_objects_unfound":0,"num_objects_dirty":8465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8611,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3772,"num_bytes_recovered":15438307,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8126},{"shards":"0,2","objects":339}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":557576950,"num_objects":136137,"num_object_clones":0,"num_object_copies":272274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40975,"num_objects_misplaced":339,"num_objects_unfound":0,"num_objects_dirty":136137,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":156850,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8146,"num_bytes_recovered":32109510,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1225969664,"data_stored":1225817000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160915,"ondisk_log_size":160915,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051653,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703872,"kb_used_data":405616,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93667968,"statfs":{"total":96636764160,"available":95915999232,"internally_reserved":0,"allocated":415350784,"data_stored":415070138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":305393324},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,0,2,2,0,0,0,0,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724599,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":606268,"kb_used_data":317736,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93765572,"statfs":{"total":96636764160,"available":96015945728,"internally_reserved":0,"allocated":325361664,"data_stored":325040811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":295431851},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,1,0,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247881,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":841564,"kb_used_data":476748,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93530276,"statfs":{"total":96636764160,"available":95775002624,"internally_reserved":0,"allocated":488189952,"data_stored":487909301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":373550760},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,0,0,3,4,0,0,0,0,0,2],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":486723584,"data_stored":486687454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":324976640,"data_stored":324900308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":414269440,"data_stored":414229238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:30:53.390 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:30:53.935 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:30:53.935 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:30:53.960 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":231,"stamp":"2024-09-17T15:30:53.684786+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":548799241,"num_objects":133996,"num_object_clones":0,"num_object_copies":267992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40952,"num_objects_misplaced":128,"num_objects_unfound":0,"num_objects_dirty":133996,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":158995,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8381,"num_bytes_recovered":32973785,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160761,"ondisk_log_size":160761,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2136304,"kb_used_data":1188856,"kb_used_omap":13,"kb_used_meta":947378,"kb_avail":280979216,"statfs":{"total":289910292480,"available":287722717184,"internally_reserved":0,"allocated":1217388544,"data_stored":1216506394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":970116095},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,8,3,6,5,7,0,8,0,0,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27201536,"num_objects":-6641,"num_object_clones":0,"num_object_copies":-13282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-80,"num_objects_misplaced":-616,"num_objects_unfound":0,"num_objects_dirty":-6641,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6641,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":696,"num_bytes_recovered":2523136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004596"},"pg_stats":[{"pgid":"2.f","version":"43'12304","reported_seq":12464,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:52.287164+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:30:52.287164+0000","last_peered":"2024-09-17T15:30:52.287164+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:30:52.287164+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'2300","ondisk_log_start":"34'2300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34177024,"num_objects":8344,"num_object_clones":0,"num_object_copies":16688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4029,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12304,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4315},{"shards":"0,2","objects":2090}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'12426","reported_seq":12529,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:52.507114+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:30:52.507114+0000","last_peered":"2024-09-17T15:30:52.507114+0000","last_clean":"2024-09-17T15:30:52.507114+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:30:52.507114+0000","last_undegraded":"2024-09-17T15:30:52.507114+0000","last_fullsized":"2024-09-17T15:30:52.507114+0000","mapping_epoch":33,"log_start":"34'2400","ondisk_log_start":"34'2400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34841542,"num_objects":8508,"num_object_clones":0,"num_object_copies":17016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8508,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10783,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'12573","reported_seq":12744,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:52.581095+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:30:52.581095+0000","last_peered":"2024-09-17T15:30:52.581095+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:30:52.581095+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'2500","ondisk_log_start":"34'2500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34242584,"num_objects":8361,"num_object_clones":0,"num_object_copies":16722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4147,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8361,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10730,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2064},{"shards":"2","objects":4215}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'12488","reported_seq":12558,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:52.470440+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:30:52.470440+0000","last_peered":"2024-09-17T15:30:52.470440+0000","last_clean":"2024-09-17T15:30:52.470440+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:30:52.470440+0000","last_undegraded":"2024-09-17T15:30:52.470440+0000","last_fullsized":"2024-09-17T15:30:52.470440+0000","mapping_epoch":33,"log_start":"34'2400","ondisk_log_start":"34'2400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34742272,"num_objects":8482,"num_object_clones":0,"num_object_copies":16964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12488,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'12308","reported_seq":12438,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:52.278973+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:30:52.278973+0000","last_peered":"2024-09-17T15:30:52.278973+0000","last_clean":"2024-09-17T15:30:52.278973+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:30:52.278973+0000","last_undegraded":"2024-09-17T15:30:52.278973+0000","last_fullsized":"2024-09-17T15:30:52.278973+0000","mapping_epoch":32,"log_start":"34'2300","ondisk_log_start":"34'2300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34181603,"num_objects":8346,"num_object_clones":0,"num_object_copies":16692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8346,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9290,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'12734","reported_seq":12947,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:52.250674+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:30:52.250674+0000","last_peered":"2024-09-17T15:30:52.250674+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:30:52.250674+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'2700","ondisk_log_start":"34'2700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34886115,"num_objects":8518,"num_object_clones":0,"num_object_copies":17036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4232,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8518,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9233,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4287},{"shards":"0,2","objects":2162}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'12455","reported_seq":12672,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:52.244178+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:30:52.244178+0000","last_peered":"2024-09-17T15:30:52.244178+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:30:52.244178+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'2400","ondisk_log_start":"34'2400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34051014,"num_objects":8315,"num_object_clones":0,"num_object_copies":16630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4135,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8315,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9224,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4180},{"shards":"0,2","objects":2106}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'12506","reported_seq":12634,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:52.235752+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:30:52.235752+0000","last_peered":"2024-09-17T15:30:52.235752+0000","last_clean":"2024-09-17T15:30:52.235752+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:30:52.235752+0000","last_undegraded":"2024-09-17T15:30:52.235752+0000","last_fullsized":"2024-09-17T15:30:52.235752+0000","mapping_epoch":32,"log_start":"34'2500","ondisk_log_start":"34'2500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34447360,"num_objects":8410,"num_object_clones":0,"num_object_copies":16820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8410,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9460,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804377+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804377+0000","last_peered":"2024-09-17T15:28:23.804377+0000","last_clean":"2024-09-17T15:28:23.804377+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804377+0000","last_undegraded":"2024-09-17T15:28:23.804377+0000","last_fullsized":"2024-09-17T15:28:23.804377+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'12400","reported_seq":20683,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:51.590308+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:30:51.590308+0000","last_peered":"2024-09-17T15:30:51.590308+0000","last_clean":"2024-09-17T15:30:51.590308+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:30:51.590308+0000","last_undegraded":"2024-09-17T15:30:51.590308+0000","last_fullsized":"2024-09-17T15:30:51.590308+0000","mapping_epoch":42,"log_start":"34'2300","ondisk_log_start":"34'2300","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34476515,"num_objects":8418,"num_object_clones":0,"num_object_copies":16836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8418,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10400,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804747+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804747+0000","last_peered":"2024-09-17T15:28:23.804747+0000","last_clean":"2024-09-17T15:28:23.804747+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804747+0000","last_undegraded":"2024-09-17T15:28:23.804747+0000","last_fullsized":"2024-09-17T15:28:23.804747+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'12263","reported_seq":12464,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:52.283081+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:30:52.283081+0000","last_peered":"2024-09-17T15:30:52.283081+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:30:52.283081+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'2200","ondisk_log_start":"34'2200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33910784,"num_objects":8279,"num_object_clones":0,"num_object_copies":16558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3977,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8859,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4304},{"shards":"0,2","objects":2022}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'12288","reported_seq":12444,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:52.559800+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:30:52.559800+0000","last_peered":"2024-09-17T15:30:52.559800+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:30:52.559800+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'2200","ondisk_log_start":"34'2200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33714659,"num_objects":8232,"num_object_clones":0,"num_object_copies":16464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4060,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8232,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10183,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2051},{"shards":"2","objects":4173}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":78,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.244375+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:25.244375+0000","last_peered":"2024-09-17T15:28:25.244375+0000","last_clean":"2024-09-17T15:28:25.244375+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:25.244375+0000","last_undegraded":"2024-09-17T15:28:25.244375+0000","last_fullsized":"2024-09-17T15:28:25.244375+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'12414","reported_seq":12579,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:52.540697+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:30:52.540697+0000","last_peered":"2024-09-17T15:30:52.540697+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:30:52.540697+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'2400","ondisk_log_start":"34'2400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34086912,"num_objects":8322,"num_object_clones":0,"num_object_copies":16644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10176,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2094},{"shards":"2","objects":4208}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'12580","reported_seq":12790,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:52.269074+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:30:52.269074+0000","last_peered":"2024-09-17T15:30:52.269074+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:30:52.269074+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'2500","ondisk_log_start":"34'2500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34349056,"num_objects":8386,"num_object_clones":0,"num_object_copies":16772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4102,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9128,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4285},{"shards":"0,2","objects":2042}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.244410+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:25.244410+0000","last_peered":"2024-09-17T15:28:25.244410+0000","last_clean":"2024-09-17T15:28:25.244410+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:25.244410+0000","last_undegraded":"2024-09-17T15:28:25.244410+0000","last_fullsized":"2024-09-17T15:28:25.244410+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'12437","reported_seq":12639,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:52.275258+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:30:52.275258+0000","last_peered":"2024-09-17T15:30:52.275258+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:30:52.275258+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'2400","ondisk_log_start":"34'2400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34140160,"num_objects":8335,"num_object_clones":0,"num_object_copies":16670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4130,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8724,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4208},{"shards":"0,2","objects":2110}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805107+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805107+0000","last_peered":"2024-09-17T15:28:23.805107+0000","last_clean":"2024-09-17T15:28:23.805107+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805107+0000","last_undegraded":"2024-09-17T15:28:23.805107+0000","last_fullsized":"2024-09-17T15:28:23.805107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'12470","reported_seq":12683,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:52.234955+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:30:52.234955+0000","last_peered":"2024-09-17T15:30:52.234955+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:30:52.234955+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'2400","ondisk_log_start":"34'2400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34471936,"num_objects":8416,"num_object_clones":0,"num_object_copies":16832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4024,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9257,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4394},{"shards":"0,2","objects":2038}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.244422+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:25.244422+0000","last_peered":"2024-09-17T15:28:25.244422+0000","last_clean":"2024-09-17T15:28:25.244422+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:25.244422+0000","last_undegraded":"2024-09-17T15:28:25.244422+0000","last_fullsized":"2024-09-17T15:28:25.244422+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:30.302636+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:30.302636+0000","last_peered":"2024-09-17T15:28:30.302636+0000","last_clean":"2024-09-17T15:28:30.302636+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:30.302636+0000","last_undegraded":"2024-09-17T15:28:30.302636+0000","last_fullsized":"2024-09-17T15:28:30.302636+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'12312","reported_seq":20422,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:30:52.303781+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:30:52.303781+0000","last_peered":"2024-09-17T15:30:52.303781+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:30:52.303781+0000","last_undegraded":"2024-09-17T15:30:52.303781+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"34'2300","ondisk_log_start":"34'2300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34079686,"num_objects":8322,"num_object_clones":0,"num_object_copies":16644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":128,"num_objects_unfound":0,"num_objects_dirty":8322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8754,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3983,"num_bytes_recovered":16302563,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8194},{"shards":"0,2","objects":128}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":548799222,"num_objects":133994,"num_object_clones":0,"num_object_copies":267988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40952,"num_objects_misplaced":128,"num_objects_unfound":0,"num_objects_dirty":133994,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":158993,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8380,"num_bytes_recovered":32973766,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1214435328,"data_stored":1214282664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160758,"ondisk_log_size":160758,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051654,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703584,"kb_used_data":401168,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93668256,"statfs":{"total":96636764160,"available":95916294144,"internally_reserved":0,"allocated":410796032,"data_stored":410515386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":309653164},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,1,0,2,0,0,0,0,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724599,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":606268,"kb_used_data":317736,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93765572,"statfs":{"total":96636764160,"available":96015945728,"internally_reserved":0,"allocated":325361664,"data_stored":325040811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":295431851},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,1,0,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247882,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":826452,"kb_used_data":469952,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93545388,"statfs":{"total":96636764160,"available":95790477312,"internally_reserved":0,"allocated":481230848,"data_stored":480950197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":365031080},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,5,3,0,0,8],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":479748096,"data_stored":479711966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":324976640,"data_stored":324900308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":409710592,"data_stored":409670390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:30:53.961 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:30:53.961 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'12304", 'reported_seq': 12464, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:52.287164+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:30:52.287164+0000', 'last_peered': '2024-09-17T15:30:52.287164+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:30:52.287164+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "34'2300", 'ondisk_log_start': "34'2300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34177024, 'num_objects': 8344, 'num_object_clones': 0, 'num_object_copies': 16688, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4029, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8344, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12304, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 41, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4315}, {'shards': '0,2', 'objects': 2090}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:53.961 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:30:53.961 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'12573", 'reported_seq': 12744, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:52.581095+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:30:52.581095+0000', 'last_peered': '2024-09-17T15:30:52.581095+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:30:52.581095+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "34'2500", 'ondisk_log_start': "34'2500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34242584, 'num_objects': 8361, 'num_object_clones': 0, 'num_object_copies': 16722, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4147, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8361, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10730, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 22, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2064}, {'shards': '2', 'objects': 4215}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:30:53.961 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:30:53.962 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'12734", 'reported_seq': 12947, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:52.250674+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:30:52.250674+0000', 'last_peered': '2024-09-17T15:30:52.250674+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:30:52.250674+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "34'2700", 'ondisk_log_start': "34'2700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34886115, 'num_objects': 8518, 'num_object_clones': 0, 'num_object_copies': 17036, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4232, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8518, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9233, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 39, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4287}, {'shards': '0,2', 'objects': 2162}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:53.962 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:30:53.962 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'12455", 'reported_seq': 12672, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:52.244178+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:30:52.244178+0000', 'last_peered': '2024-09-17T15:30:52.244178+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:30:52.244178+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "34'2400", 'ondisk_log_start': "34'2400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34051014, 'num_objects': 8315, 'num_object_clones': 0, 'num_object_copies': 16630, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4135, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8315, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 9224, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 39, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4180}, {'shards': '0,2', 'objects': 2106}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:53.962 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:30:53.962 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'12263", 'reported_seq': 12464, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:52.283081+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:30:52.283081+0000', 'last_peered': '2024-09-17T15:30:52.283081+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:30:52.283081+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "34'2200", 'ondisk_log_start': "34'2200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33910784, 'num_objects': 8279, 'num_object_clones': 0, 'num_object_copies': 16558, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3977, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8279, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8859, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 35, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4304}, {'shards': '0,2', 'objects': 2022}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:53.962 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:30:53.962 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'12288", 'reported_seq': 12444, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:52.559800+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:30:52.559800+0000', 'last_peered': '2024-09-17T15:30:52.559800+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:30:52.559800+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "34'2200", 'ondisk_log_start': "34'2200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33714659, 'num_objects': 8232, 'num_object_clones': 0, 'num_object_copies': 16464, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4060, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8232, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10183, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 18, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2051}, {'shards': '2', 'objects': 4173}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:30:53.962 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:30:53.962 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'12414", 'reported_seq': 12579, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:52.540697+0000', 'last_change': '2024-09-17T15:26:22.584680+0000', 'last_active': '2024-09-17T15:30:52.540697+0000', 'last_peered': '2024-09-17T15:30:52.540697+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:30:52.540697+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "34'2400", 'ondisk_log_start': "34'2400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34086912, 'num_objects': 8322, 'num_object_clones': 0, 'num_object_copies': 16644, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4116, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8322, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10176, 'num_write_kb': 32532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 23, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2094}, {'shards': '2', 'objects': 4208}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:30:53.962 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:30:53.962 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'12580", 'reported_seq': 12790, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:52.269074+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:30:52.269074+0000', 'last_peered': '2024-09-17T15:30:52.269074+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:30:52.269074+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "34'2500", 'ondisk_log_start': "34'2500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34349056, 'num_objects': 8386, 'num_object_clones': 0, 'num_object_copies': 16772, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4102, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8386, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9128, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 37, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4285}, {'shards': '0,2', 'objects': 2042}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:53.963 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:30:53.963 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'12437", 'reported_seq': 12639, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:52.275258+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:30:52.275258+0000', 'last_peered': '2024-09-17T15:30:52.275258+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:30:52.275258+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "34'2400", 'ondisk_log_start': "34'2400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34140160, 'num_objects': 8335, 'num_object_clones': 0, 'num_object_copies': 16670, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4130, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8335, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8724, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4208}, {'shards': '0,2', 'objects': 2110}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:53.963 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:30:53.963 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'12470", 'reported_seq': 12683, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:52.234955+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:30:52.234955+0000', 'last_peered': '2024-09-17T15:30:52.234955+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:30:52.234955+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "34'2400", 'ondisk_log_start': "34'2400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34471936, 'num_objects': 8416, 'num_object_clones': 0, 'num_object_copies': 16832, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4024, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8416, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9257, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 41, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4394}, {'shards': '0,2', 'objects': 2038}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:53.963 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:30:53.963 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'12312", 'reported_seq': 20422, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-17T15:30:52.303781+0000', 'last_change': '2024-09-17T15:28:22.509818+0000', 'last_active': '2024-09-17T15:30:52.303781+0000', 'last_peered': '2024-09-17T15:30:52.303781+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:30:52.303781+0000', 'last_undegraded': '2024-09-17T15:30:52.303781+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 40, 'log_start': "34'2300", 'ondisk_log_start': "34'2300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34079686, 'num_objects': 8322, 'num_object_clones': 0, 'num_object_copies': 16644, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 128, 'num_objects_unfound': 0, 'num_objects_dirty': 8322, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8754, 'num_write_kb': 27056, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3983, 'num_bytes_recovered': 16302563, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 8194}, {'shards': '0,2', 'objects': 128}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:53.963 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:30:54.657 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:30:54.657 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 40952/267992 objects degraded (15.281%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":391,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":133996,"data_bytes":548799241,"bytes_used":2187575296,"bytes_avail":287722717184,"bytes_total":289910292480,"degraded_objects":40952,"degraded_total":267992,"degraded_ratio":0.15281053165766142,"misplaced_objects":128,"misplaced_total":267992,"misplaced_ratio":0.00047762619779694921,"recovering_objects_per_sec":57,"recovering_bytes_per_sec":210180,"recovering_keys_per_sec":0,"num_objects_recovered":696,"num_bytes_recovered":2523136,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":553},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T15:27:19.616376+0000","services":{}},"progress_events":{}} 2024-09-17T15:30:54.686 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:30:54.687 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:30:55.178 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:30:55.178 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:30:55.199 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":231,"stamp":"2024-09-17T15:30:53.684786+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":548799241,"num_objects":133996,"num_object_clones":0,"num_object_copies":267992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40952,"num_objects_misplaced":128,"num_objects_unfound":0,"num_objects_dirty":133996,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":158995,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8381,"num_bytes_recovered":32973785,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160761,"ondisk_log_size":160761,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2136304,"kb_used_data":1188856,"kb_used_omap":13,"kb_used_meta":947378,"kb_avail":280979216,"statfs":{"total":289910292480,"available":287722717184,"internally_reserved":0,"allocated":1217388544,"data_stored":1216506394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":970116095},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,8,3,6,5,7,0,8,0,0,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27201536,"num_objects":-6641,"num_object_clones":0,"num_object_copies":-13282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-80,"num_objects_misplaced":-616,"num_objects_unfound":0,"num_objects_dirty":-6641,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6641,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":696,"num_bytes_recovered":2523136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004596"},"pg_stats":[{"pgid":"2.f","version":"43'12304","reported_seq":12464,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:52.287164+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:30:52.287164+0000","last_peered":"2024-09-17T15:30:52.287164+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:30:52.287164+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'2300","ondisk_log_start":"34'2300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34177024,"num_objects":8344,"num_object_clones":0,"num_object_copies":16688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4029,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12304,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4315},{"shards":"0,2","objects":2090}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'12426","reported_seq":12529,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:52.507114+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:30:52.507114+0000","last_peered":"2024-09-17T15:30:52.507114+0000","last_clean":"2024-09-17T15:30:52.507114+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:30:52.507114+0000","last_undegraded":"2024-09-17T15:30:52.507114+0000","last_fullsized":"2024-09-17T15:30:52.507114+0000","mapping_epoch":33,"log_start":"34'2400","ondisk_log_start":"34'2400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34841542,"num_objects":8508,"num_object_clones":0,"num_object_copies":17016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8508,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10783,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'12573","reported_seq":12744,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:52.581095+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:30:52.581095+0000","last_peered":"2024-09-17T15:30:52.581095+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:30:52.581095+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'2500","ondisk_log_start":"34'2500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34242584,"num_objects":8361,"num_object_clones":0,"num_object_copies":16722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4147,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8361,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10730,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2064},{"shards":"2","objects":4215}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'12488","reported_seq":12558,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:52.470440+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:30:52.470440+0000","last_peered":"2024-09-17T15:30:52.470440+0000","last_clean":"2024-09-17T15:30:52.470440+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:30:52.470440+0000","last_undegraded":"2024-09-17T15:30:52.470440+0000","last_fullsized":"2024-09-17T15:30:52.470440+0000","mapping_epoch":33,"log_start":"34'2400","ondisk_log_start":"34'2400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34742272,"num_objects":8482,"num_object_clones":0,"num_object_copies":16964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12488,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'12308","reported_seq":12438,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:52.278973+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:30:52.278973+0000","last_peered":"2024-09-17T15:30:52.278973+0000","last_clean":"2024-09-17T15:30:52.278973+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:30:52.278973+0000","last_undegraded":"2024-09-17T15:30:52.278973+0000","last_fullsized":"2024-09-17T15:30:52.278973+0000","mapping_epoch":32,"log_start":"34'2300","ondisk_log_start":"34'2300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34181603,"num_objects":8346,"num_object_clones":0,"num_object_copies":16692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8346,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9290,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'12734","reported_seq":12947,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:52.250674+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:30:52.250674+0000","last_peered":"2024-09-17T15:30:52.250674+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:30:52.250674+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'2700","ondisk_log_start":"34'2700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34886115,"num_objects":8518,"num_object_clones":0,"num_object_copies":17036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4232,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8518,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9233,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4287},{"shards":"0,2","objects":2162}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'12455","reported_seq":12672,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:52.244178+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:30:52.244178+0000","last_peered":"2024-09-17T15:30:52.244178+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:30:52.244178+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'2400","ondisk_log_start":"34'2400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34051014,"num_objects":8315,"num_object_clones":0,"num_object_copies":16630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4135,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8315,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9224,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4180},{"shards":"0,2","objects":2106}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'12506","reported_seq":12634,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:52.235752+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:30:52.235752+0000","last_peered":"2024-09-17T15:30:52.235752+0000","last_clean":"2024-09-17T15:30:52.235752+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:30:52.235752+0000","last_undegraded":"2024-09-17T15:30:52.235752+0000","last_fullsized":"2024-09-17T15:30:52.235752+0000","mapping_epoch":32,"log_start":"34'2500","ondisk_log_start":"34'2500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34447360,"num_objects":8410,"num_object_clones":0,"num_object_copies":16820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8410,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9460,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804377+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:28:23.804377+0000","last_peered":"2024-09-17T15:28:23.804377+0000","last_clean":"2024-09-17T15:28:23.804377+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:28:23.804377+0000","last_undegraded":"2024-09-17T15:28:23.804377+0000","last_fullsized":"2024-09-17T15:28:23.804377+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'12400","reported_seq":20683,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:30:51.590308+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:30:51.590308+0000","last_peered":"2024-09-17T15:30:51.590308+0000","last_clean":"2024-09-17T15:30:51.590308+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:30:51.590308+0000","last_undegraded":"2024-09-17T15:30:51.590308+0000","last_fullsized":"2024-09-17T15:30:51.590308+0000","mapping_epoch":42,"log_start":"34'2300","ondisk_log_start":"34'2300","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34476515,"num_objects":8418,"num_object_clones":0,"num_object_copies":16836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8418,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10400,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.804747+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:28:23.804747+0000","last_peered":"2024-09-17T15:28:23.804747+0000","last_clean":"2024-09-17T15:28:23.804747+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:28:23.804747+0000","last_undegraded":"2024-09-17T15:28:23.804747+0000","last_fullsized":"2024-09-17T15:28:23.804747+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'12263","reported_seq":12464,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:52.283081+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:30:52.283081+0000","last_peered":"2024-09-17T15:30:52.283081+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:30:52.283081+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'2200","ondisk_log_start":"34'2200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33910784,"num_objects":8279,"num_object_clones":0,"num_object_copies":16558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3977,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8859,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4304},{"shards":"0,2","objects":2022}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":103,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.814264+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:28:23.814264+0000","last_peered":"2024-09-17T15:28:23.814264+0000","last_clean":"2024-09-17T15:28:23.814264+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:28:23.814264+0000","last_undegraded":"2024-09-17T15:28:23.814264+0000","last_fullsized":"2024-09-17T15:28:23.814264+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'12288","reported_seq":12444,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:52.559800+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:30:52.559800+0000","last_peered":"2024-09-17T15:30:52.559800+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:30:52.559800+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'2200","ondisk_log_start":"34'2200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33714659,"num_objects":8232,"num_object_clones":0,"num_object_copies":16464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4060,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8232,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10183,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2051},{"shards":"2","objects":4173}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":78,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.244375+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:28:25.244375+0000","last_peered":"2024-09-17T15:28:25.244375+0000","last_clean":"2024-09-17T15:28:25.244375+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:28:25.244375+0000","last_undegraded":"2024-09-17T15:28:25.244375+0000","last_fullsized":"2024-09-17T15:28:25.244375+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'12414","reported_seq":12579,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:52.540697+0000","last_change":"2024-09-17T15:26:22.584680+0000","last_active":"2024-09-17T15:30:52.540697+0000","last_peered":"2024-09-17T15:30:52.540697+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:30:52.540697+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'2400","ondisk_log_start":"34'2400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34086912,"num_objects":8322,"num_object_clones":0,"num_object_copies":16644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10176,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2094},{"shards":"2","objects":4208}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'12580","reported_seq":12790,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:52.269074+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:30:52.269074+0000","last_peered":"2024-09-17T15:30:52.269074+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:30:52.269074+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'2500","ondisk_log_start":"34'2500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34349056,"num_objects":8386,"num_object_clones":0,"num_object_copies":16772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4102,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9128,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4285},{"shards":"0,2","objects":2042}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.244410+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:28:25.244410+0000","last_peered":"2024-09-17T15:28:25.244410+0000","last_clean":"2024-09-17T15:28:25.244410+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:28:25.244410+0000","last_undegraded":"2024-09-17T15:28:25.244410+0000","last_fullsized":"2024-09-17T15:28:25.244410+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'12437","reported_seq":12639,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:52.275258+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:30:52.275258+0000","last_peered":"2024-09-17T15:30:52.275258+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:30:52.275258+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'2400","ondisk_log_start":"34'2400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34140160,"num_objects":8335,"num_object_clones":0,"num_object_copies":16670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4130,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8724,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4208},{"shards":"0,2","objects":2110}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:23.805107+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:28:23.805107+0000","last_peered":"2024-09-17T15:28:23.805107+0000","last_clean":"2024-09-17T15:28:23.805107+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:28:23.805107+0000","last_undegraded":"2024-09-17T15:28:23.805107+0000","last_fullsized":"2024-09-17T15:28:23.805107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'12470","reported_seq":12683,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:52.234955+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:30:52.234955+0000","last_peered":"2024-09-17T15:30:52.234955+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:30:52.234955+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'2400","ondisk_log_start":"34'2400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34471936,"num_objects":8416,"num_object_clones":0,"num_object_copies":16832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4024,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9257,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4394},{"shards":"0,2","objects":2038}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:25.244422+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:28:25.244422+0000","last_peered":"2024-09-17T15:28:25.244422+0000","last_clean":"2024-09-17T15:28:25.244422+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:28:25.244422+0000","last_undegraded":"2024-09-17T15:28:25.244422+0000","last_fullsized":"2024-09-17T15:28:25.244422+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T15:28:30.302636+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:28:30.302636+0000","last_peered":"2024-09-17T15:28:30.302636+0000","last_clean":"2024-09-17T15:28:30.302636+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:28:30.302636+0000","last_undegraded":"2024-09-17T15:28:30.302636+0000","last_fullsized":"2024-09-17T15:28:30.302636+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'12312","reported_seq":20422,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-17T15:30:52.303781+0000","last_change":"2024-09-17T15:28:22.509818+0000","last_active":"2024-09-17T15:30:52.303781+0000","last_peered":"2024-09-17T15:30:52.303781+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:30:52.303781+0000","last_undegraded":"2024-09-17T15:30:52.303781+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":40,"log_start":"34'2300","ondisk_log_start":"34'2300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34079686,"num_objects":8322,"num_object_clones":0,"num_object_copies":16644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":128,"num_objects_unfound":0,"num_objects_dirty":8322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8754,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3983,"num_bytes_recovered":16302563,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8194},{"shards":"0,2","objects":128}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":548799222,"num_objects":133994,"num_object_clones":0,"num_object_copies":267988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40952,"num_objects_misplaced":128,"num_objects_unfound":0,"num_objects_dirty":133994,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":158993,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8380,"num_bytes_recovered":32973766,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1214435328,"data_stored":1214282664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160758,"ondisk_log_size":160758,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051654,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703584,"kb_used_data":401168,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93668256,"statfs":{"total":96636764160,"available":95916294144,"internally_reserved":0,"allocated":410796032,"data_stored":410515386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":309653164},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,1,0,2,0,0,0,0,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724599,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":606268,"kb_used_data":317736,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93765572,"statfs":{"total":96636764160,"available":96015945728,"internally_reserved":0,"allocated":325361664,"data_stored":325040811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":295431851},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,1,0,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247882,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":826452,"kb_used_data":469952,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93545388,"statfs":{"total":96636764160,"available":95790477312,"internally_reserved":0,"allocated":481230848,"data_stored":480950197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":365031080},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,5,3,0,0,8],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":479748096,"data_stored":479711966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":324976640,"data_stored":324900308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":409710592,"data_stored":409670390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:30:58.201 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:30:58.750 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:30:58.751 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:30:58.775 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":235,"stamp":"2024-09-17T15:30:57.686047+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":541029129,"num_objects":132099,"num_object_clones":0,"num_object_copies":264198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38934,"num_objects_misplaced":1907,"num_objects_unfound":0,"num_objects_dirty":132099,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":160892,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8620,"num_bytes_recovered":33879001,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160658,"ondisk_log_size":160658,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2138000,"kb_used_data":1178068,"kb_used_omap":13,"kb_used_meta":959858,"kb_avail":280977520,"statfs":{"total":289910292480,"available":287720980480,"internally_reserved":0,"allocated":1206341632,"data_stored":1205451063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":982895615},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,5,7,2,0,0,0,4],"upper_bound":2048},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17145856,"num_objects":-4186,"num_object_clones":0,"num_object_copies":-8372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2041,"num_objects_misplaced":1568,"num_objects_unfound":0,"num_objects_dirty":-4186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4186,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":473,"num_bytes_recovered":1769472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002650"},"pg_stats":[{"pgid":"2.f","version":"45'12414","reported_seq":12578,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:57.261999+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:30:57.261999+0000","last_peered":"2024-09-17T15:30:57.261999+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:30:57.261999+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'2400","ondisk_log_start":"34'2400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33726464,"num_objects":8234,"num_object_clones":0,"num_object_copies":16468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4029,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12414,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4205},{"shards":"0,2","objects":1980}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12546","reported_seq":12653,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.528950+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:30:57.528950+0000","last_peered":"2024-09-17T15:30:57.528950+0000","last_clean":"2024-09-17T15:30:57.528950+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:30:57.528950+0000","last_undegraded":"2024-09-17T15:30:57.528950+0000","last_fullsized":"2024-09-17T15:30:57.528950+0000","mapping_epoch":33,"log_start":"34'2500","ondisk_log_start":"34'2500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34350022,"num_objects":8388,"num_object_clones":0,"num_object_copies":16776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8388,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10903,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12702","reported_seq":12877,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:57.574855+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:30:57.574855+0000","last_peered":"2024-09-17T15:30:57.574855+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:30:57.574855+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'2700","ondisk_log_start":"34'2700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33714200,"num_objects":8232,"num_object_clones":0,"num_object_copies":16464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4147,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8232,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10859,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1935},{"shards":"2","objects":4086}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12606","reported_seq":12680,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.572435+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:30:57.572435+0000","last_peered":"2024-09-17T15:30:57.572435+0000","last_clean":"2024-09-17T15:30:57.572435+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:30:57.572435+0000","last_undegraded":"2024-09-17T15:30:57.572435+0000","last_fullsized":"2024-09-17T15:30:57.572435+0000","mapping_epoch":33,"log_start":"34'2600","ondisk_log_start":"34'2600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34258944,"num_objects":8364,"num_object_clones":0,"num_object_copies":16728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12606,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'12431","reported_seq":12565,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092639+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:30:57.092639+0000","last_peered":"2024-09-17T15:30:57.092639+0000","last_clean":"2024-09-17T15:30:57.092639+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:30:57.092639+0000","last_undegraded":"2024-09-17T15:30:57.092639+0000","last_fullsized":"2024-09-17T15:30:57.092639+0000","mapping_epoch":32,"log_start":"34'2400","ondisk_log_start":"34'2400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33677795,"num_objects":8223,"num_object_clones":0,"num_object_copies":16446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8223,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9413,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12837","reported_seq":13061,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:57.314542+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:30:57.314542+0000","last_peered":"2024-09-17T15:30:57.314542+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:30:57.314542+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'2800","ondisk_log_start":"34'2800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34464227,"num_objects":8415,"num_object_clones":0,"num_object_copies":16830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4228,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8415,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9336,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4188},{"shards":"0,2","objects":2059}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12580","reported_seq":12803,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:57.303248+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:30:57.303248+0000","last_peered":"2024-09-17T15:30:57.303248+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:30:57.303248+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'2500","ondisk_log_start":"34'2500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33539014,"num_objects":8190,"num_object_clones":0,"num_object_copies":16380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4134,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8190,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9349,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4056},{"shards":"0,2","objects":1981}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12620","reported_seq":12752,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.280105+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:30:57.280105+0000","last_peered":"2024-09-17T15:30:57.280105+0000","last_clean":"2024-09-17T15:30:57.280105+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:30:57.280105+0000","last_undegraded":"2024-09-17T15:30:57.280105+0000","last_fullsized":"2024-09-17T15:30:57.280105+0000","mapping_epoch":32,"log_start":"34'2600","ondisk_log_start":"34'2600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33980416,"num_objects":8296,"num_object_clones":0,"num_object_copies":16592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9574,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T15:30:56.094071+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:56.094071+0000","last_peered":"2024-09-17T15:30:56.094071+0000","last_clean":"2024-09-17T15:30:56.094071+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:56.094071+0000","last_undegraded":"2024-09-17T15:30:56.094071+0000","last_fullsized":"2024-09-17T15:30:56.094071+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'12522","reported_seq":20807,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T15:30:56.560349+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:30:56.560349+0000","last_peered":"2024-09-17T15:30:56.560349+0000","last_clean":"2024-09-17T15:30:56.560349+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:30:56.560349+0000","last_undegraded":"2024-09-17T15:30:56.560349+0000","last_fullsized":"2024-09-17T15:30:56.560349+0000","mapping_epoch":42,"log_start":"34'2500","ondisk_log_start":"34'2500","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33976803,"num_objects":8296,"num_object_clones":0,"num_object_copies":16592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8296,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10522,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088505+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088505+0000","last_peered":"2024-09-17T15:30:57.088505+0000","last_clean":"2024-09-17T15:30:57.088505+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088505+0000","last_undegraded":"2024-09-17T15:30:57.088505+0000","last_fullsized":"2024-09-17T15:30:57.088505+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12385","reported_seq":12594,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:57.292382+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:30:57.292382+0000","last_peered":"2024-09-17T15:30:57.292382+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:30:57.292382+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'2300","ondisk_log_start":"34'2300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33411072,"num_objects":8157,"num_object_clones":0,"num_object_copies":16314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3975,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8981,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4184},{"shards":"0,2","objects":1900}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":107,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092541+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092541+0000","last_peered":"2024-09-17T15:30:57.092541+0000","last_clean":"2024-09-17T15:30:57.092541+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092541+0000","last_undegraded":"2024-09-17T15:30:57.092541+0000","last_fullsized":"2024-09-17T15:30:57.092541+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12411","reported_seq":12574,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:57.523159+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:30:57.523159+0000","last_peered":"2024-09-17T15:30:57.523159+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:30:57.523159+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'2400","ondisk_log_start":"34'2400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33210851,"num_objects":8109,"num_object_clones":0,"num_object_copies":16218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4059,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8109,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10306,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1928},{"shards":"2","objects":4051}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":80,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T15:30:56.089532+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:56.089532+0000","last_peered":"2024-09-17T15:30:56.089532+0000","last_clean":"2024-09-17T15:30:56.089532+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:56.089532+0000","last_undegraded":"2024-09-17T15:30:56.089532+0000","last_fullsized":"2024-09-17T15:30:56.089532+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12558","reported_seq":12922,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:57.589593+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:30:57.589593+0000","last_peered":"2024-09-17T15:30:57.589593+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:30:57.589593+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'2500","ondisk_log_start":"34'2500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33497088,"num_objects":8178,"num_object_clones":0,"num_object_copies":16356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2112,"num_objects_misplaced":1907,"num_objects_unfound":0,"num_objects_dirty":8178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10320,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":380928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1907},{"shards":"2","objects":4159}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12688","reported_seq":12907,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:57.203452+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:30:57.203452+0000","last_peered":"2024-09-17T15:30:57.203452+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:30:57.203452+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'2600","ondisk_log_start":"34'2600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33906688,"num_objects":8278,"num_object_clones":0,"num_object_copies":16556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4099,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8278,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9236,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4180},{"shards":"0,2","objects":1934}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T15:30:56.082651+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:56.082651+0000","last_peered":"2024-09-17T15:30:56.082651+0000","last_clean":"2024-09-17T15:30:56.082651+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:56.082651+0000","last_undegraded":"2024-09-17T15:30:56.082651+0000","last_fullsized":"2024-09-17T15:30:56.082651+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12556","reported_seq":12762,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:57.295220+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:30:57.295220+0000","last_peered":"2024-09-17T15:30:57.295220+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:30:57.295220+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'2500","ondisk_log_start":"34'2500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33652736,"num_objects":8216,"num_object_clones":0,"num_object_copies":16432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4130,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8843,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4089},{"shards":"0,2","objects":1991}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088855+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088855+0000","last_peered":"2024-09-17T15:30:57.088855+0000","last_clean":"2024-09-17T15:30:57.088855+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088855+0000","last_undegraded":"2024-09-17T15:30:57.088855+0000","last_fullsized":"2024-09-17T15:30:57.088855+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12586","reported_seq":12810,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:57.247076+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:30:57.247076+0000","last_peered":"2024-09-17T15:30:57.247076+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:30:57.247076+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'2500","ondisk_log_start":"34'2500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33996800,"num_objects":8300,"num_object_clones":0,"num_object_copies":16600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4021,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9373,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4281},{"shards":"0,2","objects":1922}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T15:30:56.083102+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:56.083102+0000","last_peered":"2024-09-17T15:30:56.083102+0000","last_clean":"2024-09-17T15:30:56.083102+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:56.083102+0000","last_undegraded":"2024-09-17T15:30:56.083102+0000","last_fullsized":"2024-09-17T15:30:56.083102+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T15:30:56.082189+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:56.082189+0000","last_peered":"2024-09-17T15:30:56.082189+0000","last_clean":"2024-09-17T15:30:56.082189+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:56.082189+0000","last_undegraded":"2024-09-17T15:30:56.082189+0000","last_fullsized":"2024-09-17T15:30:56.082189+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'12413","reported_seq":20783,"reported_epoch":44,"state":"peering","last_fresh":"2024-09-17T15:30:55.996046+0000","last_change":"2024-09-17T15:30:56.600666+0000","last_active":"2024-09-17T15:30:56.600666+0000","last_peered":"2024-09-17T15:30:55.996046+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:30:55.996046+0000","last_undegraded":"2024-09-17T15:30:55.996046+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":44,"log_start":"34'2400","ondisk_log_start":"34'2400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33665990,"num_objects":8221,"num_object_clones":0,"num_object_copies":16442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8855,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":541029110,"num_objects":132097,"num_object_clones":0,"num_object_copies":264194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38934,"num_objects_misplaced":1907,"num_objects_unfound":0,"num_objects_dirty":132097,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":160890,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8619,"num_bytes_recovered":33878982,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1204375552,"data_stored":1204230114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160655,"ondisk_log_size":160655,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051655,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703504,"kb_used_data":396924,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93668336,"statfs":{"total":96636764160,"available":95916376064,"internally_reserved":0,"allocated":406450176,"data_stored":406164315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":313913004},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724600,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610144,"kb_used_data":317452,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93761696,"statfs":{"total":96636764160,"available":96011976704,"internally_reserved":0,"allocated":325070848,"data_stored":324752006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":299691691},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247883,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":824352,"kb_used_data":463692,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93547488,"statfs":{"total":96636764160,"available":95792627712,"internally_reserved":0,"allocated":474820608,"data_stored":474534742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":369290920},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2,2,0,0,0,4],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":473989120,"data_stored":473952990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":324677632,"data_stored":324608526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":405708800,"data_stored":405668598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:30:58.776 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:30:59.336 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:30:59.339 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:30:59.358 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":235,"stamp":"2024-09-17T15:30:57.686047+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":541029129,"num_objects":132099,"num_object_clones":0,"num_object_copies":264198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38934,"num_objects_misplaced":1907,"num_objects_unfound":0,"num_objects_dirty":132099,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":160892,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8620,"num_bytes_recovered":33879001,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160658,"ondisk_log_size":160658,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2138000,"kb_used_data":1178068,"kb_used_omap":13,"kb_used_meta":959858,"kb_avail":280977520,"statfs":{"total":289910292480,"available":287720980480,"internally_reserved":0,"allocated":1206341632,"data_stored":1205451063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":982895615},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,5,7,2,0,0,0,4],"upper_bound":2048},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17145856,"num_objects":-4186,"num_object_clones":0,"num_object_copies":-8372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2041,"num_objects_misplaced":1568,"num_objects_unfound":0,"num_objects_dirty":-4186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4186,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":473,"num_bytes_recovered":1769472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002650"},"pg_stats":[{"pgid":"2.f","version":"45'12414","reported_seq":12578,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:57.261999+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:30:57.261999+0000","last_peered":"2024-09-17T15:30:57.261999+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:30:57.261999+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'2400","ondisk_log_start":"34'2400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33726464,"num_objects":8234,"num_object_clones":0,"num_object_copies":16468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4029,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12414,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4205},{"shards":"0,2","objects":1980}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12546","reported_seq":12653,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.528950+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:30:57.528950+0000","last_peered":"2024-09-17T15:30:57.528950+0000","last_clean":"2024-09-17T15:30:57.528950+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:30:57.528950+0000","last_undegraded":"2024-09-17T15:30:57.528950+0000","last_fullsized":"2024-09-17T15:30:57.528950+0000","mapping_epoch":33,"log_start":"34'2500","ondisk_log_start":"34'2500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34350022,"num_objects":8388,"num_object_clones":0,"num_object_copies":16776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8388,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10903,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12702","reported_seq":12877,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:57.574855+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:30:57.574855+0000","last_peered":"2024-09-17T15:30:57.574855+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:30:57.574855+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'2700","ondisk_log_start":"34'2700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33714200,"num_objects":8232,"num_object_clones":0,"num_object_copies":16464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4147,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8232,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10859,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1935},{"shards":"2","objects":4086}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12606","reported_seq":12680,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.572435+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:30:57.572435+0000","last_peered":"2024-09-17T15:30:57.572435+0000","last_clean":"2024-09-17T15:30:57.572435+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:30:57.572435+0000","last_undegraded":"2024-09-17T15:30:57.572435+0000","last_fullsized":"2024-09-17T15:30:57.572435+0000","mapping_epoch":33,"log_start":"34'2600","ondisk_log_start":"34'2600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34258944,"num_objects":8364,"num_object_clones":0,"num_object_copies":16728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12606,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'12431","reported_seq":12565,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092639+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:30:57.092639+0000","last_peered":"2024-09-17T15:30:57.092639+0000","last_clean":"2024-09-17T15:30:57.092639+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:30:57.092639+0000","last_undegraded":"2024-09-17T15:30:57.092639+0000","last_fullsized":"2024-09-17T15:30:57.092639+0000","mapping_epoch":32,"log_start":"34'2400","ondisk_log_start":"34'2400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33677795,"num_objects":8223,"num_object_clones":0,"num_object_copies":16446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8223,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9413,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12837","reported_seq":13061,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:57.314542+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:30:57.314542+0000","last_peered":"2024-09-17T15:30:57.314542+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:30:57.314542+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'2800","ondisk_log_start":"34'2800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34464227,"num_objects":8415,"num_object_clones":0,"num_object_copies":16830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4228,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8415,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9336,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4188},{"shards":"0,2","objects":2059}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12580","reported_seq":12803,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:57.303248+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:30:57.303248+0000","last_peered":"2024-09-17T15:30:57.303248+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:30:57.303248+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'2500","ondisk_log_start":"34'2500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33539014,"num_objects":8190,"num_object_clones":0,"num_object_copies":16380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4134,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8190,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9349,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4056},{"shards":"0,2","objects":1981}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12620","reported_seq":12752,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.280105+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:30:57.280105+0000","last_peered":"2024-09-17T15:30:57.280105+0000","last_clean":"2024-09-17T15:30:57.280105+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:30:57.280105+0000","last_undegraded":"2024-09-17T15:30:57.280105+0000","last_fullsized":"2024-09-17T15:30:57.280105+0000","mapping_epoch":32,"log_start":"34'2600","ondisk_log_start":"34'2600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33980416,"num_objects":8296,"num_object_clones":0,"num_object_copies":16592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9574,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T15:30:56.094071+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:56.094071+0000","last_peered":"2024-09-17T15:30:56.094071+0000","last_clean":"2024-09-17T15:30:56.094071+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:56.094071+0000","last_undegraded":"2024-09-17T15:30:56.094071+0000","last_fullsized":"2024-09-17T15:30:56.094071+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'12522","reported_seq":20807,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T15:30:56.560349+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:30:56.560349+0000","last_peered":"2024-09-17T15:30:56.560349+0000","last_clean":"2024-09-17T15:30:56.560349+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:30:56.560349+0000","last_undegraded":"2024-09-17T15:30:56.560349+0000","last_fullsized":"2024-09-17T15:30:56.560349+0000","mapping_epoch":42,"log_start":"34'2500","ondisk_log_start":"34'2500","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33976803,"num_objects":8296,"num_object_clones":0,"num_object_copies":16592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8296,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10522,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088505+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088505+0000","last_peered":"2024-09-17T15:30:57.088505+0000","last_clean":"2024-09-17T15:30:57.088505+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088505+0000","last_undegraded":"2024-09-17T15:30:57.088505+0000","last_fullsized":"2024-09-17T15:30:57.088505+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12385","reported_seq":12594,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:57.292382+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:30:57.292382+0000","last_peered":"2024-09-17T15:30:57.292382+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:30:57.292382+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'2300","ondisk_log_start":"34'2300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33411072,"num_objects":8157,"num_object_clones":0,"num_object_copies":16314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3975,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8981,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4184},{"shards":"0,2","objects":1900}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":107,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092541+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092541+0000","last_peered":"2024-09-17T15:30:57.092541+0000","last_clean":"2024-09-17T15:30:57.092541+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092541+0000","last_undegraded":"2024-09-17T15:30:57.092541+0000","last_fullsized":"2024-09-17T15:30:57.092541+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12411","reported_seq":12574,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:57.523159+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:30:57.523159+0000","last_peered":"2024-09-17T15:30:57.523159+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:30:57.523159+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'2400","ondisk_log_start":"34'2400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33210851,"num_objects":8109,"num_object_clones":0,"num_object_copies":16218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4059,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8109,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10306,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1928},{"shards":"2","objects":4051}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":80,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T15:30:56.089532+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:56.089532+0000","last_peered":"2024-09-17T15:30:56.089532+0000","last_clean":"2024-09-17T15:30:56.089532+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:56.089532+0000","last_undegraded":"2024-09-17T15:30:56.089532+0000","last_fullsized":"2024-09-17T15:30:56.089532+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12558","reported_seq":12922,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:57.589593+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:30:57.589593+0000","last_peered":"2024-09-17T15:30:57.589593+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:30:57.589593+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'2500","ondisk_log_start":"34'2500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33497088,"num_objects":8178,"num_object_clones":0,"num_object_copies":16356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2112,"num_objects_misplaced":1907,"num_objects_unfound":0,"num_objects_dirty":8178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10320,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":380928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1907},{"shards":"2","objects":4159}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12688","reported_seq":12907,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:57.203452+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:30:57.203452+0000","last_peered":"2024-09-17T15:30:57.203452+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:30:57.203452+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'2600","ondisk_log_start":"34'2600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33906688,"num_objects":8278,"num_object_clones":0,"num_object_copies":16556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4099,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8278,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9236,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4180},{"shards":"0,2","objects":1934}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T15:30:56.082651+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:56.082651+0000","last_peered":"2024-09-17T15:30:56.082651+0000","last_clean":"2024-09-17T15:30:56.082651+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:56.082651+0000","last_undegraded":"2024-09-17T15:30:56.082651+0000","last_fullsized":"2024-09-17T15:30:56.082651+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12556","reported_seq":12762,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:57.295220+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:30:57.295220+0000","last_peered":"2024-09-17T15:30:57.295220+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:30:57.295220+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'2500","ondisk_log_start":"34'2500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33652736,"num_objects":8216,"num_object_clones":0,"num_object_copies":16432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4130,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8843,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4089},{"shards":"0,2","objects":1991}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088855+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088855+0000","last_peered":"2024-09-17T15:30:57.088855+0000","last_clean":"2024-09-17T15:30:57.088855+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088855+0000","last_undegraded":"2024-09-17T15:30:57.088855+0000","last_fullsized":"2024-09-17T15:30:57.088855+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12586","reported_seq":12810,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:57.247076+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:30:57.247076+0000","last_peered":"2024-09-17T15:30:57.247076+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:30:57.247076+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'2500","ondisk_log_start":"34'2500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33996800,"num_objects":8300,"num_object_clones":0,"num_object_copies":16600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4021,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9373,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4281},{"shards":"0,2","objects":1922}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T15:30:56.083102+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:56.083102+0000","last_peered":"2024-09-17T15:30:56.083102+0000","last_clean":"2024-09-17T15:30:56.083102+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:56.083102+0000","last_undegraded":"2024-09-17T15:30:56.083102+0000","last_fullsized":"2024-09-17T15:30:56.083102+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T15:30:56.082189+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:56.082189+0000","last_peered":"2024-09-17T15:30:56.082189+0000","last_clean":"2024-09-17T15:30:56.082189+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:56.082189+0000","last_undegraded":"2024-09-17T15:30:56.082189+0000","last_fullsized":"2024-09-17T15:30:56.082189+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'12413","reported_seq":20783,"reported_epoch":44,"state":"peering","last_fresh":"2024-09-17T15:30:55.996046+0000","last_change":"2024-09-17T15:30:56.600666+0000","last_active":"2024-09-17T15:30:56.600666+0000","last_peered":"2024-09-17T15:30:55.996046+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:30:55.996046+0000","last_undegraded":"2024-09-17T15:30:55.996046+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":44,"log_start":"34'2400","ondisk_log_start":"34'2400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33665990,"num_objects":8221,"num_object_clones":0,"num_object_copies":16442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8855,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":541029110,"num_objects":132097,"num_object_clones":0,"num_object_copies":264194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38934,"num_objects_misplaced":1907,"num_objects_unfound":0,"num_objects_dirty":132097,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":160890,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8619,"num_bytes_recovered":33878982,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1204375552,"data_stored":1204230114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160655,"ondisk_log_size":160655,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051655,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703504,"kb_used_data":396924,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93668336,"statfs":{"total":96636764160,"available":95916376064,"internally_reserved":0,"allocated":406450176,"data_stored":406164315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":313913004},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724600,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610144,"kb_used_data":317452,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93761696,"statfs":{"total":96636764160,"available":96011976704,"internally_reserved":0,"allocated":325070848,"data_stored":324752006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":299691691},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247883,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":824352,"kb_used_data":463692,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93547488,"statfs":{"total":96636764160,"available":95792627712,"internally_reserved":0,"allocated":474820608,"data_stored":474534742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":369290920},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2,2,0,0,0,4],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":473989120,"data_stored":473952990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":324677632,"data_stored":324608526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":405708800,"data_stored":405668598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:30:59.360 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:30:59.360 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'12414", 'reported_seq': 12578, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:57.261999+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:30:57.261999+0000', 'last_peered': '2024-09-17T15:30:57.261999+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:30:57.261999+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "34'2400", 'ondisk_log_start': "34'2400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33726464, 'num_objects': 8234, 'num_object_clones': 0, 'num_object_copies': 16468, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4029, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8234, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12414, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 41, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4205}, {'shards': '0,2', 'objects': 1980}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:59.360 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:30:59.360 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'12702", 'reported_seq': 12877, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:57.574855+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:30:57.574855+0000', 'last_peered': '2024-09-17T15:30:57.574855+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:30:57.574855+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "34'2700", 'ondisk_log_start': "34'2700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33714200, 'num_objects': 8232, 'num_object_clones': 0, 'num_object_copies': 16464, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4147, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8232, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10859, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 22, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1935}, {'shards': '2', 'objects': 4086}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:30:59.360 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:30:59.360 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'12837", 'reported_seq': 13061, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:57.314542+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:30:57.314542+0000', 'last_peered': '2024-09-17T15:30:57.314542+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:30:57.314542+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "34'2800", 'ondisk_log_start': "34'2800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34464227, 'num_objects': 8415, 'num_object_clones': 0, 'num_object_copies': 16830, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4228, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8415, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9336, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4188}, {'shards': '0,2', 'objects': 2059}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:59.360 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:30:59.360 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'12580", 'reported_seq': 12803, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:57.303248+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:30:57.303248+0000', 'last_peered': '2024-09-17T15:30:57.303248+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:30:57.303248+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "34'2500", 'ondisk_log_start': "34'2500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33539014, 'num_objects': 8190, 'num_object_clones': 0, 'num_object_copies': 16380, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4134, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8190, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 9349, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 40, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4056}, {'shards': '0,2', 'objects': 1981}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:59.361 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:30:59.361 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'12385", 'reported_seq': 12594, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:57.292382+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:30:57.292382+0000', 'last_peered': '2024-09-17T15:30:57.292382+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:30:57.292382+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "34'2300", 'ondisk_log_start': "34'2300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33411072, 'num_objects': 8157, 'num_object_clones': 0, 'num_object_copies': 16314, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3975, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8157, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8981, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 37, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4184}, {'shards': '0,2', 'objects': 1900}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:59.361 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:30:59.361 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'12411", 'reported_seq': 12574, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:57.523159+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:30:57.523159+0000', 'last_peered': '2024-09-17T15:30:57.523159+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:30:57.523159+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "34'2400", 'ondisk_log_start': "34'2400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33210851, 'num_objects': 8109, 'num_object_clones': 0, 'num_object_copies': 16218, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4059, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8109, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10306, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 19, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1928}, {'shards': '2', 'objects': 4051}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:30:59.361 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:30:59.361 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'12558", 'reported_seq': 12922, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:57.589593+0000', 'last_change': '2024-09-17T15:30:55.086473+0000', 'last_active': '2024-09-17T15:30:57.589593+0000', 'last_peered': '2024-09-17T15:30:57.589593+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:30:57.589593+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "34'2500", 'ondisk_log_start': "34'2500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33497088, 'num_objects': 8178, 'num_object_clones': 0, 'num_object_copies': 16356, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2112, 'num_objects_misplaced': 1907, 'num_objects_unfound': 0, 'num_objects_dirty': 8178, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10320, 'num_write_kb': 32532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 120, 'num_bytes_recovered': 380928, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1907}, {'shards': '2', 'objects': 4159}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:30:59.361 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:30:59.361 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'12688", 'reported_seq': 12907, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:57.203452+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:30:57.203452+0000', 'last_peered': '2024-09-17T15:30:57.203452+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:30:57.203452+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "34'2600", 'ondisk_log_start': "34'2600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33906688, 'num_objects': 8278, 'num_object_clones': 0, 'num_object_copies': 16556, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4099, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8278, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9236, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 40, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4180}, {'shards': '0,2', 'objects': 1934}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:59.361 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:30:59.361 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'12556", 'reported_seq': 12762, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:57.295220+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:30:57.295220+0000', 'last_peered': '2024-09-17T15:30:57.295220+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:30:57.295220+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "34'2500", 'ondisk_log_start': "34'2500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33652736, 'num_objects': 8216, 'num_object_clones': 0, 'num_object_copies': 16432, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4130, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8216, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8843, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4089}, {'shards': '0,2', 'objects': 1991}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:59.362 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:30:59.362 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'12586", 'reported_seq': 12810, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:30:57.247076+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:30:57.247076+0000', 'last_peered': '2024-09-17T15:30:57.247076+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:30:57.247076+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "34'2500", 'ondisk_log_start': "34'2500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33996800, 'num_objects': 8300, 'num_object_clones': 0, 'num_object_copies': 16600, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4021, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8300, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9373, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4281}, {'shards': '0,2', 'objects': 1922}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:30:59.362 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-17T15:30:59.362 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'12413", 'reported_seq': 20783, 'reported_epoch': 44, 'state': 'peering', 'last_fresh': '2024-09-17T15:30:55.996046+0000', 'last_change': '2024-09-17T15:30:56.600666+0000', 'last_active': '2024-09-17T15:30:56.600666+0000', 'last_peered': '2024-09-17T15:30:55.996046+0000', 'last_clean': '2024-09-17T15:26:20.164760+0000', 'last_became_active': '2024-09-17T15:26:22.644833+0000', 'last_became_peered': '2024-09-17T15:26:22.644833+0000', 'last_unstale': '2024-09-17T15:30:55.996046+0000', 'last_undegraded': '2024-09-17T15:30:55.996046+0000', 'last_fullsized': '2024-09-17T15:26:22.191932+0000', 'mapping_epoch': 44, 'log_start': "34'2400", 'ondisk_log_start': "34'2400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3601", 'last_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:11.852193+0000', 'objects_scrubbed': 3548, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1181, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33665990, 'num_objects': 8221, 'num_object_clones': 0, 'num_object_copies': 16442, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8221, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8855, 'num_write_kb': 27056, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4111, 'num_bytes_recovered': 16826851, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T15:30:59.362 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:31:00.056 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:31:00.056 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38934/264198 objects degraded (14.737%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":397,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"peering","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":132099,"data_bytes":541029129,"bytes_used":2189312000,"bytes_avail":287720980480,"bytes_total":289910292480,"inactive_pgs_ratio":0.041666667908430099,"degraded_objects":38934,"degraded_total":264198,"degraded_ratio":0.14736674766652283,"misplaced_objects":1907,"misplaced_total":264198,"misplaced_ratio":0.0072180712950135886,"recovering_objects_per_sec":59,"recovering_bytes_per_sec":221110,"recovering_keys_per_sec":0,"num_objects_recovered":473,"num_bytes_recovered":1769472,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":523},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T15:27:19.616376+0000","services":{}},"progress_events":{}} 2024-09-17T15:31:00.080 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:31:00.081 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:31:00.578 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:31:00.579 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:31:00.590 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":236,"stamp":"2024-09-17T15:30:59.686739+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":541029129,"num_objects":132099,"num_object_clones":0,"num_object_copies":264198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38934,"num_objects_misplaced":1907,"num_objects_unfound":0,"num_objects_dirty":132099,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":160892,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8620,"num_bytes_recovered":33879001,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160658,"ondisk_log_size":160658,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2138000,"kb_used_data":1178068,"kb_used_omap":13,"kb_used_meta":959858,"kb_avail":280977520,"statfs":{"total":289910292480,"available":287720980480,"internally_reserved":0,"allocated":1206341632,"data_stored":1205451063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":982895615},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,5,7,2,0,0,0,4],"upper_bound":2048},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16547840,"num_objects":-4040,"num_object_clones":0,"num_object_copies":-8080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2041,"num_objects_misplaced":1568,"num_objects_unfound":0,"num_objects_dirty":-4040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4040,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":473,"num_bytes_recovered":1769472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002721"},"pg_stats":[{"pgid":"2.f","version":"45'12414","reported_seq":12578,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:57.261999+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:30:57.261999+0000","last_peered":"2024-09-17T15:30:57.261999+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:30:57.261999+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'2400","ondisk_log_start":"34'2400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33726464,"num_objects":8234,"num_object_clones":0,"num_object_copies":16468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4029,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12414,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4205},{"shards":"0,2","objects":1980}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12546","reported_seq":12653,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.528950+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:30:57.528950+0000","last_peered":"2024-09-17T15:30:57.528950+0000","last_clean":"2024-09-17T15:30:57.528950+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:30:57.528950+0000","last_undegraded":"2024-09-17T15:30:57.528950+0000","last_fullsized":"2024-09-17T15:30:57.528950+0000","mapping_epoch":33,"log_start":"34'2500","ondisk_log_start":"34'2500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34350022,"num_objects":8388,"num_object_clones":0,"num_object_copies":16776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8388,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10903,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12702","reported_seq":12877,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:57.574855+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:30:57.574855+0000","last_peered":"2024-09-17T15:30:57.574855+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:30:57.574855+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'2700","ondisk_log_start":"34'2700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33714200,"num_objects":8232,"num_object_clones":0,"num_object_copies":16464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4147,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8232,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10859,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1935},{"shards":"2","objects":4086}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12606","reported_seq":12680,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.572435+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:30:57.572435+0000","last_peered":"2024-09-17T15:30:57.572435+0000","last_clean":"2024-09-17T15:30:57.572435+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:30:57.572435+0000","last_undegraded":"2024-09-17T15:30:57.572435+0000","last_fullsized":"2024-09-17T15:30:57.572435+0000","mapping_epoch":33,"log_start":"34'2600","ondisk_log_start":"34'2600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34258944,"num_objects":8364,"num_object_clones":0,"num_object_copies":16728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12606,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'12431","reported_seq":12565,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092639+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:30:57.092639+0000","last_peered":"2024-09-17T15:30:57.092639+0000","last_clean":"2024-09-17T15:30:57.092639+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:30:57.092639+0000","last_undegraded":"2024-09-17T15:30:57.092639+0000","last_fullsized":"2024-09-17T15:30:57.092639+0000","mapping_epoch":32,"log_start":"34'2400","ondisk_log_start":"34'2400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33677795,"num_objects":8223,"num_object_clones":0,"num_object_copies":16446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8223,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9413,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12837","reported_seq":13061,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:57.314542+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:30:57.314542+0000","last_peered":"2024-09-17T15:30:57.314542+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:30:57.314542+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'2800","ondisk_log_start":"34'2800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34464227,"num_objects":8415,"num_object_clones":0,"num_object_copies":16830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4228,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8415,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9336,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4188},{"shards":"0,2","objects":2059}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12580","reported_seq":12803,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:57.303248+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:30:57.303248+0000","last_peered":"2024-09-17T15:30:57.303248+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:30:57.303248+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'2500","ondisk_log_start":"34'2500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33539014,"num_objects":8190,"num_object_clones":0,"num_object_copies":16380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4134,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8190,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9349,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4056},{"shards":"0,2","objects":1981}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12620","reported_seq":12752,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.280105+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:30:57.280105+0000","last_peered":"2024-09-17T15:30:57.280105+0000","last_clean":"2024-09-17T15:30:57.280105+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:30:57.280105+0000","last_undegraded":"2024-09-17T15:30:57.280105+0000","last_fullsized":"2024-09-17T15:30:57.280105+0000","mapping_epoch":32,"log_start":"34'2600","ondisk_log_start":"34'2600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33980416,"num_objects":8296,"num_object_clones":0,"num_object_copies":16592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9574,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T15:30:56.094071+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:56.094071+0000","last_peered":"2024-09-17T15:30:56.094071+0000","last_clean":"2024-09-17T15:30:56.094071+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:56.094071+0000","last_undegraded":"2024-09-17T15:30:56.094071+0000","last_fullsized":"2024-09-17T15:30:56.094071+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'12522","reported_seq":20807,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T15:30:56.560349+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:30:56.560349+0000","last_peered":"2024-09-17T15:30:56.560349+0000","last_clean":"2024-09-17T15:30:56.560349+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:30:56.560349+0000","last_undegraded":"2024-09-17T15:30:56.560349+0000","last_fullsized":"2024-09-17T15:30:56.560349+0000","mapping_epoch":42,"log_start":"34'2500","ondisk_log_start":"34'2500","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33976803,"num_objects":8296,"num_object_clones":0,"num_object_copies":16592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8296,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10522,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088505+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088505+0000","last_peered":"2024-09-17T15:30:57.088505+0000","last_clean":"2024-09-17T15:30:57.088505+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088505+0000","last_undegraded":"2024-09-17T15:30:57.088505+0000","last_fullsized":"2024-09-17T15:30:57.088505+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12385","reported_seq":12594,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:57.292382+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:30:57.292382+0000","last_peered":"2024-09-17T15:30:57.292382+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:30:57.292382+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'2300","ondisk_log_start":"34'2300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33411072,"num_objects":8157,"num_object_clones":0,"num_object_copies":16314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3975,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8981,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4184},{"shards":"0,2","objects":1900}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":107,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092541+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092541+0000","last_peered":"2024-09-17T15:30:57.092541+0000","last_clean":"2024-09-17T15:30:57.092541+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092541+0000","last_undegraded":"2024-09-17T15:30:57.092541+0000","last_fullsized":"2024-09-17T15:30:57.092541+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12411","reported_seq":12574,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:57.523159+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:30:57.523159+0000","last_peered":"2024-09-17T15:30:57.523159+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:30:57.523159+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'2400","ondisk_log_start":"34'2400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33210851,"num_objects":8109,"num_object_clones":0,"num_object_copies":16218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4059,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8109,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10306,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1928},{"shards":"2","objects":4051}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":80,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T15:30:56.089532+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:56.089532+0000","last_peered":"2024-09-17T15:30:56.089532+0000","last_clean":"2024-09-17T15:30:56.089532+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:56.089532+0000","last_undegraded":"2024-09-17T15:30:56.089532+0000","last_fullsized":"2024-09-17T15:30:56.089532+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12558","reported_seq":12922,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:57.589593+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:30:57.589593+0000","last_peered":"2024-09-17T15:30:57.589593+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:30:57.589593+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'2500","ondisk_log_start":"34'2500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33497088,"num_objects":8178,"num_object_clones":0,"num_object_copies":16356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2112,"num_objects_misplaced":1907,"num_objects_unfound":0,"num_objects_dirty":8178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10320,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":380928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1907},{"shards":"2","objects":4159}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12688","reported_seq":12907,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:57.203452+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:30:57.203452+0000","last_peered":"2024-09-17T15:30:57.203452+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:30:57.203452+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'2600","ondisk_log_start":"34'2600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33906688,"num_objects":8278,"num_object_clones":0,"num_object_copies":16556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4099,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8278,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9236,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4180},{"shards":"0,2","objects":1934}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T15:30:56.082651+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:56.082651+0000","last_peered":"2024-09-17T15:30:56.082651+0000","last_clean":"2024-09-17T15:30:56.082651+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:56.082651+0000","last_undegraded":"2024-09-17T15:30:56.082651+0000","last_fullsized":"2024-09-17T15:30:56.082651+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12556","reported_seq":12762,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:57.295220+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:30:57.295220+0000","last_peered":"2024-09-17T15:30:57.295220+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:30:57.295220+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'2500","ondisk_log_start":"34'2500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33652736,"num_objects":8216,"num_object_clones":0,"num_object_copies":16432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4130,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8843,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4089},{"shards":"0,2","objects":1991}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088855+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088855+0000","last_peered":"2024-09-17T15:30:57.088855+0000","last_clean":"2024-09-17T15:30:57.088855+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088855+0000","last_undegraded":"2024-09-17T15:30:57.088855+0000","last_fullsized":"2024-09-17T15:30:57.088855+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12586","reported_seq":12810,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:30:57.247076+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:30:57.247076+0000","last_peered":"2024-09-17T15:30:57.247076+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:30:57.247076+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'2500","ondisk_log_start":"34'2500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33996800,"num_objects":8300,"num_object_clones":0,"num_object_copies":16600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4021,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9373,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4281},{"shards":"0,2","objects":1922}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T15:30:56.083102+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:56.083102+0000","last_peered":"2024-09-17T15:30:56.083102+0000","last_clean":"2024-09-17T15:30:56.083102+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:56.083102+0000","last_undegraded":"2024-09-17T15:30:56.083102+0000","last_fullsized":"2024-09-17T15:30:56.083102+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T15:30:56.082189+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:56.082189+0000","last_peered":"2024-09-17T15:30:56.082189+0000","last_clean":"2024-09-17T15:30:56.082189+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:56.082189+0000","last_undegraded":"2024-09-17T15:30:56.082189+0000","last_fullsized":"2024-09-17T15:30:56.082189+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'12413","reported_seq":20783,"reported_epoch":44,"state":"peering","last_fresh":"2024-09-17T15:30:55.996046+0000","last_change":"2024-09-17T15:30:56.600666+0000","last_active":"2024-09-17T15:30:56.600666+0000","last_peered":"2024-09-17T15:30:55.996046+0000","last_clean":"2024-09-17T15:26:20.164760+0000","last_became_active":"2024-09-17T15:26:22.644833+0000","last_became_peered":"2024-09-17T15:26:22.644833+0000","last_unstale":"2024-09-17T15:30:55.996046+0000","last_undegraded":"2024-09-17T15:30:55.996046+0000","last_fullsized":"2024-09-17T15:26:22.191932+0000","mapping_epoch":44,"log_start":"34'2400","ondisk_log_start":"34'2400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33665990,"num_objects":8221,"num_object_clones":0,"num_object_copies":16442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8855,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":541029110,"num_objects":132097,"num_object_clones":0,"num_object_copies":264194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38934,"num_objects_misplaced":1907,"num_objects_unfound":0,"num_objects_dirty":132097,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":160890,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8619,"num_bytes_recovered":33878982,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1204375552,"data_stored":1204230114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160655,"ondisk_log_size":160655,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051655,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703504,"kb_used_data":396924,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93668336,"statfs":{"total":96636764160,"available":95916376064,"internally_reserved":0,"allocated":406450176,"data_stored":406164315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":313913004},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724600,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610144,"kb_used_data":317452,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93761696,"statfs":{"total":96636764160,"available":96011976704,"internally_reserved":0,"allocated":325070848,"data_stored":324752006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":299691691},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247883,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":824352,"kb_used_data":463692,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93547488,"statfs":{"total":96636764160,"available":95792627712,"internally_reserved":0,"allocated":474820608,"data_stored":474534742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":369290920},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2,2,0,0,0,4],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":473989120,"data_stored":473952990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":324677632,"data_stored":324608526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":405708800,"data_stored":405668598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:31:03.592 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:31:04.154 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:31:04.155 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:31:04.179 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":238,"stamp":"2024-09-17T15:31:03.687649+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":532284169,"num_objects":129964,"num_object_clones":0,"num_object_copies":259928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38917,"num_objects_misplaced":1780,"num_objects_unfound":0,"num_objects_dirty":129964,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":163027,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8764,"num_bytes_recovered":34399193,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160793,"ondisk_log_size":160793,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2141260,"kb_used_data":1167440,"kb_used_omap":13,"kb_used_meta":973746,"kb_avail":280974260,"statfs":{"total":289910292480,"available":287717642240,"internally_reserved":0,"allocated":1195458560,"data_stored":1194552283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":997116927},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2,9,8,7,1,0,2,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16515072,"num_objects":-4032,"num_object_clones":0,"num_object_copies":-8064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2035,"num_objects_misplaced":1652,"num_objects_unfound":0,"num_objects_dirty":-4032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4032,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":383,"num_bytes_recovered":1425408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002240"},"pg_stats":[{"pgid":"2.f","version":"45'12557","reported_seq":12722,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:02.211162+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:31:02.211162+0000","last_peered":"2024-09-17T15:31:02.211162+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:31:02.211162+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'2500","ondisk_log_start":"34'2500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33140736,"num_objects":8091,"num_object_clones":0,"num_object_copies":16182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8091,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12557,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4063},{"shards":"0,2","objects":1837}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12686","reported_seq":12793,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:02.359394+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:31:02.359394+0000","last_peered":"2024-09-17T15:31:02.359394+0000","last_clean":"2024-09-17T15:31:02.359394+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:31:02.359394+0000","last_undegraded":"2024-09-17T15:31:02.359394+0000","last_fullsized":"2024-09-17T15:31:02.359394+0000","mapping_epoch":33,"log_start":"34'2600","ondisk_log_start":"34'2600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33776582,"num_objects":8248,"num_object_clones":0,"num_object_copies":16496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8248,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11043,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12823","reported_seq":13002,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:02.564574+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:31:02.564574+0000","last_peered":"2024-09-17T15:31:02.564574+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:31:02.564574+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'2800","ondisk_log_start":"34'2800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33218584,"num_objects":8111,"num_object_clones":0,"num_object_copies":16222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4145,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8111,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10980,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1814},{"shards":"2","objects":3967}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12742","reported_seq":12816,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:02.563040+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:31:02.563040+0000","last_peered":"2024-09-17T15:31:02.563040+0000","last_clean":"2024-09-17T15:31:02.563040+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:31:02.563040+0000","last_undegraded":"2024-09-17T15:31:02.563040+0000","last_fullsized":"2024-09-17T15:31:02.563040+0000","mapping_epoch":33,"log_start":"34'2700","ondisk_log_start":"34'2700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33701888,"num_objects":8228,"num_object_clones":0,"num_object_copies":16456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12742,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12562","reported_seq":12696,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:02.238414+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:31:02.238414+0000","last_peered":"2024-09-17T15:31:02.238414+0000","last_clean":"2024-09-17T15:31:02.238414+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:31:02.238414+0000","last_undegraded":"2024-09-17T15:31:02.238414+0000","last_fullsized":"2024-09-17T15:31:02.238414+0000","mapping_epoch":32,"log_start":"34'2500","ondisk_log_start":"34'2500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33141219,"num_objects":8092,"num_object_clones":0,"num_object_copies":16184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8092,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9544,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12971","reported_seq":13200,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:02.249658+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:31:02.249658+0000","last_peered":"2024-09-17T15:31:02.249658+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:31:02.249658+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'2900","ondisk_log_start":"34'2900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33915363,"num_objects":8281,"num_object_clones":0,"num_object_copies":16562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4226,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8281,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9470,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4056},{"shards":"0,2","objects":1925}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12718","reported_seq":12955,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:02.257883+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:31:02.257883+0000","last_peered":"2024-09-17T15:31:02.257883+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:31:02.257883+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'2700","ondisk_log_start":"34'2700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32973766,"num_objects":8052,"num_object_clones":0,"num_object_copies":16104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4127,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8052,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9487,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3925},{"shards":"0,2","objects":1843}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12757","reported_seq":12889,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:02.200711+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:31:02.200711+0000","last_peered":"2024-09-17T15:31:02.200711+0000","last_clean":"2024-09-17T15:31:02.200711+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:31:02.200711+0000","last_undegraded":"2024-09-17T15:31:02.200711+0000","last_fullsized":"2024-09-17T15:31:02.200711+0000","mapping_epoch":32,"log_start":"34'2700","ondisk_log_start":"34'2700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33419264,"num_objects":8159,"num_object_clones":0,"num_object_copies":16318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9711,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.086967+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.086967+0000","last_peered":"2024-09-17T15:30:57.086967+0000","last_clean":"2024-09-17T15:30:57.086967+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.086967+0000","last_undegraded":"2024-09-17T15:30:57.086967+0000","last_fullsized":"2024-09-17T15:30:57.086967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12640","reported_seq":20927,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:01.601800+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:31:01.601800+0000","last_peered":"2024-09-17T15:31:01.601800+0000","last_clean":"2024-09-17T15:31:01.601800+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:31:01.601800+0000","last_undegraded":"2024-09-17T15:31:01.601800+0000","last_fullsized":"2024-09-17T15:31:01.601800+0000","mapping_epoch":42,"log_start":"34'2600","ondisk_log_start":"34'2600","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33493475,"num_objects":8178,"num_object_clones":0,"num_object_copies":16356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8178,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10640,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088505+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088505+0000","last_peered":"2024-09-17T15:30:57.088505+0000","last_clean":"2024-09-17T15:30:57.088505+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088505+0000","last_undegraded":"2024-09-17T15:30:57.088505+0000","last_fullsized":"2024-09-17T15:30:57.088505+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12504","reported_seq":12717,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:02.234441+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:31:02.234441+0000","last_peered":"2024-09-17T15:31:02.234441+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:31:02.234441+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'2500","ondisk_log_start":"34'2500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32923648,"num_objects":8038,"num_object_clones":0,"num_object_copies":16076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9100,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4067},{"shards":"0,2","objects":1781}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":107,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092541+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092541+0000","last_peered":"2024-09-17T15:30:57.092541+0000","last_clean":"2024-09-17T15:30:57.092541+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092541+0000","last_undegraded":"2024-09-17T15:30:57.092541+0000","last_fullsized":"2024-09-17T15:30:57.092541+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12537","reported_seq":12704,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:02.564012+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:31:02.564012+0000","last_peered":"2024-09-17T15:31:02.564012+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:31:02.564012+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'2500","ondisk_log_start":"34'2500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32694755,"num_objects":7983,"num_object_clones":0,"num_object_copies":15966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4057,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7983,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10432,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1802},{"shards":"2","objects":3927}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12684","reported_seq":13302,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:02.558774+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:31:02.558774+0000","last_peered":"2024-09-17T15:31:02.558774+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:31:02.558774+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'2600","ondisk_log_start":"34'2600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32980992,"num_objects":8052,"num_object_clones":0,"num_object_copies":16104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2112,"num_objects_misplaced":1780,"num_objects_unfound":0,"num_objects_dirty":8052,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10446,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":247,"num_bytes_recovered":901120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1780},{"shards":"2","objects":4160}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12821","reported_seq":13041,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:02.300353+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:31:02.300353+0000","last_peered":"2024-09-17T15:31:02.300353+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:31:02.300353+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'2800","ondisk_log_start":"34'2800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33361920,"num_objects":8145,"num_object_clones":0,"num_object_copies":16290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4099,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9369,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4047},{"shards":"0,2","objects":1801}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12695","reported_seq":12901,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:02.268206+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:31:02.268206+0000","last_peered":"2024-09-17T15:31:02.268206+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:31:02.268206+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'2600","ondisk_log_start":"34'2600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33083392,"num_objects":8077,"num_object_clones":0,"num_object_copies":16154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4130,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8982,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3950},{"shards":"0,2","objects":1852}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088855+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088855+0000","last_peered":"2024-09-17T15:30:57.088855+0000","last_clean":"2024-09-17T15:30:57.088855+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088855+0000","last_undegraded":"2024-09-17T15:30:57.088855+0000","last_fullsized":"2024-09-17T15:30:57.088855+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12730","reported_seq":12953,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:02.326115+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:31:02.326115+0000","last_peered":"2024-09-17T15:31:02.326115+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:31:02.326115+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'2700","ondisk_log_start":"34'2700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33406976,"num_objects":8156,"num_object_clones":0,"num_object_copies":16312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4020,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9517,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4138},{"shards":"0,2","objects":1778}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12563","reported_seq":20942,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:01.576711+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:31:01.576711+0000","last_peered":"2024-09-17T15:31:01.576711+0000","last_clean":"2024-09-17T15:31:01.576711+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:31:01.576711+0000","last_undegraded":"2024-09-17T15:31:01.576711+0000","last_fullsized":"2024-09-17T15:31:01.576711+0000","mapping_epoch":44,"log_start":"34'2500","ondisk_log_start":"34'2500","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33051590,"num_objects":8071,"num_object_clones":0,"num_object_copies":16142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9005,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":532284150,"num_objects":129962,"num_object_clones":0,"num_object_copies":259924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38917,"num_objects_misplaced":1780,"num_objects_unfound":0,"num_objects_dirty":129962,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":163025,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8763,"num_bytes_recovered":34399174,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1192341504,"data_stored":1192196066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160790,"ondisk_log_size":160790,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051656,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703632,"kb_used_data":392896,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93668208,"statfs":{"total":96636764160,"available":95916244992,"internally_reserved":0,"allocated":402325504,"data_stored":402034407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":318172844},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,0,1,0,0,2],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724601,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":614808,"kb_used_data":316552,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93757032,"statfs":{"total":96636764160,"available":96007200768,"internally_reserved":0,"allocated":324149248,"data_stored":323825170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":305393323},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247884,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":822820,"kb_used_data":457992,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93549020,"statfs":{"total":96636764160,"available":95794196480,"internally_reserved":0,"allocated":468983808,"data_stored":468692706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":373550760},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,4,6,4,1,0,0,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":467263488,"data_stored":467227358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":323686400,"data_stored":323617294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":401391616,"data_stored":401351414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:31:04.181 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:31:04.711 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:31:04.711 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:31:04.732 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":238,"stamp":"2024-09-17T15:31:03.687649+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":532284169,"num_objects":129964,"num_object_clones":0,"num_object_copies":259928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38917,"num_objects_misplaced":1780,"num_objects_unfound":0,"num_objects_dirty":129964,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":163027,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8764,"num_bytes_recovered":34399193,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160793,"ondisk_log_size":160793,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2141260,"kb_used_data":1167440,"kb_used_omap":13,"kb_used_meta":973746,"kb_avail":280974260,"statfs":{"total":289910292480,"available":287717642240,"internally_reserved":0,"allocated":1195458560,"data_stored":1194552283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":997116927},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2,9,8,7,1,0,2,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16515072,"num_objects":-4032,"num_object_clones":0,"num_object_copies":-8064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2035,"num_objects_misplaced":1652,"num_objects_unfound":0,"num_objects_dirty":-4032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4032,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":383,"num_bytes_recovered":1425408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002240"},"pg_stats":[{"pgid":"2.f","version":"45'12557","reported_seq":12722,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:02.211162+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:31:02.211162+0000","last_peered":"2024-09-17T15:31:02.211162+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:31:02.211162+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'2500","ondisk_log_start":"34'2500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33140736,"num_objects":8091,"num_object_clones":0,"num_object_copies":16182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8091,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12557,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4063},{"shards":"0,2","objects":1837}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12686","reported_seq":12793,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:02.359394+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:31:02.359394+0000","last_peered":"2024-09-17T15:31:02.359394+0000","last_clean":"2024-09-17T15:31:02.359394+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:31:02.359394+0000","last_undegraded":"2024-09-17T15:31:02.359394+0000","last_fullsized":"2024-09-17T15:31:02.359394+0000","mapping_epoch":33,"log_start":"34'2600","ondisk_log_start":"34'2600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33776582,"num_objects":8248,"num_object_clones":0,"num_object_copies":16496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8248,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11043,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12823","reported_seq":13002,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:02.564574+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:31:02.564574+0000","last_peered":"2024-09-17T15:31:02.564574+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:31:02.564574+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'2800","ondisk_log_start":"34'2800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33218584,"num_objects":8111,"num_object_clones":0,"num_object_copies":16222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4145,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8111,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10980,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1814},{"shards":"2","objects":3967}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12742","reported_seq":12816,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:02.563040+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:31:02.563040+0000","last_peered":"2024-09-17T15:31:02.563040+0000","last_clean":"2024-09-17T15:31:02.563040+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:31:02.563040+0000","last_undegraded":"2024-09-17T15:31:02.563040+0000","last_fullsized":"2024-09-17T15:31:02.563040+0000","mapping_epoch":33,"log_start":"34'2700","ondisk_log_start":"34'2700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33701888,"num_objects":8228,"num_object_clones":0,"num_object_copies":16456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12742,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12562","reported_seq":12696,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:02.238414+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:31:02.238414+0000","last_peered":"2024-09-17T15:31:02.238414+0000","last_clean":"2024-09-17T15:31:02.238414+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:31:02.238414+0000","last_undegraded":"2024-09-17T15:31:02.238414+0000","last_fullsized":"2024-09-17T15:31:02.238414+0000","mapping_epoch":32,"log_start":"34'2500","ondisk_log_start":"34'2500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33141219,"num_objects":8092,"num_object_clones":0,"num_object_copies":16184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8092,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9544,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12971","reported_seq":13200,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:02.249658+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:31:02.249658+0000","last_peered":"2024-09-17T15:31:02.249658+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:31:02.249658+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'2900","ondisk_log_start":"34'2900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33915363,"num_objects":8281,"num_object_clones":0,"num_object_copies":16562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4226,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8281,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9470,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4056},{"shards":"0,2","objects":1925}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12718","reported_seq":12955,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:02.257883+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:31:02.257883+0000","last_peered":"2024-09-17T15:31:02.257883+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:31:02.257883+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'2700","ondisk_log_start":"34'2700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32973766,"num_objects":8052,"num_object_clones":0,"num_object_copies":16104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4127,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8052,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9487,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3925},{"shards":"0,2","objects":1843}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12757","reported_seq":12889,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:02.200711+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:31:02.200711+0000","last_peered":"2024-09-17T15:31:02.200711+0000","last_clean":"2024-09-17T15:31:02.200711+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:31:02.200711+0000","last_undegraded":"2024-09-17T15:31:02.200711+0000","last_fullsized":"2024-09-17T15:31:02.200711+0000","mapping_epoch":32,"log_start":"34'2700","ondisk_log_start":"34'2700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33419264,"num_objects":8159,"num_object_clones":0,"num_object_copies":16318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9711,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.086967+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.086967+0000","last_peered":"2024-09-17T15:30:57.086967+0000","last_clean":"2024-09-17T15:30:57.086967+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.086967+0000","last_undegraded":"2024-09-17T15:30:57.086967+0000","last_fullsized":"2024-09-17T15:30:57.086967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12640","reported_seq":20927,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:01.601800+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:31:01.601800+0000","last_peered":"2024-09-17T15:31:01.601800+0000","last_clean":"2024-09-17T15:31:01.601800+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:31:01.601800+0000","last_undegraded":"2024-09-17T15:31:01.601800+0000","last_fullsized":"2024-09-17T15:31:01.601800+0000","mapping_epoch":42,"log_start":"34'2600","ondisk_log_start":"34'2600","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33493475,"num_objects":8178,"num_object_clones":0,"num_object_copies":16356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8178,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10640,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088505+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088505+0000","last_peered":"2024-09-17T15:30:57.088505+0000","last_clean":"2024-09-17T15:30:57.088505+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088505+0000","last_undegraded":"2024-09-17T15:30:57.088505+0000","last_fullsized":"2024-09-17T15:30:57.088505+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12504","reported_seq":12717,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:02.234441+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:31:02.234441+0000","last_peered":"2024-09-17T15:31:02.234441+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:31:02.234441+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'2500","ondisk_log_start":"34'2500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32923648,"num_objects":8038,"num_object_clones":0,"num_object_copies":16076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9100,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4067},{"shards":"0,2","objects":1781}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":107,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092541+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092541+0000","last_peered":"2024-09-17T15:30:57.092541+0000","last_clean":"2024-09-17T15:30:57.092541+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092541+0000","last_undegraded":"2024-09-17T15:30:57.092541+0000","last_fullsized":"2024-09-17T15:30:57.092541+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12537","reported_seq":12704,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:02.564012+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:31:02.564012+0000","last_peered":"2024-09-17T15:31:02.564012+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:31:02.564012+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'2500","ondisk_log_start":"34'2500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32694755,"num_objects":7983,"num_object_clones":0,"num_object_copies":15966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4057,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7983,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10432,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1802},{"shards":"2","objects":3927}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12684","reported_seq":13302,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:02.558774+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:31:02.558774+0000","last_peered":"2024-09-17T15:31:02.558774+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:31:02.558774+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'2600","ondisk_log_start":"34'2600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32980992,"num_objects":8052,"num_object_clones":0,"num_object_copies":16104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2112,"num_objects_misplaced":1780,"num_objects_unfound":0,"num_objects_dirty":8052,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10446,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":247,"num_bytes_recovered":901120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1780},{"shards":"2","objects":4160}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12821","reported_seq":13041,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:02.300353+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:31:02.300353+0000","last_peered":"2024-09-17T15:31:02.300353+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:31:02.300353+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'2800","ondisk_log_start":"34'2800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33361920,"num_objects":8145,"num_object_clones":0,"num_object_copies":16290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4099,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9369,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4047},{"shards":"0,2","objects":1801}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12695","reported_seq":12901,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:02.268206+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:31:02.268206+0000","last_peered":"2024-09-17T15:31:02.268206+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:31:02.268206+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'2600","ondisk_log_start":"34'2600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33083392,"num_objects":8077,"num_object_clones":0,"num_object_copies":16154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4130,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8982,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3950},{"shards":"0,2","objects":1852}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088855+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088855+0000","last_peered":"2024-09-17T15:30:57.088855+0000","last_clean":"2024-09-17T15:30:57.088855+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088855+0000","last_undegraded":"2024-09-17T15:30:57.088855+0000","last_fullsized":"2024-09-17T15:30:57.088855+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12730","reported_seq":12953,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:02.326115+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:31:02.326115+0000","last_peered":"2024-09-17T15:31:02.326115+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:31:02.326115+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'2700","ondisk_log_start":"34'2700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33406976,"num_objects":8156,"num_object_clones":0,"num_object_copies":16312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4020,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9517,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4138},{"shards":"0,2","objects":1778}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12563","reported_seq":20942,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:01.576711+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:31:01.576711+0000","last_peered":"2024-09-17T15:31:01.576711+0000","last_clean":"2024-09-17T15:31:01.576711+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:31:01.576711+0000","last_undegraded":"2024-09-17T15:31:01.576711+0000","last_fullsized":"2024-09-17T15:31:01.576711+0000","mapping_epoch":44,"log_start":"34'2500","ondisk_log_start":"34'2500","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33051590,"num_objects":8071,"num_object_clones":0,"num_object_copies":16142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9005,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":532284150,"num_objects":129962,"num_object_clones":0,"num_object_copies":259924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38917,"num_objects_misplaced":1780,"num_objects_unfound":0,"num_objects_dirty":129962,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":163025,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8763,"num_bytes_recovered":34399174,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1192341504,"data_stored":1192196066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160790,"ondisk_log_size":160790,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051656,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703632,"kb_used_data":392896,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93668208,"statfs":{"total":96636764160,"available":95916244992,"internally_reserved":0,"allocated":402325504,"data_stored":402034407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":318172844},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,0,1,0,0,2],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724601,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":614808,"kb_used_data":316552,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93757032,"statfs":{"total":96636764160,"available":96007200768,"internally_reserved":0,"allocated":324149248,"data_stored":323825170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":305393323},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247884,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":822820,"kb_used_data":457992,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93549020,"statfs":{"total":96636764160,"available":95794196480,"internally_reserved":0,"allocated":468983808,"data_stored":468692706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":373550760},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,4,6,4,1,0,0,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":467263488,"data_stored":467227358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":323686400,"data_stored":323617294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":401391616,"data_stored":401351414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:31:04.733 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:31:04.733 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'12557", 'reported_seq': 12722, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:02.211162+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:31:02.211162+0000', 'last_peered': '2024-09-17T15:31:02.211162+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:31:02.211162+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "34'2500", 'ondisk_log_start': "34'2500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33140736, 'num_objects': 8091, 'num_object_clones': 0, 'num_object_copies': 16182, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4028, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8091, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12557, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 42, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4063}, {'shards': '0,2', 'objects': 1837}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:04.734 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:31:04.734 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'12823", 'reported_seq': 13002, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:02.564574+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:31:02.564574+0000', 'last_peered': '2024-09-17T15:31:02.564574+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:31:02.564574+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "34'2800", 'ondisk_log_start': "34'2800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33218584, 'num_objects': 8111, 'num_object_clones': 0, 'num_object_copies': 16222, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4145, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8111, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10980, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 24, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1814}, {'shards': '2', 'objects': 3967}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:31:04.734 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:31:04.734 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'12971", 'reported_seq': 13200, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:02.249658+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:31:02.249658+0000', 'last_peered': '2024-09-17T15:31:02.249658+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:31:02.249658+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "34'2900", 'ondisk_log_start': "34'2900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33915363, 'num_objects': 8281, 'num_object_clones': 0, 'num_object_copies': 16562, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4226, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8281, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9470, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 45, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4056}, {'shards': '0,2', 'objects': 1925}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:04.734 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:31:04.734 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'12718", 'reported_seq': 12955, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:02.257883+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:31:02.257883+0000', 'last_peered': '2024-09-17T15:31:02.257883+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:31:02.257883+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "34'2700", 'ondisk_log_start': "34'2700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32973766, 'num_objects': 8052, 'num_object_clones': 0, 'num_object_copies': 16104, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4127, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8052, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 9487, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 47, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3925}, {'shards': '0,2', 'objects': 1843}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:04.734 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:31:04.734 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'12504", 'reported_seq': 12717, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:02.234441+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:31:02.234441+0000', 'last_peered': '2024-09-17T15:31:02.234441+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:31:02.234441+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "34'2500", 'ondisk_log_start': "34'2500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32923648, 'num_objects': 8038, 'num_object_clones': 0, 'num_object_copies': 16076, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3973, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8038, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9100, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 39, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4067}, {'shards': '0,2', 'objects': 1781}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:04.734 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:31:04.735 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'12537", 'reported_seq': 12704, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:02.564012+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:31:02.564012+0000', 'last_peered': '2024-09-17T15:31:02.564012+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:31:02.564012+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "34'2500", 'ondisk_log_start': "34'2500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32694755, 'num_objects': 7983, 'num_object_clones': 0, 'num_object_copies': 15966, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4057, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7983, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10432, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 21, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1802}, {'shards': '2', 'objects': 3927}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:31:04.735 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:31:04.735 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'12684", 'reported_seq': 13302, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:02.558774+0000', 'last_change': '2024-09-17T15:30:55.086473+0000', 'last_active': '2024-09-17T15:31:02.558774+0000', 'last_peered': '2024-09-17T15:31:02.558774+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:31:02.558774+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "34'2600", 'ondisk_log_start': "34'2600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32980992, 'num_objects': 8052, 'num_object_clones': 0, 'num_object_copies': 16104, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2112, 'num_objects_misplaced': 1780, 'num_objects_unfound': 0, 'num_objects_dirty': 8052, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10446, 'num_write_kb': 32532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 247, 'num_bytes_recovered': 901120, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1780}, {'shards': '2', 'objects': 4160}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:31:04.735 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:31:04.735 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'12821", 'reported_seq': 13041, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:02.300353+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:31:02.300353+0000', 'last_peered': '2024-09-17T15:31:02.300353+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:31:02.300353+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "34'2800", 'ondisk_log_start': "34'2800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33361920, 'num_objects': 8145, 'num_object_clones': 0, 'num_object_copies': 16290, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4099, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8145, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9369, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 40, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4047}, {'shards': '0,2', 'objects': 1801}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:04.735 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:31:04.735 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'12695", 'reported_seq': 12901, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:02.268206+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:31:02.268206+0000', 'last_peered': '2024-09-17T15:31:02.268206+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:31:02.268206+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "34'2600", 'ondisk_log_start': "34'2600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33083392, 'num_objects': 8077, 'num_object_clones': 0, 'num_object_copies': 16154, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4130, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8077, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8982, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3950}, {'shards': '0,2', 'objects': 1852}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:04.735 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:31:04.735 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'12730", 'reported_seq': 12953, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:02.326115+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:31:02.326115+0000', 'last_peered': '2024-09-17T15:31:02.326115+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:31:02.326115+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "34'2700", 'ondisk_log_start': "34'2700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33406976, 'num_objects': 8156, 'num_object_clones': 0, 'num_object_copies': 16312, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4020, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8156, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9517, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 45, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4138}, {'shards': '0,2', 'objects': 1778}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:04.736 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:31:05.416 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:31:05.416 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38917/259928 objects degraded (14.972%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":402,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":129964,"data_bytes":532284169,"bytes_used":2192650240,"bytes_avail":287717642240,"bytes_total":289910292480,"degraded_objects":38917,"degraded_total":259928,"degraded_ratio":0.1497222307715983,"misplaced_objects":1780,"misplaced_total":259928,"misplaced_ratio":0.0068480502292942672,"recovering_objects_per_sec":47,"recovering_bytes_per_sec":178126,"recovering_keys_per_sec":0,"num_objects_recovered":383,"num_bytes_recovered":1425408,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":503},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T15:27:19.616376+0000","services":{}},"progress_events":{}} 2024-09-17T15:31:05.440 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:31:05.440 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:31:05.961 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:31:05.964 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:31:05.984 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":239,"stamp":"2024-09-17T15:31:05.688110+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":532284169,"num_objects":129964,"num_object_clones":0,"num_object_copies":259928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38917,"num_objects_misplaced":1780,"num_objects_unfound":0,"num_objects_dirty":129964,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":163027,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8764,"num_bytes_recovered":34399193,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160793,"ondisk_log_size":160793,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2141260,"kb_used_data":1167440,"kb_used_omap":13,"kb_used_meta":973746,"kb_avail":280974260,"statfs":{"total":289910292480,"available":287717642240,"internally_reserved":0,"allocated":1195458560,"data_stored":1194552283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":997116927},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2,9,8,7,1,0,2,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16515072,"num_objects":-4032,"num_object_clones":0,"num_object_copies":-8064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2035,"num_objects_misplaced":1652,"num_objects_unfound":0,"num_objects_dirty":-4032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4032,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":383,"num_bytes_recovered":1425408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.610388"},"pg_stats":[{"pgid":"2.f","version":"45'12557","reported_seq":12722,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:02.211162+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:31:02.211162+0000","last_peered":"2024-09-17T15:31:02.211162+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:31:02.211162+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'2500","ondisk_log_start":"34'2500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33140736,"num_objects":8091,"num_object_clones":0,"num_object_copies":16182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8091,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12557,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4063},{"shards":"0,2","objects":1837}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12686","reported_seq":12793,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:02.359394+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:31:02.359394+0000","last_peered":"2024-09-17T15:31:02.359394+0000","last_clean":"2024-09-17T15:31:02.359394+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:31:02.359394+0000","last_undegraded":"2024-09-17T15:31:02.359394+0000","last_fullsized":"2024-09-17T15:31:02.359394+0000","mapping_epoch":33,"log_start":"34'2600","ondisk_log_start":"34'2600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33776582,"num_objects":8248,"num_object_clones":0,"num_object_copies":16496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8248,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11043,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12823","reported_seq":13002,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:02.564574+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:31:02.564574+0000","last_peered":"2024-09-17T15:31:02.564574+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:31:02.564574+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'2800","ondisk_log_start":"34'2800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33218584,"num_objects":8111,"num_object_clones":0,"num_object_copies":16222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4145,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8111,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10980,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1814},{"shards":"2","objects":3967}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12742","reported_seq":12816,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:02.563040+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:31:02.563040+0000","last_peered":"2024-09-17T15:31:02.563040+0000","last_clean":"2024-09-17T15:31:02.563040+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:31:02.563040+0000","last_undegraded":"2024-09-17T15:31:02.563040+0000","last_fullsized":"2024-09-17T15:31:02.563040+0000","mapping_epoch":33,"log_start":"34'2700","ondisk_log_start":"34'2700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33701888,"num_objects":8228,"num_object_clones":0,"num_object_copies":16456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12742,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12562","reported_seq":12696,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:02.238414+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:31:02.238414+0000","last_peered":"2024-09-17T15:31:02.238414+0000","last_clean":"2024-09-17T15:31:02.238414+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:31:02.238414+0000","last_undegraded":"2024-09-17T15:31:02.238414+0000","last_fullsized":"2024-09-17T15:31:02.238414+0000","mapping_epoch":32,"log_start":"34'2500","ondisk_log_start":"34'2500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33141219,"num_objects":8092,"num_object_clones":0,"num_object_copies":16184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8092,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9544,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12971","reported_seq":13200,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:02.249658+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:31:02.249658+0000","last_peered":"2024-09-17T15:31:02.249658+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:31:02.249658+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'2900","ondisk_log_start":"34'2900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33915363,"num_objects":8281,"num_object_clones":0,"num_object_copies":16562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4226,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8281,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9470,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4056},{"shards":"0,2","objects":1925}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12718","reported_seq":12955,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:02.257883+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:31:02.257883+0000","last_peered":"2024-09-17T15:31:02.257883+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:31:02.257883+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'2700","ondisk_log_start":"34'2700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32973766,"num_objects":8052,"num_object_clones":0,"num_object_copies":16104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4127,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8052,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9487,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3925},{"shards":"0,2","objects":1843}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12757","reported_seq":12889,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:02.200711+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:31:02.200711+0000","last_peered":"2024-09-17T15:31:02.200711+0000","last_clean":"2024-09-17T15:31:02.200711+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:31:02.200711+0000","last_undegraded":"2024-09-17T15:31:02.200711+0000","last_fullsized":"2024-09-17T15:31:02.200711+0000","mapping_epoch":32,"log_start":"34'2700","ondisk_log_start":"34'2700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33419264,"num_objects":8159,"num_object_clones":0,"num_object_copies":16318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9711,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.086967+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.086967+0000","last_peered":"2024-09-17T15:30:57.086967+0000","last_clean":"2024-09-17T15:30:57.086967+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.086967+0000","last_undegraded":"2024-09-17T15:30:57.086967+0000","last_fullsized":"2024-09-17T15:30:57.086967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12640","reported_seq":20927,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:01.601800+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:31:01.601800+0000","last_peered":"2024-09-17T15:31:01.601800+0000","last_clean":"2024-09-17T15:31:01.601800+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:31:01.601800+0000","last_undegraded":"2024-09-17T15:31:01.601800+0000","last_fullsized":"2024-09-17T15:31:01.601800+0000","mapping_epoch":42,"log_start":"34'2600","ondisk_log_start":"34'2600","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33493475,"num_objects":8178,"num_object_clones":0,"num_object_copies":16356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8178,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10640,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088505+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088505+0000","last_peered":"2024-09-17T15:30:57.088505+0000","last_clean":"2024-09-17T15:30:57.088505+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088505+0000","last_undegraded":"2024-09-17T15:30:57.088505+0000","last_fullsized":"2024-09-17T15:30:57.088505+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12504","reported_seq":12717,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:02.234441+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:31:02.234441+0000","last_peered":"2024-09-17T15:31:02.234441+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:31:02.234441+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'2500","ondisk_log_start":"34'2500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32923648,"num_objects":8038,"num_object_clones":0,"num_object_copies":16076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9100,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4067},{"shards":"0,2","objects":1781}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":107,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092541+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092541+0000","last_peered":"2024-09-17T15:30:57.092541+0000","last_clean":"2024-09-17T15:30:57.092541+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092541+0000","last_undegraded":"2024-09-17T15:30:57.092541+0000","last_fullsized":"2024-09-17T15:30:57.092541+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12537","reported_seq":12704,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:02.564012+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:31:02.564012+0000","last_peered":"2024-09-17T15:31:02.564012+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:31:02.564012+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'2500","ondisk_log_start":"34'2500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32694755,"num_objects":7983,"num_object_clones":0,"num_object_copies":15966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4057,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7983,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10432,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1802},{"shards":"2","objects":3927}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12684","reported_seq":13302,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:02.558774+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:31:02.558774+0000","last_peered":"2024-09-17T15:31:02.558774+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:31:02.558774+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'2600","ondisk_log_start":"34'2600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32980992,"num_objects":8052,"num_object_clones":0,"num_object_copies":16104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2112,"num_objects_misplaced":1780,"num_objects_unfound":0,"num_objects_dirty":8052,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10446,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":247,"num_bytes_recovered":901120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1780},{"shards":"2","objects":4160}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12821","reported_seq":13041,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:02.300353+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:31:02.300353+0000","last_peered":"2024-09-17T15:31:02.300353+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:31:02.300353+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'2800","ondisk_log_start":"34'2800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33361920,"num_objects":8145,"num_object_clones":0,"num_object_copies":16290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4099,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9369,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4047},{"shards":"0,2","objects":1801}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12695","reported_seq":12901,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:02.268206+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:31:02.268206+0000","last_peered":"2024-09-17T15:31:02.268206+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:31:02.268206+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'2600","ondisk_log_start":"34'2600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33083392,"num_objects":8077,"num_object_clones":0,"num_object_copies":16154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4130,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8982,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3950},{"shards":"0,2","objects":1852}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088855+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088855+0000","last_peered":"2024-09-17T15:30:57.088855+0000","last_clean":"2024-09-17T15:30:57.088855+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088855+0000","last_undegraded":"2024-09-17T15:30:57.088855+0000","last_fullsized":"2024-09-17T15:30:57.088855+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12730","reported_seq":12953,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:02.326115+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:31:02.326115+0000","last_peered":"2024-09-17T15:31:02.326115+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:31:02.326115+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'2700","ondisk_log_start":"34'2700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33406976,"num_objects":8156,"num_object_clones":0,"num_object_copies":16312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4020,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9517,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4138},{"shards":"0,2","objects":1778}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12563","reported_seq":20942,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:01.576711+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:31:01.576711+0000","last_peered":"2024-09-17T15:31:01.576711+0000","last_clean":"2024-09-17T15:31:01.576711+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:31:01.576711+0000","last_undegraded":"2024-09-17T15:31:01.576711+0000","last_fullsized":"2024-09-17T15:31:01.576711+0000","mapping_epoch":44,"log_start":"34'2500","ondisk_log_start":"34'2500","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33051590,"num_objects":8071,"num_object_clones":0,"num_object_copies":16142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9005,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":532284150,"num_objects":129962,"num_object_clones":0,"num_object_copies":259924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38917,"num_objects_misplaced":1780,"num_objects_unfound":0,"num_objects_dirty":129962,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":163025,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8763,"num_bytes_recovered":34399174,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1192341504,"data_stored":1192196066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160790,"ondisk_log_size":160790,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051656,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703632,"kb_used_data":392896,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93668208,"statfs":{"total":96636764160,"available":95916244992,"internally_reserved":0,"allocated":402325504,"data_stored":402034407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":318172844},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,0,1,0,0,2],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724601,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":614808,"kb_used_data":316552,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93757032,"statfs":{"total":96636764160,"available":96007200768,"internally_reserved":0,"allocated":324149248,"data_stored":323825170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":305393323},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247884,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":822820,"kb_used_data":457992,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93549020,"statfs":{"total":96636764160,"available":95794196480,"internally_reserved":0,"allocated":468983808,"data_stored":468692706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":373550760},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,4,6,4,1,0,0,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":467263488,"data_stored":467227358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":323686400,"data_stored":323617294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":401391616,"data_stored":401351414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:31:08.986 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:31:09.539 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:31:09.539 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:31:09.571 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":240,"stamp":"2024-09-17T15:31:07.688585+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":523485961,"num_objects":127816,"num_object_clones":0,"num_object_copies":255632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38894,"num_objects_misplaced":1647,"num_objects_unfound":0,"num_objects_dirty":127816,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":165175,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8920,"num_bytes_recovered":34943961,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160741,"ondisk_log_size":160741,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2112892,"kb_used_data":1155720,"kb_used_omap":13,"kb_used_meta":957106,"kb_avail":281002628,"statfs":{"total":289910292480,"available":287746691072,"internally_reserved":0,"allocated":1183457280,"data_stored":1182551003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":980077567},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,6,8,10,4,0,0,4,3],"upper_bound":4096},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24399872,"num_objects":-5957,"num_object_clones":0,"num_object_copies":-11914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2058,"num_objects_misplaced":1647,"num_objects_unfound":0,"num_objects_dirty":-5957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5957,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":411,"num_bytes_recovered":1445888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.604846"},"pg_stats":[{"pgid":"2.f","version":"45'12694","reported_seq":12862,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:07.274268+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:31:07.274268+0000","last_peered":"2024-09-17T15:31:07.274268+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:31:07.274268+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'2600","ondisk_log_start":"34'2600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32579584,"num_objects":7954,"num_object_clones":0,"num_object_copies":15908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4027,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12694,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3927},{"shards":"0,2","objects":1700}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12813","reported_seq":12920,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:07.580828+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:31:07.580828+0000","last_peered":"2024-09-17T15:31:07.580828+0000","last_clean":"2024-09-17T15:31:07.580828+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:31:07.580828+0000","last_undegraded":"2024-09-17T15:31:07.580828+0000","last_fullsized":"2024-09-17T15:31:07.580828+0000","mapping_epoch":33,"log_start":"34'2800","ondisk_log_start":"34'2800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33256390,"num_objects":8121,"num_object_clones":0,"num_object_copies":16242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8121,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11170,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12939","reported_seq":13120,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:07.601259+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:31:07.601259+0000","last_peered":"2024-09-17T15:31:07.601259+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:31:07.601259+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'2900","ondisk_log_start":"34'2900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32743448,"num_objects":7995,"num_object_clones":0,"num_object_copies":15990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4143,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7995,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11096,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1698},{"shards":"2","objects":3853}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12860","reported_seq":12934,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:07.547856+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:31:07.547856+0000","last_peered":"2024-09-17T15:31:07.547856+0000","last_clean":"2024-09-17T15:31:07.547856+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:31:07.547856+0000","last_undegraded":"2024-09-17T15:31:07.547856+0000","last_fullsized":"2024-09-17T15:31:07.547856+0000","mapping_epoch":33,"log_start":"34'2800","ondisk_log_start":"34'2800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33218560,"num_objects":8110,"num_object_clones":0,"num_object_copies":16220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12860,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12701","reported_seq":12835,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:07.110022+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:31:07.110022+0000","last_peered":"2024-09-17T15:31:07.110022+0000","last_clean":"2024-09-17T15:31:07.110022+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:31:07.110022+0000","last_undegraded":"2024-09-17T15:31:07.110022+0000","last_fullsized":"2024-09-17T15:31:07.110022+0000","mapping_epoch":32,"log_start":"34'2700","ondisk_log_start":"34'2700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32571875,"num_objects":7953,"num_object_clones":0,"num_object_copies":15906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7953,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9683,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13126","reported_seq":13363,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:07.287748+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:31:07.287748+0000","last_peered":"2024-09-17T15:31:07.287748+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:31:07.287748+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'3100","ondisk_log_start":"34'3100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33280483,"num_objects":8126,"num_object_clones":0,"num_object_copies":16252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4222,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8126,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9625,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3905},{"shards":"0,2","objects":1770}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12860","reported_seq":13103,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:07.314509+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:31:07.314509+0000","last_peered":"2024-09-17T15:31:07.314509+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:31:07.314509+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'2800","ondisk_log_start":"34'2800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32392134,"num_objects":7910,"num_object_clones":0,"num_object_copies":15820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7910,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9629,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3786},{"shards":"0,2","objects":1701}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12899","reported_seq":13031,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:07.251216+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:31:07.251216+0000","last_peered":"2024-09-17T15:31:07.251216+0000","last_clean":"2024-09-17T15:31:07.251216+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:31:07.251216+0000","last_undegraded":"2024-09-17T15:31:07.251216+0000","last_fullsized":"2024-09-17T15:31:07.251216+0000","mapping_epoch":32,"log_start":"34'2800","ondisk_log_start":"34'2800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32837632,"num_objects":8017,"num_object_clones":0,"num_object_copies":16034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9853,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.086967+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.086967+0000","last_peered":"2024-09-17T15:30:57.086967+0000","last_clean":"2024-09-17T15:30:57.086967+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.086967+0000","last_undegraded":"2024-09-17T15:30:57.086967+0000","last_fullsized":"2024-09-17T15:30:57.086967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12770","reported_seq":21057,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:06.536206+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:31:06.536206+0000","last_peered":"2024-09-17T15:31:06.536206+0000","last_clean":"2024-09-17T15:31:06.536206+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:31:06.536206+0000","last_undegraded":"2024-09-17T15:31:06.536206+0000","last_fullsized":"2024-09-17T15:31:06.536206+0000","mapping_epoch":42,"log_start":"34'2700","ondisk_log_start":"34'2700","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32960995,"num_objects":8048,"num_object_clones":0,"num_object_copies":16096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8048,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10770,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088505+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088505+0000","last_peered":"2024-09-17T15:30:57.088505+0000","last_clean":"2024-09-17T15:30:57.088505+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088505+0000","last_undegraded":"2024-09-17T15:30:57.088505+0000","last_fullsized":"2024-09-17T15:30:57.088505+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12645","reported_seq":12860,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:07.225889+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:31:07.225889+0000","last_peered":"2024-09-17T15:31:07.225889+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:31:07.225889+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'2600","ondisk_log_start":"34'2600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32346112,"num_objects":7897,"num_object_clones":0,"num_object_copies":15794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3972,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9241,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3927},{"shards":"0,2","objects":1640}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":107,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092541+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092541+0000","last_peered":"2024-09-17T15:30:57.092541+0000","last_clean":"2024-09-17T15:30:57.092541+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092541+0000","last_undegraded":"2024-09-17T15:30:57.092541+0000","last_fullsized":"2024-09-17T15:30:57.092541+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12668","reported_seq":12837,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:07.509281+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:31:07.509281+0000","last_peered":"2024-09-17T15:31:07.509281+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:31:07.509281+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'2600","ondisk_log_start":"34'2600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32158179,"num_objects":7852,"num_object_clones":0,"num_object_copies":15704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4056,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7852,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10563,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1671},{"shards":"2","objects":3797}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12820","reported_seq":13704,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:07.612316+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:31:07.612316+0000","last_peered":"2024-09-17T15:31:07.612316+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:31:07.612316+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'2800","ondisk_log_start":"34'2800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32423936,"num_objects":7916,"num_object_clones":0,"num_object_copies":15832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2112,"num_objects_misplaced":1647,"num_objects_unfound":0,"num_objects_dirty":7916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10582,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":380,"num_bytes_recovered":1445888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1647},{"shards":"2","objects":4157}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12942","reported_seq":13163,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:07.292555+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:31:07.292555+0000","last_peered":"2024-09-17T15:31:07.292555+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:31:07.292555+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'2900","ondisk_log_start":"34'2900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32866304,"num_objects":8024,"num_object_clones":0,"num_object_copies":16048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9490,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3927},{"shards":"0,2","objects":1680}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12840","reported_seq":13054,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:07.298981+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:31:07.298981+0000","last_peered":"2024-09-17T15:31:07.298981+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:31:07.298981+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'2800","ondisk_log_start":"34'2800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32489472,"num_objects":7932,"num_object_clones":0,"num_object_copies":15864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7932,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9127,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3809},{"shards":"0,2","objects":1707}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088855+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088855+0000","last_peered":"2024-09-17T15:30:57.088855+0000","last_clean":"2024-09-17T15:30:57.088855+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088855+0000","last_undegraded":"2024-09-17T15:30:57.088855+0000","last_fullsized":"2024-09-17T15:30:57.088855+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12853","reported_seq":13090,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:07.271185+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:31:07.271185+0000","last_peered":"2024-09-17T15:31:07.271185+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:31:07.271185+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'2800","ondisk_log_start":"34'2800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32903168,"num_objects":8033,"num_object_clones":0,"num_object_copies":16066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4014,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9640,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4021},{"shards":"0,2","objects":1655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12708","reported_seq":21087,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:06.557831+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:31:06.557831+0000","last_peered":"2024-09-17T15:31:06.557831+0000","last_clean":"2024-09-17T15:31:06.557831+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:31:06.557831+0000","last_undegraded":"2024-09-17T15:31:06.557831+0000","last_fullsized":"2024-09-17T15:31:06.557831+0000","mapping_epoch":44,"log_start":"34'2700","ondisk_log_start":"34'2700","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32457670,"num_objects":7926,"num_object_clones":0,"num_object_copies":15852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9150,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":523485942,"num_objects":127814,"num_object_clones":0,"num_object_copies":255628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38894,"num_objects_misplaced":1647,"num_objects_unfound":0,"num_objects_dirty":127814,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":165173,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8919,"num_bytes_recovered":34943942,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1180393472,"data_stored":1180248034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160738,"ondisk_log_size":160738,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051657,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":687000,"kb_used_data":388740,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93684840,"statfs":{"total":96636764160,"available":95933276160,"internally_reserved":0,"allocated":398069760,"data_stored":397778663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":305393324},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,0,2,2,0,0,0,0,3],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724602,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":622068,"kb_used_data":315492,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93749772,"statfs":{"total":96636764160,"available":95999766528,"internally_reserved":0,"allocated":323063808,"data_stored":322739730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":313913003},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,4,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247885,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":803824,"kb_used_data":451488,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93568016,"statfs":{"total":96636764160,"available":95813648384,"internally_reserved":0,"allocated":462323712,"data_stored":462032610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":360771240},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,2,2,0,0,0,4],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":460607488,"data_stored":460571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":322633728,"data_stored":322564622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":397152256,"data_stored":397112054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:31:09.573 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:31:10.110 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:31:10.110 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:31:10.137 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":241,"stamp":"2024-09-17T15:31:09.689258+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":523485961,"num_objects":127816,"num_object_clones":0,"num_object_copies":255632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38894,"num_objects_misplaced":1647,"num_objects_unfound":0,"num_objects_dirty":127816,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":165175,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8920,"num_bytes_recovered":34943961,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160741,"ondisk_log_size":160741,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2112892,"kb_used_data":1155720,"kb_used_omap":13,"kb_used_meta":957106,"kb_avail":281002628,"statfs":{"total":289910292480,"available":287746691072,"internally_reserved":0,"allocated":1183457280,"data_stored":1182551003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":980077567},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,6,8,10,4,0,0,4,3],"upper_bound":4096},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17543168,"num_objects":-4283,"num_object_clones":0,"num_object_copies":-8566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-40,"num_objects_misplaced":-260,"num_objects_unfound":0,"num_objects_dirty":-4283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4283,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":1064960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003210"},"pg_stats":[{"pgid":"2.f","version":"45'12694","reported_seq":12862,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:07.274268+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:31:07.274268+0000","last_peered":"2024-09-17T15:31:07.274268+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:31:07.274268+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'2600","ondisk_log_start":"34'2600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32579584,"num_objects":7954,"num_object_clones":0,"num_object_copies":15908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4027,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12694,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3927},{"shards":"0,2","objects":1700}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12813","reported_seq":12920,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:07.580828+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:31:07.580828+0000","last_peered":"2024-09-17T15:31:07.580828+0000","last_clean":"2024-09-17T15:31:07.580828+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:31:07.580828+0000","last_undegraded":"2024-09-17T15:31:07.580828+0000","last_fullsized":"2024-09-17T15:31:07.580828+0000","mapping_epoch":33,"log_start":"34'2800","ondisk_log_start":"34'2800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33256390,"num_objects":8121,"num_object_clones":0,"num_object_copies":16242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8121,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11170,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12939","reported_seq":13120,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:07.601259+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:31:07.601259+0000","last_peered":"2024-09-17T15:31:07.601259+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:31:07.601259+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'2900","ondisk_log_start":"34'2900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32743448,"num_objects":7995,"num_object_clones":0,"num_object_copies":15990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4143,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7995,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11096,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1698},{"shards":"2","objects":3853}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12860","reported_seq":12934,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:07.547856+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:31:07.547856+0000","last_peered":"2024-09-17T15:31:07.547856+0000","last_clean":"2024-09-17T15:31:07.547856+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:31:07.547856+0000","last_undegraded":"2024-09-17T15:31:07.547856+0000","last_fullsized":"2024-09-17T15:31:07.547856+0000","mapping_epoch":33,"log_start":"34'2800","ondisk_log_start":"34'2800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33218560,"num_objects":8110,"num_object_clones":0,"num_object_copies":16220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12860,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12701","reported_seq":12835,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:07.110022+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:31:07.110022+0000","last_peered":"2024-09-17T15:31:07.110022+0000","last_clean":"2024-09-17T15:31:07.110022+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:31:07.110022+0000","last_undegraded":"2024-09-17T15:31:07.110022+0000","last_fullsized":"2024-09-17T15:31:07.110022+0000","mapping_epoch":32,"log_start":"34'2700","ondisk_log_start":"34'2700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32571875,"num_objects":7953,"num_object_clones":0,"num_object_copies":15906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7953,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9683,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13126","reported_seq":13363,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:07.287748+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:31:07.287748+0000","last_peered":"2024-09-17T15:31:07.287748+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:31:07.287748+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'3100","ondisk_log_start":"34'3100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33280483,"num_objects":8126,"num_object_clones":0,"num_object_copies":16252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4222,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8126,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9625,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3905},{"shards":"0,2","objects":1770}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12860","reported_seq":13103,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:07.314509+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:31:07.314509+0000","last_peered":"2024-09-17T15:31:07.314509+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:31:07.314509+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'2800","ondisk_log_start":"34'2800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32392134,"num_objects":7910,"num_object_clones":0,"num_object_copies":15820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7910,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9629,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3786},{"shards":"0,2","objects":1701}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12899","reported_seq":13031,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:07.251216+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:31:07.251216+0000","last_peered":"2024-09-17T15:31:07.251216+0000","last_clean":"2024-09-17T15:31:07.251216+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:31:07.251216+0000","last_undegraded":"2024-09-17T15:31:07.251216+0000","last_fullsized":"2024-09-17T15:31:07.251216+0000","mapping_epoch":32,"log_start":"34'2800","ondisk_log_start":"34'2800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32837632,"num_objects":8017,"num_object_clones":0,"num_object_copies":16034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9853,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.086967+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.086967+0000","last_peered":"2024-09-17T15:30:57.086967+0000","last_clean":"2024-09-17T15:30:57.086967+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.086967+0000","last_undegraded":"2024-09-17T15:30:57.086967+0000","last_fullsized":"2024-09-17T15:30:57.086967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12770","reported_seq":21057,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:06.536206+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:31:06.536206+0000","last_peered":"2024-09-17T15:31:06.536206+0000","last_clean":"2024-09-17T15:31:06.536206+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:31:06.536206+0000","last_undegraded":"2024-09-17T15:31:06.536206+0000","last_fullsized":"2024-09-17T15:31:06.536206+0000","mapping_epoch":42,"log_start":"34'2700","ondisk_log_start":"34'2700","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32960995,"num_objects":8048,"num_object_clones":0,"num_object_copies":16096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8048,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10770,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088505+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088505+0000","last_peered":"2024-09-17T15:30:57.088505+0000","last_clean":"2024-09-17T15:30:57.088505+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088505+0000","last_undegraded":"2024-09-17T15:30:57.088505+0000","last_fullsized":"2024-09-17T15:30:57.088505+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12645","reported_seq":12860,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:07.225889+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:31:07.225889+0000","last_peered":"2024-09-17T15:31:07.225889+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:31:07.225889+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'2600","ondisk_log_start":"34'2600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32346112,"num_objects":7897,"num_object_clones":0,"num_object_copies":15794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3972,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9241,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3927},{"shards":"0,2","objects":1640}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":107,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092541+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092541+0000","last_peered":"2024-09-17T15:30:57.092541+0000","last_clean":"2024-09-17T15:30:57.092541+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092541+0000","last_undegraded":"2024-09-17T15:30:57.092541+0000","last_fullsized":"2024-09-17T15:30:57.092541+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12668","reported_seq":12837,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:07.509281+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:31:07.509281+0000","last_peered":"2024-09-17T15:31:07.509281+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:31:07.509281+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'2600","ondisk_log_start":"34'2600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32158179,"num_objects":7852,"num_object_clones":0,"num_object_copies":15704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4056,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7852,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10563,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1671},{"shards":"2","objects":3797}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12820","reported_seq":13704,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:07.612316+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:31:07.612316+0000","last_peered":"2024-09-17T15:31:07.612316+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:31:07.612316+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'2800","ondisk_log_start":"34'2800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32423936,"num_objects":7916,"num_object_clones":0,"num_object_copies":15832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2112,"num_objects_misplaced":1647,"num_objects_unfound":0,"num_objects_dirty":7916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10582,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":380,"num_bytes_recovered":1445888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1647},{"shards":"2","objects":4157}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12942","reported_seq":13163,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:07.292555+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:31:07.292555+0000","last_peered":"2024-09-17T15:31:07.292555+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:31:07.292555+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'2900","ondisk_log_start":"34'2900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32866304,"num_objects":8024,"num_object_clones":0,"num_object_copies":16048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9490,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3927},{"shards":"0,2","objects":1680}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12840","reported_seq":13054,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:07.298981+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:31:07.298981+0000","last_peered":"2024-09-17T15:31:07.298981+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:31:07.298981+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'2800","ondisk_log_start":"34'2800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32489472,"num_objects":7932,"num_object_clones":0,"num_object_copies":15864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7932,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9127,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3809},{"shards":"0,2","objects":1707}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088855+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088855+0000","last_peered":"2024-09-17T15:30:57.088855+0000","last_clean":"2024-09-17T15:30:57.088855+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088855+0000","last_undegraded":"2024-09-17T15:30:57.088855+0000","last_fullsized":"2024-09-17T15:30:57.088855+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12853","reported_seq":13090,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:07.271185+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:31:07.271185+0000","last_peered":"2024-09-17T15:31:07.271185+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:31:07.271185+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'2800","ondisk_log_start":"34'2800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32903168,"num_objects":8033,"num_object_clones":0,"num_object_copies":16066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4014,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9640,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4021},{"shards":"0,2","objects":1655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12708","reported_seq":21087,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:06.557831+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:31:06.557831+0000","last_peered":"2024-09-17T15:31:06.557831+0000","last_clean":"2024-09-17T15:31:06.557831+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:31:06.557831+0000","last_undegraded":"2024-09-17T15:31:06.557831+0000","last_fullsized":"2024-09-17T15:31:06.557831+0000","mapping_epoch":44,"log_start":"34'2700","ondisk_log_start":"34'2700","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32457670,"num_objects":7926,"num_object_clones":0,"num_object_copies":15852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9150,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":523485942,"num_objects":127814,"num_object_clones":0,"num_object_copies":255628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38894,"num_objects_misplaced":1647,"num_objects_unfound":0,"num_objects_dirty":127814,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":165173,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8919,"num_bytes_recovered":34943942,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1180393472,"data_stored":1180248034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160738,"ondisk_log_size":160738,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051657,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":687000,"kb_used_data":388740,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93684840,"statfs":{"total":96636764160,"available":95933276160,"internally_reserved":0,"allocated":398069760,"data_stored":397778663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":305393324},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,0,2,2,0,0,0,0,3],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724602,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":622068,"kb_used_data":315492,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93749772,"statfs":{"total":96636764160,"available":95999766528,"internally_reserved":0,"allocated":323063808,"data_stored":322739730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":313913003},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,4,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247885,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":803824,"kb_used_data":451488,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93568016,"statfs":{"total":96636764160,"available":95813648384,"internally_reserved":0,"allocated":462323712,"data_stored":462032610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":360771240},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,2,2,0,0,0,4],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":460607488,"data_stored":460571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":322633728,"data_stored":322564622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":397152256,"data_stored":397112054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:31:10.138 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:31:10.138 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'12694", 'reported_seq': 12862, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:07.274268+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:31:07.274268+0000', 'last_peered': '2024-09-17T15:31:07.274268+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:31:07.274268+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "34'2600", 'ondisk_log_start': "34'2600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32579584, 'num_objects': 7954, 'num_object_clones': 0, 'num_object_copies': 15908, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4027, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7954, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12694, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3927}, {'shards': '0,2', 'objects': 1700}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:10.138 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:31:10.138 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'12939", 'reported_seq': 13120, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:07.601259+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:31:07.601259+0000', 'last_peered': '2024-09-17T15:31:07.601259+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:31:07.601259+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "34'2900", 'ondisk_log_start': "34'2900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32743448, 'num_objects': 7995, 'num_object_clones': 0, 'num_object_copies': 15990, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4143, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7995, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11096, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 26, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1698}, {'shards': '2', 'objects': 3853}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:31:10.139 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:31:10.139 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'13126", 'reported_seq': 13363, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:07.287748+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:31:07.287748+0000', 'last_peered': '2024-09-17T15:31:07.287748+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:31:07.287748+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "34'3100", 'ondisk_log_start': "34'3100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33280483, 'num_objects': 8126, 'num_object_clones': 0, 'num_object_copies': 16252, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4222, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8126, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9625, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 49, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3905}, {'shards': '0,2', 'objects': 1770}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:10.139 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:31:10.139 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'12860", 'reported_seq': 13103, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:07.314509+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:31:07.314509+0000', 'last_peered': '2024-09-17T15:31:07.314509+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:31:07.314509+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "34'2800", 'ondisk_log_start': "34'2800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32392134, 'num_objects': 7910, 'num_object_clones': 0, 'num_object_copies': 15820, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7910, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 9629, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 50, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3786}, {'shards': '0,2', 'objects': 1701}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:10.139 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:31:10.139 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'12645", 'reported_seq': 12860, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:07.225889+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:31:07.225889+0000', 'last_peered': '2024-09-17T15:31:07.225889+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:31:07.225889+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "34'2600", 'ondisk_log_start': "34'2600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32346112, 'num_objects': 7897, 'num_object_clones': 0, 'num_object_copies': 15794, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3972, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7897, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9241, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 40, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3927}, {'shards': '0,2', 'objects': 1640}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:10.139 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:31:10.139 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'12668", 'reported_seq': 12837, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:07.509281+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:31:07.509281+0000', 'last_peered': '2024-09-17T15:31:07.509281+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:31:07.509281+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "34'2600", 'ondisk_log_start': "34'2600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32158179, 'num_objects': 7852, 'num_object_clones': 0, 'num_object_copies': 15704, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4056, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7852, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10563, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 22, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1671}, {'shards': '2', 'objects': 3797}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:31:10.139 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:31:10.139 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'12820", 'reported_seq': 13704, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:07.612316+0000', 'last_change': '2024-09-17T15:30:55.086473+0000', 'last_active': '2024-09-17T15:31:07.612316+0000', 'last_peered': '2024-09-17T15:31:07.612316+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:31:07.612316+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "34'2800", 'ondisk_log_start': "34'2800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32423936, 'num_objects': 7916, 'num_object_clones': 0, 'num_object_copies': 15832, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2112, 'num_objects_misplaced': 1647, 'num_objects_unfound': 0, 'num_objects_dirty': 7916, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10582, 'num_write_kb': 32532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 380, 'num_bytes_recovered': 1445888, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1647}, {'shards': '2', 'objects': 4157}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:31:10.139 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:31:10.140 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'12942", 'reported_seq': 13163, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:07.292555+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:31:07.292555+0000', 'last_peered': '2024-09-17T15:31:07.292555+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:31:07.292555+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "34'2900", 'ondisk_log_start': "34'2900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32866304, 'num_objects': 8024, 'num_object_clones': 0, 'num_object_copies': 16048, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4098, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8024, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9490, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 41, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3927}, {'shards': '0,2', 'objects': 1680}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:10.140 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:31:10.140 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'12840", 'reported_seq': 13054, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:07.298981+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:31:07.298981+0000', 'last_peered': '2024-09-17T15:31:07.298981+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:31:07.298981+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "34'2800", 'ondisk_log_start': "34'2800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32489472, 'num_objects': 7932, 'num_object_clones': 0, 'num_object_copies': 15864, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7932, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9127, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 34, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3809}, {'shards': '0,2', 'objects': 1707}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:10.140 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:31:10.140 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'12853", 'reported_seq': 13090, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:07.271185+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:31:07.271185+0000', 'last_peered': '2024-09-17T15:31:07.271185+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:31:07.271185+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "34'2800", 'ondisk_log_start': "34'2800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32903168, 'num_objects': 8033, 'num_object_clones': 0, 'num_object_copies': 16066, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4014, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8033, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9640, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4021}, {'shards': '0,2', 'objects': 1655}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:10.140 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:31:10.842 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:31:10.842 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38894/255632 objects degraded (15.215%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":407,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":127816,"data_bytes":523485961,"bytes_used":2163601408,"bytes_avail":287746691072,"bytes_total":289910292480,"degraded_objects":38894,"degraded_total":255632,"degraded_ratio":0.15214840082618764,"misplaced_objects":1647,"misplaced_total":255632,"misplaced_ratio":0.0064428553545721979,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":88722,"recovering_keys_per_sec":0,"num_objects_recovered":300,"num_bytes_recovered":1064960,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":356},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T15:27:19.616376+0000","services":{}},"progress_events":{}} 2024-09-17T15:31:10.860 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:31:10.860 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:31:11.417 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:31:11.420 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:31:11.438 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":241,"stamp":"2024-09-17T15:31:09.689258+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":523485961,"num_objects":127816,"num_object_clones":0,"num_object_copies":255632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38894,"num_objects_misplaced":1647,"num_objects_unfound":0,"num_objects_dirty":127816,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":165175,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8920,"num_bytes_recovered":34943961,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160741,"ondisk_log_size":160741,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2112892,"kb_used_data":1155720,"kb_used_omap":13,"kb_used_meta":957106,"kb_avail":281002628,"statfs":{"total":289910292480,"available":287746691072,"internally_reserved":0,"allocated":1183457280,"data_stored":1182551003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":980077567},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,6,8,10,4,0,0,4,3],"upper_bound":4096},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17543168,"num_objects":-4283,"num_object_clones":0,"num_object_copies":-8566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-40,"num_objects_misplaced":-260,"num_objects_unfound":0,"num_objects_dirty":-4283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4283,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":1064960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003210"},"pg_stats":[{"pgid":"2.f","version":"45'12694","reported_seq":12862,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:07.274268+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:31:07.274268+0000","last_peered":"2024-09-17T15:31:07.274268+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:31:07.274268+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'2600","ondisk_log_start":"34'2600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32579584,"num_objects":7954,"num_object_clones":0,"num_object_copies":15908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4027,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12694,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3927},{"shards":"0,2","objects":1700}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12813","reported_seq":12920,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:07.580828+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:31:07.580828+0000","last_peered":"2024-09-17T15:31:07.580828+0000","last_clean":"2024-09-17T15:31:07.580828+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:31:07.580828+0000","last_undegraded":"2024-09-17T15:31:07.580828+0000","last_fullsized":"2024-09-17T15:31:07.580828+0000","mapping_epoch":33,"log_start":"34'2800","ondisk_log_start":"34'2800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33256390,"num_objects":8121,"num_object_clones":0,"num_object_copies":16242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8121,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11170,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12939","reported_seq":13120,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:07.601259+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:31:07.601259+0000","last_peered":"2024-09-17T15:31:07.601259+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:31:07.601259+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'2900","ondisk_log_start":"34'2900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32743448,"num_objects":7995,"num_object_clones":0,"num_object_copies":15990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4143,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7995,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11096,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1698},{"shards":"2","objects":3853}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12860","reported_seq":12934,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:07.547856+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:31:07.547856+0000","last_peered":"2024-09-17T15:31:07.547856+0000","last_clean":"2024-09-17T15:31:07.547856+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:31:07.547856+0000","last_undegraded":"2024-09-17T15:31:07.547856+0000","last_fullsized":"2024-09-17T15:31:07.547856+0000","mapping_epoch":33,"log_start":"34'2800","ondisk_log_start":"34'2800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33218560,"num_objects":8110,"num_object_clones":0,"num_object_copies":16220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12860,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12701","reported_seq":12835,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:07.110022+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:31:07.110022+0000","last_peered":"2024-09-17T15:31:07.110022+0000","last_clean":"2024-09-17T15:31:07.110022+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:31:07.110022+0000","last_undegraded":"2024-09-17T15:31:07.110022+0000","last_fullsized":"2024-09-17T15:31:07.110022+0000","mapping_epoch":32,"log_start":"34'2700","ondisk_log_start":"34'2700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32571875,"num_objects":7953,"num_object_clones":0,"num_object_copies":15906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7953,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9683,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13126","reported_seq":13363,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:07.287748+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:31:07.287748+0000","last_peered":"2024-09-17T15:31:07.287748+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:31:07.287748+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'3100","ondisk_log_start":"34'3100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33280483,"num_objects":8126,"num_object_clones":0,"num_object_copies":16252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4222,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8126,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9625,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3905},{"shards":"0,2","objects":1770}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12860","reported_seq":13103,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:07.314509+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:31:07.314509+0000","last_peered":"2024-09-17T15:31:07.314509+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:31:07.314509+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'2800","ondisk_log_start":"34'2800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32392134,"num_objects":7910,"num_object_clones":0,"num_object_copies":15820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7910,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9629,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3786},{"shards":"0,2","objects":1701}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12899","reported_seq":13031,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:07.251216+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:31:07.251216+0000","last_peered":"2024-09-17T15:31:07.251216+0000","last_clean":"2024-09-17T15:31:07.251216+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:31:07.251216+0000","last_undegraded":"2024-09-17T15:31:07.251216+0000","last_fullsized":"2024-09-17T15:31:07.251216+0000","mapping_epoch":32,"log_start":"34'2800","ondisk_log_start":"34'2800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32837632,"num_objects":8017,"num_object_clones":0,"num_object_copies":16034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9853,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.086967+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.086967+0000","last_peered":"2024-09-17T15:30:57.086967+0000","last_clean":"2024-09-17T15:30:57.086967+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.086967+0000","last_undegraded":"2024-09-17T15:30:57.086967+0000","last_fullsized":"2024-09-17T15:30:57.086967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12770","reported_seq":21057,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:06.536206+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:31:06.536206+0000","last_peered":"2024-09-17T15:31:06.536206+0000","last_clean":"2024-09-17T15:31:06.536206+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:31:06.536206+0000","last_undegraded":"2024-09-17T15:31:06.536206+0000","last_fullsized":"2024-09-17T15:31:06.536206+0000","mapping_epoch":42,"log_start":"34'2700","ondisk_log_start":"34'2700","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32960995,"num_objects":8048,"num_object_clones":0,"num_object_copies":16096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8048,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10770,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088505+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088505+0000","last_peered":"2024-09-17T15:30:57.088505+0000","last_clean":"2024-09-17T15:30:57.088505+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088505+0000","last_undegraded":"2024-09-17T15:30:57.088505+0000","last_fullsized":"2024-09-17T15:30:57.088505+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12645","reported_seq":12860,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:07.225889+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:31:07.225889+0000","last_peered":"2024-09-17T15:31:07.225889+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:31:07.225889+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'2600","ondisk_log_start":"34'2600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32346112,"num_objects":7897,"num_object_clones":0,"num_object_copies":15794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3972,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9241,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3927},{"shards":"0,2","objects":1640}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":107,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092541+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092541+0000","last_peered":"2024-09-17T15:30:57.092541+0000","last_clean":"2024-09-17T15:30:57.092541+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092541+0000","last_undegraded":"2024-09-17T15:30:57.092541+0000","last_fullsized":"2024-09-17T15:30:57.092541+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12668","reported_seq":12837,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:07.509281+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:31:07.509281+0000","last_peered":"2024-09-17T15:31:07.509281+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:31:07.509281+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'2600","ondisk_log_start":"34'2600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32158179,"num_objects":7852,"num_object_clones":0,"num_object_copies":15704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4056,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7852,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10563,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1671},{"shards":"2","objects":3797}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12820","reported_seq":13704,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:07.612316+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:31:07.612316+0000","last_peered":"2024-09-17T15:31:07.612316+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:31:07.612316+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'2800","ondisk_log_start":"34'2800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32423936,"num_objects":7916,"num_object_clones":0,"num_object_copies":15832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2112,"num_objects_misplaced":1647,"num_objects_unfound":0,"num_objects_dirty":7916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10582,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":380,"num_bytes_recovered":1445888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1647},{"shards":"2","objects":4157}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12942","reported_seq":13163,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:07.292555+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:31:07.292555+0000","last_peered":"2024-09-17T15:31:07.292555+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:31:07.292555+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'2900","ondisk_log_start":"34'2900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32866304,"num_objects":8024,"num_object_clones":0,"num_object_copies":16048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9490,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3927},{"shards":"0,2","objects":1680}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12840","reported_seq":13054,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:07.298981+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:31:07.298981+0000","last_peered":"2024-09-17T15:31:07.298981+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:31:07.298981+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'2800","ondisk_log_start":"34'2800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32489472,"num_objects":7932,"num_object_clones":0,"num_object_copies":15864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7932,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9127,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3809},{"shards":"0,2","objects":1707}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088855+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088855+0000","last_peered":"2024-09-17T15:30:57.088855+0000","last_clean":"2024-09-17T15:30:57.088855+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088855+0000","last_undegraded":"2024-09-17T15:30:57.088855+0000","last_fullsized":"2024-09-17T15:30:57.088855+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12853","reported_seq":13090,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:07.271185+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:31:07.271185+0000","last_peered":"2024-09-17T15:31:07.271185+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:31:07.271185+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'2800","ondisk_log_start":"34'2800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32903168,"num_objects":8033,"num_object_clones":0,"num_object_copies":16066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4014,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9640,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4021},{"shards":"0,2","objects":1655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12708","reported_seq":21087,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:06.557831+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:31:06.557831+0000","last_peered":"2024-09-17T15:31:06.557831+0000","last_clean":"2024-09-17T15:31:06.557831+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:31:06.557831+0000","last_undegraded":"2024-09-17T15:31:06.557831+0000","last_fullsized":"2024-09-17T15:31:06.557831+0000","mapping_epoch":44,"log_start":"34'2700","ondisk_log_start":"34'2700","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32457670,"num_objects":7926,"num_object_clones":0,"num_object_copies":15852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9150,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":523485942,"num_objects":127814,"num_object_clones":0,"num_object_copies":255628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38894,"num_objects_misplaced":1647,"num_objects_unfound":0,"num_objects_dirty":127814,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":165173,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8919,"num_bytes_recovered":34943942,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1180393472,"data_stored":1180248034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160738,"ondisk_log_size":160738,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051657,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":687000,"kb_used_data":388740,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93684840,"statfs":{"total":96636764160,"available":95933276160,"internally_reserved":0,"allocated":398069760,"data_stored":397778663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":305393324},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,0,2,2,0,0,0,0,3],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724602,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":622068,"kb_used_data":315492,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93749772,"statfs":{"total":96636764160,"available":95999766528,"internally_reserved":0,"allocated":323063808,"data_stored":322739730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":313913003},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,4,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247885,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":803824,"kb_used_data":451488,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93568016,"statfs":{"total":96636764160,"available":95813648384,"internally_reserved":0,"allocated":462323712,"data_stored":462032610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":360771240},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,2,2,0,0,0,4],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":460607488,"data_stored":460571358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":322633728,"data_stored":322564622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":397152256,"data_stored":397112054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:31:14.440 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:31:14.987 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:31:14.990 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:31:15.011 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":243,"stamp":"2024-09-17T15:31:13.690394+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":514401033,"num_objects":125598,"num_object_clones":0,"num_object_copies":251196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38880,"num_objects_misplaced":1537,"num_objects_unfound":0,"num_objects_dirty":125598,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":167393,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9044,"num_bytes_recovered":35394521,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160959,"ondisk_log_size":160959,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2117204,"kb_used_data":1143384,"kb_used_omap":13,"kb_used_meta":973746,"kb_avail":280998316,"statfs":{"total":289910292480,"available":287742275584,"internally_reserved":0,"allocated":1170825216,"data_stored":1169918939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":997116927},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,1,6,12,5,0,0,0,0,3,3],"upper_bound":8192},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25530368,"num_objects":-6233,"num_object_clones":0,"num_object_copies":-12466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-54,"num_objects_misplaced":-370,"num_objects_unfound":0,"num_objects_dirty":-6233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6233,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":424,"num_bytes_recovered":1515520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003223"},"pg_stats":[{"pgid":"2.f","version":"45'12827","reported_seq":12999,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:12.206117+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:31:12.206117+0000","last_peered":"2024-09-17T15:31:12.206117+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:31:12.206117+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'2800","ondisk_log_start":"34'2800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32034816,"num_objects":7821,"num_object_clones":0,"num_object_copies":15642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4025,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7821,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12827,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3796},{"shards":"0,2","objects":1567}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12944","reported_seq":13051,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:12.519147+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:31:12.519147+0000","last_peered":"2024-09-17T15:31:12.519147+0000","last_clean":"2024-09-17T15:31:12.519147+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:31:12.519147+0000","last_undegraded":"2024-09-17T15:31:12.519147+0000","last_fullsized":"2024-09-17T15:31:12.519147+0000","mapping_epoch":33,"log_start":"34'2900","ondisk_log_start":"34'2900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32719814,"num_objects":7990,"num_object_clones":0,"num_object_copies":15980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7990,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11301,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13096","reported_seq":13281,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:12.493686+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:31:12.493686+0000","last_peered":"2024-09-17T15:31:12.493686+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:31:12.493686+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'3000","ondisk_log_start":"34'3000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32100376,"num_objects":7838,"num_object_clones":0,"num_object_copies":15676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7838,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11253,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1541},{"shards":"2","objects":3697}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13001","reported_seq":13075,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:12.610519+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:31:12.610519+0000","last_peered":"2024-09-17T15:31:12.610519+0000","last_clean":"2024-09-17T15:31:12.610519+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:31:12.610519+0000","last_undegraded":"2024-09-17T15:31:12.610519+0000","last_fullsized":"2024-09-17T15:31:12.610519+0000","mapping_epoch":33,"log_start":"34'3000","ondisk_log_start":"34'3000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32641024,"num_objects":7969,"num_object_clones":0,"num_object_copies":15938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7969,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13001,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12817","reported_seq":12951,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:12.291411+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:31:12.291411+0000","last_peered":"2024-09-17T15:31:12.291411+0000","last_clean":"2024-09-17T15:31:12.291411+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:31:12.291411+0000","last_undegraded":"2024-09-17T15:31:12.291411+0000","last_fullsized":"2024-09-17T15:31:12.291411+0000","mapping_epoch":32,"log_start":"34'2800","ondisk_log_start":"34'2800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32096739,"num_objects":7837,"num_object_clones":0,"num_object_copies":15674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7837,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9799,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13285","reported_seq":13526,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:12.279994+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:31:12.279994+0000","last_peered":"2024-09-17T15:31:12.279994+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:31:12.279994+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'3200","ondisk_log_start":"34'3200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32629219,"num_objects":7967,"num_object_clones":0,"num_object_copies":15934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4220,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7967,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9784,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3748},{"shards":"0,2","objects":1611}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12995","reported_seq":13240,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:12.302308+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:31:12.302308+0000","last_peered":"2024-09-17T15:31:12.302308+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:31:12.302308+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'2900","ondisk_log_start":"34'2900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31839174,"num_objects":7775,"num_object_clones":0,"num_object_copies":15550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7775,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9764,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3652},{"shards":"0,2","objects":1566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13041","reported_seq":13173,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:12.278511+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:31:12.278511+0000","last_peered":"2024-09-17T15:31:12.278511+0000","last_clean":"2024-09-17T15:31:12.278511+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:31:12.278511+0000","last_undegraded":"2024-09-17T15:31:12.278511+0000","last_fullsized":"2024-09-17T15:31:12.278511+0000","mapping_epoch":32,"log_start":"34'3000","ondisk_log_start":"34'3000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32256000,"num_objects":7875,"num_object_clones":0,"num_object_copies":15750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9995,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.086967+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.086967+0000","last_peered":"2024-09-17T15:30:57.086967+0000","last_clean":"2024-09-17T15:30:57.086967+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.086967+0000","last_undegraded":"2024-09-17T15:30:57.086967+0000","last_fullsized":"2024-09-17T15:30:57.086967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12904","reported_seq":21191,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:11.605333+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:31:11.605333+0000","last_peered":"2024-09-17T15:31:11.605333+0000","last_clean":"2024-09-17T15:31:11.605333+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:31:11.605333+0000","last_undegraded":"2024-09-17T15:31:11.605333+0000","last_fullsized":"2024-09-17T15:31:11.605333+0000","mapping_epoch":42,"log_start":"34'2900","ondisk_log_start":"34'2900","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32412131,"num_objects":7914,"num_object_clones":0,"num_object_copies":15828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7914,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10904,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088505+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088505+0000","last_peered":"2024-09-17T15:30:57.088505+0000","last_clean":"2024-09-17T15:30:57.088505+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088505+0000","last_undegraded":"2024-09-17T15:30:57.088505+0000","last_fullsized":"2024-09-17T15:30:57.088505+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12786","reported_seq":13004,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:12.282900+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:31:12.282900+0000","last_peered":"2024-09-17T15:31:12.282900+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:31:12.282900+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'2700","ondisk_log_start":"34'2700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31768576,"num_objects":7756,"num_object_clones":0,"num_object_copies":15512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3970,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9382,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3788},{"shards":"0,2","objects":1499}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":107,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092541+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092541+0000","last_peered":"2024-09-17T15:30:57.092541+0000","last_clean":"2024-09-17T15:30:57.092541+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092541+0000","last_undegraded":"2024-09-17T15:30:57.092541+0000","last_fullsized":"2024-09-17T15:30:57.092541+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12783","reported_seq":12956,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:12.604901+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:31:12.604901+0000","last_peered":"2024-09-17T15:31:12.604901+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:31:12.604901+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'2700","ondisk_log_start":"34'2700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31687139,"num_objects":7737,"num_object_clones":0,"num_object_copies":15474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4054,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7737,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10678,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1556},{"shards":"2","objects":3684}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12959","reported_seq":14063,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:12.605103+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:31:12.605103+0000","last_peered":"2024-09-17T15:31:12.605103+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:31:12.605103+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'2900","ondisk_log_start":"34'2900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31854592,"num_objects":7777,"num_object_clones":0,"num_object_copies":15554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2112,"num_objects_misplaced":1537,"num_objects_unfound":0,"num_objects_dirty":7777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10721,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":490,"num_bytes_recovered":1896448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1537},{"shards":"2","objects":4128}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13088","reported_seq":13312,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:12.317321+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:31:12.317321+0000","last_peered":"2024-09-17T15:31:12.317321+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:31:12.317321+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'3000","ondisk_log_start":"34'3000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32268288,"num_objects":7878,"num_object_clones":0,"num_object_copies":15756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4097,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9636,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3782},{"shards":"0,2","objects":1534}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12990","reported_seq":13208,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:12.319706+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:31:12.319706+0000","last_peered":"2024-09-17T15:31:12.319706+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:31:12.319706+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'2900","ondisk_log_start":"34'2900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31875072,"num_objects":7782,"num_object_clones":0,"num_object_copies":15564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9277,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3661},{"shards":"0,2","objects":1557}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088855+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088855+0000","last_peered":"2024-09-17T15:30:57.088855+0000","last_clean":"2024-09-17T15:30:57.088855+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088855+0000","last_undegraded":"2024-09-17T15:30:57.088855+0000","last_fullsized":"2024-09-17T15:30:57.088855+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12985","reported_seq":13225,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:12.236883+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:31:12.236883+0000","last_peered":"2024-09-17T15:31:12.236883+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:31:12.236883+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'2900","ondisk_log_start":"34'2900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32362496,"num_objects":7901,"num_object_clones":0,"num_object_copies":15802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4013,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9772,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3890},{"shards":"0,2","objects":1523}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12855","reported_seq":21234,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:11.666295+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:31:11.666295+0000","last_peered":"2024-09-17T15:31:11.666295+0000","last_clean":"2024-09-17T15:31:11.666295+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:31:11.666295+0000","last_undegraded":"2024-09-17T15:31:11.666295+0000","last_fullsized":"2024-09-17T15:31:11.666295+0000","mapping_epoch":44,"log_start":"34'2800","ondisk_log_start":"34'2800","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31855558,"num_objects":7779,"num_object_clones":0,"num_object_copies":15558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9297,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":514401014,"num_objects":125596,"num_object_clones":0,"num_object_copies":251192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38880,"num_objects_misplaced":1537,"num_objects_unfound":0,"num_objects_dirty":125596,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":167391,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9043,"num_bytes_recovered":35394502,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1167749120,"data_stored":1167603682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160956,"ondisk_log_size":160956,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051658,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":686764,"kb_used_data":384344,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93685076,"statfs":{"total":96636764160,"available":95933517824,"internally_reserved":0,"allocated":393568256,"data_stored":393277159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":309653164},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,1,3,0,0,0,0,0,3],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724603,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":624972,"kb_used_data":314228,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93746868,"statfs":{"total":96636764160,"available":95996792832,"internally_reserved":0,"allocated":321769472,"data_stored":321445394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":318172843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,3,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247886,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":805468,"kb_used_data":444812,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93566372,"statfs":{"total":96636764160,"available":95811964928,"internally_reserved":0,"allocated":455487488,"data_stored":455196386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":369290920},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,5,4,0,0,0,0,0,3],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":453718016,"data_stored":453681886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":321331200,"data_stored":321262094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":392699904,"data_stored":392659702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:31:15.013 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:31:15.575 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:31:15.575 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:31:15.607 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":243,"stamp":"2024-09-17T15:31:13.690394+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":514401033,"num_objects":125598,"num_object_clones":0,"num_object_copies":251196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38880,"num_objects_misplaced":1537,"num_objects_unfound":0,"num_objects_dirty":125598,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":167393,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9044,"num_bytes_recovered":35394521,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160959,"ondisk_log_size":160959,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2117204,"kb_used_data":1143384,"kb_used_omap":13,"kb_used_meta":973746,"kb_avail":280998316,"statfs":{"total":289910292480,"available":287742275584,"internally_reserved":0,"allocated":1170825216,"data_stored":1169918939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":997116927},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,1,6,12,5,0,0,0,0,3,3],"upper_bound":8192},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25530368,"num_objects":-6233,"num_object_clones":0,"num_object_copies":-12466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-54,"num_objects_misplaced":-370,"num_objects_unfound":0,"num_objects_dirty":-6233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6233,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":424,"num_bytes_recovered":1515520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003223"},"pg_stats":[{"pgid":"2.f","version":"45'12827","reported_seq":12999,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:12.206117+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:31:12.206117+0000","last_peered":"2024-09-17T15:31:12.206117+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:31:12.206117+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'2800","ondisk_log_start":"34'2800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32034816,"num_objects":7821,"num_object_clones":0,"num_object_copies":15642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4025,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7821,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12827,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3796},{"shards":"0,2","objects":1567}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12944","reported_seq":13051,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:12.519147+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:31:12.519147+0000","last_peered":"2024-09-17T15:31:12.519147+0000","last_clean":"2024-09-17T15:31:12.519147+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:31:12.519147+0000","last_undegraded":"2024-09-17T15:31:12.519147+0000","last_fullsized":"2024-09-17T15:31:12.519147+0000","mapping_epoch":33,"log_start":"34'2900","ondisk_log_start":"34'2900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32719814,"num_objects":7990,"num_object_clones":0,"num_object_copies":15980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7990,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11301,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13096","reported_seq":13281,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:12.493686+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:31:12.493686+0000","last_peered":"2024-09-17T15:31:12.493686+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:31:12.493686+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'3000","ondisk_log_start":"34'3000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32100376,"num_objects":7838,"num_object_clones":0,"num_object_copies":15676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7838,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11253,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1541},{"shards":"2","objects":3697}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13001","reported_seq":13075,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:12.610519+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:31:12.610519+0000","last_peered":"2024-09-17T15:31:12.610519+0000","last_clean":"2024-09-17T15:31:12.610519+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:31:12.610519+0000","last_undegraded":"2024-09-17T15:31:12.610519+0000","last_fullsized":"2024-09-17T15:31:12.610519+0000","mapping_epoch":33,"log_start":"34'3000","ondisk_log_start":"34'3000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32641024,"num_objects":7969,"num_object_clones":0,"num_object_copies":15938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7969,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13001,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12817","reported_seq":12951,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:12.291411+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:31:12.291411+0000","last_peered":"2024-09-17T15:31:12.291411+0000","last_clean":"2024-09-17T15:31:12.291411+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:31:12.291411+0000","last_undegraded":"2024-09-17T15:31:12.291411+0000","last_fullsized":"2024-09-17T15:31:12.291411+0000","mapping_epoch":32,"log_start":"34'2800","ondisk_log_start":"34'2800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32096739,"num_objects":7837,"num_object_clones":0,"num_object_copies":15674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7837,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9799,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13285","reported_seq":13526,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:12.279994+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:31:12.279994+0000","last_peered":"2024-09-17T15:31:12.279994+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:31:12.279994+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'3200","ondisk_log_start":"34'3200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32629219,"num_objects":7967,"num_object_clones":0,"num_object_copies":15934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4220,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7967,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9784,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3748},{"shards":"0,2","objects":1611}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12995","reported_seq":13240,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:12.302308+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:31:12.302308+0000","last_peered":"2024-09-17T15:31:12.302308+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:31:12.302308+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'2900","ondisk_log_start":"34'2900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31839174,"num_objects":7775,"num_object_clones":0,"num_object_copies":15550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7775,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9764,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3652},{"shards":"0,2","objects":1566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13041","reported_seq":13173,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:12.278511+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:31:12.278511+0000","last_peered":"2024-09-17T15:31:12.278511+0000","last_clean":"2024-09-17T15:31:12.278511+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:31:12.278511+0000","last_undegraded":"2024-09-17T15:31:12.278511+0000","last_fullsized":"2024-09-17T15:31:12.278511+0000","mapping_epoch":32,"log_start":"34'3000","ondisk_log_start":"34'3000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32256000,"num_objects":7875,"num_object_clones":0,"num_object_copies":15750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9995,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.086967+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.086967+0000","last_peered":"2024-09-17T15:30:57.086967+0000","last_clean":"2024-09-17T15:30:57.086967+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.086967+0000","last_undegraded":"2024-09-17T15:30:57.086967+0000","last_fullsized":"2024-09-17T15:30:57.086967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12904","reported_seq":21191,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:11.605333+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:31:11.605333+0000","last_peered":"2024-09-17T15:31:11.605333+0000","last_clean":"2024-09-17T15:31:11.605333+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:31:11.605333+0000","last_undegraded":"2024-09-17T15:31:11.605333+0000","last_fullsized":"2024-09-17T15:31:11.605333+0000","mapping_epoch":42,"log_start":"34'2900","ondisk_log_start":"34'2900","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32412131,"num_objects":7914,"num_object_clones":0,"num_object_copies":15828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7914,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10904,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088505+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088505+0000","last_peered":"2024-09-17T15:30:57.088505+0000","last_clean":"2024-09-17T15:30:57.088505+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088505+0000","last_undegraded":"2024-09-17T15:30:57.088505+0000","last_fullsized":"2024-09-17T15:30:57.088505+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12786","reported_seq":13004,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:12.282900+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:31:12.282900+0000","last_peered":"2024-09-17T15:31:12.282900+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:31:12.282900+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'2700","ondisk_log_start":"34'2700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31768576,"num_objects":7756,"num_object_clones":0,"num_object_copies":15512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3970,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9382,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3788},{"shards":"0,2","objects":1499}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":107,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092541+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092541+0000","last_peered":"2024-09-17T15:30:57.092541+0000","last_clean":"2024-09-17T15:30:57.092541+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092541+0000","last_undegraded":"2024-09-17T15:30:57.092541+0000","last_fullsized":"2024-09-17T15:30:57.092541+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12783","reported_seq":12956,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:12.604901+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:31:12.604901+0000","last_peered":"2024-09-17T15:31:12.604901+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:31:12.604901+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'2700","ondisk_log_start":"34'2700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31687139,"num_objects":7737,"num_object_clones":0,"num_object_copies":15474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4054,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7737,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10678,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1556},{"shards":"2","objects":3684}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12959","reported_seq":14063,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:12.605103+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:31:12.605103+0000","last_peered":"2024-09-17T15:31:12.605103+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:31:12.605103+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'2900","ondisk_log_start":"34'2900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31854592,"num_objects":7777,"num_object_clones":0,"num_object_copies":15554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2112,"num_objects_misplaced":1537,"num_objects_unfound":0,"num_objects_dirty":7777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10721,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":490,"num_bytes_recovered":1896448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1537},{"shards":"2","objects":4128}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13088","reported_seq":13312,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:12.317321+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:31:12.317321+0000","last_peered":"2024-09-17T15:31:12.317321+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:31:12.317321+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'3000","ondisk_log_start":"34'3000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32268288,"num_objects":7878,"num_object_clones":0,"num_object_copies":15756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4097,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9636,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3782},{"shards":"0,2","objects":1534}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12990","reported_seq":13208,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:12.319706+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:31:12.319706+0000","last_peered":"2024-09-17T15:31:12.319706+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:31:12.319706+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'2900","ondisk_log_start":"34'2900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31875072,"num_objects":7782,"num_object_clones":0,"num_object_copies":15564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9277,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3661},{"shards":"0,2","objects":1557}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088855+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088855+0000","last_peered":"2024-09-17T15:30:57.088855+0000","last_clean":"2024-09-17T15:30:57.088855+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088855+0000","last_undegraded":"2024-09-17T15:30:57.088855+0000","last_fullsized":"2024-09-17T15:30:57.088855+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12985","reported_seq":13225,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:12.236883+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:31:12.236883+0000","last_peered":"2024-09-17T15:31:12.236883+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:31:12.236883+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'2900","ondisk_log_start":"34'2900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32362496,"num_objects":7901,"num_object_clones":0,"num_object_copies":15802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4013,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9772,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3890},{"shards":"0,2","objects":1523}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12855","reported_seq":21234,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:11.666295+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:31:11.666295+0000","last_peered":"2024-09-17T15:31:11.666295+0000","last_clean":"2024-09-17T15:31:11.666295+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:31:11.666295+0000","last_undegraded":"2024-09-17T15:31:11.666295+0000","last_fullsized":"2024-09-17T15:31:11.666295+0000","mapping_epoch":44,"log_start":"34'2800","ondisk_log_start":"34'2800","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31855558,"num_objects":7779,"num_object_clones":0,"num_object_copies":15558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9297,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":514401014,"num_objects":125596,"num_object_clones":0,"num_object_copies":251192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38880,"num_objects_misplaced":1537,"num_objects_unfound":0,"num_objects_dirty":125596,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":167391,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9043,"num_bytes_recovered":35394502,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1167749120,"data_stored":1167603682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160956,"ondisk_log_size":160956,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051658,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":686764,"kb_used_data":384344,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93685076,"statfs":{"total":96636764160,"available":95933517824,"internally_reserved":0,"allocated":393568256,"data_stored":393277159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":309653164},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,1,3,0,0,0,0,0,3],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724603,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":624972,"kb_used_data":314228,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93746868,"statfs":{"total":96636764160,"available":95996792832,"internally_reserved":0,"allocated":321769472,"data_stored":321445394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":318172843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,3,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247886,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":805468,"kb_used_data":444812,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93566372,"statfs":{"total":96636764160,"available":95811964928,"internally_reserved":0,"allocated":455487488,"data_stored":455196386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":369290920},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,5,4,0,0,0,0,0,3],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":453718016,"data_stored":453681886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":321331200,"data_stored":321262094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":392699904,"data_stored":392659702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:31:15.608 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:31:15.608 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'12827", 'reported_seq': 12999, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:12.206117+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:31:12.206117+0000', 'last_peered': '2024-09-17T15:31:12.206117+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:31:12.206117+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "34'2800", 'ondisk_log_start': "34'2800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32034816, 'num_objects': 7821, 'num_object_clones': 0, 'num_object_copies': 15642, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4025, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7821, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12827, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 45, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3796}, {'shards': '0,2', 'objects': 1567}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:15.608 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:31:15.609 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'13096", 'reported_seq': 13281, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:12.493686+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:31:12.493686+0000', 'last_peered': '2024-09-17T15:31:12.493686+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:31:12.493686+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "34'3000", 'ondisk_log_start': "34'3000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32100376, 'num_objects': 7838, 'num_object_clones': 0, 'num_object_copies': 15676, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7838, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11253, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1541}, {'shards': '2', 'objects': 3697}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:31:15.609 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:31:15.609 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'13285", 'reported_seq': 13526, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:12.279994+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:31:12.279994+0000', 'last_peered': '2024-09-17T15:31:12.279994+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:31:12.279994+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "34'3200", 'ondisk_log_start': "34'3200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32629219, 'num_objects': 7967, 'num_object_clones': 0, 'num_object_copies': 15934, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4220, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7967, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9784, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3748}, {'shards': '0,2', 'objects': 1611}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:15.609 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:31:15.609 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'12995", 'reported_seq': 13240, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:12.302308+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:31:12.302308+0000', 'last_peered': '2024-09-17T15:31:12.302308+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:31:12.302308+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "34'2900", 'ondisk_log_start': "34'2900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31839174, 'num_objects': 7775, 'num_object_clones': 0, 'num_object_copies': 15550, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4123, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7775, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 9764, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3652}, {'shards': '0,2', 'objects': 1566}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:15.609 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:31:15.609 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'12786", 'reported_seq': 13004, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:12.282900+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:31:12.282900+0000', 'last_peered': '2024-09-17T15:31:12.282900+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:31:12.282900+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "34'2700", 'ondisk_log_start': "34'2700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31768576, 'num_objects': 7756, 'num_object_clones': 0, 'num_object_copies': 15512, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3970, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7756, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9382, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 42, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3788}, {'shards': '0,2', 'objects': 1499}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:15.609 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:31:15.609 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'12783", 'reported_seq': 12956, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:12.604901+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:31:12.604901+0000', 'last_peered': '2024-09-17T15:31:12.604901+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:31:12.604901+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "34'2700", 'ondisk_log_start': "34'2700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31687139, 'num_objects': 7737, 'num_object_clones': 0, 'num_object_copies': 15474, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4054, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7737, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10678, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 24, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1556}, {'shards': '2', 'objects': 3684}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:31:15.609 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:31:15.610 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'12959", 'reported_seq': 14063, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:12.605103+0000', 'last_change': '2024-09-17T15:30:55.086473+0000', 'last_active': '2024-09-17T15:31:12.605103+0000', 'last_peered': '2024-09-17T15:31:12.605103+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:31:12.605103+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "34'2900", 'ondisk_log_start': "34'2900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31854592, 'num_objects': 7777, 'num_object_clones': 0, 'num_object_copies': 15554, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2112, 'num_objects_misplaced': 1537, 'num_objects_unfound': 0, 'num_objects_dirty': 7777, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10721, 'num_write_kb': 32532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 490, 'num_bytes_recovered': 1896448, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1537}, {'shards': '2', 'objects': 4128}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:31:15.610 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:31:15.610 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'13088", 'reported_seq': 13312, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:12.317321+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:31:12.317321+0000', 'last_peered': '2024-09-17T15:31:12.317321+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:31:12.317321+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "34'3000", 'ondisk_log_start': "34'3000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32268288, 'num_objects': 7878, 'num_object_clones': 0, 'num_object_copies': 15756, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4097, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7878, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9636, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 42, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3782}, {'shards': '0,2', 'objects': 1534}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:15.610 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:31:15.610 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'12990", 'reported_seq': 13208, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:12.319706+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:31:12.319706+0000', 'last_peered': '2024-09-17T15:31:12.319706+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:31:12.319706+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "34'2900", 'ondisk_log_start': "34'2900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31875072, 'num_objects': 7782, 'num_object_clones': 0, 'num_object_copies': 15564, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7782, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9277, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 36, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3661}, {'shards': '0,2', 'objects': 1557}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:15.610 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:31:15.610 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'12985", 'reported_seq': 13225, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:12.236883+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:31:12.236883+0000', 'last_peered': '2024-09-17T15:31:12.236883+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:31:12.236883+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "34'2900", 'ondisk_log_start': "34'2900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32362496, 'num_objects': 7901, 'num_object_clones': 0, 'num_object_copies': 15802, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4013, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7901, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9772, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3890}, {'shards': '0,2', 'objects': 1523}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:15.610 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:31:16.260 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:31:16.260 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38880/251196 objects degraded (15.478%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":413,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":125598,"data_bytes":514401033,"bytes_used":2168016896,"bytes_avail":287742275584,"bytes_total":289910292480,"degraded_objects":38880,"degraded_total":251196,"degraded_ratio":0.15477953470596664,"misplaced_objects":1537,"misplaced_total":251196,"misplaced_ratio":0.0061187280052230134,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":82920,"recovering_keys_per_sec":0,"num_objects_recovered":280,"num_bytes_recovered":995328,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":363},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T15:27:19.616376+0000","services":{}},"progress_events":{}} 2024-09-17T15:31:16.285 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:31:16.285 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:31:16.834 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:31:16.835 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:31:16.858 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":244,"stamp":"2024-09-17T15:31:15.691010+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":514401033,"num_objects":125598,"num_object_clones":0,"num_object_copies":251196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38880,"num_objects_misplaced":1537,"num_objects_unfound":0,"num_objects_dirty":125598,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":167393,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9044,"num_bytes_recovered":35394521,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160959,"ondisk_log_size":160959,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2117204,"kb_used_data":1143384,"kb_used_omap":13,"kb_used_meta":973746,"kb_avail":280998316,"statfs":{"total":289910292480,"available":287742275584,"internally_reserved":0,"allocated":1170825216,"data_stored":1169918939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":997116927},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,1,6,12,5,0,0,0,0,3,3],"upper_bound":8192},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17883136,"num_objects":-4366,"num_object_clones":0,"num_object_copies":-8732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-37,"num_objects_misplaced":-243,"num_objects_unfound":0,"num_objects_dirty":-4366,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4366,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":995328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003361"},"pg_stats":[{"pgid":"2.f","version":"45'12827","reported_seq":12999,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:12.206117+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:31:12.206117+0000","last_peered":"2024-09-17T15:31:12.206117+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:31:12.206117+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'2800","ondisk_log_start":"34'2800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32034816,"num_objects":7821,"num_object_clones":0,"num_object_copies":15642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4025,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7821,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12827,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3796},{"shards":"0,2","objects":1567}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12944","reported_seq":13051,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:12.519147+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:31:12.519147+0000","last_peered":"2024-09-17T15:31:12.519147+0000","last_clean":"2024-09-17T15:31:12.519147+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:31:12.519147+0000","last_undegraded":"2024-09-17T15:31:12.519147+0000","last_fullsized":"2024-09-17T15:31:12.519147+0000","mapping_epoch":33,"log_start":"34'2900","ondisk_log_start":"34'2900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32719814,"num_objects":7990,"num_object_clones":0,"num_object_copies":15980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7990,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11301,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13096","reported_seq":13281,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:12.493686+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:31:12.493686+0000","last_peered":"2024-09-17T15:31:12.493686+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:31:12.493686+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'3000","ondisk_log_start":"34'3000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32100376,"num_objects":7838,"num_object_clones":0,"num_object_copies":15676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7838,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11253,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1541},{"shards":"2","objects":3697}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13001","reported_seq":13075,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:12.610519+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:31:12.610519+0000","last_peered":"2024-09-17T15:31:12.610519+0000","last_clean":"2024-09-17T15:31:12.610519+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:31:12.610519+0000","last_undegraded":"2024-09-17T15:31:12.610519+0000","last_fullsized":"2024-09-17T15:31:12.610519+0000","mapping_epoch":33,"log_start":"34'3000","ondisk_log_start":"34'3000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32641024,"num_objects":7969,"num_object_clones":0,"num_object_copies":15938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7969,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13001,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12817","reported_seq":12951,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:12.291411+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:31:12.291411+0000","last_peered":"2024-09-17T15:31:12.291411+0000","last_clean":"2024-09-17T15:31:12.291411+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:31:12.291411+0000","last_undegraded":"2024-09-17T15:31:12.291411+0000","last_fullsized":"2024-09-17T15:31:12.291411+0000","mapping_epoch":32,"log_start":"34'2800","ondisk_log_start":"34'2800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32096739,"num_objects":7837,"num_object_clones":0,"num_object_copies":15674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7837,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9799,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13285","reported_seq":13526,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:12.279994+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:31:12.279994+0000","last_peered":"2024-09-17T15:31:12.279994+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:31:12.279994+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'3200","ondisk_log_start":"34'3200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32629219,"num_objects":7967,"num_object_clones":0,"num_object_copies":15934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4220,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7967,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9784,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3748},{"shards":"0,2","objects":1611}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12995","reported_seq":13240,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:12.302308+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:31:12.302308+0000","last_peered":"2024-09-17T15:31:12.302308+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:31:12.302308+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'2900","ondisk_log_start":"34'2900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31839174,"num_objects":7775,"num_object_clones":0,"num_object_copies":15550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7775,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9764,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3652},{"shards":"0,2","objects":1566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13041","reported_seq":13173,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:12.278511+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:31:12.278511+0000","last_peered":"2024-09-17T15:31:12.278511+0000","last_clean":"2024-09-17T15:31:12.278511+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:31:12.278511+0000","last_undegraded":"2024-09-17T15:31:12.278511+0000","last_fullsized":"2024-09-17T15:31:12.278511+0000","mapping_epoch":32,"log_start":"34'3000","ondisk_log_start":"34'3000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32256000,"num_objects":7875,"num_object_clones":0,"num_object_copies":15750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9995,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.086967+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.086967+0000","last_peered":"2024-09-17T15:30:57.086967+0000","last_clean":"2024-09-17T15:30:57.086967+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.086967+0000","last_undegraded":"2024-09-17T15:30:57.086967+0000","last_fullsized":"2024-09-17T15:30:57.086967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12904","reported_seq":21191,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:11.605333+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:31:11.605333+0000","last_peered":"2024-09-17T15:31:11.605333+0000","last_clean":"2024-09-17T15:31:11.605333+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:31:11.605333+0000","last_undegraded":"2024-09-17T15:31:11.605333+0000","last_fullsized":"2024-09-17T15:31:11.605333+0000","mapping_epoch":42,"log_start":"34'2900","ondisk_log_start":"34'2900","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32412131,"num_objects":7914,"num_object_clones":0,"num_object_copies":15828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7914,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10904,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088505+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088505+0000","last_peered":"2024-09-17T15:30:57.088505+0000","last_clean":"2024-09-17T15:30:57.088505+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088505+0000","last_undegraded":"2024-09-17T15:30:57.088505+0000","last_fullsized":"2024-09-17T15:30:57.088505+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12786","reported_seq":13004,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:12.282900+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:31:12.282900+0000","last_peered":"2024-09-17T15:31:12.282900+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:31:12.282900+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'2700","ondisk_log_start":"34'2700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31768576,"num_objects":7756,"num_object_clones":0,"num_object_copies":15512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3970,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9382,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3788},{"shards":"0,2","objects":1499}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":107,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092541+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092541+0000","last_peered":"2024-09-17T15:30:57.092541+0000","last_clean":"2024-09-17T15:30:57.092541+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092541+0000","last_undegraded":"2024-09-17T15:30:57.092541+0000","last_fullsized":"2024-09-17T15:30:57.092541+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12783","reported_seq":12956,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:12.604901+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:31:12.604901+0000","last_peered":"2024-09-17T15:31:12.604901+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:31:12.604901+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'2700","ondisk_log_start":"34'2700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31687139,"num_objects":7737,"num_object_clones":0,"num_object_copies":15474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4054,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7737,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10678,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1556},{"shards":"2","objects":3684}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12959","reported_seq":14063,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:12.605103+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:31:12.605103+0000","last_peered":"2024-09-17T15:31:12.605103+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:31:12.605103+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'2900","ondisk_log_start":"34'2900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31854592,"num_objects":7777,"num_object_clones":0,"num_object_copies":15554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2112,"num_objects_misplaced":1537,"num_objects_unfound":0,"num_objects_dirty":7777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10721,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":490,"num_bytes_recovered":1896448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1537},{"shards":"2","objects":4128}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13088","reported_seq":13312,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:12.317321+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:31:12.317321+0000","last_peered":"2024-09-17T15:31:12.317321+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:31:12.317321+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'3000","ondisk_log_start":"34'3000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32268288,"num_objects":7878,"num_object_clones":0,"num_object_copies":15756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4097,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9636,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3782},{"shards":"0,2","objects":1534}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12990","reported_seq":13208,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:12.319706+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:31:12.319706+0000","last_peered":"2024-09-17T15:31:12.319706+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:31:12.319706+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'2900","ondisk_log_start":"34'2900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31875072,"num_objects":7782,"num_object_clones":0,"num_object_copies":15564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9277,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3661},{"shards":"0,2","objects":1557}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088855+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088855+0000","last_peered":"2024-09-17T15:30:57.088855+0000","last_clean":"2024-09-17T15:30:57.088855+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088855+0000","last_undegraded":"2024-09-17T15:30:57.088855+0000","last_fullsized":"2024-09-17T15:30:57.088855+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12985","reported_seq":13225,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:12.236883+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:31:12.236883+0000","last_peered":"2024-09-17T15:31:12.236883+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:31:12.236883+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'2900","ondisk_log_start":"34'2900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32362496,"num_objects":7901,"num_object_clones":0,"num_object_copies":15802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4013,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9772,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3890},{"shards":"0,2","objects":1523}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12855","reported_seq":21234,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:11.666295+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:31:11.666295+0000","last_peered":"2024-09-17T15:31:11.666295+0000","last_clean":"2024-09-17T15:31:11.666295+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:31:11.666295+0000","last_undegraded":"2024-09-17T15:31:11.666295+0000","last_fullsized":"2024-09-17T15:31:11.666295+0000","mapping_epoch":44,"log_start":"34'2800","ondisk_log_start":"34'2800","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31855558,"num_objects":7779,"num_object_clones":0,"num_object_copies":15558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9297,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":514401014,"num_objects":125596,"num_object_clones":0,"num_object_copies":251192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38880,"num_objects_misplaced":1537,"num_objects_unfound":0,"num_objects_dirty":125596,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":167391,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9043,"num_bytes_recovered":35394502,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1167749120,"data_stored":1167603682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160956,"ondisk_log_size":160956,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051658,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":686764,"kb_used_data":384344,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93685076,"statfs":{"total":96636764160,"available":95933517824,"internally_reserved":0,"allocated":393568256,"data_stored":393277159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":309653164},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,1,3,0,0,0,0,0,3],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724603,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":624972,"kb_used_data":314228,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93746868,"statfs":{"total":96636764160,"available":95996792832,"internally_reserved":0,"allocated":321769472,"data_stored":321445394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":318172843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,3,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247886,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":805468,"kb_used_data":444812,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93566372,"statfs":{"total":96636764160,"available":95811964928,"internally_reserved":0,"allocated":455487488,"data_stored":455196386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":369290920},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,5,4,0,0,0,0,0,3],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":453718016,"data_stored":453681886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":321331200,"data_stored":321262094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":392699904,"data_stored":392659702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:31:19.860 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:31:20.419 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:31:20.419 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:31:20.444 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":246,"stamp":"2024-09-17T15:31:19.692619+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":505279241,"num_objects":123371,"num_object_clones":0,"num_object_copies":246742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38844,"num_objects_misplaced":1410,"num_objects_unfound":0,"num_objects_dirty":123371,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":169620,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9207,"num_bytes_recovered":35910617,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160686,"ondisk_log_size":160686,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2122800,"kb_used_data":1130936,"kb_used_omap":13,"kb_used_meta":991794,"kb_avail":280992720,"statfs":{"total":289910292480,"available":287736545280,"internally_reserved":0,"allocated":1158078464,"data_stored":1157172187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1015598079},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,2,7,7,6,0,0,6],"upper_bound":1024},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18206720,"num_objects":-4445,"num_object_clones":0,"num_object_copies":-8890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-50,"num_objects_misplaced":-237,"num_objects_unfound":0,"num_objects_dirty":-4445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4445,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":966656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004033"},"pg_stats":[{"pgid":"2.f","version":"45'12966","reported_seq":13156,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:17.259635+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:31:17.259635+0000","last_peered":"2024-09-17T15:31:17.259635+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:31:17.259635+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'2900","ondisk_log_start":"34'2900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31465472,"num_objects":7682,"num_object_clones":0,"num_object_copies":15364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4016,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12966,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3666},{"shards":"0,2","objects":1428}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13081","reported_seq":13188,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:17.603850+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:31:17.603850+0000","last_peered":"2024-09-17T15:31:17.603850+0000","last_clean":"2024-09-17T15:31:17.603850+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:31:17.603850+0000","last_undegraded":"2024-09-17T15:31:17.603850+0000","last_fullsized":"2024-09-17T15:31:17.603850+0000","mapping_epoch":33,"log_start":"34'3000","ondisk_log_start":"34'3000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32158662,"num_objects":7853,"num_object_clones":0,"num_object_copies":15706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7853,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11438,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13236","reported_seq":13427,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:17.609054+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:31:17.609054+0000","last_peered":"2024-09-17T15:31:17.609054+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:31:17.609054+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'3200","ondisk_log_start":"34'3200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31526936,"num_objects":7698,"num_object_clones":0,"num_object_copies":15396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7698,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11393,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1401},{"shards":"2","objects":3560}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13121","reported_seq":13195,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:17.582585+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:31:17.582585+0000","last_peered":"2024-09-17T15:31:17.582585+0000","last_clean":"2024-09-17T15:31:17.582585+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:31:17.582585+0000","last_undegraded":"2024-09-17T15:31:17.582585+0000","last_fullsized":"2024-09-17T15:31:17.582585+0000","mapping_epoch":33,"log_start":"34'3100","ondisk_log_start":"34'3100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32149504,"num_objects":7849,"num_object_clones":0,"num_object_copies":15698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7849,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13121,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12950","reported_seq":13084,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:17.253666+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:31:17.253666+0000","last_peered":"2024-09-17T15:31:17.253666+0000","last_clean":"2024-09-17T15:31:17.253666+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:31:17.253666+0000","last_undegraded":"2024-09-17T15:31:17.253666+0000","last_fullsized":"2024-09-17T15:31:17.253666+0000","mapping_epoch":32,"log_start":"34'2900","ondisk_log_start":"34'2900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31551971,"num_objects":7704,"num_object_clones":0,"num_object_copies":15408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7704,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9932,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13430","reported_seq":13681,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:17.299033+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:31:17.299033+0000","last_peered":"2024-09-17T15:31:17.299033+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:31:17.299033+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'3400","ondisk_log_start":"34'3400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32035299,"num_objects":7822,"num_object_clones":0,"num_object_copies":15644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4215,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7822,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9929,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3608},{"shards":"0,2","objects":1466}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13136","reported_seq":13391,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:17.336423+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:31:17.336423+0000","last_peered":"2024-09-17T15:31:17.336423+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:31:17.336423+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'3100","ondisk_log_start":"34'3100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31261638,"num_objects":7634,"num_object_clones":0,"num_object_copies":15268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4118,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7634,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9905,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3516},{"shards":"0,2","objects":1425}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13185","reported_seq":13317,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:17.300267+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:31:17.300267+0000","last_peered":"2024-09-17T15:31:17.300267+0000","last_clean":"2024-09-17T15:31:17.300267+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:31:17.300267+0000","last_undegraded":"2024-09-17T15:31:17.300267+0000","last_fullsized":"2024-09-17T15:31:17.300267+0000","mapping_epoch":32,"log_start":"34'3100","ondisk_log_start":"34'3100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31666176,"num_objects":7731,"num_object_clones":0,"num_object_copies":15462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10139,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.086967+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.086967+0000","last_peered":"2024-09-17T15:30:57.086967+0000","last_clean":"2024-09-17T15:30:57.086967+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.086967+0000","last_undegraded":"2024-09-17T15:30:57.086967+0000","last_fullsized":"2024-09-17T15:30:57.086967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13035","reported_seq":21322,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:16.678219+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:31:16.678219+0000","last_peered":"2024-09-17T15:31:16.678219+0000","last_clean":"2024-09-17T15:31:16.678219+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:31:16.678219+0000","last_undegraded":"2024-09-17T15:31:16.678219+0000","last_fullsized":"2024-09-17T15:31:16.678219+0000","mapping_epoch":42,"log_start":"34'3000","ondisk_log_start":"34'3000","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31875555,"num_objects":7783,"num_object_clones":0,"num_object_copies":15566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7783,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11035,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088505+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088505+0000","last_peered":"2024-09-17T15:30:57.088505+0000","last_clean":"2024-09-17T15:30:57.088505+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088505+0000","last_undegraded":"2024-09-17T15:30:57.088505+0000","last_fullsized":"2024-09-17T15:30:57.088505+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12920","reported_seq":13139,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:17.242857+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:31:17.242857+0000","last_peered":"2024-09-17T15:31:17.242857+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:31:17.242857+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'2900","ondisk_log_start":"34'2900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31219712,"num_objects":7622,"num_object_clones":0,"num_object_copies":15244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3970,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7622,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9516,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3654},{"shards":"0,2","objects":1365}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":107,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092541+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092541+0000","last_peered":"2024-09-17T15:30:57.092541+0000","last_clean":"2024-09-17T15:30:57.092541+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092541+0000","last_undegraded":"2024-09-17T15:30:57.092541+0000","last_fullsized":"2024-09-17T15:30:57.092541+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12916","reported_seq":13091,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:17.546180+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:31:17.546180+0000","last_peered":"2024-09-17T15:31:17.546180+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:31:17.546180+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'2900","ondisk_log_start":"34'2900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31142371,"num_objects":7604,"num_object_clones":0,"num_object_copies":15208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4053,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7604,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10811,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1423},{"shards":"2","objects":3552}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13094","reported_seq":14450,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:17.637215+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:31:17.637215+0000","last_peered":"2024-09-17T15:31:17.637215+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:31:17.637215+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'3000","ondisk_log_start":"34'3000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31301632,"num_objects":7642,"num_object_clones":0,"num_object_copies":15284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2113,"num_objects_misplaced":1410,"num_objects_unfound":0,"num_objects_dirty":7642,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10856,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":616,"num_bytes_recovered":2412544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1410},{"shards":"2","objects":4119}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13239","reported_seq":13477,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:17.340236+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:31:17.340236+0000","last_peered":"2024-09-17T15:31:17.340236+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:31:17.340236+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'3200","ondisk_log_start":"34'3200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31649792,"num_objects":7727,"num_object_clones":0,"num_object_copies":15454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4090,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9787,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3638},{"shards":"0,2","objects":1383}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13141","reported_seq":13362,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:17.280362+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:31:17.280362+0000","last_peered":"2024-09-17T15:31:17.280362+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:31:17.280362+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'3100","ondisk_log_start":"34'3100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31256576,"num_objects":7631,"num_object_clones":0,"num_object_copies":15262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9428,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3514},{"shards":"0,2","objects":1406}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088855+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088855+0000","last_peered":"2024-09-17T15:30:57.088855+0000","last_clean":"2024-09-17T15:30:57.088855+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088855+0000","last_undegraded":"2024-09-17T15:30:57.088855+0000","last_fullsized":"2024-09-17T15:30:57.088855+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13123","reported_seq":13369,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:17.347783+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:31:17.347783+0000","last_peered":"2024-09-17T15:31:17.347783+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:31:17.347783+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'3100","ondisk_log_start":"34'3100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31797248,"num_objects":7763,"num_object_clones":0,"num_object_copies":15526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4010,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9910,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3755},{"shards":"0,2","objects":1385}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13010","reported_seq":21389,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:16.559420+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:31:16.559420+0000","last_peered":"2024-09-17T15:31:16.559420+0000","last_clean":"2024-09-17T15:31:16.559420+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:31:16.559420+0000","last_undegraded":"2024-09-17T15:31:16.559420+0000","last_fullsized":"2024-09-17T15:31:16.559420+0000","mapping_epoch":44,"log_start":"34'3000","ondisk_log_start":"34'3000","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31220678,"num_objects":7624,"num_object_clones":0,"num_object_copies":15248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9452,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":505279222,"num_objects":123369,"num_object_clones":0,"num_object_copies":246738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38844,"num_objects_misplaced":1410,"num_objects_unfound":0,"num_objects_dirty":123369,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":169618,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9206,"num_bytes_recovered":35910598,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1155280896,"data_stored":1155135458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160683,"ondisk_log_size":160683,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051659,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":686564,"kb_used_data":379988,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93685276,"statfs":{"total":96636764160,"available":95933722624,"internally_reserved":0,"allocated":389107712,"data_stored":388816615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":313913004},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724604,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":615436,"kb_used_data":313020,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93756404,"statfs":{"total":96636764160,"available":96006557696,"internally_reserved":0,"allocated":320532480,"data_stored":320208402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":309653163},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247887,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":820800,"kb_used_data":437928,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93551040,"statfs":{"total":96636764160,"available":95796264960,"internally_reserved":0,"allocated":448438272,"data_stored":448147170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":392031912},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,1,2,0,3,0,0,6],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":446828544,"data_stored":446792414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":320102400,"data_stored":320033294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":388349952,"data_stored":388309750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:31:20.445 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:31:20.991 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:31:20.991 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:31:21.016 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":246,"stamp":"2024-09-17T15:31:19.692619+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":505279241,"num_objects":123371,"num_object_clones":0,"num_object_copies":246742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38844,"num_objects_misplaced":1410,"num_objects_unfound":0,"num_objects_dirty":123371,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":169620,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9207,"num_bytes_recovered":35910617,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160686,"ondisk_log_size":160686,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2122800,"kb_used_data":1130936,"kb_used_omap":13,"kb_used_meta":991794,"kb_avail":280992720,"statfs":{"total":289910292480,"available":287736545280,"internally_reserved":0,"allocated":1158078464,"data_stored":1157172187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1015598079},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,2,7,7,6,0,0,6],"upper_bound":1024},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18206720,"num_objects":-4445,"num_object_clones":0,"num_object_copies":-8890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-50,"num_objects_misplaced":-237,"num_objects_unfound":0,"num_objects_dirty":-4445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4445,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":966656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004033"},"pg_stats":[{"pgid":"2.f","version":"45'12966","reported_seq":13156,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:17.259635+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:31:17.259635+0000","last_peered":"2024-09-17T15:31:17.259635+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:31:17.259635+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'2900","ondisk_log_start":"34'2900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31465472,"num_objects":7682,"num_object_clones":0,"num_object_copies":15364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4016,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12966,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3666},{"shards":"0,2","objects":1428}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13081","reported_seq":13188,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:17.603850+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:31:17.603850+0000","last_peered":"2024-09-17T15:31:17.603850+0000","last_clean":"2024-09-17T15:31:17.603850+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:31:17.603850+0000","last_undegraded":"2024-09-17T15:31:17.603850+0000","last_fullsized":"2024-09-17T15:31:17.603850+0000","mapping_epoch":33,"log_start":"34'3000","ondisk_log_start":"34'3000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32158662,"num_objects":7853,"num_object_clones":0,"num_object_copies":15706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7853,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11438,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13236","reported_seq":13427,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:17.609054+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:31:17.609054+0000","last_peered":"2024-09-17T15:31:17.609054+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:31:17.609054+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'3200","ondisk_log_start":"34'3200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31526936,"num_objects":7698,"num_object_clones":0,"num_object_copies":15396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7698,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11393,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1401},{"shards":"2","objects":3560}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13121","reported_seq":13195,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:17.582585+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:31:17.582585+0000","last_peered":"2024-09-17T15:31:17.582585+0000","last_clean":"2024-09-17T15:31:17.582585+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:31:17.582585+0000","last_undegraded":"2024-09-17T15:31:17.582585+0000","last_fullsized":"2024-09-17T15:31:17.582585+0000","mapping_epoch":33,"log_start":"34'3100","ondisk_log_start":"34'3100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32149504,"num_objects":7849,"num_object_clones":0,"num_object_copies":15698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7849,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13121,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12950","reported_seq":13084,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:17.253666+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:31:17.253666+0000","last_peered":"2024-09-17T15:31:17.253666+0000","last_clean":"2024-09-17T15:31:17.253666+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:31:17.253666+0000","last_undegraded":"2024-09-17T15:31:17.253666+0000","last_fullsized":"2024-09-17T15:31:17.253666+0000","mapping_epoch":32,"log_start":"34'2900","ondisk_log_start":"34'2900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31551971,"num_objects":7704,"num_object_clones":0,"num_object_copies":15408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7704,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9932,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13430","reported_seq":13681,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:17.299033+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:31:17.299033+0000","last_peered":"2024-09-17T15:31:17.299033+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:31:17.299033+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'3400","ondisk_log_start":"34'3400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32035299,"num_objects":7822,"num_object_clones":0,"num_object_copies":15644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4215,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7822,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9929,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3608},{"shards":"0,2","objects":1466}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13136","reported_seq":13391,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:17.336423+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:31:17.336423+0000","last_peered":"2024-09-17T15:31:17.336423+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:31:17.336423+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'3100","ondisk_log_start":"34'3100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31261638,"num_objects":7634,"num_object_clones":0,"num_object_copies":15268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4118,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7634,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9905,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3516},{"shards":"0,2","objects":1425}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13185","reported_seq":13317,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:17.300267+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:31:17.300267+0000","last_peered":"2024-09-17T15:31:17.300267+0000","last_clean":"2024-09-17T15:31:17.300267+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:31:17.300267+0000","last_undegraded":"2024-09-17T15:31:17.300267+0000","last_fullsized":"2024-09-17T15:31:17.300267+0000","mapping_epoch":32,"log_start":"34'3100","ondisk_log_start":"34'3100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31666176,"num_objects":7731,"num_object_clones":0,"num_object_copies":15462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10139,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.086967+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.086967+0000","last_peered":"2024-09-17T15:30:57.086967+0000","last_clean":"2024-09-17T15:30:57.086967+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.086967+0000","last_undegraded":"2024-09-17T15:30:57.086967+0000","last_fullsized":"2024-09-17T15:30:57.086967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13035","reported_seq":21322,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:16.678219+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:31:16.678219+0000","last_peered":"2024-09-17T15:31:16.678219+0000","last_clean":"2024-09-17T15:31:16.678219+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:31:16.678219+0000","last_undegraded":"2024-09-17T15:31:16.678219+0000","last_fullsized":"2024-09-17T15:31:16.678219+0000","mapping_epoch":42,"log_start":"34'3000","ondisk_log_start":"34'3000","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31875555,"num_objects":7783,"num_object_clones":0,"num_object_copies":15566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7783,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11035,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088505+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088505+0000","last_peered":"2024-09-17T15:30:57.088505+0000","last_clean":"2024-09-17T15:30:57.088505+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088505+0000","last_undegraded":"2024-09-17T15:30:57.088505+0000","last_fullsized":"2024-09-17T15:30:57.088505+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12920","reported_seq":13139,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:17.242857+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:31:17.242857+0000","last_peered":"2024-09-17T15:31:17.242857+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:31:17.242857+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'2900","ondisk_log_start":"34'2900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31219712,"num_objects":7622,"num_object_clones":0,"num_object_copies":15244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3970,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7622,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9516,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3654},{"shards":"0,2","objects":1365}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":107,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092541+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092541+0000","last_peered":"2024-09-17T15:30:57.092541+0000","last_clean":"2024-09-17T15:30:57.092541+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092541+0000","last_undegraded":"2024-09-17T15:30:57.092541+0000","last_fullsized":"2024-09-17T15:30:57.092541+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12916","reported_seq":13091,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:17.546180+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:31:17.546180+0000","last_peered":"2024-09-17T15:31:17.546180+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:31:17.546180+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'2900","ondisk_log_start":"34'2900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31142371,"num_objects":7604,"num_object_clones":0,"num_object_copies":15208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4053,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7604,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10811,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1423},{"shards":"2","objects":3552}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13094","reported_seq":14450,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:17.637215+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:31:17.637215+0000","last_peered":"2024-09-17T15:31:17.637215+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:31:17.637215+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'3000","ondisk_log_start":"34'3000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31301632,"num_objects":7642,"num_object_clones":0,"num_object_copies":15284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2113,"num_objects_misplaced":1410,"num_objects_unfound":0,"num_objects_dirty":7642,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10856,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":616,"num_bytes_recovered":2412544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1410},{"shards":"2","objects":4119}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13239","reported_seq":13477,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:17.340236+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:31:17.340236+0000","last_peered":"2024-09-17T15:31:17.340236+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:31:17.340236+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'3200","ondisk_log_start":"34'3200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31649792,"num_objects":7727,"num_object_clones":0,"num_object_copies":15454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4090,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9787,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3638},{"shards":"0,2","objects":1383}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13141","reported_seq":13362,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:17.280362+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:31:17.280362+0000","last_peered":"2024-09-17T15:31:17.280362+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:31:17.280362+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'3100","ondisk_log_start":"34'3100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31256576,"num_objects":7631,"num_object_clones":0,"num_object_copies":15262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9428,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3514},{"shards":"0,2","objects":1406}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088855+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088855+0000","last_peered":"2024-09-17T15:30:57.088855+0000","last_clean":"2024-09-17T15:30:57.088855+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088855+0000","last_undegraded":"2024-09-17T15:30:57.088855+0000","last_fullsized":"2024-09-17T15:30:57.088855+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13123","reported_seq":13369,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:17.347783+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:31:17.347783+0000","last_peered":"2024-09-17T15:31:17.347783+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:31:17.347783+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'3100","ondisk_log_start":"34'3100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31797248,"num_objects":7763,"num_object_clones":0,"num_object_copies":15526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4010,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9910,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3755},{"shards":"0,2","objects":1385}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13010","reported_seq":21389,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:16.559420+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:31:16.559420+0000","last_peered":"2024-09-17T15:31:16.559420+0000","last_clean":"2024-09-17T15:31:16.559420+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:31:16.559420+0000","last_undegraded":"2024-09-17T15:31:16.559420+0000","last_fullsized":"2024-09-17T15:31:16.559420+0000","mapping_epoch":44,"log_start":"34'3000","ondisk_log_start":"34'3000","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31220678,"num_objects":7624,"num_object_clones":0,"num_object_copies":15248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9452,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":505279222,"num_objects":123369,"num_object_clones":0,"num_object_copies":246738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38844,"num_objects_misplaced":1410,"num_objects_unfound":0,"num_objects_dirty":123369,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":169618,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9206,"num_bytes_recovered":35910598,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1155280896,"data_stored":1155135458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160683,"ondisk_log_size":160683,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051659,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":686564,"kb_used_data":379988,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93685276,"statfs":{"total":96636764160,"available":95933722624,"internally_reserved":0,"allocated":389107712,"data_stored":388816615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":313913004},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724604,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":615436,"kb_used_data":313020,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93756404,"statfs":{"total":96636764160,"available":96006557696,"internally_reserved":0,"allocated":320532480,"data_stored":320208402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":309653163},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247887,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":820800,"kb_used_data":437928,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93551040,"statfs":{"total":96636764160,"available":95796264960,"internally_reserved":0,"allocated":448438272,"data_stored":448147170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":392031912},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,1,2,0,3,0,0,6],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":446828544,"data_stored":446792414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":320102400,"data_stored":320033294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":388349952,"data_stored":388309750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:31:21.017 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:31:21.017 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'12966", 'reported_seq': 13156, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:17.259635+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:31:17.259635+0000', 'last_peered': '2024-09-17T15:31:17.259635+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:31:17.259635+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "34'2900", 'ondisk_log_start': "34'2900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31465472, 'num_objects': 7682, 'num_object_clones': 0, 'num_object_copies': 15364, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4016, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7682, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12966, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 54, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3666}, {'shards': '0,2', 'objects': 1428}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:21.018 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:31:21.018 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'13236", 'reported_seq': 13427, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:17.609054+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:31:17.609054+0000', 'last_peered': '2024-09-17T15:31:17.609054+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:31:17.609054+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "34'3200", 'ondisk_log_start': "34'3200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31526936, 'num_objects': 7698, 'num_object_clones': 0, 'num_object_copies': 15396, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7698, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11393, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1401}, {'shards': '2', 'objects': 3560}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:31:21.018 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:31:21.018 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'13430", 'reported_seq': 13681, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:17.299033+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:31:17.299033+0000', 'last_peered': '2024-09-17T15:31:17.299033+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:31:17.299033+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "34'3400", 'ondisk_log_start': "34'3400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32035299, 'num_objects': 7822, 'num_object_clones': 0, 'num_object_copies': 15644, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4215, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7822, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9929, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3608}, {'shards': '0,2', 'objects': 1466}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:21.018 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:31:21.018 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'13136", 'reported_seq': 13391, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:17.336423+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:31:17.336423+0000', 'last_peered': '2024-09-17T15:31:17.336423+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:31:17.336423+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "34'3100", 'ondisk_log_start': "34'3100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31261638, 'num_objects': 7634, 'num_object_clones': 0, 'num_object_copies': 15268, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4118, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7634, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 9905, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3516}, {'shards': '0,2', 'objects': 1425}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:21.018 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:31:21.019 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'12920", 'reported_seq': 13139, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:17.242857+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:31:17.242857+0000', 'last_peered': '2024-09-17T15:31:17.242857+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:31:17.242857+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "34'2900", 'ondisk_log_start': "34'2900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31219712, 'num_objects': 7622, 'num_object_clones': 0, 'num_object_copies': 15244, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3970, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7622, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9516, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 42, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3654}, {'shards': '0,2', 'objects': 1365}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:21.019 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:31:21.019 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'12916", 'reported_seq': 13091, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:17.546180+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:31:17.546180+0000', 'last_peered': '2024-09-17T15:31:17.546180+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:31:17.546180+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "34'2900", 'ondisk_log_start': "34'2900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31142371, 'num_objects': 7604, 'num_object_clones': 0, 'num_object_copies': 15208, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4053, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7604, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10811, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 25, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1423}, {'shards': '2', 'objects': 3552}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:31:21.019 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:31:21.019 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'13094", 'reported_seq': 14450, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:17.637215+0000', 'last_change': '2024-09-17T15:30:55.086473+0000', 'last_active': '2024-09-17T15:31:17.637215+0000', 'last_peered': '2024-09-17T15:31:17.637215+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:31:17.637215+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "34'3000", 'ondisk_log_start': "34'3000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31301632, 'num_objects': 7642, 'num_object_clones': 0, 'num_object_copies': 15284, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2113, 'num_objects_misplaced': 1410, 'num_objects_unfound': 0, 'num_objects_dirty': 7642, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10856, 'num_write_kb': 32532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 616, 'num_bytes_recovered': 2412544, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1410}, {'shards': '2', 'objects': 4119}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:31:21.019 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:31:21.019 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'13239", 'reported_seq': 13477, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:17.340236+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:31:17.340236+0000', 'last_peered': '2024-09-17T15:31:17.340236+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:31:17.340236+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "34'3200", 'ondisk_log_start': "34'3200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31649792, 'num_objects': 7727, 'num_object_clones': 0, 'num_object_copies': 15454, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4090, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7727, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9787, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 49, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3638}, {'shards': '0,2', 'objects': 1383}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:21.020 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:31:21.020 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'13141", 'reported_seq': 13362, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:17.280362+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:31:17.280362+0000', 'last_peered': '2024-09-17T15:31:17.280362+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:31:17.280362+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "34'3100", 'ondisk_log_start': "34'3100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31256576, 'num_objects': 7631, 'num_object_clones': 0, 'num_object_copies': 15262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7631, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9428, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 40, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3514}, {'shards': '0,2', 'objects': 1406}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:21.020 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:31:21.020 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'13123", 'reported_seq': 13369, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:17.347783+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:31:17.347783+0000', 'last_peered': '2024-09-17T15:31:17.347783+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:31:17.347783+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "34'3100", 'ondisk_log_start': "34'3100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31797248, 'num_objects': 7763, 'num_object_clones': 0, 'num_object_copies': 15526, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4010, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7763, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9910, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 55, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3755}, {'shards': '0,2', 'objects': 1385}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:21.020 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:31:21.700 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:31:21.700 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38844/246742 objects degraded (15.743%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":418,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":123371,"data_bytes":505279241,"bytes_used":2173747200,"bytes_avail":287736545280,"bytes_total":289910292480,"degraded_objects":38844,"degraded_total":246742,"degraded_ratio":0.15742759643676391,"misplaced_objects":1410,"misplaced_total":246742,"misplaced_ratio":0.0057144709858880935,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":80527,"recovering_keys_per_sec":0,"num_objects_recovered":287,"num_bytes_recovered":966656,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":370},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T15:27:19.616376+0000","services":{}},"progress_events":{}} 2024-09-17T15:31:21.720 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:31:21.721 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:31:22.310 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:31:22.311 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:31:22.331 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":247,"stamp":"2024-09-17T15:31:21.693190+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":505279241,"num_objects":123371,"num_object_clones":0,"num_object_copies":246742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38844,"num_objects_misplaced":1410,"num_objects_unfound":0,"num_objects_dirty":123371,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":169620,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9207,"num_bytes_recovered":35910617,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160686,"ondisk_log_size":160686,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2122800,"kb_used_data":1130936,"kb_used_omap":13,"kb_used_meta":991794,"kb_avail":280992720,"statfs":{"total":289910292480,"available":287736545280,"internally_reserved":0,"allocated":1158078464,"data_stored":1157172187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1015598079},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,2,7,7,6,0,0,6],"upper_bound":1024},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18206720,"num_objects":-4445,"num_object_clones":0,"num_object_copies":-8890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-50,"num_objects_misplaced":-237,"num_objects_unfound":0,"num_objects_dirty":-4445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4445,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":966656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003932"},"pg_stats":[{"pgid":"2.f","version":"45'12966","reported_seq":13156,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:17.259635+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:31:17.259635+0000","last_peered":"2024-09-17T15:31:17.259635+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:31:17.259635+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'2900","ondisk_log_start":"34'2900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31465472,"num_objects":7682,"num_object_clones":0,"num_object_copies":15364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4016,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12966,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3666},{"shards":"0,2","objects":1428}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13081","reported_seq":13188,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:17.603850+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:31:17.603850+0000","last_peered":"2024-09-17T15:31:17.603850+0000","last_clean":"2024-09-17T15:31:17.603850+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:31:17.603850+0000","last_undegraded":"2024-09-17T15:31:17.603850+0000","last_fullsized":"2024-09-17T15:31:17.603850+0000","mapping_epoch":33,"log_start":"34'3000","ondisk_log_start":"34'3000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32158662,"num_objects":7853,"num_object_clones":0,"num_object_copies":15706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7853,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11438,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13236","reported_seq":13427,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:17.609054+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:31:17.609054+0000","last_peered":"2024-09-17T15:31:17.609054+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:31:17.609054+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'3200","ondisk_log_start":"34'3200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31526936,"num_objects":7698,"num_object_clones":0,"num_object_copies":15396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7698,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11393,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1401},{"shards":"2","objects":3560}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13121","reported_seq":13195,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:17.582585+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:31:17.582585+0000","last_peered":"2024-09-17T15:31:17.582585+0000","last_clean":"2024-09-17T15:31:17.582585+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:31:17.582585+0000","last_undegraded":"2024-09-17T15:31:17.582585+0000","last_fullsized":"2024-09-17T15:31:17.582585+0000","mapping_epoch":33,"log_start":"34'3100","ondisk_log_start":"34'3100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32149504,"num_objects":7849,"num_object_clones":0,"num_object_copies":15698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7849,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13121,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12950","reported_seq":13084,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:17.253666+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:31:17.253666+0000","last_peered":"2024-09-17T15:31:17.253666+0000","last_clean":"2024-09-17T15:31:17.253666+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:31:17.253666+0000","last_undegraded":"2024-09-17T15:31:17.253666+0000","last_fullsized":"2024-09-17T15:31:17.253666+0000","mapping_epoch":32,"log_start":"34'2900","ondisk_log_start":"34'2900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31551971,"num_objects":7704,"num_object_clones":0,"num_object_copies":15408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7704,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9932,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13430","reported_seq":13681,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:17.299033+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:31:17.299033+0000","last_peered":"2024-09-17T15:31:17.299033+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:31:17.299033+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'3400","ondisk_log_start":"34'3400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32035299,"num_objects":7822,"num_object_clones":0,"num_object_copies":15644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4215,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7822,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9929,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3608},{"shards":"0,2","objects":1466}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13136","reported_seq":13391,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:17.336423+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:31:17.336423+0000","last_peered":"2024-09-17T15:31:17.336423+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:31:17.336423+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'3100","ondisk_log_start":"34'3100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31261638,"num_objects":7634,"num_object_clones":0,"num_object_copies":15268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4118,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7634,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9905,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3516},{"shards":"0,2","objects":1425}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13185","reported_seq":13317,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:17.300267+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:31:17.300267+0000","last_peered":"2024-09-17T15:31:17.300267+0000","last_clean":"2024-09-17T15:31:17.300267+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:31:17.300267+0000","last_undegraded":"2024-09-17T15:31:17.300267+0000","last_fullsized":"2024-09-17T15:31:17.300267+0000","mapping_epoch":32,"log_start":"34'3100","ondisk_log_start":"34'3100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31666176,"num_objects":7731,"num_object_clones":0,"num_object_copies":15462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10139,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.086967+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.086967+0000","last_peered":"2024-09-17T15:30:57.086967+0000","last_clean":"2024-09-17T15:30:57.086967+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.086967+0000","last_undegraded":"2024-09-17T15:30:57.086967+0000","last_fullsized":"2024-09-17T15:30:57.086967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13035","reported_seq":21322,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:16.678219+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:31:16.678219+0000","last_peered":"2024-09-17T15:31:16.678219+0000","last_clean":"2024-09-17T15:31:16.678219+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:31:16.678219+0000","last_undegraded":"2024-09-17T15:31:16.678219+0000","last_fullsized":"2024-09-17T15:31:16.678219+0000","mapping_epoch":42,"log_start":"34'3000","ondisk_log_start":"34'3000","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31875555,"num_objects":7783,"num_object_clones":0,"num_object_copies":15566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7783,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11035,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088505+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088505+0000","last_peered":"2024-09-17T15:30:57.088505+0000","last_clean":"2024-09-17T15:30:57.088505+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088505+0000","last_undegraded":"2024-09-17T15:30:57.088505+0000","last_fullsized":"2024-09-17T15:30:57.088505+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12920","reported_seq":13139,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:17.242857+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:31:17.242857+0000","last_peered":"2024-09-17T15:31:17.242857+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:31:17.242857+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'2900","ondisk_log_start":"34'2900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31219712,"num_objects":7622,"num_object_clones":0,"num_object_copies":15244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3970,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7622,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9516,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3654},{"shards":"0,2","objects":1365}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":107,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092541+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092541+0000","last_peered":"2024-09-17T15:30:57.092541+0000","last_clean":"2024-09-17T15:30:57.092541+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092541+0000","last_undegraded":"2024-09-17T15:30:57.092541+0000","last_fullsized":"2024-09-17T15:30:57.092541+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12916","reported_seq":13091,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:17.546180+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:31:17.546180+0000","last_peered":"2024-09-17T15:31:17.546180+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:31:17.546180+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'2900","ondisk_log_start":"34'2900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31142371,"num_objects":7604,"num_object_clones":0,"num_object_copies":15208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4053,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7604,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10811,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1423},{"shards":"2","objects":3552}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13094","reported_seq":14450,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:17.637215+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:31:17.637215+0000","last_peered":"2024-09-17T15:31:17.637215+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:31:17.637215+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'3000","ondisk_log_start":"34'3000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31301632,"num_objects":7642,"num_object_clones":0,"num_object_copies":15284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2113,"num_objects_misplaced":1410,"num_objects_unfound":0,"num_objects_dirty":7642,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10856,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":616,"num_bytes_recovered":2412544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1410},{"shards":"2","objects":4119}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13239","reported_seq":13477,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:17.340236+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:31:17.340236+0000","last_peered":"2024-09-17T15:31:17.340236+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:31:17.340236+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'3200","ondisk_log_start":"34'3200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31649792,"num_objects":7727,"num_object_clones":0,"num_object_copies":15454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4090,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9787,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3638},{"shards":"0,2","objects":1383}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13141","reported_seq":13362,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:17.280362+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:31:17.280362+0000","last_peered":"2024-09-17T15:31:17.280362+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:31:17.280362+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'3100","ondisk_log_start":"34'3100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31256576,"num_objects":7631,"num_object_clones":0,"num_object_copies":15262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9428,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3514},{"shards":"0,2","objects":1406}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088855+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088855+0000","last_peered":"2024-09-17T15:30:57.088855+0000","last_clean":"2024-09-17T15:30:57.088855+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088855+0000","last_undegraded":"2024-09-17T15:30:57.088855+0000","last_fullsized":"2024-09-17T15:30:57.088855+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13123","reported_seq":13369,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:17.347783+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:31:17.347783+0000","last_peered":"2024-09-17T15:31:17.347783+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:31:17.347783+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'3100","ondisk_log_start":"34'3100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31797248,"num_objects":7763,"num_object_clones":0,"num_object_copies":15526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4010,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9910,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3755},{"shards":"0,2","objects":1385}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13010","reported_seq":21389,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:16.559420+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:31:16.559420+0000","last_peered":"2024-09-17T15:31:16.559420+0000","last_clean":"2024-09-17T15:31:16.559420+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:31:16.559420+0000","last_undegraded":"2024-09-17T15:31:16.559420+0000","last_fullsized":"2024-09-17T15:31:16.559420+0000","mapping_epoch":44,"log_start":"34'3000","ondisk_log_start":"34'3000","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31220678,"num_objects":7624,"num_object_clones":0,"num_object_copies":15248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9452,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":505279222,"num_objects":123369,"num_object_clones":0,"num_object_copies":246738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38844,"num_objects_misplaced":1410,"num_objects_unfound":0,"num_objects_dirty":123369,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":169618,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9206,"num_bytes_recovered":35910598,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1155280896,"data_stored":1155135458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160683,"ondisk_log_size":160683,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051659,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":686564,"kb_used_data":379988,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93685276,"statfs":{"total":96636764160,"available":95933722624,"internally_reserved":0,"allocated":389107712,"data_stored":388816615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":313913004},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724604,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":615436,"kb_used_data":313020,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93756404,"statfs":{"total":96636764160,"available":96006557696,"internally_reserved":0,"allocated":320532480,"data_stored":320208402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":309653163},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247887,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":820800,"kb_used_data":437928,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93551040,"statfs":{"total":96636764160,"available":95796264960,"internally_reserved":0,"allocated":448438272,"data_stored":448147170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":392031912},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,1,2,0,3,0,0,6],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":446828544,"data_stored":446792414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":320102400,"data_stored":320033294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":388349952,"data_stored":388309750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:31:25.333 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:31:25.860 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:31:25.861 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:31:25.883 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":249,"stamp":"2024-09-17T15:31:25.694437+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":495997705,"num_objects":121105,"num_object_clones":0,"num_object_copies":242210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38836,"num_objects_misplaced":1281,"num_objects_unfound":0,"num_objects_dirty":121105,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":171886,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9344,"num_bytes_recovered":36398041,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161052,"ondisk_log_size":161052,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2110164,"kb_used_data":1118308,"kb_used_omap":13,"kb_used_meta":991794,"kb_avail":281005356,"statfs":{"total":289910292480,"available":287749484544,"internally_reserved":0,"allocated":1145147392,"data_stored":1144241115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1015598079},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,5,6,5,8,0,0,0,2,0,0,1],"upper_bound":16384},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18403328,"num_objects":-4493,"num_object_clones":0,"num_object_copies":-8986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-44,"num_objects_misplaced":-256,"num_objects_unfound":0,"num_objects_dirty":-4493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4493,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":1003520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004042"},"pg_stats":[{"pgid":"2.f","version":"45'13091","reported_seq":13287,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:22.322994+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:31:22.322994+0000","last_peered":"2024-09-17T15:31:22.322994+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:31:22.322994+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'3000","ondisk_log_start":"34'3000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30953472,"num_objects":7557,"num_object_clones":0,"num_object_copies":15114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7557,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13091,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3545},{"shards":"0,2","objects":1303}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13250","reported_seq":13357,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:22.628947+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:31:22.628947+0000","last_peered":"2024-09-17T15:31:22.628947+0000","last_clean":"2024-09-17T15:31:22.628947+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:31:22.628947+0000","last_undegraded":"2024-09-17T15:31:22.628947+0000","last_fullsized":"2024-09-17T15:31:22.628947+0000","mapping_epoch":33,"log_start":"34'3200","ondisk_log_start":"34'3200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31466438,"num_objects":7684,"num_object_clones":0,"num_object_copies":15368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7684,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11607,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13370","reported_seq":13560,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:22.559125+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:31:22.559125+0000","last_peered":"2024-09-17T15:31:22.559125+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:31:22.559125+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'3300","ondisk_log_start":"34'3300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30978072,"num_objects":7564,"num_object_clones":0,"num_object_copies":15128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7564,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11527,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1267},{"shards":"2","objects":3426}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13267","reported_seq":13341,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:22.556349+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:31:22.556349+0000","last_peered":"2024-09-17T15:31:22.556349+0000","last_clean":"2024-09-17T15:31:22.556349+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:31:22.556349+0000","last_undegraded":"2024-09-17T15:31:22.556349+0000","last_fullsized":"2024-09-17T15:31:22.556349+0000","mapping_epoch":33,"log_start":"34'3200","ondisk_log_start":"34'3200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31551488,"num_objects":7703,"num_object_clones":0,"num_object_copies":15406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13267,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13089","reported_seq":13223,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:22.294215+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:31:22.294215+0000","last_peered":"2024-09-17T15:31:22.294215+0000","last_clean":"2024-09-17T15:31:22.294215+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:31:22.294215+0000","last_undegraded":"2024-09-17T15:31:22.294215+0000","last_fullsized":"2024-09-17T15:31:22.294215+0000","mapping_epoch":32,"log_start":"34'3000","ondisk_log_start":"34'3000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30982627,"num_objects":7565,"num_object_clones":0,"num_object_copies":15130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7565,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10071,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13583","reported_seq":13833,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:22.365528+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:31:22.365528+0000","last_peered":"2024-09-17T15:31:22.365528+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:31:22.365528+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'3500","ondisk_log_start":"34'3500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31408611,"num_objects":7669,"num_object_clones":0,"num_object_copies":15338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4215,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7669,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10082,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3455},{"shards":"0,2","objects":1313}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13270","reported_seq":13531,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:22.287736+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:31:22.287736+0000","last_peered":"2024-09-17T15:31:22.287736+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:31:22.287736+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'3200","ondisk_log_start":"34'3200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30712774,"num_objects":7500,"num_object_clones":0,"num_object_copies":15000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4115,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7500,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10039,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3385},{"shards":"0,2","objects":1291}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13348","reported_seq":13480,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:22.270266+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:31:22.270266+0000","last_peered":"2024-09-17T15:31:22.270266+0000","last_clean":"2024-09-17T15:31:22.270266+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:31:22.270266+0000","last_undegraded":"2024-09-17T15:31:22.270266+0000","last_fullsized":"2024-09-17T15:31:22.270266+0000","mapping_epoch":32,"log_start":"34'3300","ondisk_log_start":"34'3300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30998528,"num_objects":7568,"num_object_clones":0,"num_object_copies":15136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10302,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.086967+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.086967+0000","last_peered":"2024-09-17T15:30:57.086967+0000","last_clean":"2024-09-17T15:30:57.086967+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.086967+0000","last_undegraded":"2024-09-17T15:30:57.086967+0000","last_fullsized":"2024-09-17T15:30:57.086967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13179","reported_seq":21466,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:21.658293+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:31:21.658293+0000","last_peered":"2024-09-17T15:31:21.658293+0000","last_clean":"2024-09-17T15:31:21.658293+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:31:21.658293+0000","last_undegraded":"2024-09-17T15:31:21.658293+0000","last_fullsized":"2024-09-17T15:31:21.658293+0000","mapping_epoch":42,"log_start":"34'3100","ondisk_log_start":"34'3100","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31285731,"num_objects":7639,"num_object_clones":0,"num_object_copies":15278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7639,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11179,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088505+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088505+0000","last_peered":"2024-09-17T15:30:57.088505+0000","last_clean":"2024-09-17T15:30:57.088505+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088505+0000","last_undegraded":"2024-09-17T15:30:57.088505+0000","last_fullsized":"2024-09-17T15:30:57.088505+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13066","reported_seq":13291,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:22.344532+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:31:22.344532+0000","last_peered":"2024-09-17T15:31:22.344532+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:31:22.344532+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'3000","ondisk_log_start":"34'3000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30621696,"num_objects":7476,"num_object_clones":0,"num_object_copies":14952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3967,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9662,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3511},{"shards":"0,2","objects":1219}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13058","reported_seq":13233,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:22.539215+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:31:22.539215+0000","last_peered":"2024-09-17T15:31:22.539215+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:31:22.539215+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'3000","ondisk_log_start":"34'3000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30560739,"num_objects":7462,"num_object_clones":0,"num_object_copies":14924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4053,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7462,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10953,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1281},{"shards":"2","objects":3410}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13227","reported_seq":14821,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:22.664436+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:31:22.664436+0000","last_peered":"2024-09-17T15:31:22.664436+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:31:22.664436+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'3200","ondisk_log_start":"34'3200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30756864,"num_objects":7509,"num_object_clones":0,"num_object_copies":15018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2123,"num_objects_misplaced":1281,"num_objects_unfound":0,"num_objects_dirty":7509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10989,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":735,"num_bytes_recovered":2899968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1281},{"shards":"2","objects":4105}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13366","reported_seq":13614,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:22.372140+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:31:22.372140+0000","last_peered":"2024-09-17T15:31:22.372140+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:31:22.372140+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'3300","ondisk_log_start":"34'3300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31129600,"num_objects":7600,"num_object_clones":0,"num_object_copies":15200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7600,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9914,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3517},{"shards":"0,2","objects":1256}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13283","reported_seq":13513,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:22.336267+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:31:22.336267+0000","last_peered":"2024-09-17T15:31:22.336267+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:31:22.336267+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'3200","ondisk_log_start":"34'3200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30674944,"num_objects":7489,"num_object_clones":0,"num_object_copies":14978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4118,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7489,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9570,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3374},{"shards":"0,2","objects":1264}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088855+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088855+0000","last_peered":"2024-09-17T15:30:57.088855+0000","last_clean":"2024-09-17T15:30:57.088855+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088855+0000","last_undegraded":"2024-09-17T15:30:57.088855+0000","last_fullsized":"2024-09-17T15:30:57.088855+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13254","reported_seq":13500,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:22.341606+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:31:22.341606+0000","last_peered":"2024-09-17T15:31:22.341606+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:31:22.341606+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'3200","ondisk_log_start":"34'3200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31260672,"num_objects":7632,"num_object_clones":0,"num_object_copies":15264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4010,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10041,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3624},{"shards":"0,2","objects":1254}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13148","reported_seq":21527,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:21.701605+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:31:21.701605+0000","last_peered":"2024-09-17T15:31:21.701605+0000","last_clean":"2024-09-17T15:31:21.701605+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:31:21.701605+0000","last_undegraded":"2024-09-17T15:31:21.701605+0000","last_fullsized":"2024-09-17T15:31:21.701605+0000","mapping_epoch":44,"log_start":"34'3100","ondisk_log_start":"34'3100","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30655430,"num_objects":7486,"num_object_clones":0,"num_object_copies":14972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7486,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9590,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":495997686,"num_objects":121103,"num_object_clones":0,"num_object_copies":242206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38836,"num_objects_misplaced":1281,"num_objects_unfound":0,"num_objects_dirty":121103,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":171884,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9343,"num_bytes_recovered":36398022,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1142267904,"data_stored":1142122466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161049,"ondisk_log_size":161049,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051660,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":686056,"kb_used_data":375320,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93685784,"statfs":{"total":96636764160,"available":95934242816,"internally_reserved":0,"allocated":384327680,"data_stored":384036583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":318172844},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,3,1,2,0,0,0,0,0,0,1],"upper_bound":16384},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724605,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":618520,"kb_used_data":311944,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93753320,"statfs":{"total":96636764160,"available":96003399680,"internally_reserved":0,"allocated":319430656,"data_stored":319106578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":313913003},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247888,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":805588,"kb_used_data":431044,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93566252,"statfs":{"total":96636764160,"available":95811842048,"internally_reserved":0,"allocated":441389056,"data_stored":441097954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":383512232},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,2,2,4,0,0,0,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":439775232,"data_stored":439739102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":318955520,"data_stored":318886414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":383537152,"data_stored":383496950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:31:25.884 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:31:26.449 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:31:26.450 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:31:26.474 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":249,"stamp":"2024-09-17T15:31:25.694437+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":495997705,"num_objects":121105,"num_object_clones":0,"num_object_copies":242210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38836,"num_objects_misplaced":1281,"num_objects_unfound":0,"num_objects_dirty":121105,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":171886,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9344,"num_bytes_recovered":36398041,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161052,"ondisk_log_size":161052,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2110164,"kb_used_data":1118308,"kb_used_omap":13,"kb_used_meta":991794,"kb_avail":281005356,"statfs":{"total":289910292480,"available":287749484544,"internally_reserved":0,"allocated":1145147392,"data_stored":1144241115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1015598079},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,5,6,5,8,0,0,0,2,0,0,1],"upper_bound":16384},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18403328,"num_objects":-4493,"num_object_clones":0,"num_object_copies":-8986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-44,"num_objects_misplaced":-256,"num_objects_unfound":0,"num_objects_dirty":-4493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4493,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":1003520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004042"},"pg_stats":[{"pgid":"2.f","version":"45'13091","reported_seq":13287,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:22.322994+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:31:22.322994+0000","last_peered":"2024-09-17T15:31:22.322994+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:31:22.322994+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'3000","ondisk_log_start":"34'3000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30953472,"num_objects":7557,"num_object_clones":0,"num_object_copies":15114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7557,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13091,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3545},{"shards":"0,2","objects":1303}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13250","reported_seq":13357,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:22.628947+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:31:22.628947+0000","last_peered":"2024-09-17T15:31:22.628947+0000","last_clean":"2024-09-17T15:31:22.628947+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:31:22.628947+0000","last_undegraded":"2024-09-17T15:31:22.628947+0000","last_fullsized":"2024-09-17T15:31:22.628947+0000","mapping_epoch":33,"log_start":"34'3200","ondisk_log_start":"34'3200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31466438,"num_objects":7684,"num_object_clones":0,"num_object_copies":15368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7684,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11607,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13370","reported_seq":13560,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:22.559125+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:31:22.559125+0000","last_peered":"2024-09-17T15:31:22.559125+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:31:22.559125+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'3300","ondisk_log_start":"34'3300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30978072,"num_objects":7564,"num_object_clones":0,"num_object_copies":15128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7564,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11527,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1267},{"shards":"2","objects":3426}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13267","reported_seq":13341,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:22.556349+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:31:22.556349+0000","last_peered":"2024-09-17T15:31:22.556349+0000","last_clean":"2024-09-17T15:31:22.556349+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:31:22.556349+0000","last_undegraded":"2024-09-17T15:31:22.556349+0000","last_fullsized":"2024-09-17T15:31:22.556349+0000","mapping_epoch":33,"log_start":"34'3200","ondisk_log_start":"34'3200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31551488,"num_objects":7703,"num_object_clones":0,"num_object_copies":15406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13267,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13089","reported_seq":13223,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:22.294215+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:31:22.294215+0000","last_peered":"2024-09-17T15:31:22.294215+0000","last_clean":"2024-09-17T15:31:22.294215+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:31:22.294215+0000","last_undegraded":"2024-09-17T15:31:22.294215+0000","last_fullsized":"2024-09-17T15:31:22.294215+0000","mapping_epoch":32,"log_start":"34'3000","ondisk_log_start":"34'3000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30982627,"num_objects":7565,"num_object_clones":0,"num_object_copies":15130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7565,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10071,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13583","reported_seq":13833,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:22.365528+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:31:22.365528+0000","last_peered":"2024-09-17T15:31:22.365528+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:31:22.365528+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'3500","ondisk_log_start":"34'3500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31408611,"num_objects":7669,"num_object_clones":0,"num_object_copies":15338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4215,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7669,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10082,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3455},{"shards":"0,2","objects":1313}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13270","reported_seq":13531,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:22.287736+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:31:22.287736+0000","last_peered":"2024-09-17T15:31:22.287736+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:31:22.287736+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'3200","ondisk_log_start":"34'3200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30712774,"num_objects":7500,"num_object_clones":0,"num_object_copies":15000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4115,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7500,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10039,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3385},{"shards":"0,2","objects":1291}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13348","reported_seq":13480,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:22.270266+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:31:22.270266+0000","last_peered":"2024-09-17T15:31:22.270266+0000","last_clean":"2024-09-17T15:31:22.270266+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:31:22.270266+0000","last_undegraded":"2024-09-17T15:31:22.270266+0000","last_fullsized":"2024-09-17T15:31:22.270266+0000","mapping_epoch":32,"log_start":"34'3300","ondisk_log_start":"34'3300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30998528,"num_objects":7568,"num_object_clones":0,"num_object_copies":15136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10302,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.086967+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.086967+0000","last_peered":"2024-09-17T15:30:57.086967+0000","last_clean":"2024-09-17T15:30:57.086967+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.086967+0000","last_undegraded":"2024-09-17T15:30:57.086967+0000","last_fullsized":"2024-09-17T15:30:57.086967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13179","reported_seq":21466,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:21.658293+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:31:21.658293+0000","last_peered":"2024-09-17T15:31:21.658293+0000","last_clean":"2024-09-17T15:31:21.658293+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:31:21.658293+0000","last_undegraded":"2024-09-17T15:31:21.658293+0000","last_fullsized":"2024-09-17T15:31:21.658293+0000","mapping_epoch":42,"log_start":"34'3100","ondisk_log_start":"34'3100","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31285731,"num_objects":7639,"num_object_clones":0,"num_object_copies":15278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7639,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11179,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088505+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088505+0000","last_peered":"2024-09-17T15:30:57.088505+0000","last_clean":"2024-09-17T15:30:57.088505+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088505+0000","last_undegraded":"2024-09-17T15:30:57.088505+0000","last_fullsized":"2024-09-17T15:30:57.088505+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13066","reported_seq":13291,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:22.344532+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:31:22.344532+0000","last_peered":"2024-09-17T15:31:22.344532+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:31:22.344532+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'3000","ondisk_log_start":"34'3000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30621696,"num_objects":7476,"num_object_clones":0,"num_object_copies":14952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3967,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9662,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3511},{"shards":"0,2","objects":1219}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13058","reported_seq":13233,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:22.539215+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:31:22.539215+0000","last_peered":"2024-09-17T15:31:22.539215+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:31:22.539215+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'3000","ondisk_log_start":"34'3000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30560739,"num_objects":7462,"num_object_clones":0,"num_object_copies":14924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4053,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7462,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10953,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1281},{"shards":"2","objects":3410}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13227","reported_seq":14821,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:22.664436+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:31:22.664436+0000","last_peered":"2024-09-17T15:31:22.664436+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:31:22.664436+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'3200","ondisk_log_start":"34'3200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30756864,"num_objects":7509,"num_object_clones":0,"num_object_copies":15018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2123,"num_objects_misplaced":1281,"num_objects_unfound":0,"num_objects_dirty":7509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10989,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":735,"num_bytes_recovered":2899968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1281},{"shards":"2","objects":4105}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13366","reported_seq":13614,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:22.372140+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:31:22.372140+0000","last_peered":"2024-09-17T15:31:22.372140+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:31:22.372140+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'3300","ondisk_log_start":"34'3300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31129600,"num_objects":7600,"num_object_clones":0,"num_object_copies":15200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7600,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9914,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3517},{"shards":"0,2","objects":1256}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13283","reported_seq":13513,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:22.336267+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:31:22.336267+0000","last_peered":"2024-09-17T15:31:22.336267+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:31:22.336267+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'3200","ondisk_log_start":"34'3200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30674944,"num_objects":7489,"num_object_clones":0,"num_object_copies":14978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4118,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7489,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9570,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3374},{"shards":"0,2","objects":1264}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088855+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088855+0000","last_peered":"2024-09-17T15:30:57.088855+0000","last_clean":"2024-09-17T15:30:57.088855+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088855+0000","last_undegraded":"2024-09-17T15:30:57.088855+0000","last_fullsized":"2024-09-17T15:30:57.088855+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13254","reported_seq":13500,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:22.341606+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:31:22.341606+0000","last_peered":"2024-09-17T15:31:22.341606+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:31:22.341606+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'3200","ondisk_log_start":"34'3200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31260672,"num_objects":7632,"num_object_clones":0,"num_object_copies":15264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4010,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10041,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3624},{"shards":"0,2","objects":1254}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13148","reported_seq":21527,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:21.701605+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:31:21.701605+0000","last_peered":"2024-09-17T15:31:21.701605+0000","last_clean":"2024-09-17T15:31:21.701605+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:31:21.701605+0000","last_undegraded":"2024-09-17T15:31:21.701605+0000","last_fullsized":"2024-09-17T15:31:21.701605+0000","mapping_epoch":44,"log_start":"34'3100","ondisk_log_start":"34'3100","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30655430,"num_objects":7486,"num_object_clones":0,"num_object_copies":14972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7486,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9590,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":495997686,"num_objects":121103,"num_object_clones":0,"num_object_copies":242206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38836,"num_objects_misplaced":1281,"num_objects_unfound":0,"num_objects_dirty":121103,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":171884,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9343,"num_bytes_recovered":36398022,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1142267904,"data_stored":1142122466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161049,"ondisk_log_size":161049,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051660,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":686056,"kb_used_data":375320,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93685784,"statfs":{"total":96636764160,"available":95934242816,"internally_reserved":0,"allocated":384327680,"data_stored":384036583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":318172844},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,3,1,2,0,0,0,0,0,0,1],"upper_bound":16384},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724605,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":618520,"kb_used_data":311944,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93753320,"statfs":{"total":96636764160,"available":96003399680,"internally_reserved":0,"allocated":319430656,"data_stored":319106578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":313913003},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247888,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":805588,"kb_used_data":431044,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93566252,"statfs":{"total":96636764160,"available":95811842048,"internally_reserved":0,"allocated":441389056,"data_stored":441097954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":383512232},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,2,2,4,0,0,0,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":439775232,"data_stored":439739102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":318955520,"data_stored":318886414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":383537152,"data_stored":383496950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:31:26.476 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:31:26.476 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'13091", 'reported_seq': 13287, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:22.322994+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:31:22.322994+0000', 'last_peered': '2024-09-17T15:31:22.322994+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:31:22.322994+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "34'3000", 'ondisk_log_start': "34'3000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30953472, 'num_objects': 7557, 'num_object_clones': 0, 'num_object_copies': 15114, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4012, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7557, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13091, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3545}, {'shards': '0,2', 'objects': 1303}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:26.476 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:31:26.476 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'13370", 'reported_seq': 13560, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:22.559125+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:31:22.559125+0000', 'last_peered': '2024-09-17T15:31:22.559125+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:31:22.559125+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "34'3300", 'ondisk_log_start': "34'3300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30978072, 'num_objects': 7564, 'num_object_clones': 0, 'num_object_copies': 15128, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7564, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11527, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1267}, {'shards': '2', 'objects': 3426}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:31:26.476 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:31:26.476 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'13583", 'reported_seq': 13833, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:22.365528+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:31:22.365528+0000', 'last_peered': '2024-09-17T15:31:22.365528+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:31:22.365528+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "34'3500", 'ondisk_log_start': "34'3500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31408611, 'num_objects': 7669, 'num_object_clones': 0, 'num_object_copies': 15338, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4215, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7669, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10082, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3455}, {'shards': '0,2', 'objects': 1313}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:26.477 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:31:26.477 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'13270", 'reported_seq': 13531, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:22.287736+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:31:22.287736+0000', 'last_peered': '2024-09-17T15:31:22.287736+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:31:22.287736+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "34'3200", 'ondisk_log_start': "34'3200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30712774, 'num_objects': 7500, 'num_object_clones': 0, 'num_object_copies': 15000, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4115, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7500, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 10039, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 59, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3385}, {'shards': '0,2', 'objects': 1291}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:26.477 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:31:26.477 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'13066", 'reported_seq': 13291, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:22.344532+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:31:22.344532+0000', 'last_peered': '2024-09-17T15:31:22.344532+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:31:22.344532+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "34'3000", 'ondisk_log_start': "34'3000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30621696, 'num_objects': 7476, 'num_object_clones': 0, 'num_object_copies': 14952, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3967, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7476, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9662, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 45, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3511}, {'shards': '0,2', 'objects': 1219}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:26.477 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:31:26.477 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'13058", 'reported_seq': 13233, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:22.539215+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:31:22.539215+0000', 'last_peered': '2024-09-17T15:31:22.539215+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:31:22.539215+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "34'3000", 'ondisk_log_start': "34'3000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30560739, 'num_objects': 7462, 'num_object_clones': 0, 'num_object_copies': 14924, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4053, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7462, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10953, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 25, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1281}, {'shards': '2', 'objects': 3410}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:31:26.477 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:31:26.477 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'13227", 'reported_seq': 14821, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:22.664436+0000', 'last_change': '2024-09-17T15:30:55.086473+0000', 'last_active': '2024-09-17T15:31:22.664436+0000', 'last_peered': '2024-09-17T15:31:22.664436+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:31:22.664436+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "34'3200", 'ondisk_log_start': "34'3200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30756864, 'num_objects': 7509, 'num_object_clones': 0, 'num_object_copies': 15018, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2123, 'num_objects_misplaced': 1281, 'num_objects_unfound': 0, 'num_objects_dirty': 7509, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10989, 'num_write_kb': 32532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 735, 'num_bytes_recovered': 2899968, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1281}, {'shards': '2', 'objects': 4105}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:31:26.477 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:31:26.477 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'13366", 'reported_seq': 13614, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:22.372140+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:31:22.372140+0000', 'last_peered': '2024-09-17T15:31:22.372140+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:31:22.372140+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "34'3300", 'ondisk_log_start': "34'3300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31129600, 'num_objects': 7600, 'num_object_clones': 0, 'num_object_copies': 15200, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4084, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7600, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9914, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 55, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3517}, {'shards': '0,2', 'objects': 1256}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:26.478 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:31:26.478 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'13283", 'reported_seq': 13513, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:22.336267+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:31:22.336267+0000', 'last_peered': '2024-09-17T15:31:22.336267+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:31:22.336267+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "34'3200", 'ondisk_log_start': "34'3200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30674944, 'num_objects': 7489, 'num_object_clones': 0, 'num_object_copies': 14978, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4118, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7489, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9570, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 42, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3374}, {'shards': '0,2', 'objects': 1264}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:26.478 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:31:26.478 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'13254", 'reported_seq': 13500, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:22.341606+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:31:22.341606+0000', 'last_peered': '2024-09-17T15:31:22.341606+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:31:22.341606+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "34'3200", 'ondisk_log_start': "34'3200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31260672, 'num_objects': 7632, 'num_object_clones': 0, 'num_object_copies': 15264, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4010, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7632, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10041, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 55, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3624}, {'shards': '0,2', 'objects': 1254}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:26.478 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:31:27.165 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:31:27.165 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38836/242210 objects degraded (16.034%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":424,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":121105,"data_bytes":495997705,"bytes_used":2160807936,"bytes_avail":287749484544,"bytes_total":289910292480,"degraded_objects":38836,"degraded_total":242210,"degraded_ratio":0.16034020065232649,"misplaced_objects":1281,"misplaced_total":242210,"misplaced_ratio":0.0052887989760951239,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":83598,"recovering_keys_per_sec":0,"num_objects_recovered":300,"num_bytes_recovered":1003520,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":374},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T15:27:19.616376+0000","services":{}},"progress_events":{}} 2024-09-17T15:31:27.183 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:31:27.184 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:31:27.726 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:31:27.726 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:31:27.757 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":250,"stamp":"2024-09-17T15:31:27.695196+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":486781705,"num_objects":118855,"num_object_clones":0,"num_object_copies":237710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38836,"num_objects_misplaced":1128,"num_objects_unfound":0,"num_objects_dirty":118855,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":174136,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9497,"num_bytes_recovered":36951001,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160802,"ondisk_log_size":160802,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2093740,"kb_used_data":1106040,"kb_used_omap":13,"kb_used_meta":987634,"kb_avail":281021780,"statfs":{"total":289910292480,"available":287766302720,"internally_reserved":0,"allocated":1132584960,"data_stored":1131678683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1011338239},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,3,1,6,10,4,0,0,0,0,2,0,1],"upper_bound":16384},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27619328,"num_objects":-6743,"num_object_clones":0,"num_object_copies":-13486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-44,"num_objects_misplaced":-409,"num_objects_unfound":0,"num_objects_dirty":-6743,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6743,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":453,"num_bytes_recovered":1556480,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004186"},"pg_stats":[{"pgid":"2.f","version":"45'13215","reported_seq":13417,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:27.361641+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:31:27.361641+0000","last_peered":"2024-09-17T15:31:27.361641+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:31:27.361641+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'3200","ondisk_log_start":"34'3200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30445568,"num_objects":7433,"num_object_clones":0,"num_object_copies":14866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4010,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13215,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3423},{"shards":"0,2","objects":1179}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13394","reported_seq":13501,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:27.626631+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:31:27.626631+0000","last_peered":"2024-09-17T15:31:27.626631+0000","last_clean":"2024-09-17T15:31:27.626631+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:31:27.626631+0000","last_undegraded":"2024-09-17T15:31:27.626631+0000","last_fullsized":"2024-09-17T15:31:27.626631+0000","mapping_epoch":33,"log_start":"34'3300","ondisk_log_start":"34'3300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30876614,"num_objects":7540,"num_object_clones":0,"num_object_copies":15080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7540,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11751,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13497","reported_seq":13690,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:27.579155+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:31:27.579155+0000","last_peered":"2024-09-17T15:31:27.579155+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:31:27.579155+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'3400","ondisk_log_start":"34'3400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30457880,"num_objects":7437,"num_object_clones":0,"num_object_copies":14874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4138,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7437,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11654,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1140},{"shards":"2","objects":3300}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13426","reported_seq":13500,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:27.578458+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:31:27.578458+0000","last_peered":"2024-09-17T15:31:27.578458+0000","last_clean":"2024-09-17T15:31:27.578458+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:31:27.578458+0000","last_undegraded":"2024-09-17T15:31:27.578458+0000","last_fullsized":"2024-09-17T15:31:27.578458+0000","mapping_epoch":33,"log_start":"34'3400","ondisk_log_start":"34'3400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30900224,"num_objects":7544,"num_object_clones":0,"num_object_copies":15088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13426,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13237","reported_seq":13371,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:27.335825+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:31:27.335825+0000","last_peered":"2024-09-17T15:31:27.335825+0000","last_clean":"2024-09-17T15:31:27.335825+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:31:27.335825+0000","last_undegraded":"2024-09-17T15:31:27.335825+0000","last_fullsized":"2024-09-17T15:31:27.335825+0000","mapping_epoch":32,"log_start":"34'3200","ondisk_log_start":"34'3200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30376419,"num_objects":7417,"num_object_clones":0,"num_object_copies":14834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7417,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10219,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13734","reported_seq":13991,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:27.322767+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:31:27.322767+0000","last_peered":"2024-09-17T15:31:27.322767+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:31:27.322767+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'3700","ondisk_log_start":"34'3700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30790115,"num_objects":7518,"num_object_clones":0,"num_object_copies":15036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4212,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7518,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10233,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3307},{"shards":"0,2","objects":1162}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13415","reported_seq":13678,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:27.346510+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:31:27.346510+0000","last_peered":"2024-09-17T15:31:27.346510+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:31:27.346510+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'3400","ondisk_log_start":"34'3400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30118854,"num_objects":7355,"num_object_clones":0,"num_object_copies":14710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7355,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10184,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3241},{"shards":"0,2","objects":1146}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13464","reported_seq":13596,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:27.358436+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:31:27.358436+0000","last_peered":"2024-09-17T15:31:27.358436+0000","last_clean":"2024-09-17T15:31:27.358436+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:31:27.358436+0000","last_undegraded":"2024-09-17T15:31:27.358436+0000","last_fullsized":"2024-09-17T15:31:27.358436+0000","mapping_epoch":32,"log_start":"34'3400","ondisk_log_start":"34'3400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30523392,"num_objects":7452,"num_object_clones":0,"num_object_copies":14904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10418,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.086967+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.086967+0000","last_peered":"2024-09-17T15:30:57.086967+0000","last_clean":"2024-09-17T15:30:57.086967+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.086967+0000","last_undegraded":"2024-09-17T15:30:57.086967+0000","last_fullsized":"2024-09-17T15:30:57.086967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13328","reported_seq":21614,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:26.748821+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:31:26.748821+0000","last_peered":"2024-09-17T15:31:26.748821+0000","last_clean":"2024-09-17T15:31:26.748821+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:31:26.748821+0000","last_undegraded":"2024-09-17T15:31:26.748821+0000","last_fullsized":"2024-09-17T15:31:26.748821+0000","mapping_epoch":42,"log_start":"34'3300","ondisk_log_start":"34'3300","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30675427,"num_objects":7490,"num_object_clones":0,"num_object_copies":14980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7490,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11328,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088505+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088505+0000","last_peered":"2024-09-17T15:30:57.088505+0000","last_clean":"2024-09-17T15:30:57.088505+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088505+0000","last_undegraded":"2024-09-17T15:30:57.088505+0000","last_fullsized":"2024-09-17T15:30:57.088505+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13190","reported_seq":13414,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:27.430229+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:31:27.430229+0000","last_peered":"2024-09-17T15:31:27.430229+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:31:27.430229+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'3100","ondisk_log_start":"34'3100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30113792,"num_objects":7352,"num_object_clones":0,"num_object_copies":14704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3967,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9786,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3387},{"shards":"0,2","objects":1095}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13200","reported_seq":13377,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:27.466083+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:31:27.466083+0000","last_peered":"2024-09-17T15:31:27.466083+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:31:27.466083+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'3100","ondisk_log_start":"34'3100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29979107,"num_objects":7320,"num_object_clones":0,"num_object_copies":14640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4052,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7320,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11095,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1139},{"shards":"2","objects":3269}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13377","reported_seq":15242,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:27.654346+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:31:27.654346+0000","last_peered":"2024-09-17T15:31:27.654346+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:31:27.654346+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'3300","ondisk_log_start":"34'3300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30142464,"num_objects":7359,"num_object_clones":0,"num_object_copies":14718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2141,"num_objects_misplaced":1128,"num_objects_unfound":0,"num_objects_dirty":7359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11139,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":870,"num_bytes_recovered":3452928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1128},{"shards":"2","objects":4090}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13521","reported_seq":13781,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:27.411649+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:31:27.411649+0000","last_peered":"2024-09-17T15:31:27.411649+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:31:27.411649+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'3500","ondisk_log_start":"34'3500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30494720,"num_objects":7445,"num_object_clones":0,"num_object_copies":14890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4079,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10069,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3367},{"shards":"0,2","objects":1101}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13419","reported_seq":13656,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:27.387205+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:31:27.387205+0000","last_peered":"2024-09-17T15:31:27.387205+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:31:27.387205+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'3400","ondisk_log_start":"34'3400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30117888,"num_objects":7353,"num_object_clones":0,"num_object_copies":14706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7353,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9706,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3242},{"shards":"0,2","objects":1128}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088855+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088855+0000","last_peered":"2024-09-17T15:30:57.088855+0000","last_clean":"2024-09-17T15:30:57.088855+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088855+0000","last_undegraded":"2024-09-17T15:30:57.088855+0000","last_fullsized":"2024-09-17T15:30:57.088855+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13403","reported_seq":13651,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:27.404280+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:31:27.404280+0000","last_peered":"2024-09-17T15:31:27.404280+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:31:27.404280+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'3400","ondisk_log_start":"34'3400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30650368,"num_objects":7483,"num_object_clones":0,"num_object_copies":14966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4009,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10190,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3476},{"shards":"0,2","objects":1105}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13279","reported_seq":21658,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:26.740938+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:31:26.740938+0000","last_peered":"2024-09-17T15:31:26.740938+0000","last_clean":"2024-09-17T15:31:26.740938+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:31:26.740938+0000","last_undegraded":"2024-09-17T15:31:26.740938+0000","last_fullsized":"2024-09-17T15:31:26.740938+0000","mapping_epoch":44,"log_start":"34'3200","ondisk_log_start":"34'3200","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30118854,"num_objects":7355,"num_object_clones":0,"num_object_copies":14710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7355,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9721,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":486781686,"num_objects":118853,"num_object_clones":0,"num_object_copies":237706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38836,"num_objects_misplaced":1128,"num_objects_unfound":0,"num_objects_dirty":118853,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":174134,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9496,"num_bytes_recovered":36950982,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1129668608,"data_stored":1129523170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160799,"ondisk_log_size":160799,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051661,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":669076,"kb_used_data":370816,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93702764,"statfs":{"total":96636764160,"available":95951630336,"internally_reserved":0,"allocated":379715584,"data_stored":379424487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":305393324},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,3,0,0,0,0,0,0,0,1],"upper_bound":16384},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724606,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":621592,"kb_used_data":310856,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93750248,"statfs":{"total":96636764160,"available":96000253952,"internally_reserved":0,"allocated":318316544,"data_stored":317992466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":318172843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247889,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":803072,"kb_used_data":424368,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93568768,"statfs":{"total":96636764160,"available":95814418432,"internally_reserved":0,"allocated":434552832,"data_stored":434261730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":387772072},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,3,7,4,0,0,0,0,2],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":432910336,"data_stored":432874206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":317812736,"data_stored":317743630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":378945536,"data_stored":378905334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:31:30.759 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:31:31.281 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:31:31.282 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:31:31.304 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":251,"stamp":"2024-09-17T15:31:29.695901+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":486781705,"num_objects":118855,"num_object_clones":0,"num_object_copies":237710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38836,"num_objects_misplaced":1128,"num_objects_unfound":0,"num_objects_dirty":118855,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":174136,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9497,"num_bytes_recovered":36951001,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160802,"ondisk_log_size":160802,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2093740,"kb_used_data":1106040,"kb_used_omap":13,"kb_used_meta":987634,"kb_avail":281021780,"statfs":{"total":289910292480,"available":287766302720,"internally_reserved":0,"allocated":1132584960,"data_stored":1131678683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1011338239},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,3,1,6,10,4,0,0,0,0,2,0,1],"upper_bound":16384},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18497536,"num_objects":-4516,"num_object_clones":0,"num_object_copies":-9032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-8,"num_objects_misplaced":-282,"num_objects_unfound":0,"num_objects_dirty":-4516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4516,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":1040384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004154"},"pg_stats":[{"pgid":"2.f","version":"45'13215","reported_seq":13417,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:27.361641+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:31:27.361641+0000","last_peered":"2024-09-17T15:31:27.361641+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:31:27.361641+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'3200","ondisk_log_start":"34'3200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30445568,"num_objects":7433,"num_object_clones":0,"num_object_copies":14866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4010,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13215,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3423},{"shards":"0,2","objects":1179}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13394","reported_seq":13501,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:27.626631+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:31:27.626631+0000","last_peered":"2024-09-17T15:31:27.626631+0000","last_clean":"2024-09-17T15:31:27.626631+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:31:27.626631+0000","last_undegraded":"2024-09-17T15:31:27.626631+0000","last_fullsized":"2024-09-17T15:31:27.626631+0000","mapping_epoch":33,"log_start":"34'3300","ondisk_log_start":"34'3300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30876614,"num_objects":7540,"num_object_clones":0,"num_object_copies":15080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7540,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11751,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13497","reported_seq":13690,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:27.579155+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:31:27.579155+0000","last_peered":"2024-09-17T15:31:27.579155+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:31:27.579155+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'3400","ondisk_log_start":"34'3400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30457880,"num_objects":7437,"num_object_clones":0,"num_object_copies":14874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4138,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7437,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11654,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1140},{"shards":"2","objects":3300}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13426","reported_seq":13500,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:27.578458+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:31:27.578458+0000","last_peered":"2024-09-17T15:31:27.578458+0000","last_clean":"2024-09-17T15:31:27.578458+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:31:27.578458+0000","last_undegraded":"2024-09-17T15:31:27.578458+0000","last_fullsized":"2024-09-17T15:31:27.578458+0000","mapping_epoch":33,"log_start":"34'3400","ondisk_log_start":"34'3400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30900224,"num_objects":7544,"num_object_clones":0,"num_object_copies":15088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13426,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13237","reported_seq":13371,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:27.335825+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:31:27.335825+0000","last_peered":"2024-09-17T15:31:27.335825+0000","last_clean":"2024-09-17T15:31:27.335825+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:31:27.335825+0000","last_undegraded":"2024-09-17T15:31:27.335825+0000","last_fullsized":"2024-09-17T15:31:27.335825+0000","mapping_epoch":32,"log_start":"34'3200","ondisk_log_start":"34'3200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30376419,"num_objects":7417,"num_object_clones":0,"num_object_copies":14834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7417,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10219,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13734","reported_seq":13991,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:27.322767+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:31:27.322767+0000","last_peered":"2024-09-17T15:31:27.322767+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:31:27.322767+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'3700","ondisk_log_start":"34'3700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30790115,"num_objects":7518,"num_object_clones":0,"num_object_copies":15036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4212,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7518,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10233,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3307},{"shards":"0,2","objects":1162}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13415","reported_seq":13678,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:27.346510+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:31:27.346510+0000","last_peered":"2024-09-17T15:31:27.346510+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:31:27.346510+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'3400","ondisk_log_start":"34'3400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30118854,"num_objects":7355,"num_object_clones":0,"num_object_copies":14710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7355,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10184,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3241},{"shards":"0,2","objects":1146}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13464","reported_seq":13596,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:27.358436+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:31:27.358436+0000","last_peered":"2024-09-17T15:31:27.358436+0000","last_clean":"2024-09-17T15:31:27.358436+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:31:27.358436+0000","last_undegraded":"2024-09-17T15:31:27.358436+0000","last_fullsized":"2024-09-17T15:31:27.358436+0000","mapping_epoch":32,"log_start":"34'3400","ondisk_log_start":"34'3400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30523392,"num_objects":7452,"num_object_clones":0,"num_object_copies":14904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10418,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.086967+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.086967+0000","last_peered":"2024-09-17T15:30:57.086967+0000","last_clean":"2024-09-17T15:30:57.086967+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.086967+0000","last_undegraded":"2024-09-17T15:30:57.086967+0000","last_fullsized":"2024-09-17T15:30:57.086967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13328","reported_seq":21614,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:26.748821+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:31:26.748821+0000","last_peered":"2024-09-17T15:31:26.748821+0000","last_clean":"2024-09-17T15:31:26.748821+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:31:26.748821+0000","last_undegraded":"2024-09-17T15:31:26.748821+0000","last_fullsized":"2024-09-17T15:31:26.748821+0000","mapping_epoch":42,"log_start":"34'3300","ondisk_log_start":"34'3300","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30675427,"num_objects":7490,"num_object_clones":0,"num_object_copies":14980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7490,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11328,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088505+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088505+0000","last_peered":"2024-09-17T15:30:57.088505+0000","last_clean":"2024-09-17T15:30:57.088505+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088505+0000","last_undegraded":"2024-09-17T15:30:57.088505+0000","last_fullsized":"2024-09-17T15:30:57.088505+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13190","reported_seq":13414,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:27.430229+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:31:27.430229+0000","last_peered":"2024-09-17T15:31:27.430229+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:31:27.430229+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'3100","ondisk_log_start":"34'3100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30113792,"num_objects":7352,"num_object_clones":0,"num_object_copies":14704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3967,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9786,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3387},{"shards":"0,2","objects":1095}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13200","reported_seq":13377,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:27.466083+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:31:27.466083+0000","last_peered":"2024-09-17T15:31:27.466083+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:31:27.466083+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'3100","ondisk_log_start":"34'3100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29979107,"num_objects":7320,"num_object_clones":0,"num_object_copies":14640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4052,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7320,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11095,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1139},{"shards":"2","objects":3269}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13377","reported_seq":15242,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:27.654346+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:31:27.654346+0000","last_peered":"2024-09-17T15:31:27.654346+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:31:27.654346+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'3300","ondisk_log_start":"34'3300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30142464,"num_objects":7359,"num_object_clones":0,"num_object_copies":14718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2141,"num_objects_misplaced":1128,"num_objects_unfound":0,"num_objects_dirty":7359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11139,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":870,"num_bytes_recovered":3452928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1128},{"shards":"2","objects":4090}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13521","reported_seq":13781,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:27.411649+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:31:27.411649+0000","last_peered":"2024-09-17T15:31:27.411649+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:31:27.411649+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'3500","ondisk_log_start":"34'3500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30494720,"num_objects":7445,"num_object_clones":0,"num_object_copies":14890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4079,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10069,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3367},{"shards":"0,2","objects":1101}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13419","reported_seq":13656,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:27.387205+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:31:27.387205+0000","last_peered":"2024-09-17T15:31:27.387205+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:31:27.387205+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'3400","ondisk_log_start":"34'3400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30117888,"num_objects":7353,"num_object_clones":0,"num_object_copies":14706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7353,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9706,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3242},{"shards":"0,2","objects":1128}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088855+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088855+0000","last_peered":"2024-09-17T15:30:57.088855+0000","last_clean":"2024-09-17T15:30:57.088855+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088855+0000","last_undegraded":"2024-09-17T15:30:57.088855+0000","last_fullsized":"2024-09-17T15:30:57.088855+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13403","reported_seq":13651,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:27.404280+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:31:27.404280+0000","last_peered":"2024-09-17T15:31:27.404280+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:31:27.404280+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'3400","ondisk_log_start":"34'3400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30650368,"num_objects":7483,"num_object_clones":0,"num_object_copies":14966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4009,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10190,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3476},{"shards":"0,2","objects":1105}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13279","reported_seq":21658,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:26.740938+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:31:26.740938+0000","last_peered":"2024-09-17T15:31:26.740938+0000","last_clean":"2024-09-17T15:31:26.740938+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:31:26.740938+0000","last_undegraded":"2024-09-17T15:31:26.740938+0000","last_fullsized":"2024-09-17T15:31:26.740938+0000","mapping_epoch":44,"log_start":"34'3200","ondisk_log_start":"34'3200","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30118854,"num_objects":7355,"num_object_clones":0,"num_object_copies":14710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7355,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9721,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":486781686,"num_objects":118853,"num_object_clones":0,"num_object_copies":237706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38836,"num_objects_misplaced":1128,"num_objects_unfound":0,"num_objects_dirty":118853,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":174134,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9496,"num_bytes_recovered":36950982,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1129668608,"data_stored":1129523170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160799,"ondisk_log_size":160799,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051661,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":669076,"kb_used_data":370816,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93702764,"statfs":{"total":96636764160,"available":95951630336,"internally_reserved":0,"allocated":379715584,"data_stored":379424487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":305393324},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,3,0,0,0,0,0,0,0,1],"upper_bound":16384},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724606,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":621592,"kb_used_data":310856,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93750248,"statfs":{"total":96636764160,"available":96000253952,"internally_reserved":0,"allocated":318316544,"data_stored":317992466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":318172843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247889,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":803072,"kb_used_data":424368,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93568768,"statfs":{"total":96636764160,"available":95814418432,"internally_reserved":0,"allocated":434552832,"data_stored":434261730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":387772072},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,3,7,4,0,0,0,0,2],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":432910336,"data_stored":432874206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":317812736,"data_stored":317743630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":378945536,"data_stored":378905334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:31:31.305 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:31:31.820 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:31:31.821 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:31:31.843 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":252,"stamp":"2024-09-17T15:31:31.699106+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":486781705,"num_objects":118855,"num_object_clones":0,"num_object_copies":237710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38836,"num_objects_misplaced":1128,"num_objects_unfound":0,"num_objects_dirty":118855,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":174136,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9497,"num_bytes_recovered":36951001,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160802,"ondisk_log_size":160802,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2093740,"kb_used_data":1106040,"kb_used_omap":13,"kb_used_meta":987634,"kb_avail":281021780,"statfs":{"total":289910292480,"available":287766302720,"internally_reserved":0,"allocated":1132584960,"data_stored":1131678683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1011338239},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,3,1,6,10,4,0,0,0,0,2,0,1],"upper_bound":16384},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18497536,"num_objects":-4516,"num_object_clones":0,"num_object_copies":-9032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-8,"num_objects_misplaced":-282,"num_objects_unfound":0,"num_objects_dirty":-4516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4516,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":1040384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006487"},"pg_stats":[{"pgid":"2.f","version":"45'13215","reported_seq":13417,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:27.361641+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:31:27.361641+0000","last_peered":"2024-09-17T15:31:27.361641+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:31:27.361641+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'3200","ondisk_log_start":"34'3200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30445568,"num_objects":7433,"num_object_clones":0,"num_object_copies":14866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4010,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13215,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3423},{"shards":"0,2","objects":1179}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13394","reported_seq":13501,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:27.626631+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:31:27.626631+0000","last_peered":"2024-09-17T15:31:27.626631+0000","last_clean":"2024-09-17T15:31:27.626631+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:31:27.626631+0000","last_undegraded":"2024-09-17T15:31:27.626631+0000","last_fullsized":"2024-09-17T15:31:27.626631+0000","mapping_epoch":33,"log_start":"34'3300","ondisk_log_start":"34'3300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30876614,"num_objects":7540,"num_object_clones":0,"num_object_copies":15080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7540,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11751,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13497","reported_seq":13690,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:27.579155+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:31:27.579155+0000","last_peered":"2024-09-17T15:31:27.579155+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:31:27.579155+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'3400","ondisk_log_start":"34'3400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30457880,"num_objects":7437,"num_object_clones":0,"num_object_copies":14874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4138,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7437,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11654,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1140},{"shards":"2","objects":3300}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13426","reported_seq":13500,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:27.578458+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:31:27.578458+0000","last_peered":"2024-09-17T15:31:27.578458+0000","last_clean":"2024-09-17T15:31:27.578458+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:31:27.578458+0000","last_undegraded":"2024-09-17T15:31:27.578458+0000","last_fullsized":"2024-09-17T15:31:27.578458+0000","mapping_epoch":33,"log_start":"34'3400","ondisk_log_start":"34'3400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30900224,"num_objects":7544,"num_object_clones":0,"num_object_copies":15088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13426,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13237","reported_seq":13371,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:27.335825+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:31:27.335825+0000","last_peered":"2024-09-17T15:31:27.335825+0000","last_clean":"2024-09-17T15:31:27.335825+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:31:27.335825+0000","last_undegraded":"2024-09-17T15:31:27.335825+0000","last_fullsized":"2024-09-17T15:31:27.335825+0000","mapping_epoch":32,"log_start":"34'3200","ondisk_log_start":"34'3200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30376419,"num_objects":7417,"num_object_clones":0,"num_object_copies":14834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7417,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10219,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13734","reported_seq":13991,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:27.322767+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:31:27.322767+0000","last_peered":"2024-09-17T15:31:27.322767+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:31:27.322767+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'3700","ondisk_log_start":"34'3700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30790115,"num_objects":7518,"num_object_clones":0,"num_object_copies":15036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4212,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7518,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10233,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3307},{"shards":"0,2","objects":1162}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13415","reported_seq":13678,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:27.346510+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:31:27.346510+0000","last_peered":"2024-09-17T15:31:27.346510+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:31:27.346510+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'3400","ondisk_log_start":"34'3400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30118854,"num_objects":7355,"num_object_clones":0,"num_object_copies":14710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7355,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10184,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3241},{"shards":"0,2","objects":1146}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13464","reported_seq":13596,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:27.358436+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:31:27.358436+0000","last_peered":"2024-09-17T15:31:27.358436+0000","last_clean":"2024-09-17T15:31:27.358436+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:31:27.358436+0000","last_undegraded":"2024-09-17T15:31:27.358436+0000","last_fullsized":"2024-09-17T15:31:27.358436+0000","mapping_epoch":32,"log_start":"34'3400","ondisk_log_start":"34'3400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30523392,"num_objects":7452,"num_object_clones":0,"num_object_copies":14904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10418,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.086967+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.086967+0000","last_peered":"2024-09-17T15:30:57.086967+0000","last_clean":"2024-09-17T15:30:57.086967+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.086967+0000","last_undegraded":"2024-09-17T15:30:57.086967+0000","last_fullsized":"2024-09-17T15:30:57.086967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13328","reported_seq":21614,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:26.748821+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:31:26.748821+0000","last_peered":"2024-09-17T15:31:26.748821+0000","last_clean":"2024-09-17T15:31:26.748821+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:31:26.748821+0000","last_undegraded":"2024-09-17T15:31:26.748821+0000","last_fullsized":"2024-09-17T15:31:26.748821+0000","mapping_epoch":42,"log_start":"34'3300","ondisk_log_start":"34'3300","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30675427,"num_objects":7490,"num_object_clones":0,"num_object_copies":14980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7490,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11328,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088505+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088505+0000","last_peered":"2024-09-17T15:30:57.088505+0000","last_clean":"2024-09-17T15:30:57.088505+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088505+0000","last_undegraded":"2024-09-17T15:30:57.088505+0000","last_fullsized":"2024-09-17T15:30:57.088505+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13190","reported_seq":13414,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:27.430229+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:31:27.430229+0000","last_peered":"2024-09-17T15:31:27.430229+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:31:27.430229+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'3100","ondisk_log_start":"34'3100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30113792,"num_objects":7352,"num_object_clones":0,"num_object_copies":14704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3967,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9786,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3387},{"shards":"0,2","objects":1095}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13200","reported_seq":13377,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:27.466083+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:31:27.466083+0000","last_peered":"2024-09-17T15:31:27.466083+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:31:27.466083+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'3100","ondisk_log_start":"34'3100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29979107,"num_objects":7320,"num_object_clones":0,"num_object_copies":14640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4052,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7320,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11095,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1139},{"shards":"2","objects":3269}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13377","reported_seq":15242,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:27.654346+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:31:27.654346+0000","last_peered":"2024-09-17T15:31:27.654346+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:31:27.654346+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'3300","ondisk_log_start":"34'3300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30142464,"num_objects":7359,"num_object_clones":0,"num_object_copies":14718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2141,"num_objects_misplaced":1128,"num_objects_unfound":0,"num_objects_dirty":7359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11139,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":870,"num_bytes_recovered":3452928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1128},{"shards":"2","objects":4090}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13521","reported_seq":13781,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:27.411649+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:31:27.411649+0000","last_peered":"2024-09-17T15:31:27.411649+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:31:27.411649+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'3500","ondisk_log_start":"34'3500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30494720,"num_objects":7445,"num_object_clones":0,"num_object_copies":14890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4079,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10069,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3367},{"shards":"0,2","objects":1101}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13419","reported_seq":13656,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:27.387205+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:31:27.387205+0000","last_peered":"2024-09-17T15:31:27.387205+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:31:27.387205+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'3400","ondisk_log_start":"34'3400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30117888,"num_objects":7353,"num_object_clones":0,"num_object_copies":14706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7353,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9706,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3242},{"shards":"0,2","objects":1128}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088855+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088855+0000","last_peered":"2024-09-17T15:30:57.088855+0000","last_clean":"2024-09-17T15:30:57.088855+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088855+0000","last_undegraded":"2024-09-17T15:30:57.088855+0000","last_fullsized":"2024-09-17T15:30:57.088855+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13403","reported_seq":13651,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:27.404280+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:31:27.404280+0000","last_peered":"2024-09-17T15:31:27.404280+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:31:27.404280+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'3400","ondisk_log_start":"34'3400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30650368,"num_objects":7483,"num_object_clones":0,"num_object_copies":14966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4009,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10190,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3476},{"shards":"0,2","objects":1105}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13279","reported_seq":21658,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:26.740938+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:31:26.740938+0000","last_peered":"2024-09-17T15:31:26.740938+0000","last_clean":"2024-09-17T15:31:26.740938+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:31:26.740938+0000","last_undegraded":"2024-09-17T15:31:26.740938+0000","last_fullsized":"2024-09-17T15:31:26.740938+0000","mapping_epoch":44,"log_start":"34'3200","ondisk_log_start":"34'3200","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30118854,"num_objects":7355,"num_object_clones":0,"num_object_copies":14710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7355,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9721,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":486781686,"num_objects":118853,"num_object_clones":0,"num_object_copies":237706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38836,"num_objects_misplaced":1128,"num_objects_unfound":0,"num_objects_dirty":118853,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":174134,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9496,"num_bytes_recovered":36950982,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1129668608,"data_stored":1129523170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160799,"ondisk_log_size":160799,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051661,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":669076,"kb_used_data":370816,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93702764,"statfs":{"total":96636764160,"available":95951630336,"internally_reserved":0,"allocated":379715584,"data_stored":379424487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":305393324},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,3,0,0,0,0,0,0,0,1],"upper_bound":16384},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724606,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":621592,"kb_used_data":310856,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93750248,"statfs":{"total":96636764160,"available":96000253952,"internally_reserved":0,"allocated":318316544,"data_stored":317992466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":318172843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247889,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":803072,"kb_used_data":424368,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93568768,"statfs":{"total":96636764160,"available":95814418432,"internally_reserved":0,"allocated":434552832,"data_stored":434261730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":387772072},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,3,7,4,0,0,0,0,2],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":432910336,"data_stored":432874206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":317812736,"data_stored":317743630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":378945536,"data_stored":378905334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:31:31.844 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:31:31.844 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'13215", 'reported_seq': 13417, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:27.361641+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:31:27.361641+0000', 'last_peered': '2024-09-17T15:31:27.361641+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:31:27.361641+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "34'3200", 'ondisk_log_start': "34'3200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30445568, 'num_objects': 7433, 'num_object_clones': 0, 'num_object_copies': 14866, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4010, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7433, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13215, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 60, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3423}, {'shards': '0,2', 'objects': 1179}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:31.845 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:31:31.845 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'13497", 'reported_seq': 13690, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:27.579155+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:31:27.579155+0000', 'last_peered': '2024-09-17T15:31:27.579155+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:31:27.579155+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "34'3400", 'ondisk_log_start': "34'3400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30457880, 'num_objects': 7437, 'num_object_clones': 0, 'num_object_copies': 14874, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4138, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7437, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11654, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 31, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1140}, {'shards': '2', 'objects': 3300}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:31:31.845 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:31:31.845 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'13734", 'reported_seq': 13991, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:27.322767+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:31:27.322767+0000', 'last_peered': '2024-09-17T15:31:27.322767+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:31:27.322767+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "34'3700", 'ondisk_log_start': "34'3700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30790115, 'num_objects': 7518, 'num_object_clones': 0, 'num_object_copies': 15036, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4212, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7518, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10233, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 59, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3307}, {'shards': '0,2', 'objects': 1162}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:31.845 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:31:31.845 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'13415", 'reported_seq': 13678, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:27.346510+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:31:27.346510+0000', 'last_peered': '2024-09-17T15:31:27.346510+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:31:27.346510+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "34'3400", 'ondisk_log_start': "34'3400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30118854, 'num_objects': 7355, 'num_object_clones': 0, 'num_object_copies': 14710, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4114, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7355, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 10184, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 60, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3241}, {'shards': '0,2', 'objects': 1146}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:31.845 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:31:31.845 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'13190", 'reported_seq': 13414, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:27.430229+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:31:27.430229+0000', 'last_peered': '2024-09-17T15:31:27.430229+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:31:27.430229+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "34'3100", 'ondisk_log_start': "34'3100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30113792, 'num_objects': 7352, 'num_object_clones': 0, 'num_object_copies': 14704, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3967, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7352, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9786, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 45, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3387}, {'shards': '0,2', 'objects': 1095}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:31.845 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:31:31.845 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'13200", 'reported_seq': 13377, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:27.466083+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:31:27.466083+0000', 'last_peered': '2024-09-17T15:31:27.466083+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:31:27.466083+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "34'3100", 'ondisk_log_start': "34'3100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29979107, 'num_objects': 7320, 'num_object_clones': 0, 'num_object_copies': 14640, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4052, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7320, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11095, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 26, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1139}, {'shards': '2', 'objects': 3269}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:31:31.845 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:31:31.846 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'13377", 'reported_seq': 15242, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:27.654346+0000', 'last_change': '2024-09-17T15:30:55.086473+0000', 'last_active': '2024-09-17T15:31:27.654346+0000', 'last_peered': '2024-09-17T15:31:27.654346+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:31:27.654346+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "34'3300", 'ondisk_log_start': "34'3300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30142464, 'num_objects': 7359, 'num_object_clones': 0, 'num_object_copies': 14718, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2141, 'num_objects_misplaced': 1128, 'num_objects_unfound': 0, 'num_objects_dirty': 7359, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11139, 'num_write_kb': 32532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 870, 'num_bytes_recovered': 3452928, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1128}, {'shards': '2', 'objects': 4090}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:31:31.846 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:31:31.846 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'13521", 'reported_seq': 13781, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:27.411649+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:31:27.411649+0000', 'last_peered': '2024-09-17T15:31:27.411649+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:31:27.411649+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "34'3500", 'ondisk_log_start': "34'3500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30494720, 'num_objects': 7445, 'num_object_clones': 0, 'num_object_copies': 14890, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4079, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7445, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10069, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 60, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3367}, {'shards': '0,2', 'objects': 1101}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:31.846 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:31:31.846 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'13419", 'reported_seq': 13656, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:27.387205+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:31:27.387205+0000', 'last_peered': '2024-09-17T15:31:27.387205+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:31:27.387205+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "34'3400", 'ondisk_log_start': "34'3400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30117888, 'num_objects': 7353, 'num_object_clones': 0, 'num_object_copies': 14706, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4114, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7353, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9706, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 46, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3242}, {'shards': '0,2', 'objects': 1128}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:31.846 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:31:31.846 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'13403", 'reported_seq': 13651, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:27.404280+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:31:27.404280+0000', 'last_peered': '2024-09-17T15:31:27.404280+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:31:27.404280+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "34'3400", 'ondisk_log_start': "34'3400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30650368, 'num_objects': 7483, 'num_object_clones': 0, 'num_object_copies': 14966, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4009, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7483, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10190, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3476}, {'shards': '0,2', 'objects': 1105}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:31.846 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:31:32.511 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:31:32.512 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38836/237710 objects degraded (16.338%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":429,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":118855,"data_bytes":486781705,"bytes_used":2143989760,"bytes_avail":287766302720,"bytes_total":289910292480,"degraded_objects":38836,"degraded_total":237710,"degraded_ratio":0.16337554162635143,"misplaced_objects":1128,"misplaced_total":237710,"misplaced_ratio":0.0047452778595767952,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":86651,"recovering_keys_per_sec":0,"num_objects_recovered":290,"num_bytes_recovered":1040384,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":376},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-17T15:31:29.695933+0000","services":{}},"progress_events":{}} 2024-09-17T15:31:32.539 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:31:32.539 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:31:33.073 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:31:33.076 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:31:33.096 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":252,"stamp":"2024-09-17T15:31:31.699106+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":486781705,"num_objects":118855,"num_object_clones":0,"num_object_copies":237710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38836,"num_objects_misplaced":1128,"num_objects_unfound":0,"num_objects_dirty":118855,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":174136,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9497,"num_bytes_recovered":36951001,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160802,"ondisk_log_size":160802,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2093740,"kb_used_data":1106040,"kb_used_omap":13,"kb_used_meta":987634,"kb_avail":281021780,"statfs":{"total":289910292480,"available":287766302720,"internally_reserved":0,"allocated":1132584960,"data_stored":1131678683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1011338239},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,3,1,6,10,4,0,0,0,0,2,0,1],"upper_bound":16384},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18497536,"num_objects":-4516,"num_object_clones":0,"num_object_copies":-9032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-8,"num_objects_misplaced":-282,"num_objects_unfound":0,"num_objects_dirty":-4516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4516,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":1040384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006487"},"pg_stats":[{"pgid":"2.f","version":"45'13215","reported_seq":13417,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:27.361641+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:31:27.361641+0000","last_peered":"2024-09-17T15:31:27.361641+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:31:27.361641+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'3200","ondisk_log_start":"34'3200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30445568,"num_objects":7433,"num_object_clones":0,"num_object_copies":14866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4010,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13215,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3423},{"shards":"0,2","objects":1179}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13394","reported_seq":13501,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:27.626631+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:31:27.626631+0000","last_peered":"2024-09-17T15:31:27.626631+0000","last_clean":"2024-09-17T15:31:27.626631+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:31:27.626631+0000","last_undegraded":"2024-09-17T15:31:27.626631+0000","last_fullsized":"2024-09-17T15:31:27.626631+0000","mapping_epoch":33,"log_start":"34'3300","ondisk_log_start":"34'3300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30876614,"num_objects":7540,"num_object_clones":0,"num_object_copies":15080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7540,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11751,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13497","reported_seq":13690,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:27.579155+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:31:27.579155+0000","last_peered":"2024-09-17T15:31:27.579155+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:31:27.579155+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'3400","ondisk_log_start":"34'3400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30457880,"num_objects":7437,"num_object_clones":0,"num_object_copies":14874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4138,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7437,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11654,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1140},{"shards":"2","objects":3300}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13426","reported_seq":13500,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:27.578458+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:31:27.578458+0000","last_peered":"2024-09-17T15:31:27.578458+0000","last_clean":"2024-09-17T15:31:27.578458+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:31:27.578458+0000","last_undegraded":"2024-09-17T15:31:27.578458+0000","last_fullsized":"2024-09-17T15:31:27.578458+0000","mapping_epoch":33,"log_start":"34'3400","ondisk_log_start":"34'3400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30900224,"num_objects":7544,"num_object_clones":0,"num_object_copies":15088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13426,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13237","reported_seq":13371,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:27.335825+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:31:27.335825+0000","last_peered":"2024-09-17T15:31:27.335825+0000","last_clean":"2024-09-17T15:31:27.335825+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:31:27.335825+0000","last_undegraded":"2024-09-17T15:31:27.335825+0000","last_fullsized":"2024-09-17T15:31:27.335825+0000","mapping_epoch":32,"log_start":"34'3200","ondisk_log_start":"34'3200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30376419,"num_objects":7417,"num_object_clones":0,"num_object_copies":14834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7417,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10219,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13734","reported_seq":13991,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:27.322767+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:31:27.322767+0000","last_peered":"2024-09-17T15:31:27.322767+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:31:27.322767+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'3700","ondisk_log_start":"34'3700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30790115,"num_objects":7518,"num_object_clones":0,"num_object_copies":15036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4212,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7518,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10233,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3307},{"shards":"0,2","objects":1162}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13415","reported_seq":13678,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:27.346510+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:31:27.346510+0000","last_peered":"2024-09-17T15:31:27.346510+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:31:27.346510+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'3400","ondisk_log_start":"34'3400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30118854,"num_objects":7355,"num_object_clones":0,"num_object_copies":14710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7355,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10184,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3241},{"shards":"0,2","objects":1146}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13464","reported_seq":13596,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:27.358436+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:31:27.358436+0000","last_peered":"2024-09-17T15:31:27.358436+0000","last_clean":"2024-09-17T15:31:27.358436+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:31:27.358436+0000","last_undegraded":"2024-09-17T15:31:27.358436+0000","last_fullsized":"2024-09-17T15:31:27.358436+0000","mapping_epoch":32,"log_start":"34'3400","ondisk_log_start":"34'3400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30523392,"num_objects":7452,"num_object_clones":0,"num_object_copies":14904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10418,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.086967+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.086967+0000","last_peered":"2024-09-17T15:30:57.086967+0000","last_clean":"2024-09-17T15:30:57.086967+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.086967+0000","last_undegraded":"2024-09-17T15:30:57.086967+0000","last_fullsized":"2024-09-17T15:30:57.086967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13328","reported_seq":21614,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:26.748821+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:31:26.748821+0000","last_peered":"2024-09-17T15:31:26.748821+0000","last_clean":"2024-09-17T15:31:26.748821+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:31:26.748821+0000","last_undegraded":"2024-09-17T15:31:26.748821+0000","last_fullsized":"2024-09-17T15:31:26.748821+0000","mapping_epoch":42,"log_start":"34'3300","ondisk_log_start":"34'3300","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30675427,"num_objects":7490,"num_object_clones":0,"num_object_copies":14980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7490,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11328,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088505+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088505+0000","last_peered":"2024-09-17T15:30:57.088505+0000","last_clean":"2024-09-17T15:30:57.088505+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088505+0000","last_undegraded":"2024-09-17T15:30:57.088505+0000","last_fullsized":"2024-09-17T15:30:57.088505+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13190","reported_seq":13414,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:27.430229+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:31:27.430229+0000","last_peered":"2024-09-17T15:31:27.430229+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:31:27.430229+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'3100","ondisk_log_start":"34'3100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30113792,"num_objects":7352,"num_object_clones":0,"num_object_copies":14704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3967,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9786,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3387},{"shards":"0,2","objects":1095}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13200","reported_seq":13377,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:27.466083+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:31:27.466083+0000","last_peered":"2024-09-17T15:31:27.466083+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:31:27.466083+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'3100","ondisk_log_start":"34'3100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29979107,"num_objects":7320,"num_object_clones":0,"num_object_copies":14640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4052,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7320,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11095,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1139},{"shards":"2","objects":3269}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13377","reported_seq":15242,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:27.654346+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:31:27.654346+0000","last_peered":"2024-09-17T15:31:27.654346+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:31:27.654346+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'3300","ondisk_log_start":"34'3300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30142464,"num_objects":7359,"num_object_clones":0,"num_object_copies":14718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2141,"num_objects_misplaced":1128,"num_objects_unfound":0,"num_objects_dirty":7359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11139,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":870,"num_bytes_recovered":3452928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1128},{"shards":"2","objects":4090}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13521","reported_seq":13781,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:27.411649+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:31:27.411649+0000","last_peered":"2024-09-17T15:31:27.411649+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:31:27.411649+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'3500","ondisk_log_start":"34'3500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30494720,"num_objects":7445,"num_object_clones":0,"num_object_copies":14890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4079,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10069,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3367},{"shards":"0,2","objects":1101}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13419","reported_seq":13656,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:27.387205+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:31:27.387205+0000","last_peered":"2024-09-17T15:31:27.387205+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:31:27.387205+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'3400","ondisk_log_start":"34'3400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30117888,"num_objects":7353,"num_object_clones":0,"num_object_copies":14706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7353,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9706,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3242},{"shards":"0,2","objects":1128}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088855+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088855+0000","last_peered":"2024-09-17T15:30:57.088855+0000","last_clean":"2024-09-17T15:30:57.088855+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088855+0000","last_undegraded":"2024-09-17T15:30:57.088855+0000","last_fullsized":"2024-09-17T15:30:57.088855+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13403","reported_seq":13651,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:27.404280+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:31:27.404280+0000","last_peered":"2024-09-17T15:31:27.404280+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:31:27.404280+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'3400","ondisk_log_start":"34'3400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30650368,"num_objects":7483,"num_object_clones":0,"num_object_copies":14966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4009,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10190,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3476},{"shards":"0,2","objects":1105}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13279","reported_seq":21658,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:26.740938+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:31:26.740938+0000","last_peered":"2024-09-17T15:31:26.740938+0000","last_clean":"2024-09-17T15:31:26.740938+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:31:26.740938+0000","last_undegraded":"2024-09-17T15:31:26.740938+0000","last_fullsized":"2024-09-17T15:31:26.740938+0000","mapping_epoch":44,"log_start":"34'3200","ondisk_log_start":"34'3200","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30118854,"num_objects":7355,"num_object_clones":0,"num_object_copies":14710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7355,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9721,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":486781686,"num_objects":118853,"num_object_clones":0,"num_object_copies":237706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38836,"num_objects_misplaced":1128,"num_objects_unfound":0,"num_objects_dirty":118853,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":174134,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9496,"num_bytes_recovered":36950982,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1129668608,"data_stored":1129523170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160799,"ondisk_log_size":160799,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051661,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":669076,"kb_used_data":370816,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93702764,"statfs":{"total":96636764160,"available":95951630336,"internally_reserved":0,"allocated":379715584,"data_stored":379424487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":305393324},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,3,0,0,0,0,0,0,0,1],"upper_bound":16384},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724606,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":621592,"kb_used_data":310856,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93750248,"statfs":{"total":96636764160,"available":96000253952,"internally_reserved":0,"allocated":318316544,"data_stored":317992466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":318172843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247889,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":803072,"kb_used_data":424368,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93568768,"statfs":{"total":96636764160,"available":95814418432,"internally_reserved":0,"allocated":434552832,"data_stored":434261730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":387772072},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,3,7,4,0,0,0,0,2],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":432910336,"data_stored":432874206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":317812736,"data_stored":317743630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":378945536,"data_stored":378905334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:31:36.099 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:31:36.630 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:31:36.633 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:31:36.652 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":254,"stamp":"2024-09-17T15:31:35.701883+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":477893385,"num_objects":116685,"num_object_clones":0,"num_object_copies":233370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38831,"num_objects_misplaced":961,"num_objects_unfound":0,"num_objects_dirty":116685,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":176306,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9669,"num_bytes_recovered":37635033,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160672,"ondisk_log_size":160672,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2081584,"kb_used_data":1093888,"kb_used_omap":13,"kb_used_meta":987634,"kb_avail":281033936,"statfs":{"total":289910292480,"available":287778750464,"internally_reserved":0,"allocated":1120141312,"data_stored":1119235035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1011338239},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,4,3,5,4,2,0,6,0,0,0,4],"upper_bound":8192},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18104320,"num_objects":-4420,"num_object_clones":0,"num_object_copies":-8840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-5,"num_objects_misplaced":-320,"num_objects_unfound":0,"num_objects_dirty":-4420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4420,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":325,"num_bytes_recovered":1236992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008091"},"pg_stats":[{"pgid":"2.f","version":"45'13349","reported_seq":13553,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:32.370236+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:31:32.370236+0000","last_peered":"2024-09-17T15:31:32.370236+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:31:32.370236+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'3300","ondisk_log_start":"34'3300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29896704,"num_objects":7299,"num_object_clones":0,"num_object_copies":14598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4009,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7299,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13349,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3290},{"shards":"0,2","objects":1045}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13538","reported_seq":13645,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:32.582289+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:31:32.582289+0000","last_peered":"2024-09-17T15:31:32.582289+0000","last_clean":"2024-09-17T15:31:32.582289+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:31:32.582289+0000","last_undegraded":"2024-09-17T15:31:32.582289+0000","last_fullsized":"2024-09-17T15:31:32.582289+0000","mapping_epoch":33,"log_start":"34'3500","ondisk_log_start":"34'3500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30286790,"num_objects":7396,"num_object_clones":0,"num_object_copies":14792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7396,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11895,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13645","reported_seq":13838,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:32.610381+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:31:32.610381+0000","last_peered":"2024-09-17T15:31:32.610381+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:31:32.610381+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'3600","ondisk_log_start":"34'3600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29851672,"num_objects":7289,"num_object_clones":0,"num_object_copies":14578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4138,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7289,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11802,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":992},{"shards":"2","objects":3152}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13551","reported_seq":13625,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:32.600961+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:31:32.600961+0000","last_peered":"2024-09-17T15:31:32.600961+0000","last_clean":"2024-09-17T15:31:32.600961+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:31:32.600961+0000","last_undegraded":"2024-09-17T15:31:32.600961+0000","last_fullsized":"2024-09-17T15:31:32.600961+0000","mapping_epoch":33,"log_start":"34'3500","ondisk_log_start":"34'3500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30388224,"num_objects":7419,"num_object_clones":0,"num_object_copies":14838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7419,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13551,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13368","reported_seq":13502,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:32.346002+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:31:32.346002+0000","last_peered":"2024-09-17T15:31:32.346002+0000","last_clean":"2024-09-17T15:31:32.346002+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:31:32.346002+0000","last_undegraded":"2024-09-17T15:31:32.346002+0000","last_fullsized":"2024-09-17T15:31:32.346002+0000","mapping_epoch":32,"log_start":"34'3300","ondisk_log_start":"34'3300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29839843,"num_objects":7286,"num_object_clones":0,"num_object_copies":14572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7286,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10350,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13876","reported_seq":14133,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:32.447675+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:31:32.447675+0000","last_peered":"2024-09-17T15:31:32.447675+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:31:32.447675+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'3800","ondisk_log_start":"34'3800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30208483,"num_objects":7376,"num_object_clones":0,"num_object_copies":14752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4212,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7376,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10375,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3165},{"shards":"0,2","objects":1020}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13549","reported_seq":13814,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:32.430447+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:31:32.430447+0000","last_peered":"2024-09-17T15:31:32.430447+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:31:32.430447+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'3500","ondisk_log_start":"34'3500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29569990,"num_objects":7221,"num_object_clones":0,"num_object_copies":14442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4113,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7221,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10318,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3108},{"shards":"0,2","objects":1012}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13614","reported_seq":13746,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:32.323416+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:31:32.323416+0000","last_peered":"2024-09-17T15:31:32.323416+0000","last_clean":"2024-09-17T15:31:32.323416+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:31:32.323416+0000","last_undegraded":"2024-09-17T15:31:32.323416+0000","last_fullsized":"2024-09-17T15:31:32.323416+0000","mapping_epoch":32,"log_start":"34'3600","ondisk_log_start":"34'3600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29908992,"num_objects":7302,"num_object_clones":0,"num_object_copies":14604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10568,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.086967+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.086967+0000","last_peered":"2024-09-17T15:30:57.086967+0000","last_clean":"2024-09-17T15:30:57.086967+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.086967+0000","last_undegraded":"2024-09-17T15:30:57.086967+0000","last_fullsized":"2024-09-17T15:30:57.086967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13461","reported_seq":21748,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:31.786085+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:31:31.786085+0000","last_peered":"2024-09-17T15:31:31.786085+0000","last_clean":"2024-09-17T15:31:31.786085+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:31:31.786085+0000","last_undegraded":"2024-09-17T15:31:31.786085+0000","last_fullsized":"2024-09-17T15:31:31.786085+0000","mapping_epoch":42,"log_start":"34'3400","ondisk_log_start":"34'3400","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30130659,"num_objects":7357,"num_object_clones":0,"num_object_copies":14714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7357,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11461,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088505+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088505+0000","last_peered":"2024-09-17T15:30:57.088505+0000","last_clean":"2024-09-17T15:30:57.088505+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088505+0000","last_undegraded":"2024-09-17T15:30:57.088505+0000","last_fullsized":"2024-09-17T15:30:57.088505+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13328","reported_seq":13553,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:32.327427+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:31:32.327427+0000","last_peered":"2024-09-17T15:31:32.327427+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:31:32.327427+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'3300","ondisk_log_start":"34'3300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29548544,"num_objects":7214,"num_object_clones":0,"num_object_copies":14428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3967,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9924,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3249},{"shards":"0,2","objects":957}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13339","reported_seq":13516,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:32.612441+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:31:32.612441+0000","last_peered":"2024-09-17T15:31:32.612441+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:31:32.612441+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'3300","ondisk_log_start":"34'3300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29409763,"num_objects":7181,"num_object_clones":0,"num_object_copies":14362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4052,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7181,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11234,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1000},{"shards":"2","objects":3130}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13501","reported_seq":15700,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:32.655190+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:31:32.655190+0000","last_peered":"2024-09-17T15:31:32.655190+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:31:32.655190+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'3500","ondisk_log_start":"34'3500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29634560,"num_objects":7235,"num_object_clones":0,"num_object_copies":14470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2141,"num_objects_misplaced":961,"num_objects_unfound":0,"num_objects_dirty":7235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11263,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1037,"num_bytes_recovered":4136960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":961},{"shards":"2","objects":4133}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13644","reported_seq":13906,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:32.400725+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:31:32.400725+0000","last_peered":"2024-09-17T15:31:32.400725+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:31:32.400725+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'3600","ondisk_log_start":"34'3600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29990912,"num_objects":7322,"num_object_clones":0,"num_object_copies":14644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10192,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3245},{"shards":"0,2","objects":978}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13561","reported_seq":13799,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:32.383835+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:31:32.383835+0000","last_peered":"2024-09-17T15:31:32.383835+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:31:32.383835+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'3500","ondisk_log_start":"34'3500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29536256,"num_objects":7211,"num_object_clones":0,"num_object_copies":14422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7211,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9848,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3100},{"shards":"0,2","objects":986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088855+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088855+0000","last_peered":"2024-09-17T15:30:57.088855+0000","last_clean":"2024-09-17T15:30:57.088855+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088855+0000","last_undegraded":"2024-09-17T15:30:57.088855+0000","last_fullsized":"2024-09-17T15:30:57.088855+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13538","reported_seq":13791,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:32.470813+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:31:32.470813+0000","last_peered":"2024-09-17T15:31:32.470813+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:31:32.470813+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'3500","ondisk_log_start":"34'3500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30097408,"num_objects":7348,"num_object_clones":0,"num_object_copies":14696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4007,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7348,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10325,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3343},{"shards":"0,2","objects":970}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13407","reported_seq":21786,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:31.720635+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:31:31.720635+0000","last_peered":"2024-09-17T15:31:31.720635+0000","last_clean":"2024-09-17T15:31:31.720635+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:31:31.720635+0000","last_undegraded":"2024-09-17T15:31:31.720635+0000","last_fullsized":"2024-09-17T15:31:31.720635+0000","mapping_epoch":44,"log_start":"34'3400","ondisk_log_start":"34'3400","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29594566,"num_objects":7227,"num_object_clones":0,"num_object_copies":14454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9849,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":477893366,"num_objects":116683,"num_object_clones":0,"num_object_copies":233366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38831,"num_objects_misplaced":961,"num_objects_unfound":0,"num_objects_dirty":116683,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":176304,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9668,"num_bytes_recovered":37635014,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1117528064,"data_stored":1117382626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160669,"ondisk_log_size":160669,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051662,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":668792,"kb_used_data":366376,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93703048,"statfs":{"total":96636764160,"available":95951921152,"internally_reserved":0,"allocated":375169024,"data_stored":374877927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":309653164},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,0,1,2,0,0,0,0,0,4],"upper_bound":8192},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724607,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":612340,"kb_used_data":309924,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93759500,"statfs":{"total":96636764160,"available":96009728000,"internally_reserved":0,"allocated":317362176,"data_stored":317038098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":309653163},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247890,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":800452,"kb_used_data":417588,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93571388,"statfs":{"total":96636764160,"available":95817101312,"internally_reserved":0,"allocated":427610112,"data_stored":427319010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":392031912},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,4,1,0,0,6],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":426242048,"data_stored":426205918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":316895232,"data_stored":316826126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":374390784,"data_stored":374350582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:31:36.654 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:31:37.209 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:31:37.212 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:31:37.239 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":254,"stamp":"2024-09-17T15:31:35.701883+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":477893385,"num_objects":116685,"num_object_clones":0,"num_object_copies":233370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38831,"num_objects_misplaced":961,"num_objects_unfound":0,"num_objects_dirty":116685,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":176306,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9669,"num_bytes_recovered":37635033,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160672,"ondisk_log_size":160672,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2081584,"kb_used_data":1093888,"kb_used_omap":13,"kb_used_meta":987634,"kb_avail":281033936,"statfs":{"total":289910292480,"available":287778750464,"internally_reserved":0,"allocated":1120141312,"data_stored":1119235035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1011338239},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,4,3,5,4,2,0,6,0,0,0,4],"upper_bound":8192},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18104320,"num_objects":-4420,"num_object_clones":0,"num_object_copies":-8840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-5,"num_objects_misplaced":-320,"num_objects_unfound":0,"num_objects_dirty":-4420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4420,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":325,"num_bytes_recovered":1236992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008091"},"pg_stats":[{"pgid":"2.f","version":"45'13349","reported_seq":13553,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:32.370236+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:31:32.370236+0000","last_peered":"2024-09-17T15:31:32.370236+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:31:32.370236+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'3300","ondisk_log_start":"34'3300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29896704,"num_objects":7299,"num_object_clones":0,"num_object_copies":14598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4009,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7299,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13349,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3290},{"shards":"0,2","objects":1045}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13538","reported_seq":13645,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:32.582289+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:31:32.582289+0000","last_peered":"2024-09-17T15:31:32.582289+0000","last_clean":"2024-09-17T15:31:32.582289+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:31:32.582289+0000","last_undegraded":"2024-09-17T15:31:32.582289+0000","last_fullsized":"2024-09-17T15:31:32.582289+0000","mapping_epoch":33,"log_start":"34'3500","ondisk_log_start":"34'3500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30286790,"num_objects":7396,"num_object_clones":0,"num_object_copies":14792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7396,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11895,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13645","reported_seq":13838,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:32.610381+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:31:32.610381+0000","last_peered":"2024-09-17T15:31:32.610381+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:31:32.610381+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'3600","ondisk_log_start":"34'3600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29851672,"num_objects":7289,"num_object_clones":0,"num_object_copies":14578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4138,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7289,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11802,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":992},{"shards":"2","objects":3152}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13551","reported_seq":13625,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:32.600961+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:31:32.600961+0000","last_peered":"2024-09-17T15:31:32.600961+0000","last_clean":"2024-09-17T15:31:32.600961+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:31:32.600961+0000","last_undegraded":"2024-09-17T15:31:32.600961+0000","last_fullsized":"2024-09-17T15:31:32.600961+0000","mapping_epoch":33,"log_start":"34'3500","ondisk_log_start":"34'3500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30388224,"num_objects":7419,"num_object_clones":0,"num_object_copies":14838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7419,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13551,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13368","reported_seq":13502,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:32.346002+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:31:32.346002+0000","last_peered":"2024-09-17T15:31:32.346002+0000","last_clean":"2024-09-17T15:31:32.346002+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:31:32.346002+0000","last_undegraded":"2024-09-17T15:31:32.346002+0000","last_fullsized":"2024-09-17T15:31:32.346002+0000","mapping_epoch":32,"log_start":"34'3300","ondisk_log_start":"34'3300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29839843,"num_objects":7286,"num_object_clones":0,"num_object_copies":14572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7286,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10350,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13876","reported_seq":14133,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:32.447675+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:31:32.447675+0000","last_peered":"2024-09-17T15:31:32.447675+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:31:32.447675+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"34'3800","ondisk_log_start":"34'3800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30208483,"num_objects":7376,"num_object_clones":0,"num_object_copies":14752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4212,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7376,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10375,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3165},{"shards":"0,2","objects":1020}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13549","reported_seq":13814,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:32.430447+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:31:32.430447+0000","last_peered":"2024-09-17T15:31:32.430447+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:31:32.430447+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'3500","ondisk_log_start":"34'3500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29569990,"num_objects":7221,"num_object_clones":0,"num_object_copies":14442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4113,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7221,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10318,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3108},{"shards":"0,2","objects":1012}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13614","reported_seq":13746,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:32.323416+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:31:32.323416+0000","last_peered":"2024-09-17T15:31:32.323416+0000","last_clean":"2024-09-17T15:31:32.323416+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:31:32.323416+0000","last_undegraded":"2024-09-17T15:31:32.323416+0000","last_fullsized":"2024-09-17T15:31:32.323416+0000","mapping_epoch":32,"log_start":"34'3600","ondisk_log_start":"34'3600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29908992,"num_objects":7302,"num_object_clones":0,"num_object_copies":14604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10568,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.086967+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.086967+0000","last_peered":"2024-09-17T15:30:57.086967+0000","last_clean":"2024-09-17T15:30:57.086967+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.086967+0000","last_undegraded":"2024-09-17T15:30:57.086967+0000","last_fullsized":"2024-09-17T15:30:57.086967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13461","reported_seq":21748,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:31.786085+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:31:31.786085+0000","last_peered":"2024-09-17T15:31:31.786085+0000","last_clean":"2024-09-17T15:31:31.786085+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:31:31.786085+0000","last_undegraded":"2024-09-17T15:31:31.786085+0000","last_fullsized":"2024-09-17T15:31:31.786085+0000","mapping_epoch":42,"log_start":"34'3400","ondisk_log_start":"34'3400","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30130659,"num_objects":7357,"num_object_clones":0,"num_object_copies":14714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7357,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11461,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088505+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088505+0000","last_peered":"2024-09-17T15:30:57.088505+0000","last_clean":"2024-09-17T15:30:57.088505+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088505+0000","last_undegraded":"2024-09-17T15:30:57.088505+0000","last_fullsized":"2024-09-17T15:30:57.088505+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13328","reported_seq":13553,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:32.327427+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:31:32.327427+0000","last_peered":"2024-09-17T15:31:32.327427+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:31:32.327427+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'3300","ondisk_log_start":"34'3300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29548544,"num_objects":7214,"num_object_clones":0,"num_object_copies":14428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3967,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9924,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3249},{"shards":"0,2","objects":957}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13339","reported_seq":13516,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:32.612441+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:31:32.612441+0000","last_peered":"2024-09-17T15:31:32.612441+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:31:32.612441+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'3300","ondisk_log_start":"34'3300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29409763,"num_objects":7181,"num_object_clones":0,"num_object_copies":14362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4052,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7181,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11234,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1000},{"shards":"2","objects":3130}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13501","reported_seq":15700,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:32.655190+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:31:32.655190+0000","last_peered":"2024-09-17T15:31:32.655190+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:31:32.655190+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'3500","ondisk_log_start":"34'3500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29634560,"num_objects":7235,"num_object_clones":0,"num_object_copies":14470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2141,"num_objects_misplaced":961,"num_objects_unfound":0,"num_objects_dirty":7235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11263,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1037,"num_bytes_recovered":4136960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":961},{"shards":"2","objects":4133}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13644","reported_seq":13906,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:32.400725+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:31:32.400725+0000","last_peered":"2024-09-17T15:31:32.400725+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:31:32.400725+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'3600","ondisk_log_start":"34'3600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29990912,"num_objects":7322,"num_object_clones":0,"num_object_copies":14644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10192,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3245},{"shards":"0,2","objects":978}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13561","reported_seq":13799,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:32.383835+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:31:32.383835+0000","last_peered":"2024-09-17T15:31:32.383835+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:31:32.383835+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'3500","ondisk_log_start":"34'3500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29536256,"num_objects":7211,"num_object_clones":0,"num_object_copies":14422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7211,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9848,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3100},{"shards":"0,2","objects":986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088855+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088855+0000","last_peered":"2024-09-17T15:30:57.088855+0000","last_clean":"2024-09-17T15:30:57.088855+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088855+0000","last_undegraded":"2024-09-17T15:30:57.088855+0000","last_fullsized":"2024-09-17T15:30:57.088855+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13538","reported_seq":13791,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:32.470813+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:31:32.470813+0000","last_peered":"2024-09-17T15:31:32.470813+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:31:32.470813+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'3500","ondisk_log_start":"34'3500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30097408,"num_objects":7348,"num_object_clones":0,"num_object_copies":14696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4007,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7348,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10325,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3343},{"shards":"0,2","objects":970}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13407","reported_seq":21786,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:31.720635+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:31:31.720635+0000","last_peered":"2024-09-17T15:31:31.720635+0000","last_clean":"2024-09-17T15:31:31.720635+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:31:31.720635+0000","last_undegraded":"2024-09-17T15:31:31.720635+0000","last_fullsized":"2024-09-17T15:31:31.720635+0000","mapping_epoch":44,"log_start":"34'3400","ondisk_log_start":"34'3400","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29594566,"num_objects":7227,"num_object_clones":0,"num_object_copies":14454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9849,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":477893366,"num_objects":116683,"num_object_clones":0,"num_object_copies":233366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38831,"num_objects_misplaced":961,"num_objects_unfound":0,"num_objects_dirty":116683,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":176304,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9668,"num_bytes_recovered":37635014,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1117528064,"data_stored":1117382626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160669,"ondisk_log_size":160669,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051662,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":668792,"kb_used_data":366376,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93703048,"statfs":{"total":96636764160,"available":95951921152,"internally_reserved":0,"allocated":375169024,"data_stored":374877927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":309653164},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,0,1,2,0,0,0,0,0,4],"upper_bound":8192},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724607,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":612340,"kb_used_data":309924,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93759500,"statfs":{"total":96636764160,"available":96009728000,"internally_reserved":0,"allocated":317362176,"data_stored":317038098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":309653163},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247890,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":800452,"kb_used_data":417588,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93571388,"statfs":{"total":96636764160,"available":95817101312,"internally_reserved":0,"allocated":427610112,"data_stored":427319010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":392031912},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,4,1,0,0,6],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":426242048,"data_stored":426205918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":316895232,"data_stored":316826126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":374390784,"data_stored":374350582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:31:37.241 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:31:37.241 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'13349", 'reported_seq': 13553, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:32.370236+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:31:32.370236+0000', 'last_peered': '2024-09-17T15:31:32.370236+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:31:32.370236+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "34'3300", 'ondisk_log_start': "34'3300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29896704, 'num_objects': 7299, 'num_object_clones': 0, 'num_object_copies': 14598, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4009, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7299, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13349, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 61, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3290}, {'shards': '0,2', 'objects': 1045}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:37.241 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:31:37.241 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'13645", 'reported_seq': 13838, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:32.610381+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:31:32.610381+0000', 'last_peered': '2024-09-17T15:31:32.610381+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:31:32.610381+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "34'3600", 'ondisk_log_start': "34'3600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29851672, 'num_objects': 7289, 'num_object_clones': 0, 'num_object_copies': 14578, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4138, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7289, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11802, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 31, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 992}, {'shards': '2', 'objects': 3152}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:31:37.241 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:31:37.241 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'13876", 'reported_seq': 14133, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:32.447675+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:31:32.447675+0000', 'last_peered': '2024-09-17T15:31:32.447675+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:31:32.447675+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "34'3800", 'ondisk_log_start': "34'3800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30208483, 'num_objects': 7376, 'num_object_clones': 0, 'num_object_copies': 14752, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4212, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7376, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10375, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 59, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3165}, {'shards': '0,2', 'objects': 1020}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:37.241 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:31:37.241 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'13549", 'reported_seq': 13814, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:32.430447+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:31:32.430447+0000', 'last_peered': '2024-09-17T15:31:32.430447+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:31:32.430447+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "34'3500", 'ondisk_log_start': "34'3500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29569990, 'num_objects': 7221, 'num_object_clones': 0, 'num_object_copies': 14442, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4113, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7221, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 10318, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 61, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3108}, {'shards': '0,2', 'objects': 1012}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:37.242 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:31:37.242 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'13328", 'reported_seq': 13553, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:32.327427+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:31:32.327427+0000', 'last_peered': '2024-09-17T15:31:32.327427+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:31:32.327427+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "34'3300", 'ondisk_log_start': "34'3300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29548544, 'num_objects': 7214, 'num_object_clones': 0, 'num_object_copies': 14428, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3967, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7214, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9924, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 45, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3249}, {'shards': '0,2', 'objects': 957}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:37.242 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:31:37.242 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'13339", 'reported_seq': 13516, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:32.612441+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:31:32.612441+0000', 'last_peered': '2024-09-17T15:31:32.612441+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:31:32.612441+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "34'3300", 'ondisk_log_start': "34'3300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29409763, 'num_objects': 7181, 'num_object_clones': 0, 'num_object_copies': 14362, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4052, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7181, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11234, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 26, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1000}, {'shards': '2', 'objects': 3130}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:31:37.242 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:31:37.242 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'13501", 'reported_seq': 15700, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:32.655190+0000', 'last_change': '2024-09-17T15:30:55.086473+0000', 'last_active': '2024-09-17T15:31:32.655190+0000', 'last_peered': '2024-09-17T15:31:32.655190+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:31:32.655190+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "34'3500", 'ondisk_log_start': "34'3500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29634560, 'num_objects': 7235, 'num_object_clones': 0, 'num_object_copies': 14470, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2141, 'num_objects_misplaced': 961, 'num_objects_unfound': 0, 'num_objects_dirty': 7235, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11263, 'num_write_kb': 32532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1037, 'num_bytes_recovered': 4136960, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 961}, {'shards': '2', 'objects': 4133}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:31:37.242 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:31:37.242 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'13644", 'reported_seq': 13906, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:32.400725+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:31:32.400725+0000', 'last_peered': '2024-09-17T15:31:32.400725+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:31:32.400725+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "34'3600", 'ondisk_log_start': "34'3600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29990912, 'num_objects': 7322, 'num_object_clones': 0, 'num_object_copies': 14644, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4078, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7322, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10192, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 61, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3245}, {'shards': '0,2', 'objects': 978}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:37.242 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:31:37.242 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'13561", 'reported_seq': 13799, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:32.383835+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:31:32.383835+0000', 'last_peered': '2024-09-17T15:31:32.383835+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:31:32.383835+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "34'3500", 'ondisk_log_start': "34'3500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29536256, 'num_objects': 7211, 'num_object_clones': 0, 'num_object_copies': 14422, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4114, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7211, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9848, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 46, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3100}, {'shards': '0,2', 'objects': 986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:37.242 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:31:37.243 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'13538", 'reported_seq': 13791, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:32.470813+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:31:32.470813+0000', 'last_peered': '2024-09-17T15:31:32.470813+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:31:32.470813+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "34'3500", 'ondisk_log_start': "34'3500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30097408, 'num_objects': 7348, 'num_object_clones': 0, 'num_object_copies': 14696, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4007, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7348, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10325, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3343}, {'shards': '0,2', 'objects': 970}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:37.243 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:31:37.909 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:31:37.910 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38831/233370 objects degraded (16.639%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":434,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":116685,"data_bytes":477893385,"bytes_used":2131542016,"bytes_avail":287778750464,"bytes_total":289910292480,"degraded_objects":38831,"degraded_total":233370,"degraded_ratio":0.16639242404764965,"misplaced_objects":961,"misplaced_total":233370,"misplaced_ratio":0.0041179243261773146,"recovering_objects_per_sec":27,"recovering_bytes_per_sec":103013,"recovering_keys_per_sec":0,"num_objects_recovered":325,"num_bytes_recovered":1236992,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":368},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-17T15:31:29.695933+0000","services":{}},"progress_events":{}} 2024-09-17T15:31:37.931 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:31:37.931 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:31:38.466 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:31:38.466 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:31:38.487 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":255,"stamp":"2024-09-17T15:31:37.702586+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":468628233,"num_objects":114423,"num_object_clones":0,"num_object_copies":228846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38811,"num_objects_misplaced":818,"num_objects_unfound":0,"num_objects_dirty":114423,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":178568,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9832,"num_bytes_recovered":38220761,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160934,"ondisk_log_size":160934,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2069244,"kb_used_data":1081540,"kb_used_omap":13,"kb_used_meta":987634,"kb_avail":281046276,"statfs":{"total":289910292480,"available":287791386624,"internally_reserved":0,"allocated":1107496960,"data_stored":1106590683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1011338239},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,4,5,4,0,2,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27369472,"num_objects":-6682,"num_object_clones":0,"num_object_copies":-13364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-25,"num_objects_misplaced":-463,"num_objects_unfound":0,"num_objects_dirty":-6682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6682,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":488,"num_bytes_recovered":1822720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008149"},"pg_stats":[{"pgid":"2.f","version":"45'13499","reported_seq":13703,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:37.499804+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:31:37.499804+0000","last_peered":"2024-09-17T15:31:37.499804+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:31:37.499804+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'3400","ondisk_log_start":"34'3400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29282304,"num_objects":7149,"num_object_clones":0,"num_object_copies":14298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4008,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13499,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3141},{"shards":"0,2","objects":895}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13688","reported_seq":13795,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:37.649250+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:31:37.649250+0000","last_peered":"2024-09-17T15:31:37.649250+0000","last_clean":"2024-09-17T15:31:37.649250+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:31:37.649250+0000","last_undegraded":"2024-09-17T15:31:37.649250+0000","last_fullsized":"2024-09-17T15:31:37.649250+0000","mapping_epoch":33,"log_start":"34'3600","ondisk_log_start":"34'3600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29672390,"num_objects":7246,"num_object_clones":0,"num_object_copies":14492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7246,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12045,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13757","reported_seq":13950,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:37.630612+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:31:37.630612+0000","last_peered":"2024-09-17T15:31:37.630612+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:31:37.630612+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'3700","ondisk_log_start":"34'3700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29392920,"num_objects":7177,"num_object_clones":0,"num_object_copies":14354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4138,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7177,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11914,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":880},{"shards":"2","objects":3040}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13679","reported_seq":13753,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:37.486121+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:31:37.486121+0000","last_peered":"2024-09-17T15:31:37.486121+0000","last_clean":"2024-09-17T15:31:37.486121+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:31:37.486121+0000","last_undegraded":"2024-09-17T15:31:37.486121+0000","last_fullsized":"2024-09-17T15:31:37.486121+0000","mapping_epoch":33,"log_start":"34'3600","ondisk_log_start":"34'3600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29863936,"num_objects":7291,"num_object_clones":0,"num_object_copies":14582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7291,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13679,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13525","reported_seq":13659,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:37.465704+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:31:37.465704+0000","last_peered":"2024-09-17T15:31:37.465704+0000","last_clean":"2024-09-17T15:31:37.465704+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:31:37.465704+0000","last_undegraded":"2024-09-17T15:31:37.465704+0000","last_fullsized":"2024-09-17T15:31:37.465704+0000","mapping_epoch":32,"log_start":"34'3500","ondisk_log_start":"34'3500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29196771,"num_objects":7129,"num_object_clones":0,"num_object_copies":14258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7129,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10507,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14031","reported_seq":14288,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:37.421975+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:31:37.421975+0000","last_peered":"2024-09-17T15:31:37.421975+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:31:37.421975+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"37'4000","ondisk_log_start":"37'4000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29573603,"num_objects":7221,"num_object_clones":0,"num_object_copies":14442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4212,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7221,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10530,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3010},{"shards":"0,2","objects":865}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13694","reported_seq":13969,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:37.418035+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:31:37.418035+0000","last_peered":"2024-09-17T15:31:37.418035+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:31:37.418035+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'3600","ondisk_log_start":"34'3600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28976070,"num_objects":7076,"num_object_clones":0,"num_object_copies":14152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7076,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10463,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2968},{"shards":"0,2","objects":867}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13759","reported_seq":13892,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:37.451066+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:31:37.451066+0000","last_peered":"2024-09-17T15:31:37.451066+0000","last_clean":"2024-09-17T15:31:37.451066+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:31:37.451066+0000","last_undegraded":"2024-09-17T15:31:37.451066+0000","last_fullsized":"2024-09-17T15:31:37.451066+0000","mapping_epoch":32,"log_start":"34'3700","ondisk_log_start":"34'3700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29315072,"num_objects":7157,"num_object_clones":0,"num_object_copies":14314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10713,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.086967+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.086967+0000","last_peered":"2024-09-17T15:30:57.086967+0000","last_clean":"2024-09-17T15:30:57.086967+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.086967+0000","last_undegraded":"2024-09-17T15:30:57.086967+0000","last_fullsized":"2024-09-17T15:30:57.086967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13613","reported_seq":21900,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:36.868030+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:31:36.868030+0000","last_peered":"2024-09-17T15:31:36.868030+0000","last_clean":"2024-09-17T15:31:36.868030+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:31:36.868030+0000","last_undegraded":"2024-09-17T15:31:36.868030+0000","last_fullsized":"2024-09-17T15:31:36.868030+0000","mapping_epoch":42,"log_start":"34'3600","ondisk_log_start":"34'3600","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29508067,"num_objects":7205,"num_object_clones":0,"num_object_copies":14410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7205,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11613,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13465","reported_seq":13696,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:37.374200+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:31:37.374200+0000","last_peered":"2024-09-17T15:31:37.374200+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:31:37.374200+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'3400","ondisk_log_start":"34'3400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28987392,"num_objects":7077,"num_object_clones":0,"num_object_copies":14154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10061,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3115},{"shards":"0,2","objects":820}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13488","reported_seq":13673,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:37.651641+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:31:37.651641+0000","last_peered":"2024-09-17T15:31:37.651641+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:31:37.651641+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'3400","ondisk_log_start":"34'3400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28799459,"num_objects":7032,"num_object_clones":0,"num_object_copies":14064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7032,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11383,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":851},{"shards":"2","objects":2985}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13632","reported_seq":16117,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:37.661219+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:31:37.661219+0000","last_peered":"2024-09-17T15:31:37.661219+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:31:37.661219+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'3600","ondisk_log_start":"34'3600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29097984,"num_objects":7104,"num_object_clones":0,"num_object_copies":14208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2141,"num_objects_misplaced":818,"num_objects_unfound":0,"num_objects_dirty":7104,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11394,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1180,"num_bytes_recovered":4722688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":818},{"shards":"2","objects":4145}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13777","reported_seq":14038,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:37.495516+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:31:37.495516+0000","last_peered":"2024-09-17T15:31:37.495516+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:31:37.495516+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'3700","ondisk_log_start":"34'3700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29446144,"num_objects":7189,"num_object_clones":0,"num_object_copies":14378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10325,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3112},{"shards":"0,2","objects":845}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13707","reported_seq":13950,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:37.450311+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:31:37.450311+0000","last_peered":"2024-09-17T15:31:37.450311+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:31:37.450311+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'3700","ondisk_log_start":"34'3700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28938240,"num_objects":7065,"num_object_clones":0,"num_object_copies":14130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9994,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2957},{"shards":"0,2","objects":840}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13662","reported_seq":13923,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:37.384472+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:31:37.384472+0000","last_peered":"2024-09-17T15:31:37.384472+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:31:37.384472+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'3600","ondisk_log_start":"34'3600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29589504,"num_objects":7224,"num_object_clones":0,"num_object_copies":14448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4003,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10449,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3223},{"shards":"0,2","objects":846}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13555","reported_seq":21934,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:36.868509+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:31:36.868509+0000","last_peered":"2024-09-17T15:31:36.868509+0000","last_clean":"2024-09-17T15:31:36.868509+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:31:36.868509+0000","last_undegraded":"2024-09-17T15:31:36.868509+0000","last_fullsized":"2024-09-17T15:31:36.868509+0000","mapping_epoch":44,"log_start":"34'3500","ondisk_log_start":"34'3500","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28988358,"num_objects":7079,"num_object_clones":0,"num_object_copies":14158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9997,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":468628214,"num_objects":114421,"num_object_clones":0,"num_object_copies":228842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38811,"num_objects_misplaced":818,"num_objects_unfound":0,"num_objects_dirty":114421,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":178566,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9831,"num_bytes_recovered":38220742,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1104801792,"data_stored":1104656354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051663,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":668540,"kb_used_data":361960,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93703300,"statfs":{"total":96636764160,"available":95952179200,"internally_reserved":0,"allocated":370647040,"data_stored":370355943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":313913004},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724608,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":615388,"kb_used_data":308812,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93756452,"statfs":{"total":96636764160,"available":96006606848,"internally_reserved":0,"allocated":316223488,"data_stored":315899410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":313913003},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247891,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":785316,"kb_used_data":410768,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93586524,"statfs":{"total":96636764160,"available":95832600576,"internally_reserved":0,"allocated":420626432,"data_stored":420335330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":383512232},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,1,0,2,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":419184640,"data_stored":419148510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":315727872,"data_stored":315658766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":369889280,"data_stored":369849078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:31:41.489 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:31:42.043 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:31:42.046 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:31:42.069 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":257,"stamp":"2024-09-17T15:31:41.704005+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":468628233,"num_objects":114423,"num_object_clones":0,"num_object_copies":228846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38811,"num_objects_misplaced":818,"num_objects_unfound":0,"num_objects_dirty":114423,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":178568,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9832,"num_bytes_recovered":38220761,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160934,"ondisk_log_size":160934,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2069244,"kb_used_data":1081540,"kb_used_omap":13,"kb_used_meta":987634,"kb_avail":281046276,"statfs":{"total":289910292480,"available":287791386624,"internally_reserved":0,"allocated":1107496960,"data_stored":1106590683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1011338239},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,4,5,4,0,2,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18153472,"num_objects":-4432,"num_object_clones":0,"num_object_copies":-8864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-25,"num_objects_misplaced":-310,"num_objects_unfound":0,"num_objects_dirty":-4432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4432,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":335,"num_bytes_recovered":1269760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008103"},"pg_stats":[{"pgid":"2.f","version":"45'13499","reported_seq":13703,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:37.499804+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:31:37.499804+0000","last_peered":"2024-09-17T15:31:37.499804+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:31:37.499804+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'3400","ondisk_log_start":"34'3400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29282304,"num_objects":7149,"num_object_clones":0,"num_object_copies":14298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4008,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13499,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3141},{"shards":"0,2","objects":895}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13688","reported_seq":13795,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:37.649250+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:31:37.649250+0000","last_peered":"2024-09-17T15:31:37.649250+0000","last_clean":"2024-09-17T15:31:37.649250+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:31:37.649250+0000","last_undegraded":"2024-09-17T15:31:37.649250+0000","last_fullsized":"2024-09-17T15:31:37.649250+0000","mapping_epoch":33,"log_start":"34'3600","ondisk_log_start":"34'3600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29672390,"num_objects":7246,"num_object_clones":0,"num_object_copies":14492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7246,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12045,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13757","reported_seq":13950,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:37.630612+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:31:37.630612+0000","last_peered":"2024-09-17T15:31:37.630612+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:31:37.630612+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'3700","ondisk_log_start":"34'3700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29392920,"num_objects":7177,"num_object_clones":0,"num_object_copies":14354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4138,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7177,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11914,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":880},{"shards":"2","objects":3040}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13679","reported_seq":13753,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:37.486121+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:31:37.486121+0000","last_peered":"2024-09-17T15:31:37.486121+0000","last_clean":"2024-09-17T15:31:37.486121+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:31:37.486121+0000","last_undegraded":"2024-09-17T15:31:37.486121+0000","last_fullsized":"2024-09-17T15:31:37.486121+0000","mapping_epoch":33,"log_start":"34'3600","ondisk_log_start":"34'3600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29863936,"num_objects":7291,"num_object_clones":0,"num_object_copies":14582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7291,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13679,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13525","reported_seq":13659,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:37.465704+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:31:37.465704+0000","last_peered":"2024-09-17T15:31:37.465704+0000","last_clean":"2024-09-17T15:31:37.465704+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:31:37.465704+0000","last_undegraded":"2024-09-17T15:31:37.465704+0000","last_fullsized":"2024-09-17T15:31:37.465704+0000","mapping_epoch":32,"log_start":"34'3500","ondisk_log_start":"34'3500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29196771,"num_objects":7129,"num_object_clones":0,"num_object_copies":14258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7129,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10507,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14031","reported_seq":14288,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:37.421975+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:31:37.421975+0000","last_peered":"2024-09-17T15:31:37.421975+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:31:37.421975+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"37'4000","ondisk_log_start":"37'4000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29573603,"num_objects":7221,"num_object_clones":0,"num_object_copies":14442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4212,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7221,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10530,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3010},{"shards":"0,2","objects":865}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13694","reported_seq":13969,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:37.418035+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:31:37.418035+0000","last_peered":"2024-09-17T15:31:37.418035+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:31:37.418035+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'3600","ondisk_log_start":"34'3600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28976070,"num_objects":7076,"num_object_clones":0,"num_object_copies":14152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7076,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10463,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2968},{"shards":"0,2","objects":867}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13759","reported_seq":13892,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:37.451066+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:31:37.451066+0000","last_peered":"2024-09-17T15:31:37.451066+0000","last_clean":"2024-09-17T15:31:37.451066+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:31:37.451066+0000","last_undegraded":"2024-09-17T15:31:37.451066+0000","last_fullsized":"2024-09-17T15:31:37.451066+0000","mapping_epoch":32,"log_start":"34'3700","ondisk_log_start":"34'3700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29315072,"num_objects":7157,"num_object_clones":0,"num_object_copies":14314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10713,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.086967+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.086967+0000","last_peered":"2024-09-17T15:30:57.086967+0000","last_clean":"2024-09-17T15:30:57.086967+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.086967+0000","last_undegraded":"2024-09-17T15:30:57.086967+0000","last_fullsized":"2024-09-17T15:30:57.086967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13613","reported_seq":21900,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:36.868030+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:31:36.868030+0000","last_peered":"2024-09-17T15:31:36.868030+0000","last_clean":"2024-09-17T15:31:36.868030+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:31:36.868030+0000","last_undegraded":"2024-09-17T15:31:36.868030+0000","last_fullsized":"2024-09-17T15:31:36.868030+0000","mapping_epoch":42,"log_start":"34'3600","ondisk_log_start":"34'3600","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29508067,"num_objects":7205,"num_object_clones":0,"num_object_copies":14410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7205,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11613,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13465","reported_seq":13696,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:37.374200+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:31:37.374200+0000","last_peered":"2024-09-17T15:31:37.374200+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:31:37.374200+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'3400","ondisk_log_start":"34'3400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28987392,"num_objects":7077,"num_object_clones":0,"num_object_copies":14154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10061,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3115},{"shards":"0,2","objects":820}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13488","reported_seq":13673,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:37.651641+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:31:37.651641+0000","last_peered":"2024-09-17T15:31:37.651641+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:31:37.651641+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'3400","ondisk_log_start":"34'3400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28799459,"num_objects":7032,"num_object_clones":0,"num_object_copies":14064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7032,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11383,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":851},{"shards":"2","objects":2985}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13632","reported_seq":16117,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:37.661219+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:31:37.661219+0000","last_peered":"2024-09-17T15:31:37.661219+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:31:37.661219+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'3600","ondisk_log_start":"34'3600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29097984,"num_objects":7104,"num_object_clones":0,"num_object_copies":14208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2141,"num_objects_misplaced":818,"num_objects_unfound":0,"num_objects_dirty":7104,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11394,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1180,"num_bytes_recovered":4722688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":818},{"shards":"2","objects":4145}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13777","reported_seq":14038,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:37.495516+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:31:37.495516+0000","last_peered":"2024-09-17T15:31:37.495516+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:31:37.495516+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'3700","ondisk_log_start":"34'3700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29446144,"num_objects":7189,"num_object_clones":0,"num_object_copies":14378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10325,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3112},{"shards":"0,2","objects":845}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13707","reported_seq":13950,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:37.450311+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:31:37.450311+0000","last_peered":"2024-09-17T15:31:37.450311+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:31:37.450311+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'3700","ondisk_log_start":"34'3700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28938240,"num_objects":7065,"num_object_clones":0,"num_object_copies":14130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9994,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2957},{"shards":"0,2","objects":840}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13662","reported_seq":13923,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:37.384472+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:31:37.384472+0000","last_peered":"2024-09-17T15:31:37.384472+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:31:37.384472+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'3600","ondisk_log_start":"34'3600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29589504,"num_objects":7224,"num_object_clones":0,"num_object_copies":14448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4003,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10449,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3223},{"shards":"0,2","objects":846}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13555","reported_seq":21934,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:36.868509+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:31:36.868509+0000","last_peered":"2024-09-17T15:31:36.868509+0000","last_clean":"2024-09-17T15:31:36.868509+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:31:36.868509+0000","last_undegraded":"2024-09-17T15:31:36.868509+0000","last_fullsized":"2024-09-17T15:31:36.868509+0000","mapping_epoch":44,"log_start":"34'3500","ondisk_log_start":"34'3500","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28988358,"num_objects":7079,"num_object_clones":0,"num_object_copies":14158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9997,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":468628214,"num_objects":114421,"num_object_clones":0,"num_object_copies":228842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38811,"num_objects_misplaced":818,"num_objects_unfound":0,"num_objects_dirty":114421,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":178566,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9831,"num_bytes_recovered":38220742,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1104801792,"data_stored":1104656354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051663,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":668540,"kb_used_data":361960,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93703300,"statfs":{"total":96636764160,"available":95952179200,"internally_reserved":0,"allocated":370647040,"data_stored":370355943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":313913004},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724608,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":615388,"kb_used_data":308812,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93756452,"statfs":{"total":96636764160,"available":96006606848,"internally_reserved":0,"allocated":316223488,"data_stored":315899410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":313913003},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247891,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":785316,"kb_used_data":410768,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93586524,"statfs":{"total":96636764160,"available":95832600576,"internally_reserved":0,"allocated":420626432,"data_stored":420335330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":383512232},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,1,0,2,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":419184640,"data_stored":419148510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":315727872,"data_stored":315658766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":369889280,"data_stored":369849078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:31:42.070 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:31:42.608 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:31:42.610 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:31:42.637 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":257,"stamp":"2024-09-17T15:31:41.704005+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":468628233,"num_objects":114423,"num_object_clones":0,"num_object_copies":228846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38811,"num_objects_misplaced":818,"num_objects_unfound":0,"num_objects_dirty":114423,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":178568,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9832,"num_bytes_recovered":38220761,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160934,"ondisk_log_size":160934,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2069244,"kb_used_data":1081540,"kb_used_omap":13,"kb_used_meta":987634,"kb_avail":281046276,"statfs":{"total":289910292480,"available":287791386624,"internally_reserved":0,"allocated":1107496960,"data_stored":1106590683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1011338239},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,4,5,4,0,2,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18153472,"num_objects":-4432,"num_object_clones":0,"num_object_copies":-8864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-25,"num_objects_misplaced":-310,"num_objects_unfound":0,"num_objects_dirty":-4432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4432,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":335,"num_bytes_recovered":1269760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008103"},"pg_stats":[{"pgid":"2.f","version":"45'13499","reported_seq":13703,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:37.499804+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:31:37.499804+0000","last_peered":"2024-09-17T15:31:37.499804+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:31:37.499804+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'3400","ondisk_log_start":"34'3400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29282304,"num_objects":7149,"num_object_clones":0,"num_object_copies":14298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4008,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13499,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3141},{"shards":"0,2","objects":895}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13688","reported_seq":13795,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:37.649250+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:31:37.649250+0000","last_peered":"2024-09-17T15:31:37.649250+0000","last_clean":"2024-09-17T15:31:37.649250+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:31:37.649250+0000","last_undegraded":"2024-09-17T15:31:37.649250+0000","last_fullsized":"2024-09-17T15:31:37.649250+0000","mapping_epoch":33,"log_start":"34'3600","ondisk_log_start":"34'3600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29672390,"num_objects":7246,"num_object_clones":0,"num_object_copies":14492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7246,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12045,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13757","reported_seq":13950,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:37.630612+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:31:37.630612+0000","last_peered":"2024-09-17T15:31:37.630612+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:31:37.630612+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"34'3700","ondisk_log_start":"34'3700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29392920,"num_objects":7177,"num_object_clones":0,"num_object_copies":14354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4138,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7177,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11914,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":880},{"shards":"2","objects":3040}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13679","reported_seq":13753,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:37.486121+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:31:37.486121+0000","last_peered":"2024-09-17T15:31:37.486121+0000","last_clean":"2024-09-17T15:31:37.486121+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:31:37.486121+0000","last_undegraded":"2024-09-17T15:31:37.486121+0000","last_fullsized":"2024-09-17T15:31:37.486121+0000","mapping_epoch":33,"log_start":"34'3600","ondisk_log_start":"34'3600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29863936,"num_objects":7291,"num_object_clones":0,"num_object_copies":14582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7291,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13679,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13525","reported_seq":13659,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:37.465704+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:31:37.465704+0000","last_peered":"2024-09-17T15:31:37.465704+0000","last_clean":"2024-09-17T15:31:37.465704+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:31:37.465704+0000","last_undegraded":"2024-09-17T15:31:37.465704+0000","last_fullsized":"2024-09-17T15:31:37.465704+0000","mapping_epoch":32,"log_start":"34'3500","ondisk_log_start":"34'3500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29196771,"num_objects":7129,"num_object_clones":0,"num_object_copies":14258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7129,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10507,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14031","reported_seq":14288,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:37.421975+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:31:37.421975+0000","last_peered":"2024-09-17T15:31:37.421975+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:31:37.421975+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"37'4000","ondisk_log_start":"37'4000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29573603,"num_objects":7221,"num_object_clones":0,"num_object_copies":14442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4212,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7221,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10530,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3010},{"shards":"0,2","objects":865}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13694","reported_seq":13969,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:37.418035+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:31:37.418035+0000","last_peered":"2024-09-17T15:31:37.418035+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:31:37.418035+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"34'3600","ondisk_log_start":"34'3600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28976070,"num_objects":7076,"num_object_clones":0,"num_object_copies":14152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7076,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10463,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2968},{"shards":"0,2","objects":867}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13759","reported_seq":13892,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:37.451066+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:31:37.451066+0000","last_peered":"2024-09-17T15:31:37.451066+0000","last_clean":"2024-09-17T15:31:37.451066+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:31:37.451066+0000","last_undegraded":"2024-09-17T15:31:37.451066+0000","last_fullsized":"2024-09-17T15:31:37.451066+0000","mapping_epoch":32,"log_start":"34'3700","ondisk_log_start":"34'3700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29315072,"num_objects":7157,"num_object_clones":0,"num_object_copies":14314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10713,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.086967+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.086967+0000","last_peered":"2024-09-17T15:30:57.086967+0000","last_clean":"2024-09-17T15:30:57.086967+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.086967+0000","last_undegraded":"2024-09-17T15:30:57.086967+0000","last_fullsized":"2024-09-17T15:30:57.086967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13613","reported_seq":21900,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:36.868030+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:31:36.868030+0000","last_peered":"2024-09-17T15:31:36.868030+0000","last_clean":"2024-09-17T15:31:36.868030+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:31:36.868030+0000","last_undegraded":"2024-09-17T15:31:36.868030+0000","last_fullsized":"2024-09-17T15:31:36.868030+0000","mapping_epoch":42,"log_start":"34'3600","ondisk_log_start":"34'3600","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29508067,"num_objects":7205,"num_object_clones":0,"num_object_copies":14410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7205,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11613,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13465","reported_seq":13696,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:37.374200+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:31:37.374200+0000","last_peered":"2024-09-17T15:31:37.374200+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:31:37.374200+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'3400","ondisk_log_start":"34'3400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28987392,"num_objects":7077,"num_object_clones":0,"num_object_copies":14154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10061,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3115},{"shards":"0,2","objects":820}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13488","reported_seq":13673,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:37.651641+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:31:37.651641+0000","last_peered":"2024-09-17T15:31:37.651641+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:31:37.651641+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'3400","ondisk_log_start":"34'3400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28799459,"num_objects":7032,"num_object_clones":0,"num_object_copies":14064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7032,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11383,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":851},{"shards":"2","objects":2985}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13632","reported_seq":16117,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:37.661219+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:31:37.661219+0000","last_peered":"2024-09-17T15:31:37.661219+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:31:37.661219+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'3600","ondisk_log_start":"34'3600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29097984,"num_objects":7104,"num_object_clones":0,"num_object_copies":14208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2141,"num_objects_misplaced":818,"num_objects_unfound":0,"num_objects_dirty":7104,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11394,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1180,"num_bytes_recovered":4722688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":818},{"shards":"2","objects":4145}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13777","reported_seq":14038,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:37.495516+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:31:37.495516+0000","last_peered":"2024-09-17T15:31:37.495516+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:31:37.495516+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"34'3700","ondisk_log_start":"34'3700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29446144,"num_objects":7189,"num_object_clones":0,"num_object_copies":14378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10325,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3112},{"shards":"0,2","objects":845}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13707","reported_seq":13950,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:37.450311+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:31:37.450311+0000","last_peered":"2024-09-17T15:31:37.450311+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:31:37.450311+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"34'3700","ondisk_log_start":"34'3700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28938240,"num_objects":7065,"num_object_clones":0,"num_object_copies":14130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9994,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2957},{"shards":"0,2","objects":840}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13662","reported_seq":13923,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:37.384472+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:31:37.384472+0000","last_peered":"2024-09-17T15:31:37.384472+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:31:37.384472+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"34'3600","ondisk_log_start":"34'3600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29589504,"num_objects":7224,"num_object_clones":0,"num_object_copies":14448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4003,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10449,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3223},{"shards":"0,2","objects":846}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13555","reported_seq":21934,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:36.868509+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:31:36.868509+0000","last_peered":"2024-09-17T15:31:36.868509+0000","last_clean":"2024-09-17T15:31:36.868509+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:31:36.868509+0000","last_undegraded":"2024-09-17T15:31:36.868509+0000","last_fullsized":"2024-09-17T15:31:36.868509+0000","mapping_epoch":44,"log_start":"34'3500","ondisk_log_start":"34'3500","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28988358,"num_objects":7079,"num_object_clones":0,"num_object_copies":14158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9997,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":468628214,"num_objects":114421,"num_object_clones":0,"num_object_copies":228842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38811,"num_objects_misplaced":818,"num_objects_unfound":0,"num_objects_dirty":114421,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":178566,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9831,"num_bytes_recovered":38220742,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1104801792,"data_stored":1104656354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051663,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":668540,"kb_used_data":361960,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93703300,"statfs":{"total":96636764160,"available":95952179200,"internally_reserved":0,"allocated":370647040,"data_stored":370355943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":313913004},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724608,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":615388,"kb_used_data":308812,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93756452,"statfs":{"total":96636764160,"available":96006606848,"internally_reserved":0,"allocated":316223488,"data_stored":315899410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":313913003},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247891,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":785316,"kb_used_data":410768,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93586524,"statfs":{"total":96636764160,"available":95832600576,"internally_reserved":0,"allocated":420626432,"data_stored":420335330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":383512232},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,1,0,2,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":419184640,"data_stored":419148510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":315727872,"data_stored":315658766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":369889280,"data_stored":369849078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:31:42.639 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:31:42.639 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'13499", 'reported_seq': 13703, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:37.499804+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:31:37.499804+0000', 'last_peered': '2024-09-17T15:31:37.499804+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:31:37.499804+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "34'3400", 'ondisk_log_start': "34'3400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29282304, 'num_objects': 7149, 'num_object_clones': 0, 'num_object_copies': 14298, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4008, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7149, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13499, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 62, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3141}, {'shards': '0,2', 'objects': 895}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:42.639 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:31:42.639 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'13757", 'reported_seq': 13950, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:37.630612+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:31:37.630612+0000', 'last_peered': '2024-09-17T15:31:37.630612+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:31:37.630612+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "34'3700", 'ondisk_log_start': "34'3700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29392920, 'num_objects': 7177, 'num_object_clones': 0, 'num_object_copies': 14354, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4138, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7177, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11914, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 31, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 880}, {'shards': '2', 'objects': 3040}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:31:42.639 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:31:42.639 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'14031", 'reported_seq': 14288, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:37.421975+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:31:37.421975+0000', 'last_peered': '2024-09-17T15:31:37.421975+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:31:37.421975+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "37'4000", 'ondisk_log_start': "37'4000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29573603, 'num_objects': 7221, 'num_object_clones': 0, 'num_object_copies': 14442, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4212, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7221, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10530, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 59, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3010}, {'shards': '0,2', 'objects': 865}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:42.639 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:31:42.639 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'13694", 'reported_seq': 13969, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:37.418035+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:31:37.418035+0000', 'last_peered': '2024-09-17T15:31:37.418035+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:31:37.418035+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "34'3600", 'ondisk_log_start': "34'3600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28976070, 'num_objects': 7076, 'num_object_clones': 0, 'num_object_copies': 14152, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4108, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7076, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 10463, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2968}, {'shards': '0,2', 'objects': 867}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:42.640 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:31:42.640 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'13465", 'reported_seq': 13696, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:37.374200+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:31:37.374200+0000', 'last_peered': '2024-09-17T15:31:37.374200+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:31:37.374200+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "34'3400", 'ondisk_log_start': "34'3400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28987392, 'num_objects': 7077, 'num_object_clones': 0, 'num_object_copies': 14154, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7077, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10061, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 48, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3115}, {'shards': '0,2', 'objects': 820}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:42.640 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:31:42.640 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'13488", 'reported_seq': 13673, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:37.651641+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:31:37.651641+0000', 'last_peered': '2024-09-17T15:31:37.651641+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:31:37.651641+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "34'3400", 'ondisk_log_start': "34'3400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28799459, 'num_objects': 7032, 'num_object_clones': 0, 'num_object_copies': 14064, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4048, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7032, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11383, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 851}, {'shards': '2', 'objects': 2985}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:31:42.640 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:31:42.640 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'13632", 'reported_seq': 16117, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:37.661219+0000', 'last_change': '2024-09-17T15:30:55.086473+0000', 'last_active': '2024-09-17T15:31:37.661219+0000', 'last_peered': '2024-09-17T15:31:37.661219+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:31:37.661219+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "34'3600", 'ondisk_log_start': "34'3600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29097984, 'num_objects': 7104, 'num_object_clones': 0, 'num_object_copies': 14208, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2141, 'num_objects_misplaced': 818, 'num_objects_unfound': 0, 'num_objects_dirty': 7104, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11394, 'num_write_kb': 32532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1180, 'num_bytes_recovered': 4722688, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 818}, {'shards': '2', 'objects': 4145}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:31:42.640 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:31:42.640 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'13777", 'reported_seq': 14038, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:37.495516+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:31:37.495516+0000', 'last_peered': '2024-09-17T15:31:37.495516+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:31:37.495516+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "34'3700", 'ondisk_log_start': "34'3700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29446144, 'num_objects': 7189, 'num_object_clones': 0, 'num_object_copies': 14378, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4078, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7189, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10325, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 61, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3112}, {'shards': '0,2', 'objects': 845}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:42.640 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:31:42.640 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'13707", 'reported_seq': 13950, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:37.450311+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:31:37.450311+0000', 'last_peered': '2024-09-17T15:31:37.450311+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:31:37.450311+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "34'3700", 'ondisk_log_start': "34'3700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28938240, 'num_objects': 7065, 'num_object_clones': 0, 'num_object_copies': 14130, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4111, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7065, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9994, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 49, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2957}, {'shards': '0,2', 'objects': 840}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:42.641 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:31:42.641 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'13662", 'reported_seq': 13923, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:37.384472+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:31:37.384472+0000', 'last_peered': '2024-09-17T15:31:37.384472+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:31:37.384472+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "34'3600", 'ondisk_log_start': "34'3600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29589504, 'num_objects': 7224, 'num_object_clones': 0, 'num_object_copies': 14448, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4003, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7224, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10449, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 62, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3223}, {'shards': '0,2', 'objects': 846}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:42.641 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:31:43.311 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:31:43.311 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38811/228846 objects degraded (16.959%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":440,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":114423,"data_bytes":468628233,"bytes_used":2118905856,"bytes_avail":287791386624,"bytes_total":289910292480,"degraded_objects":38811,"degraded_total":228846,"degraded_ratio":0.16959439972732754,"misplaced_objects":818,"misplaced_total":228846,"misplaced_ratio":0.0035744561845083593,"recovering_objects_per_sec":27,"recovering_bytes_per_sec":105741,"recovering_keys_per_sec":0,"num_objects_recovered":335,"num_bytes_recovered":1269760,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":369},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-17T15:31:29.695933+0000","services":{}},"progress_events":{}} 2024-09-17T15:31:43.338 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:31:43.338 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:31:43.880 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:31:43.882 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:31:43.908 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":258,"stamp":"2024-09-17T15:31:43.704579+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459346697,"num_objects":112157,"num_object_clones":0,"num_object_copies":224314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38787,"num_objects_misplaced":664,"num_objects_unfound":0,"num_objects_dirty":112157,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":180834,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10010,"num_bytes_recovered":38851545,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160700,"ondisk_log_size":160700,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2074868,"kb_used_data":1069116,"kb_used_omap":13,"kb_used_meta":1005682,"kb_avail":281040652,"statfs":{"total":289910292480,"available":287785627648,"internally_reserved":0,"allocated":1094774784,"data_stored":1093868507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1029819391},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,2,6,10,1,2,0,0,8],"upper_bound":2048},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27435008,"num_objects":-6698,"num_object_clones":0,"num_object_copies":-13396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-49,"num_objects_misplaced":-464,"num_objects_unfound":0,"num_objects_dirty":-6698,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6698,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":513,"num_bytes_recovered":1900544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005473"},"pg_stats":[{"pgid":"2.f","version":"45'13634","reported_seq":13847,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:42.451925+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:31:42.451925+0000","last_peered":"2024-09-17T15:31:42.451925+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:31:42.451925+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'3600","ondisk_log_start":"34'3600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28729344,"num_objects":7014,"num_object_clones":0,"num_object_copies":14028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4004,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13634,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3010},{"shards":"0,2","objects":760}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13829","reported_seq":13936,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:42.669671+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:31:42.669671+0000","last_peered":"2024-09-17T15:31:42.669671+0000","last_clean":"2024-09-17T15:31:42.669671+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:31:42.669671+0000","last_undegraded":"2024-09-17T15:31:42.669671+0000","last_fullsized":"2024-09-17T15:31:42.669671+0000","mapping_epoch":33,"log_start":"37'3800","ondisk_log_start":"37'3800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29094854,"num_objects":7105,"num_object_clones":0,"num_object_copies":14210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7105,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12186,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13910","reported_seq":14113,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:42.600990+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:31:42.600990+0000","last_peered":"2024-09-17T15:31:42.600990+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:31:42.600990+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"37'3900","ondisk_log_start":"37'3900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28766232,"num_objects":7024,"num_object_clones":0,"num_object_copies":14048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4133,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7024,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12067,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":727},{"shards":"2","objects":2892}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13858","reported_seq":13932,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:42.622771+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:31:42.622771+0000","last_peered":"2024-09-17T15:31:42.622771+0000","last_clean":"2024-09-17T15:31:42.622771+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:31:42.622771+0000","last_undegraded":"2024-09-17T15:31:42.622771+0000","last_fullsized":"2024-09-17T15:31:42.622771+0000","mapping_epoch":33,"log_start":"37'3800","ondisk_log_start":"37'3800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29130752,"num_objects":7112,"num_object_clones":0,"num_object_copies":14224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13858,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13632","reported_seq":13766,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:42.514594+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:31:42.514594+0000","last_peered":"2024-09-17T15:31:42.514594+0000","last_clean":"2024-09-17T15:31:42.514594+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:31:42.514594+0000","last_undegraded":"2024-09-17T15:31:42.514594+0000","last_fullsized":"2024-09-17T15:31:42.514594+0000","mapping_epoch":32,"log_start":"34'3600","ondisk_log_start":"34'3600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28758499,"num_objects":7022,"num_object_clones":0,"num_object_copies":14044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7022,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10614,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14180","reported_seq":14437,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:42.481929+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:31:42.481929+0000","last_peered":"2024-09-17T15:31:42.481929+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:31:42.481929+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"37'4100","ondisk_log_start":"37'4100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28963299,"num_objects":7072,"num_object_clones":0,"num_object_copies":14144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4212,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7072,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10679,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2861},{"shards":"0,2","objects":716}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13836","reported_seq":14115,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:42.425355+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:31:42.425355+0000","last_peered":"2024-09-17T15:31:42.425355+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:31:42.425355+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"37'3800","ondisk_log_start":"37'3800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28394438,"num_objects":6934,"num_object_clones":0,"num_object_copies":13868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4106,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6934,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10605,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2828},{"shards":"0,2","objects":725}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13909","reported_seq":14042,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:42.377626+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:31:42.377626+0000","last_peered":"2024-09-17T15:31:42.377626+0000","last_clean":"2024-09-17T15:31:42.377626+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:31:42.377626+0000","last_undegraded":"2024-09-17T15:31:42.377626+0000","last_fullsized":"2024-09-17T15:31:42.377626+0000","mapping_epoch":32,"log_start":"37'3900","ondisk_log_start":"37'3900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28700672,"num_objects":7007,"num_object_clones":0,"num_object_copies":14014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10863,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.086967+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.086967+0000","last_peered":"2024-09-17T15:30:57.086967+0000","last_clean":"2024-09-17T15:30:57.086967+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.086967+0000","last_undegraded":"2024-09-17T15:30:57.086967+0000","last_fullsized":"2024-09-17T15:30:57.086967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13740","reported_seq":22027,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:41.853228+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:31:41.853228+0000","last_peered":"2024-09-17T15:31:41.853228+0000","last_clean":"2024-09-17T15:31:41.853228+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:31:41.853228+0000","last_undegraded":"2024-09-17T15:31:41.853228+0000","last_fullsized":"2024-09-17T15:31:41.853228+0000","mapping_epoch":42,"log_start":"37'3700","ondisk_log_start":"37'3700","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28987875,"num_objects":7078,"num_object_clones":0,"num_object_copies":14156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7078,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11740,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13595","reported_seq":13828,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:42.520196+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:31:42.520196+0000","last_peered":"2024-09-17T15:31:42.520196+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:31:42.520196+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'3500","ondisk_log_start":"34'3500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28454912,"num_objects":6947,"num_object_clones":0,"num_object_copies":13894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3963,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10191,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2986},{"shards":"0,2","objects":690}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13632","reported_seq":13830,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:42.640183+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:31:42.640183+0000","last_peered":"2024-09-17T15:31:42.640183+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:31:42.640183+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'3600","ondisk_log_start":"34'3600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28209635,"num_objects":6888,"num_object_clones":0,"num_object_copies":13776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4042,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6888,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11527,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":707},{"shards":"2","objects":2847}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13772","reported_seq":16565,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:42.680629+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:31:42.680629+0000","last_peered":"2024-09-17T15:31:42.680629+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:31:42.680629+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'3700","ondisk_log_start":"34'3700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28524544,"num_objects":6964,"num_object_clones":0,"num_object_copies":13928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2141,"num_objects_misplaced":664,"num_objects_unfound":0,"num_objects_dirty":6964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11534,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1334,"num_bytes_recovered":5353472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":664},{"shards":"2","objects":4159}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13912","reported_seq":14180,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:42.517767+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:31:42.517767+0000","last_peered":"2024-09-17T15:31:42.517767+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:31:42.517767+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"37'3900","ondisk_log_start":"37'3900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28893184,"num_objects":7054,"num_object_clones":0,"num_object_copies":14108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4075,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7054,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10460,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2980},{"shards":"0,2","objects":710}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13855","reported_seq":14102,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:42.494942+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:31:42.494942+0000","last_peered":"2024-09-17T15:31:42.494942+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:31:42.494942+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"37'3800","ondisk_log_start":"37'3800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28332032,"num_objects":6917,"num_object_clones":0,"num_object_copies":13834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4110,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10142,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2810},{"shards":"0,2","objects":692}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13800","reported_seq":14065,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:42.437134+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:31:42.437134+0000","last_peered":"2024-09-17T15:31:42.437134+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:31:42.437134+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"37'3700","ondisk_log_start":"37'3700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29024256,"num_objects":7086,"num_object_clones":0,"num_object_copies":14172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4001,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10587,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3087},{"shards":"0,2","objects":708}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13703","reported_seq":22082,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:41.869687+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:31:41.869687+0000","last_peered":"2024-09-17T15:31:41.869687+0000","last_clean":"2024-09-17T15:31:41.869687+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:31:41.869687+0000","last_undegraded":"2024-09-17T15:31:41.869687+0000","last_fullsized":"2024-09-17T15:31:41.869687+0000","mapping_epoch":44,"log_start":"37'3700","ondisk_log_start":"37'3700","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28382150,"num_objects":6931,"num_object_clones":0,"num_object_copies":13862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6931,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10145,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":459346678,"num_objects":112155,"num_object_clones":0,"num_object_copies":224310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38787,"num_objects_misplaced":664,"num_objects_unfound":0,"num_objects_dirty":112155,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":180832,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10009,"num_bytes_recovered":38851526,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1092075520,"data_stored":1091930082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160697,"ondisk_log_size":160697,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051664,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":668116,"kb_used_data":357376,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93703724,"statfs":{"total":96636764160,"available":95952613376,"internally_reserved":0,"allocated":365953024,"data_stored":365661927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":318172844},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,3,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724609,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":624060,"kb_used_data":307756,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93747780,"statfs":{"total":96636764160,"available":95997726720,"internally_reserved":0,"allocated":315142144,"data_stored":314818066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":323874475},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247892,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":782692,"kb_used_data":403984,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93589148,"statfs":{"total":96636764160,"available":95835287552,"internally_reserved":0,"allocated":413679616,"data_stored":413388514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":387772072},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,3,3,0,0,0,0,8],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":412217344,"data_stored":412181214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":314654720,"data_stored":314585614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":365203456,"data_stored":365163254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:31:46.911 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:31:47.460 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:31:47.461 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:31:47.490 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2024-09-17T15:31:45.707125+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459346697,"num_objects":112157,"num_object_clones":0,"num_object_copies":224314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38787,"num_objects_misplaced":664,"num_objects_unfound":0,"num_objects_dirty":112157,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":180834,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10010,"num_bytes_recovered":38851545,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160700,"ondisk_log_size":160700,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2074868,"kb_used_data":1069116,"kb_used_omap":13,"kb_used_meta":1005682,"kb_avail":281040652,"statfs":{"total":289910292480,"available":287785627648,"internally_reserved":0,"allocated":1094774784,"data_stored":1093868507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1029819391},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,2,6,10,1,2,0,0,8],"upper_bound":2048},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18546688,"num_objects":-4528,"num_object_clones":0,"num_object_copies":-9056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-44,"num_objects_misplaced":-297,"num_objects_unfound":0,"num_objects_dirty":-4528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4528,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":341,"num_bytes_recovered":1216512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007429"},"pg_stats":[{"pgid":"2.f","version":"45'13634","reported_seq":13847,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:42.451925+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:31:42.451925+0000","last_peered":"2024-09-17T15:31:42.451925+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:31:42.451925+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"34'3600","ondisk_log_start":"34'3600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28729344,"num_objects":7014,"num_object_clones":0,"num_object_copies":14028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4004,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13634,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3010},{"shards":"0,2","objects":760}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13829","reported_seq":13936,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:42.669671+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:31:42.669671+0000","last_peered":"2024-09-17T15:31:42.669671+0000","last_clean":"2024-09-17T15:31:42.669671+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:31:42.669671+0000","last_undegraded":"2024-09-17T15:31:42.669671+0000","last_fullsized":"2024-09-17T15:31:42.669671+0000","mapping_epoch":33,"log_start":"37'3800","ondisk_log_start":"37'3800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29094854,"num_objects":7105,"num_object_clones":0,"num_object_copies":14210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7105,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12186,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13910","reported_seq":14113,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:42.600990+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:31:42.600990+0000","last_peered":"2024-09-17T15:31:42.600990+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:31:42.600990+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"37'3900","ondisk_log_start":"37'3900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28766232,"num_objects":7024,"num_object_clones":0,"num_object_copies":14048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4133,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7024,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12067,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":727},{"shards":"2","objects":2892}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13858","reported_seq":13932,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:42.622771+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:31:42.622771+0000","last_peered":"2024-09-17T15:31:42.622771+0000","last_clean":"2024-09-17T15:31:42.622771+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:31:42.622771+0000","last_undegraded":"2024-09-17T15:31:42.622771+0000","last_fullsized":"2024-09-17T15:31:42.622771+0000","mapping_epoch":33,"log_start":"37'3800","ondisk_log_start":"37'3800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29130752,"num_objects":7112,"num_object_clones":0,"num_object_copies":14224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13858,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13632","reported_seq":13766,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:42.514594+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:31:42.514594+0000","last_peered":"2024-09-17T15:31:42.514594+0000","last_clean":"2024-09-17T15:31:42.514594+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:31:42.514594+0000","last_undegraded":"2024-09-17T15:31:42.514594+0000","last_fullsized":"2024-09-17T15:31:42.514594+0000","mapping_epoch":32,"log_start":"34'3600","ondisk_log_start":"34'3600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28758499,"num_objects":7022,"num_object_clones":0,"num_object_copies":14044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7022,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10614,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14180","reported_seq":14437,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:42.481929+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:31:42.481929+0000","last_peered":"2024-09-17T15:31:42.481929+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:31:42.481929+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"37'4100","ondisk_log_start":"37'4100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28963299,"num_objects":7072,"num_object_clones":0,"num_object_copies":14144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4212,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7072,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10679,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2861},{"shards":"0,2","objects":716}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13836","reported_seq":14115,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:42.425355+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:31:42.425355+0000","last_peered":"2024-09-17T15:31:42.425355+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:31:42.425355+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"37'3800","ondisk_log_start":"37'3800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28394438,"num_objects":6934,"num_object_clones":0,"num_object_copies":13868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4106,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6934,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10605,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2828},{"shards":"0,2","objects":725}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13909","reported_seq":14042,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:42.377626+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:31:42.377626+0000","last_peered":"2024-09-17T15:31:42.377626+0000","last_clean":"2024-09-17T15:31:42.377626+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:31:42.377626+0000","last_undegraded":"2024-09-17T15:31:42.377626+0000","last_fullsized":"2024-09-17T15:31:42.377626+0000","mapping_epoch":32,"log_start":"37'3900","ondisk_log_start":"37'3900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28700672,"num_objects":7007,"num_object_clones":0,"num_object_copies":14014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10863,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.086967+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.086967+0000","last_peered":"2024-09-17T15:30:57.086967+0000","last_clean":"2024-09-17T15:30:57.086967+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.086967+0000","last_undegraded":"2024-09-17T15:30:57.086967+0000","last_fullsized":"2024-09-17T15:30:57.086967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13740","reported_seq":22027,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:41.853228+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:31:41.853228+0000","last_peered":"2024-09-17T15:31:41.853228+0000","last_clean":"2024-09-17T15:31:41.853228+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:31:41.853228+0000","last_undegraded":"2024-09-17T15:31:41.853228+0000","last_fullsized":"2024-09-17T15:31:41.853228+0000","mapping_epoch":42,"log_start":"37'3700","ondisk_log_start":"37'3700","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28987875,"num_objects":7078,"num_object_clones":0,"num_object_copies":14156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7078,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11740,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13595","reported_seq":13828,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:42.520196+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:31:42.520196+0000","last_peered":"2024-09-17T15:31:42.520196+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:31:42.520196+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"34'3500","ondisk_log_start":"34'3500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28454912,"num_objects":6947,"num_object_clones":0,"num_object_copies":13894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3963,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10191,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2986},{"shards":"0,2","objects":690}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13632","reported_seq":13830,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:42.640183+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:31:42.640183+0000","last_peered":"2024-09-17T15:31:42.640183+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:31:42.640183+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'3600","ondisk_log_start":"34'3600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28209635,"num_objects":6888,"num_object_clones":0,"num_object_copies":13776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4042,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6888,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11527,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":707},{"shards":"2","objects":2847}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13772","reported_seq":16565,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:42.680629+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:31:42.680629+0000","last_peered":"2024-09-17T15:31:42.680629+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:31:42.680629+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'3700","ondisk_log_start":"34'3700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28524544,"num_objects":6964,"num_object_clones":0,"num_object_copies":13928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2141,"num_objects_misplaced":664,"num_objects_unfound":0,"num_objects_dirty":6964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11534,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1334,"num_bytes_recovered":5353472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":664},{"shards":"2","objects":4159}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13912","reported_seq":14180,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:42.517767+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:31:42.517767+0000","last_peered":"2024-09-17T15:31:42.517767+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:31:42.517767+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"37'3900","ondisk_log_start":"37'3900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28893184,"num_objects":7054,"num_object_clones":0,"num_object_copies":14108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4075,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7054,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10460,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2980},{"shards":"0,2","objects":710}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13855","reported_seq":14102,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:42.494942+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:31:42.494942+0000","last_peered":"2024-09-17T15:31:42.494942+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:31:42.494942+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"37'3800","ondisk_log_start":"37'3800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28332032,"num_objects":6917,"num_object_clones":0,"num_object_copies":13834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4110,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10142,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2810},{"shards":"0,2","objects":692}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13800","reported_seq":14065,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:42.437134+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:31:42.437134+0000","last_peered":"2024-09-17T15:31:42.437134+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:31:42.437134+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"37'3700","ondisk_log_start":"37'3700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29024256,"num_objects":7086,"num_object_clones":0,"num_object_copies":14172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4001,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10587,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3087},{"shards":"0,2","objects":708}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13703","reported_seq":22082,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:41.869687+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:31:41.869687+0000","last_peered":"2024-09-17T15:31:41.869687+0000","last_clean":"2024-09-17T15:31:41.869687+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:31:41.869687+0000","last_undegraded":"2024-09-17T15:31:41.869687+0000","last_fullsized":"2024-09-17T15:31:41.869687+0000","mapping_epoch":44,"log_start":"37'3700","ondisk_log_start":"37'3700","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28382150,"num_objects":6931,"num_object_clones":0,"num_object_copies":13862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6931,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10145,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":459346678,"num_objects":112155,"num_object_clones":0,"num_object_copies":224310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38787,"num_objects_misplaced":664,"num_objects_unfound":0,"num_objects_dirty":112155,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":180832,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10009,"num_bytes_recovered":38851526,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1092075520,"data_stored":1091930082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160697,"ondisk_log_size":160697,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051664,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":668116,"kb_used_data":357376,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93703724,"statfs":{"total":96636764160,"available":95952613376,"internally_reserved":0,"allocated":365953024,"data_stored":365661927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":318172844},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,3,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724609,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":624060,"kb_used_data":307756,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93747780,"statfs":{"total":96636764160,"available":95997726720,"internally_reserved":0,"allocated":315142144,"data_stored":314818066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":323874475},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247892,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":782692,"kb_used_data":403984,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93589148,"statfs":{"total":96636764160,"available":95835287552,"internally_reserved":0,"allocated":413679616,"data_stored":413388514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":387772072},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,3,3,0,0,0,0,8],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":412217344,"data_stored":412181214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":314654720,"data_stored":314585614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":365203456,"data_stored":365163254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:31:47.492 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:31:48.068 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:31:48.070 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:31:48.089 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":260,"stamp":"2024-09-17T15:31:47.709000+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":452899593,"num_objects":110583,"num_object_clones":0,"num_object_copies":221166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38760,"num_objects_misplaced":664,"num_objects_unfound":0,"num_objects_dirty":110583,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":182408,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10037,"num_bytes_recovered":38851545,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160874,"ondisk_log_size":160874,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2075348,"kb_used_data":1061280,"kb_used_omap":13,"kb_used_meta":1014002,"kb_avail":281040172,"statfs":{"total":289910292480,"available":287785136128,"internally_reserved":0,"allocated":1086750720,"data_stored":1085844443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1038339071},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,1,8,9,7,2,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24993792,"num_objects":-6102,"num_object_clones":0,"num_object_copies":-12204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-71,"num_objects_misplaced":-297,"num_objects_unfound":0,"num_objects_dirty":-6102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6102,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":368,"num_bytes_recovered":1216512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007117"},"pg_stats":[{"pgid":"2.f","version":"45'13789","reported_seq":14017,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:47.514503+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:31:47.514503+0000","last_peered":"2024-09-17T15:31:47.514503+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:31:47.514503+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"37'3700","ondisk_log_start":"37'3700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28094464,"num_objects":6859,"num_object_clones":0,"num_object_copies":13718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3997,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13789,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2862},{"shards":"0,2","objects":605}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13829","reported_seq":13936,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:42.669671+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:31:42.669671+0000","last_peered":"2024-09-17T15:31:42.669671+0000","last_clean":"2024-09-17T15:31:42.669671+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:31:42.669671+0000","last_undegraded":"2024-09-17T15:31:42.669671+0000","last_fullsized":"2024-09-17T15:31:42.669671+0000","mapping_epoch":33,"log_start":"37'3800","ondisk_log_start":"37'3800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29094854,"num_objects":7105,"num_object_clones":0,"num_object_copies":14210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7105,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12186,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13910","reported_seq":14113,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:42.600990+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:31:42.600990+0000","last_peered":"2024-09-17T15:31:42.600990+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:31:42.600990+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"37'3900","ondisk_log_start":"37'3900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28766232,"num_objects":7024,"num_object_clones":0,"num_object_copies":14048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4133,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7024,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12067,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":727},{"shards":"2","objects":2892}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13858","reported_seq":13932,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:42.622771+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:31:42.622771+0000","last_peered":"2024-09-17T15:31:42.622771+0000","last_clean":"2024-09-17T15:31:42.622771+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:31:42.622771+0000","last_undegraded":"2024-09-17T15:31:42.622771+0000","last_fullsized":"2024-09-17T15:31:42.622771+0000","mapping_epoch":33,"log_start":"37'3800","ondisk_log_start":"37'3800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29130752,"num_objects":7112,"num_object_clones":0,"num_object_copies":14224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13858,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13783","reported_seq":13917,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:47.520489+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:31:47.520489+0000","last_peered":"2024-09-17T15:31:47.520489+0000","last_clean":"2024-09-17T15:31:47.520489+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:31:47.520489+0000","last_undegraded":"2024-09-17T15:31:47.520489+0000","last_fullsized":"2024-09-17T15:31:47.520489+0000","mapping_epoch":32,"log_start":"37'3700","ondisk_log_start":"37'3700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28140003,"num_objects":6871,"num_object_clones":0,"num_object_copies":13742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6871,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10765,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14322","reported_seq":14586,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:47.546042+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:31:47.546042+0000","last_peered":"2024-09-17T15:31:47.546042+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:31:47.546042+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'4300","ondisk_log_start":"41'4300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28381667,"num_objects":6930,"num_object_clones":0,"num_object_copies":13860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6930,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10821,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2723},{"shards":"0,2","objects":574}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13987","reported_seq":14269,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:47.542501+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:31:47.542501+0000","last_peered":"2024-09-17T15:31:47.542501+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:31:47.542501+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"37'3900","ondisk_log_start":"37'3900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27775942,"num_objects":6783,"num_object_clones":0,"num_object_copies":13566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4105,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6783,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10756,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2678},{"shards":"0,2","objects":574}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14042","reported_seq":14175,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:47.520366+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:31:47.520366+0000","last_peered":"2024-09-17T15:31:47.520366+0000","last_clean":"2024-09-17T15:31:47.520366+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:31:47.520366+0000","last_undegraded":"2024-09-17T15:31:47.520366+0000","last_fullsized":"2024-09-17T15:31:47.520366+0000","mapping_epoch":32,"log_start":"37'4000","ondisk_log_start":"37'4000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28155904,"num_objects":6874,"num_object_clones":0,"num_object_copies":13748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10996,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.086967+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.086967+0000","last_peered":"2024-09-17T15:30:57.086967+0000","last_clean":"2024-09-17T15:30:57.086967+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.086967+0000","last_undegraded":"2024-09-17T15:30:57.086967+0000","last_fullsized":"2024-09-17T15:30:57.086967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13884","reported_seq":22171,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:46.896719+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:31:46.896719+0000","last_peered":"2024-09-17T15:31:46.896719+0000","last_clean":"2024-09-17T15:31:46.896719+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:31:46.896719+0000","last_undegraded":"2024-09-17T15:31:46.896719+0000","last_fullsized":"2024-09-17T15:31:46.896719+0000","mapping_epoch":42,"log_start":"37'3800","ondisk_log_start":"37'3800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28398051,"num_objects":6934,"num_object_clones":0,"num_object_copies":13868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6934,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11884,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13732","reported_seq":13973,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:47.542524+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:31:47.542524+0000","last_peered":"2024-09-17T15:31:47.542524+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:31:47.542524+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"37'3700","ondisk_log_start":"37'3700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27893760,"num_objects":6810,"num_object_clones":0,"num_object_copies":13620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3959,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10328,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2853},{"shards":"0,2","objects":553}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13632","reported_seq":13830,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:42.640183+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:31:42.640183+0000","last_peered":"2024-09-17T15:31:42.640183+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:31:42.640183+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'3600","ondisk_log_start":"34'3600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28209635,"num_objects":6888,"num_object_clones":0,"num_object_copies":13776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4042,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6888,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11527,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":707},{"shards":"2","objects":2847}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13772","reported_seq":16565,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:42.680629+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:31:42.680629+0000","last_peered":"2024-09-17T15:31:42.680629+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:31:42.680629+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'3700","ondisk_log_start":"34'3700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28524544,"num_objects":6964,"num_object_clones":0,"num_object_copies":13928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2141,"num_objects_misplaced":664,"num_objects_unfound":0,"num_objects_dirty":6964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11534,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1334,"num_bytes_recovered":5353472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":664},{"shards":"2","objects":4159}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14059","reported_seq":14331,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:47.544705+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:31:47.544705+0000","last_peered":"2024-09-17T15:31:47.544705+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:31:47.544705+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"37'4000","ondisk_log_start":"37'4000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28291072,"num_objects":6907,"num_object_clones":0,"num_object_copies":13814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4073,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10607,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2835},{"shards":"0,2","objects":563}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13997","reported_seq":14248,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:47.349453+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:31:47.349453+0000","last_peered":"2024-09-17T15:31:47.349453+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:31:47.349453+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"37'3900","ondisk_log_start":"37'3900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27750400,"num_objects":6775,"num_object_clones":0,"num_object_copies":13550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10284,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2670},{"shards":"0,2","objects":550}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13934","reported_seq":14213,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:47.512919+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:31:47.512919+0000","last_peered":"2024-09-17T15:31:47.512919+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:31:47.512919+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"37'3900","ondisk_log_start":"37'3900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28475392,"num_objects":6952,"num_object_clones":0,"num_object_copies":13904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3994,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10721,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":71,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2960},{"shards":"0,2","objects":574}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13841","reported_seq":22220,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:46.872797+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:31:46.872797+0000","last_peered":"2024-09-17T15:31:46.872797+0000","last_clean":"2024-09-17T15:31:46.872797+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:31:46.872797+0000","last_undegraded":"2024-09-17T15:31:46.872797+0000","last_fullsized":"2024-09-17T15:31:46.872797+0000","mapping_epoch":44,"log_start":"37'3800","ondisk_log_start":"37'3800","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27816902,"num_objects":6793,"num_object_clones":0,"num_object_copies":13586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10283,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":452899574,"num_objects":110581,"num_object_clones":0,"num_object_copies":221162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38760,"num_objects_misplaced":664,"num_objects_unfound":0,"num_objects_dirty":110581,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":182406,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10036,"num_bytes_recovered":38851526,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1083957248,"data_stored":1083811810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051664,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":668116,"kb_used_data":357376,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93703724,"statfs":{"total":96636764160,"available":95952613376,"internally_reserved":0,"allocated":365953024,"data_stored":365661927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":318172844},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,3,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724610,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627188,"kb_used_data":306724,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93744652,"statfs":{"total":96636764160,"available":95994523648,"internally_reserved":0,"allocated":314085376,"data_stored":313761298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":328134315},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247893,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":780044,"kb_used_data":397180,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93591796,"statfs":{"total":96636764160,"available":95837999104,"internally_reserved":0,"allocated":406712320,"data_stored":406421218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":392031912},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,3,4,4,0,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":405233664,"data_stored":405197534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":313520128,"data_stored":313451022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":365203456,"data_stored":365163254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:31:48.090 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:31:48.091 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'13789", 'reported_seq': 14017, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:47.514503+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:31:47.514503+0000', 'last_peered': '2024-09-17T15:31:47.514503+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:31:47.514503+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "37'3700", 'ondisk_log_start': "37'3700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28094464, 'num_objects': 6859, 'num_object_clones': 0, 'num_object_copies': 13718, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3997, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6859, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13789, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 73, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2862}, {'shards': '0,2', 'objects': 605}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:48.091 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:31:48.091 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'13910", 'reported_seq': 14113, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:42.600990+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:31:42.600990+0000', 'last_peered': '2024-09-17T15:31:42.600990+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:31:42.600990+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "37'3900", 'ondisk_log_start': "37'3900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28766232, 'num_objects': 7024, 'num_object_clones': 0, 'num_object_copies': 14048, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4133, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7024, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12067, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 36, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 727}, {'shards': '2', 'objects': 2892}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:31:48.091 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:31:48.091 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'14322", 'reported_seq': 14586, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:47.546042+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:31:47.546042+0000', 'last_peered': '2024-09-17T15:31:47.546042+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:31:47.546042+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "41'4300", 'ondisk_log_start': "41'4300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28381667, 'num_objects': 6930, 'num_object_clones': 0, 'num_object_copies': 13860, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4208, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6930, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10821, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 63, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2723}, {'shards': '0,2', 'objects': 574}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:48.091 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:31:48.091 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'13987", 'reported_seq': 14269, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:47.542501+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:31:47.542501+0000', 'last_peered': '2024-09-17T15:31:47.542501+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:31:47.542501+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "37'3900", 'ondisk_log_start': "37'3900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27775942, 'num_objects': 6783, 'num_object_clones': 0, 'num_object_copies': 13566, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4105, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6783, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 10756, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2678}, {'shards': '0,2', 'objects': 574}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:48.091 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:31:48.092 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'13732", 'reported_seq': 13973, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:47.542524+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:31:47.542524+0000', 'last_peered': '2024-09-17T15:31:47.542524+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:31:47.542524+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "37'3700", 'ondisk_log_start': "37'3700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27893760, 'num_objects': 6810, 'num_object_clones': 0, 'num_object_copies': 13620, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3959, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6810, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10328, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2853}, {'shards': '0,2', 'objects': 553}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:48.092 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:31:48.092 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'13632", 'reported_seq': 13830, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:42.640183+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:31:42.640183+0000', 'last_peered': '2024-09-17T15:31:42.640183+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:31:42.640183+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "34'3600", 'ondisk_log_start': "34'3600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28209635, 'num_objects': 6888, 'num_object_clones': 0, 'num_object_copies': 13776, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4042, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6888, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11527, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 36, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 707}, {'shards': '2', 'objects': 2847}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:31:48.092 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:31:48.092 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'13772", 'reported_seq': 16565, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:42.680629+0000', 'last_change': '2024-09-17T15:30:55.086473+0000', 'last_active': '2024-09-17T15:31:42.680629+0000', 'last_peered': '2024-09-17T15:31:42.680629+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:31:42.680629+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "34'3700", 'ondisk_log_start': "34'3700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28524544, 'num_objects': 6964, 'num_object_clones': 0, 'num_object_copies': 13928, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2141, 'num_objects_misplaced': 664, 'num_objects_unfound': 0, 'num_objects_dirty': 6964, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11534, 'num_write_kb': 32532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1334, 'num_bytes_recovered': 5353472, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 664}, {'shards': '2', 'objects': 4159}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:31:48.092 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:31:48.092 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'14059", 'reported_seq': 14331, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:47.544705+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:31:47.544705+0000', 'last_peered': '2024-09-17T15:31:47.544705+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:31:47.544705+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "37'4000", 'ondisk_log_start': "37'4000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28291072, 'num_objects': 6907, 'num_object_clones': 0, 'num_object_copies': 13814, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4073, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6907, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10607, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2835}, {'shards': '0,2', 'objects': 563}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:48.092 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:31:48.092 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'13997", 'reported_seq': 14248, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:47.349453+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:31:47.349453+0000', 'last_peered': '2024-09-17T15:31:47.349453+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:31:47.349453+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "37'3900", 'ondisk_log_start': "37'3900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27750400, 'num_objects': 6775, 'num_object_clones': 0, 'num_object_copies': 13550, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4108, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6775, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10284, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2670}, {'shards': '0,2', 'objects': 550}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:48.092 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:31:48.093 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'13934", 'reported_seq': 14213, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:47.512919+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:31:47.512919+0000', 'last_peered': '2024-09-17T15:31:47.512919+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:31:47.512919+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "37'3900", 'ondisk_log_start': "37'3900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28475392, 'num_objects': 6952, 'num_object_clones': 0, 'num_object_copies': 13904, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3994, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6952, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10721, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 71, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2960}, {'shards': '0,2', 'objects': 574}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:48.093 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:31:48.795 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:31:48.795 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38760/221166 objects degraded (17.525%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":445,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":110583,"data_bytes":452899593,"bytes_used":2125156352,"bytes_avail":287785136128,"bytes_total":289910292480,"degraded_objects":38760,"degraded_total":221166,"degraded_ratio":0.17525297740158977,"misplaced_objects":664,"misplaced_total":221166,"misplaced_ratio":0.0030022697883038078,"recovering_objects_per_sec":30,"recovering_bytes_per_sec":101315,"recovering_keys_per_sec":0,"num_objects_recovered":368,"num_bytes_recovered":1216512,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":508},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-17T15:31:29.695933+0000","services":{}},"progress_events":{}} 2024-09-17T15:31:48.816 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:31:48.816 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:31:49.399 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:31:49.399 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:31:49.433 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":260,"stamp":"2024-09-17T15:31:47.709000+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":452899593,"num_objects":110583,"num_object_clones":0,"num_object_copies":221166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38760,"num_objects_misplaced":664,"num_objects_unfound":0,"num_objects_dirty":110583,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":182408,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10037,"num_bytes_recovered":38851545,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160874,"ondisk_log_size":160874,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2075348,"kb_used_data":1061280,"kb_used_omap":13,"kb_used_meta":1014002,"kb_avail":281040172,"statfs":{"total":289910292480,"available":287785136128,"internally_reserved":0,"allocated":1086750720,"data_stored":1085844443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1038339071},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,1,8,9,7,2,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24993792,"num_objects":-6102,"num_object_clones":0,"num_object_copies":-12204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-71,"num_objects_misplaced":-297,"num_objects_unfound":0,"num_objects_dirty":-6102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6102,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":368,"num_bytes_recovered":1216512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007117"},"pg_stats":[{"pgid":"2.f","version":"45'13789","reported_seq":14017,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:47.514503+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:31:47.514503+0000","last_peered":"2024-09-17T15:31:47.514503+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:31:47.514503+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"37'3700","ondisk_log_start":"37'3700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28094464,"num_objects":6859,"num_object_clones":0,"num_object_copies":13718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3997,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13789,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2862},{"shards":"0,2","objects":605}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13829","reported_seq":13936,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:42.669671+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:31:42.669671+0000","last_peered":"2024-09-17T15:31:42.669671+0000","last_clean":"2024-09-17T15:31:42.669671+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:31:42.669671+0000","last_undegraded":"2024-09-17T15:31:42.669671+0000","last_fullsized":"2024-09-17T15:31:42.669671+0000","mapping_epoch":33,"log_start":"37'3800","ondisk_log_start":"37'3800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29094854,"num_objects":7105,"num_object_clones":0,"num_object_copies":14210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7105,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12186,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13910","reported_seq":14113,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:42.600990+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:31:42.600990+0000","last_peered":"2024-09-17T15:31:42.600990+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:31:42.600990+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"37'3900","ondisk_log_start":"37'3900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28766232,"num_objects":7024,"num_object_clones":0,"num_object_copies":14048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4133,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7024,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12067,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":727},{"shards":"2","objects":2892}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13858","reported_seq":13932,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:42.622771+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:31:42.622771+0000","last_peered":"2024-09-17T15:31:42.622771+0000","last_clean":"2024-09-17T15:31:42.622771+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:31:42.622771+0000","last_undegraded":"2024-09-17T15:31:42.622771+0000","last_fullsized":"2024-09-17T15:31:42.622771+0000","mapping_epoch":33,"log_start":"37'3800","ondisk_log_start":"37'3800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29130752,"num_objects":7112,"num_object_clones":0,"num_object_copies":14224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13858,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13783","reported_seq":13917,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:47.520489+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:31:47.520489+0000","last_peered":"2024-09-17T15:31:47.520489+0000","last_clean":"2024-09-17T15:31:47.520489+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:31:47.520489+0000","last_undegraded":"2024-09-17T15:31:47.520489+0000","last_fullsized":"2024-09-17T15:31:47.520489+0000","mapping_epoch":32,"log_start":"37'3700","ondisk_log_start":"37'3700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28140003,"num_objects":6871,"num_object_clones":0,"num_object_copies":13742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6871,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10765,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14322","reported_seq":14586,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:47.546042+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:31:47.546042+0000","last_peered":"2024-09-17T15:31:47.546042+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:31:47.546042+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'4300","ondisk_log_start":"41'4300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28381667,"num_objects":6930,"num_object_clones":0,"num_object_copies":13860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6930,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10821,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2723},{"shards":"0,2","objects":574}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13987","reported_seq":14269,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:47.542501+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:31:47.542501+0000","last_peered":"2024-09-17T15:31:47.542501+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:31:47.542501+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"37'3900","ondisk_log_start":"37'3900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27775942,"num_objects":6783,"num_object_clones":0,"num_object_copies":13566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4105,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6783,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10756,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2678},{"shards":"0,2","objects":574}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14042","reported_seq":14175,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:47.520366+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:31:47.520366+0000","last_peered":"2024-09-17T15:31:47.520366+0000","last_clean":"2024-09-17T15:31:47.520366+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:31:47.520366+0000","last_undegraded":"2024-09-17T15:31:47.520366+0000","last_fullsized":"2024-09-17T15:31:47.520366+0000","mapping_epoch":32,"log_start":"37'4000","ondisk_log_start":"37'4000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28155904,"num_objects":6874,"num_object_clones":0,"num_object_copies":13748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10996,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.086967+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.086967+0000","last_peered":"2024-09-17T15:30:57.086967+0000","last_clean":"2024-09-17T15:30:57.086967+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.086967+0000","last_undegraded":"2024-09-17T15:30:57.086967+0000","last_fullsized":"2024-09-17T15:30:57.086967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13884","reported_seq":22171,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:46.896719+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:31:46.896719+0000","last_peered":"2024-09-17T15:31:46.896719+0000","last_clean":"2024-09-17T15:31:46.896719+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:31:46.896719+0000","last_undegraded":"2024-09-17T15:31:46.896719+0000","last_fullsized":"2024-09-17T15:31:46.896719+0000","mapping_epoch":42,"log_start":"37'3800","ondisk_log_start":"37'3800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28398051,"num_objects":6934,"num_object_clones":0,"num_object_copies":13868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6934,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11884,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13732","reported_seq":13973,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:47.542524+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:31:47.542524+0000","last_peered":"2024-09-17T15:31:47.542524+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:31:47.542524+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"37'3700","ondisk_log_start":"37'3700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27893760,"num_objects":6810,"num_object_clones":0,"num_object_copies":13620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3959,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10328,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2853},{"shards":"0,2","objects":553}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13632","reported_seq":13830,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:42.640183+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:31:42.640183+0000","last_peered":"2024-09-17T15:31:42.640183+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:31:42.640183+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"34'3600","ondisk_log_start":"34'3600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28209635,"num_objects":6888,"num_object_clones":0,"num_object_copies":13776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4042,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6888,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11527,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":707},{"shards":"2","objects":2847}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13772","reported_seq":16565,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:42.680629+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:31:42.680629+0000","last_peered":"2024-09-17T15:31:42.680629+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:31:42.680629+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"34'3700","ondisk_log_start":"34'3700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28524544,"num_objects":6964,"num_object_clones":0,"num_object_copies":13928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2141,"num_objects_misplaced":664,"num_objects_unfound":0,"num_objects_dirty":6964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11534,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1334,"num_bytes_recovered":5353472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":664},{"shards":"2","objects":4159}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14059","reported_seq":14331,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:47.544705+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:31:47.544705+0000","last_peered":"2024-09-17T15:31:47.544705+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:31:47.544705+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"37'4000","ondisk_log_start":"37'4000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28291072,"num_objects":6907,"num_object_clones":0,"num_object_copies":13814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4073,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10607,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2835},{"shards":"0,2","objects":563}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13997","reported_seq":14248,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:47.349453+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:31:47.349453+0000","last_peered":"2024-09-17T15:31:47.349453+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:31:47.349453+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"37'3900","ondisk_log_start":"37'3900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27750400,"num_objects":6775,"num_object_clones":0,"num_object_copies":13550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10284,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2670},{"shards":"0,2","objects":550}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13934","reported_seq":14213,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:47.512919+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:31:47.512919+0000","last_peered":"2024-09-17T15:31:47.512919+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:31:47.512919+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"37'3900","ondisk_log_start":"37'3900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28475392,"num_objects":6952,"num_object_clones":0,"num_object_copies":13904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3994,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10721,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":71,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2960},{"shards":"0,2","objects":574}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13841","reported_seq":22220,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:46.872797+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:31:46.872797+0000","last_peered":"2024-09-17T15:31:46.872797+0000","last_clean":"2024-09-17T15:31:46.872797+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:31:46.872797+0000","last_undegraded":"2024-09-17T15:31:46.872797+0000","last_fullsized":"2024-09-17T15:31:46.872797+0000","mapping_epoch":44,"log_start":"37'3800","ondisk_log_start":"37'3800","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27816902,"num_objects":6793,"num_object_clones":0,"num_object_copies":13586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10283,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":452899574,"num_objects":110581,"num_object_clones":0,"num_object_copies":221162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38760,"num_objects_misplaced":664,"num_objects_unfound":0,"num_objects_dirty":110581,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":182406,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10036,"num_bytes_recovered":38851526,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1083957248,"data_stored":1083811810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051664,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":668116,"kb_used_data":357376,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93703724,"statfs":{"total":96636764160,"available":95952613376,"internally_reserved":0,"allocated":365953024,"data_stored":365661927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":318172844},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,3,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724610,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627188,"kb_used_data":306724,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93744652,"statfs":{"total":96636764160,"available":95994523648,"internally_reserved":0,"allocated":314085376,"data_stored":313761298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":328134315},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247893,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":780044,"kb_used_data":397180,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93591796,"statfs":{"total":96636764160,"available":95837999104,"internally_reserved":0,"allocated":406712320,"data_stored":406421218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":392031912},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,3,4,4,0,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":405233664,"data_stored":405197534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":313520128,"data_stored":313451022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":365203456,"data_stored":365163254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:31:52.435 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:31:52.993 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:31:52.993 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:31:53.023 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":262,"stamp":"2024-09-17T15:31:51.709973+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":449979145,"num_objects":109870,"num_object_clones":0,"num_object_copies":219740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38758,"num_objects_misplaced":520,"num_objects_unfound":0,"num_objects_dirty":109870,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":183121,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10183,"num_bytes_recovered":39441369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160987,"ondisk_log_size":160987,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2076312,"kb_used_data":1056680,"kb_used_omap":13,"kb_used_meta":1019570,"kb_avail":281039208,"statfs":{"total":289910292480,"available":287784148992,"internally_reserved":0,"allocated":1082040320,"data_stored":1081134043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1044040703},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,4,4,5,6,6,0,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18649088,"num_objects":-4553,"num_object_clones":0,"num_object_copies":-9106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-53,"num_objects_misplaced":-298,"num_objects_unfound":0,"num_objects_dirty":-4553,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4553,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":351,"num_bytes_recovered":1220608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006517"},"pg_stats":[{"pgid":"2.f","version":"45'13789","reported_seq":14017,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:47.514503+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:31:47.514503+0000","last_peered":"2024-09-17T15:31:47.514503+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:31:47.514503+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"37'3700","ondisk_log_start":"37'3700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28094464,"num_objects":6859,"num_object_clones":0,"num_object_copies":13718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3997,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13789,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2862},{"shards":"0,2","objects":605}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13976","reported_seq":14083,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:47.675326+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:31:47.675326+0000","last_peered":"2024-09-17T15:31:47.675326+0000","last_clean":"2024-09-17T15:31:47.675326+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:31:47.675326+0000","last_undegraded":"2024-09-17T15:31:47.675326+0000","last_fullsized":"2024-09-17T15:31:47.675326+0000","mapping_epoch":33,"log_start":"37'3900","ondisk_log_start":"37'3900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28492742,"num_objects":6958,"num_object_clones":0,"num_object_copies":13916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6958,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12333,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14052","reported_seq":14257,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:47.692185+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:31:47.692185+0000","last_peered":"2024-09-17T15:31:47.692185+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:31:47.692185+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"37'4000","ondisk_log_start":"37'4000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28184600,"num_objects":6882,"num_object_clones":0,"num_object_copies":13764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6882,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12209,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":585},{"shards":"2","objects":2751}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13985","reported_seq":14059,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:47.565476+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:31:47.565476+0000","last_peered":"2024-09-17T15:31:47.565476+0000","last_clean":"2024-09-17T15:31:47.565476+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:31:47.565476+0000","last_undegraded":"2024-09-17T15:31:47.565476+0000","last_fullsized":"2024-09-17T15:31:47.565476+0000","mapping_epoch":33,"log_start":"37'3900","ondisk_log_start":"37'3900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28610560,"num_objects":6985,"num_object_clones":0,"num_object_copies":13970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13985,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13783","reported_seq":13917,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:47.520489+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:31:47.520489+0000","last_peered":"2024-09-17T15:31:47.520489+0000","last_clean":"2024-09-17T15:31:47.520489+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:31:47.520489+0000","last_undegraded":"2024-09-17T15:31:47.520489+0000","last_fullsized":"2024-09-17T15:31:47.520489+0000","mapping_epoch":32,"log_start":"37'3700","ondisk_log_start":"37'3700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28140003,"num_objects":6871,"num_object_clones":0,"num_object_copies":13742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6871,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10765,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14322","reported_seq":14586,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:47.546042+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:31:47.546042+0000","last_peered":"2024-09-17T15:31:47.546042+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:31:47.546042+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'4300","ondisk_log_start":"41'4300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28381667,"num_objects":6930,"num_object_clones":0,"num_object_copies":13860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6930,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10821,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2723},{"shards":"0,2","objects":574}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13987","reported_seq":14269,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:47.542501+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:31:47.542501+0000","last_peered":"2024-09-17T15:31:47.542501+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:31:47.542501+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"37'3900","ondisk_log_start":"37'3900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27775942,"num_objects":6783,"num_object_clones":0,"num_object_copies":13566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4105,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6783,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10756,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2678},{"shards":"0,2","objects":574}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14042","reported_seq":14175,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:47.520366+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:31:47.520366+0000","last_peered":"2024-09-17T15:31:47.520366+0000","last_clean":"2024-09-17T15:31:47.520366+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:31:47.520366+0000","last_undegraded":"2024-09-17T15:31:47.520366+0000","last_fullsized":"2024-09-17T15:31:47.520366+0000","mapping_epoch":32,"log_start":"37'4000","ondisk_log_start":"37'4000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28155904,"num_objects":6874,"num_object_clones":0,"num_object_copies":13748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10996,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.086967+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.086967+0000","last_peered":"2024-09-17T15:30:57.086967+0000","last_clean":"2024-09-17T15:30:57.086967+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.086967+0000","last_undegraded":"2024-09-17T15:30:57.086967+0000","last_fullsized":"2024-09-17T15:30:57.086967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13884","reported_seq":22171,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:46.896719+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:31:46.896719+0000","last_peered":"2024-09-17T15:31:46.896719+0000","last_clean":"2024-09-17T15:31:46.896719+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:31:46.896719+0000","last_undegraded":"2024-09-17T15:31:46.896719+0000","last_fullsized":"2024-09-17T15:31:46.896719+0000","mapping_epoch":42,"log_start":"37'3800","ondisk_log_start":"37'3800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28398051,"num_objects":6934,"num_object_clones":0,"num_object_copies":13868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6934,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11884,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13732","reported_seq":13973,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:47.542524+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:31:47.542524+0000","last_peered":"2024-09-17T15:31:47.542524+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:31:47.542524+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"37'3700","ondisk_log_start":"37'3700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27893760,"num_objects":6810,"num_object_clones":0,"num_object_copies":13620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3959,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10328,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2853},{"shards":"0,2","objects":553}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13787","reported_seq":13987,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:47.590234+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:31:47.590234+0000","last_peered":"2024-09-17T15:31:47.590234+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:31:47.590234+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"37'3700","ondisk_log_start":"37'3700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27574755,"num_objects":6733,"num_object_clones":0,"num_object_copies":13466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6733,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11682,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":552},{"shards":"2","objects":2693}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13914","reported_seq":16995,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:47.708269+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:31:47.708269+0000","last_peered":"2024-09-17T15:31:47.708269+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:31:47.708269+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"37'3900","ondisk_log_start":"37'3900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27942912,"num_objects":6822,"num_object_clones":0,"num_object_copies":13644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2141,"num_objects_misplaced":520,"num_objects_unfound":0,"num_objects_dirty":6822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11676,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1478,"num_bytes_recovered":5943296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":520},{"shards":"2","objects":4161}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14059","reported_seq":14331,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:47.544705+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:31:47.544705+0000","last_peered":"2024-09-17T15:31:47.544705+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:31:47.544705+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"37'4000","ondisk_log_start":"37'4000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28291072,"num_objects":6907,"num_object_clones":0,"num_object_copies":13814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4073,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10607,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2835},{"shards":"0,2","objects":563}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13997","reported_seq":14248,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:47.349453+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:31:47.349453+0000","last_peered":"2024-09-17T15:31:47.349453+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:31:47.349453+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"37'3900","ondisk_log_start":"37'3900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27750400,"num_objects":6775,"num_object_clones":0,"num_object_copies":13550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10284,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2670},{"shards":"0,2","objects":550}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13934","reported_seq":14213,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:47.512919+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:31:47.512919+0000","last_peered":"2024-09-17T15:31:47.512919+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:31:47.512919+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"37'3900","ondisk_log_start":"37'3900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28475392,"num_objects":6952,"num_object_clones":0,"num_object_copies":13904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3994,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10721,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":71,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2960},{"shards":"0,2","objects":574}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13841","reported_seq":22220,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:46.872797+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:31:46.872797+0000","last_peered":"2024-09-17T15:31:46.872797+0000","last_clean":"2024-09-17T15:31:46.872797+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:31:46.872797+0000","last_undegraded":"2024-09-17T15:31:46.872797+0000","last_fullsized":"2024-09-17T15:31:46.872797+0000","mapping_epoch":44,"log_start":"37'3800","ondisk_log_start":"37'3800","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27816902,"num_objects":6793,"num_object_clones":0,"num_object_copies":13586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10283,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":449979126,"num_objects":109868,"num_object_clones":0,"num_object_copies":219736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38758,"num_objects_misplaced":520,"num_objects_unfound":0,"num_objects_dirty":109868,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":183119,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10182,"num_bytes_recovered":39441350,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1079271424,"data_stored":1079125986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160984,"ondisk_log_size":160984,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051665,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":669080,"kb_used_data":352776,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93702760,"statfs":{"total":96636764160,"available":95951626240,"internally_reserved":0,"allocated":361242624,"data_stored":360951527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":323874476},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724610,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627188,"kb_used_data":306724,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93744652,"statfs":{"total":96636764160,"available":95994523648,"internally_reserved":0,"allocated":314085376,"data_stored":313761298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":328134315},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247893,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":780044,"kb_used_data":397180,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93591796,"statfs":{"total":96636764160,"available":95837999104,"internally_reserved":0,"allocated":406712320,"data_stored":406421218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":392031912},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,3,4,4,0,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":405233664,"data_stored":405197534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":313520128,"data_stored":313451022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":360517632,"data_stored":360477430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:31:53.025 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:31:53.554 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:31:53.556 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:31:53.585 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":262,"stamp":"2024-09-17T15:31:51.709973+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":449979145,"num_objects":109870,"num_object_clones":0,"num_object_copies":219740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38758,"num_objects_misplaced":520,"num_objects_unfound":0,"num_objects_dirty":109870,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":183121,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10183,"num_bytes_recovered":39441369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160987,"ondisk_log_size":160987,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2076312,"kb_used_data":1056680,"kb_used_omap":13,"kb_used_meta":1019570,"kb_avail":281039208,"statfs":{"total":289910292480,"available":287784148992,"internally_reserved":0,"allocated":1082040320,"data_stored":1081134043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1044040703},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,4,4,5,6,6,0,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18649088,"num_objects":-4553,"num_object_clones":0,"num_object_copies":-9106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-53,"num_objects_misplaced":-298,"num_objects_unfound":0,"num_objects_dirty":-4553,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4553,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":351,"num_bytes_recovered":1220608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006517"},"pg_stats":[{"pgid":"2.f","version":"45'13789","reported_seq":14017,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:47.514503+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:31:47.514503+0000","last_peered":"2024-09-17T15:31:47.514503+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:31:47.514503+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"37'3700","ondisk_log_start":"37'3700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28094464,"num_objects":6859,"num_object_clones":0,"num_object_copies":13718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3997,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13789,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2862},{"shards":"0,2","objects":605}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13976","reported_seq":14083,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:47.675326+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:31:47.675326+0000","last_peered":"2024-09-17T15:31:47.675326+0000","last_clean":"2024-09-17T15:31:47.675326+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:31:47.675326+0000","last_undegraded":"2024-09-17T15:31:47.675326+0000","last_fullsized":"2024-09-17T15:31:47.675326+0000","mapping_epoch":33,"log_start":"37'3900","ondisk_log_start":"37'3900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28492742,"num_objects":6958,"num_object_clones":0,"num_object_copies":13916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6958,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12333,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14052","reported_seq":14257,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:47.692185+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:31:47.692185+0000","last_peered":"2024-09-17T15:31:47.692185+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:31:47.692185+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"37'4000","ondisk_log_start":"37'4000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28184600,"num_objects":6882,"num_object_clones":0,"num_object_copies":13764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6882,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12209,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":585},{"shards":"2","objects":2751}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13985","reported_seq":14059,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:47.565476+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:31:47.565476+0000","last_peered":"2024-09-17T15:31:47.565476+0000","last_clean":"2024-09-17T15:31:47.565476+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:31:47.565476+0000","last_undegraded":"2024-09-17T15:31:47.565476+0000","last_fullsized":"2024-09-17T15:31:47.565476+0000","mapping_epoch":33,"log_start":"37'3900","ondisk_log_start":"37'3900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28610560,"num_objects":6985,"num_object_clones":0,"num_object_copies":13970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13985,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13783","reported_seq":13917,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:47.520489+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:31:47.520489+0000","last_peered":"2024-09-17T15:31:47.520489+0000","last_clean":"2024-09-17T15:31:47.520489+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:31:47.520489+0000","last_undegraded":"2024-09-17T15:31:47.520489+0000","last_fullsized":"2024-09-17T15:31:47.520489+0000","mapping_epoch":32,"log_start":"37'3700","ondisk_log_start":"37'3700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28140003,"num_objects":6871,"num_object_clones":0,"num_object_copies":13742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6871,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10765,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14322","reported_seq":14586,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:47.546042+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:31:47.546042+0000","last_peered":"2024-09-17T15:31:47.546042+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:31:47.546042+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'4300","ondisk_log_start":"41'4300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28381667,"num_objects":6930,"num_object_clones":0,"num_object_copies":13860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6930,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10821,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2723},{"shards":"0,2","objects":574}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13987","reported_seq":14269,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:47.542501+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:31:47.542501+0000","last_peered":"2024-09-17T15:31:47.542501+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:31:47.542501+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"37'3900","ondisk_log_start":"37'3900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27775942,"num_objects":6783,"num_object_clones":0,"num_object_copies":13566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4105,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6783,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10756,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2678},{"shards":"0,2","objects":574}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14042","reported_seq":14175,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:47.520366+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:31:47.520366+0000","last_peered":"2024-09-17T15:31:47.520366+0000","last_clean":"2024-09-17T15:31:47.520366+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:31:47.520366+0000","last_undegraded":"2024-09-17T15:31:47.520366+0000","last_fullsized":"2024-09-17T15:31:47.520366+0000","mapping_epoch":32,"log_start":"37'4000","ondisk_log_start":"37'4000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28155904,"num_objects":6874,"num_object_clones":0,"num_object_copies":13748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10996,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.086967+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.086967+0000","last_peered":"2024-09-17T15:30:57.086967+0000","last_clean":"2024-09-17T15:30:57.086967+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.086967+0000","last_undegraded":"2024-09-17T15:30:57.086967+0000","last_fullsized":"2024-09-17T15:30:57.086967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13884","reported_seq":22171,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:46.896719+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:31:46.896719+0000","last_peered":"2024-09-17T15:31:46.896719+0000","last_clean":"2024-09-17T15:31:46.896719+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:31:46.896719+0000","last_undegraded":"2024-09-17T15:31:46.896719+0000","last_fullsized":"2024-09-17T15:31:46.896719+0000","mapping_epoch":42,"log_start":"37'3800","ondisk_log_start":"37'3800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28398051,"num_objects":6934,"num_object_clones":0,"num_object_copies":13868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6934,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11884,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13732","reported_seq":13973,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:47.542524+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:31:47.542524+0000","last_peered":"2024-09-17T15:31:47.542524+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:31:47.542524+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"37'3700","ondisk_log_start":"37'3700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27893760,"num_objects":6810,"num_object_clones":0,"num_object_copies":13620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3959,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10328,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2853},{"shards":"0,2","objects":553}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13787","reported_seq":13987,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:47.590234+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:31:47.590234+0000","last_peered":"2024-09-17T15:31:47.590234+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:31:47.590234+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"37'3700","ondisk_log_start":"37'3700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27574755,"num_objects":6733,"num_object_clones":0,"num_object_copies":13466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6733,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11682,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":552},{"shards":"2","objects":2693}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13914","reported_seq":16995,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:47.708269+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:31:47.708269+0000","last_peered":"2024-09-17T15:31:47.708269+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:31:47.708269+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"37'3900","ondisk_log_start":"37'3900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27942912,"num_objects":6822,"num_object_clones":0,"num_object_copies":13644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2141,"num_objects_misplaced":520,"num_objects_unfound":0,"num_objects_dirty":6822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11676,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1478,"num_bytes_recovered":5943296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":520},{"shards":"2","objects":4161}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14059","reported_seq":14331,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:47.544705+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:31:47.544705+0000","last_peered":"2024-09-17T15:31:47.544705+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:31:47.544705+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"37'4000","ondisk_log_start":"37'4000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28291072,"num_objects":6907,"num_object_clones":0,"num_object_copies":13814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4073,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10607,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2835},{"shards":"0,2","objects":563}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13997","reported_seq":14248,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:47.349453+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:31:47.349453+0000","last_peered":"2024-09-17T15:31:47.349453+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:31:47.349453+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"37'3900","ondisk_log_start":"37'3900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27750400,"num_objects":6775,"num_object_clones":0,"num_object_copies":13550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10284,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2670},{"shards":"0,2","objects":550}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13934","reported_seq":14213,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:47.512919+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:31:47.512919+0000","last_peered":"2024-09-17T15:31:47.512919+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:31:47.512919+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"37'3900","ondisk_log_start":"37'3900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28475392,"num_objects":6952,"num_object_clones":0,"num_object_copies":13904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3994,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10721,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":71,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2960},{"shards":"0,2","objects":574}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13841","reported_seq":22220,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:46.872797+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:31:46.872797+0000","last_peered":"2024-09-17T15:31:46.872797+0000","last_clean":"2024-09-17T15:31:46.872797+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:31:46.872797+0000","last_undegraded":"2024-09-17T15:31:46.872797+0000","last_fullsized":"2024-09-17T15:31:46.872797+0000","mapping_epoch":44,"log_start":"37'3800","ondisk_log_start":"37'3800","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27816902,"num_objects":6793,"num_object_clones":0,"num_object_copies":13586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10283,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":449979126,"num_objects":109868,"num_object_clones":0,"num_object_copies":219736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38758,"num_objects_misplaced":520,"num_objects_unfound":0,"num_objects_dirty":109868,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":183119,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10182,"num_bytes_recovered":39441350,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1079271424,"data_stored":1079125986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160984,"ondisk_log_size":160984,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051665,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":669080,"kb_used_data":352776,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93702760,"statfs":{"total":96636764160,"available":95951626240,"internally_reserved":0,"allocated":361242624,"data_stored":360951527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":323874476},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724610,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627188,"kb_used_data":306724,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93744652,"statfs":{"total":96636764160,"available":95994523648,"internally_reserved":0,"allocated":314085376,"data_stored":313761298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":328134315},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247893,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":780044,"kb_used_data":397180,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93591796,"statfs":{"total":96636764160,"available":95837999104,"internally_reserved":0,"allocated":406712320,"data_stored":406421218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":392031912},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,3,4,4,0,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":405233664,"data_stored":405197534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":313520128,"data_stored":313451022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":360517632,"data_stored":360477430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:31:53.586 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:31:53.587 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'13789", 'reported_seq': 14017, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:47.514503+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:31:47.514503+0000', 'last_peered': '2024-09-17T15:31:47.514503+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:31:47.514503+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "37'3700", 'ondisk_log_start': "37'3700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28094464, 'num_objects': 6859, 'num_object_clones': 0, 'num_object_copies': 13718, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3997, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6859, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13789, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 73, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2862}, {'shards': '0,2', 'objects': 605}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:53.587 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:31:53.587 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'14052", 'reported_seq': 14257, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:47.692185+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:31:47.692185+0000', 'last_peered': '2024-09-17T15:31:47.692185+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:31:47.692185+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "37'4000", 'ondisk_log_start': "37'4000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28184600, 'num_objects': 6882, 'num_object_clones': 0, 'num_object_copies': 13764, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6882, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12209, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 37, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 585}, {'shards': '2', 'objects': 2751}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:31:53.587 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:31:53.587 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'14322", 'reported_seq': 14586, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:47.546042+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:31:47.546042+0000', 'last_peered': '2024-09-17T15:31:47.546042+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:31:47.546042+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "41'4300", 'ondisk_log_start': "41'4300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28381667, 'num_objects': 6930, 'num_object_clones': 0, 'num_object_copies': 13860, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4208, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6930, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10821, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 63, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2723}, {'shards': '0,2', 'objects': 574}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:53.587 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:31:53.587 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'13987", 'reported_seq': 14269, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:47.542501+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:31:47.542501+0000', 'last_peered': '2024-09-17T15:31:47.542501+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:31:47.542501+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "37'3900", 'ondisk_log_start': "37'3900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27775942, 'num_objects': 6783, 'num_object_clones': 0, 'num_object_copies': 13566, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4105, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6783, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 10756, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2678}, {'shards': '0,2', 'objects': 574}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:53.587 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:31:53.587 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'13732", 'reported_seq': 13973, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:47.542524+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:31:47.542524+0000', 'last_peered': '2024-09-17T15:31:47.542524+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:31:47.542524+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "37'3700", 'ondisk_log_start': "37'3700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27893760, 'num_objects': 6810, 'num_object_clones': 0, 'num_object_copies': 13620, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3959, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6810, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10328, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2853}, {'shards': '0,2', 'objects': 553}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:53.588 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:31:53.588 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'13787", 'reported_seq': 13987, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:47.590234+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:31:47.590234+0000', 'last_peered': '2024-09-17T15:31:47.590234+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:31:47.590234+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "37'3700", 'ondisk_log_start': "37'3700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27574755, 'num_objects': 6733, 'num_object_clones': 0, 'num_object_copies': 13466, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6733, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11682, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 37, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 552}, {'shards': '2', 'objects': 2693}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:31:53.588 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:31:53.588 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'13914", 'reported_seq': 16995, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:47.708269+0000', 'last_change': '2024-09-17T15:30:55.086473+0000', 'last_active': '2024-09-17T15:31:47.708269+0000', 'last_peered': '2024-09-17T15:31:47.708269+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:31:47.708269+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "37'3900", 'ondisk_log_start': "37'3900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27942912, 'num_objects': 6822, 'num_object_clones': 0, 'num_object_copies': 13644, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2141, 'num_objects_misplaced': 520, 'num_objects_unfound': 0, 'num_objects_dirty': 6822, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11676, 'num_write_kb': 32532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1478, 'num_bytes_recovered': 5943296, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 520}, {'shards': '2', 'objects': 4161}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:31:53.588 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:31:53.588 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'14059", 'reported_seq': 14331, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:47.544705+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:31:47.544705+0000', 'last_peered': '2024-09-17T15:31:47.544705+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:31:47.544705+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "37'4000", 'ondisk_log_start': "37'4000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28291072, 'num_objects': 6907, 'num_object_clones': 0, 'num_object_copies': 13814, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4073, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6907, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10607, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2835}, {'shards': '0,2', 'objects': 563}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:53.588 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:31:53.588 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'13997", 'reported_seq': 14248, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:47.349453+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:31:47.349453+0000', 'last_peered': '2024-09-17T15:31:47.349453+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:31:47.349453+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "37'3900", 'ondisk_log_start': "37'3900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27750400, 'num_objects': 6775, 'num_object_clones': 0, 'num_object_copies': 13550, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4108, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6775, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10284, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2670}, {'shards': '0,2', 'objects': 550}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:53.588 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:31:53.588 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'13934", 'reported_seq': 14213, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:47.512919+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:31:47.512919+0000', 'last_peered': '2024-09-17T15:31:47.512919+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:31:47.512919+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "37'3900", 'ondisk_log_start': "37'3900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28475392, 'num_objects': 6952, 'num_object_clones': 0, 'num_object_copies': 13904, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3994, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6952, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10721, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 71, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2960}, {'shards': '0,2', 'objects': 574}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:53.589 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:31:54.276 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:31:54.276 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38758/219740 objects degraded (17.638%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":451,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":109870,"data_bytes":449979145,"bytes_used":2126143488,"bytes_avail":287784148992,"bytes_total":289910292480,"degraded_objects":38758,"degraded_total":219740,"degraded_ratio":0.17638117775552925,"misplaced_objects":520,"misplaced_total":219740,"misplaced_ratio":0.0023664330572494767,"recovering_objects_per_sec":29,"recovering_bytes_per_sec":101662,"recovering_keys_per_sec":0,"num_objects_recovered":351,"num_bytes_recovered":1220608,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":379},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-17T15:31:29.695933+0000","services":{}},"progress_events":{}} 2024-09-17T15:31:54.299 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:31:54.300 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:31:54.844 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:31:54.846 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:31:54.865 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":263,"stamp":"2024-09-17T15:31:53.710665+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":440398601,"num_objects":107531,"num_object_clones":0,"num_object_copies":215062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38738,"num_objects_misplaced":392,"num_objects_unfound":0,"num_objects_dirty":107531,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":185460,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10331,"num_bytes_recovered":39965657,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160926,"ondisk_log_size":160926,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2067336,"kb_used_data":1043536,"kb_used_omap":13,"kb_used_meta":1023730,"kb_avail":281048184,"statfs":{"total":289910292480,"available":287793340416,"internally_reserved":0,"allocated":1068580864,"data_stored":1067674587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1048300543},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,8,6,12,4,0,0,0,3,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28229632,"num_objects":-6892,"num_object_clones":0,"num_object_copies":-13784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-73,"num_objects_misplaced":-426,"num_objects_unfound":0,"num_objects_dirty":-6892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6892,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":499,"num_bytes_recovered":1744896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006660"},"pg_stats":[{"pgid":"2.f","version":"45'13932","reported_seq":14169,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:52.631045+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:31:52.631045+0000","last_peered":"2024-09-17T15:31:52.631045+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:31:52.631045+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"37'3900","ondisk_log_start":"37'3900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27508736,"num_objects":6716,"num_object_clones":0,"num_object_copies":13432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3993,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13932,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2723},{"shards":"0,2","objects":462}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14136","reported_seq":14243,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:52.630798+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:31:52.630798+0000","last_peered":"2024-09-17T15:31:52.630798+0000","last_clean":"2024-09-17T15:31:52.630798+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:31:52.630798+0000","last_undegraded":"2024-09-17T15:31:52.630798+0000","last_fullsized":"2024-09-17T15:31:52.630798+0000","mapping_epoch":33,"log_start":"41'4100","ondisk_log_start":"41'4100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27837382,"num_objects":6798,"num_object_clones":0,"num_object_copies":13596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6798,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12493,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14191","reported_seq":14400,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:52.726250+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:31:52.726250+0000","last_peered":"2024-09-17T15:31:52.726250+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:31:52.726250+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"38'4100","ondisk_log_start":"38'4100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27615256,"num_objects":6743,"num_object_clones":0,"num_object_copies":13486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4130,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6743,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12348,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":446},{"shards":"2","objects":2614}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'14122","reported_seq":14196,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:52.734503+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:31:52.734503+0000","last_peered":"2024-09-17T15:31:52.734503+0000","last_clean":"2024-09-17T15:31:52.734503+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:31:52.734503+0000","last_undegraded":"2024-09-17T15:31:52.734503+0000","last_fullsized":"2024-09-17T15:31:52.734503+0000","mapping_epoch":33,"log_start":"41'4100","ondisk_log_start":"41'4100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28049408,"num_objects":6848,"num_object_clones":0,"num_object_copies":13696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14122,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13922","reported_seq":14056,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:52.580675+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:31:52.580675+0000","last_peered":"2024-09-17T15:31:52.580675+0000","last_clean":"2024-09-17T15:31:52.580675+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:31:52.580675+0000","last_undegraded":"2024-09-17T15:31:52.580675+0000","last_fullsized":"2024-09-17T15:31:52.580675+0000","mapping_epoch":32,"log_start":"37'3900","ondisk_log_start":"37'3900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27570659,"num_objects":6732,"num_object_clones":0,"num_object_copies":13464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6732,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10904,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14476","reported_seq":14744,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:52.610317+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:31:52.610317+0000","last_peered":"2024-09-17T15:31:52.610317+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:31:52.610317+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'4400","ondisk_log_start":"41'4400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27750883,"num_objects":6776,"num_object_clones":0,"num_object_copies":13552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4207,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6776,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10975,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2570},{"shards":"0,2","objects":420}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14133","reported_seq":14421,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:52.465160+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:31:52.465160+0000","last_peered":"2024-09-17T15:31:52.465160+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:31:52.465160+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"38'4100","ondisk_log_start":"38'4100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27177926,"num_objects":6637,"num_object_clones":0,"num_object_copies":13274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4102,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6637,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10902,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2535},{"shards":"0,2","objects":428}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14185","reported_seq":14318,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:52.582791+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:31:52.582791+0000","last_peered":"2024-09-17T15:31:52.582791+0000","last_clean":"2024-09-17T15:31:52.582791+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:31:52.582791+0000","last_undegraded":"2024-09-17T15:31:52.582791+0000","last_fullsized":"2024-09-17T15:31:52.582791+0000","mapping_epoch":32,"log_start":"38'4100","ondisk_log_start":"38'4100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27570176,"num_objects":6731,"num_object_clones":0,"num_object_copies":13462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11139,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.086967+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.086967+0000","last_peered":"2024-09-17T15:30:57.086967+0000","last_clean":"2024-09-17T15:30:57.086967+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.086967+0000","last_undegraded":"2024-09-17T15:30:57.086967+0000","last_fullsized":"2024-09-17T15:30:57.086967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14048","reported_seq":22335,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:51.857369+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:31:51.857369+0000","last_peered":"2024-09-17T15:31:51.857369+0000","last_clean":"2024-09-17T15:31:51.857369+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:31:51.857369+0000","last_undegraded":"2024-09-17T15:31:51.857369+0000","last_fullsized":"2024-09-17T15:31:51.857369+0000","mapping_epoch":42,"log_start":"38'4000","ondisk_log_start":"38'4000","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27726307,"num_objects":6770,"num_object_clones":0,"num_object_copies":13540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6770,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12048,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13872","reported_seq":14119,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:52.487473+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:31:52.487473+0000","last_peered":"2024-09-17T15:31:52.487473+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:31:52.487473+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"37'3800","ondisk_log_start":"37'3800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27320320,"num_objects":6670,"num_object_clones":0,"num_object_copies":13340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3956,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10468,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2716},{"shards":"0,2","objects":413}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13920","reported_seq":14122,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:52.466268+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:31:52.466268+0000","last_peered":"2024-09-17T15:31:52.466268+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:31:52.466268+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"37'3900","ondisk_log_start":"37'3900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27029987,"num_objects":6600,"num_object_clones":0,"num_object_copies":13200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4040,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6600,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11815,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":419},{"shards":"2","objects":2561}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14090","reported_seq":17427,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:52.711610+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:31:52.711610+0000","last_peered":"2024-09-17T15:31:52.711610+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:31:52.711610+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"37'4000","ondisk_log_start":"37'4000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27222016,"num_objects":6646,"num_object_clones":0,"num_object_copies":13292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2141,"num_objects_misplaced":392,"num_objects_unfound":0,"num_objects_dirty":6646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11852,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1606,"num_bytes_recovered":6467584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":392},{"shards":"2","objects":4113}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14196","reported_seq":14472,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:52.560466+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:31:52.560466+0000","last_peered":"2024-09-17T15:31:52.560466+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:31:52.560466+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"38'4100","ondisk_log_start":"38'4100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27729920,"num_objects":6770,"num_object_clones":0,"num_object_copies":13540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4071,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10744,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2700},{"shards":"0,2","objects":426}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14138","reported_seq":14391,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:52.482495+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:31:52.482495+0000","last_peered":"2024-09-17T15:31:52.482495+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:31:52.482495+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"38'4100","ondisk_log_start":"38'4100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27172864,"num_objects":6634,"num_object_clones":0,"num_object_copies":13268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10425,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2530},{"shards":"0,2","objects":409}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14085","reported_seq":14370,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:52.605675+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:31:52.605675+0000","last_peered":"2024-09-17T15:31:52.605675+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:31:52.605675+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"38'4000","ondisk_log_start":"38'4000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27856896,"num_objects":6801,"num_object_clones":0,"num_object_copies":13602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3991,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6801,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10872,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2812},{"shards":"0,2","objects":423}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13977","reported_seq":22356,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:51.958014+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:31:51.958014+0000","last_peered":"2024-09-17T15:31:51.958014+0000","last_clean":"2024-09-17T15:31:51.958014+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:31:51.958014+0000","last_undegraded":"2024-09-17T15:31:51.958014+0000","last_fullsized":"2024-09-17T15:31:51.958014+0000","mapping_epoch":44,"log_start":"37'3900","ondisk_log_start":"37'3900","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27259846,"num_objects":6657,"num_object_clones":0,"num_object_copies":13314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10419,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":440398582,"num_objects":107529,"num_object_clones":0,"num_object_copies":215058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38738,"num_objects_misplaced":392,"num_objects_unfound":0,"num_objects_dirty":107529,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":185458,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10330,"num_bytes_recovered":39965638,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1065922560,"data_stored":1065777122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160923,"ondisk_log_size":160923,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051666,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":668448,"kb_used_data":347984,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93703392,"statfs":{"total":96636764160,"available":95952273408,"internally_reserved":0,"allocated":356335616,"data_stored":356044519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":328134316},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,1,3,0,0,0,0,0,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724611,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":634144,"kb_used_data":305360,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93737696,"statfs":{"total":96636764160,"available":95987400704,"internally_reserved":0,"allocated":312688640,"data_stored":312364562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":336653995},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,0,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247894,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":764744,"kb_used_data":390192,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93607096,"statfs":{"total":96636764160,"available":95853666304,"internally_reserved":0,"allocated":399556608,"data_stored":399265506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":383512232},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,6,2,0,0,0,3],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":398147584,"data_stored":398111454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":312143872,"data_stored":312074766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":355631104,"data_stored":355590902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:31:57.867 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:31:58.429 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:31:58.432 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:31:58.460 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":265,"stamp":"2024-09-17T15:31:57.712108+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":434049801,"num_objects":105981,"num_object_clones":0,"num_object_copies":211962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38726,"num_objects_misplaced":392,"num_objects_unfound":0,"num_objects_dirty":105981,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":187010,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10343,"num_bytes_recovered":39965657,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160776,"ondisk_log_size":160776,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2051292,"kb_used_data":1035800,"kb_used_omap":13,"kb_used_meta":1015410,"kb_avail":281064228,"statfs":{"total":289910292480,"available":287809769472,"internally_reserved":0,"allocated":1060659200,"data_stored":1059752923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1039780863},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,7,7,5,10,2,0,0,0,0,0,7],"upper_bound":8192},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25296896,"num_objects":-6176,"num_object_clones":0,"num_object_copies":-12352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-61,"num_objects_misplaced":-272,"num_objects_unfound":0,"num_objects_dirty":-6176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6176,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":333,"num_bytes_recovered":1114112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004983"},"pg_stats":[{"pgid":"2.f","version":"45'14087","reported_seq":14324,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:57.669690+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:31:57.669690+0000","last_peered":"2024-09-17T15:31:57.669690+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:31:57.669690+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"38'4000","ondisk_log_start":"38'4000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26873856,"num_objects":6561,"num_object_clones":0,"num_object_copies":13122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3993,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14087,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2568},{"shards":"0,2","objects":307}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14136","reported_seq":14243,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:52.630798+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:31:52.630798+0000","last_peered":"2024-09-17T15:31:52.630798+0000","last_clean":"2024-09-17T15:31:52.630798+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:31:52.630798+0000","last_undegraded":"2024-09-17T15:31:52.630798+0000","last_fullsized":"2024-09-17T15:31:52.630798+0000","mapping_epoch":33,"log_start":"41'4100","ondisk_log_start":"41'4100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27837382,"num_objects":6798,"num_object_clones":0,"num_object_copies":13596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6798,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12493,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14191","reported_seq":14400,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:52.726250+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:31:52.726250+0000","last_peered":"2024-09-17T15:31:52.726250+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:31:52.726250+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"38'4100","ondisk_log_start":"38'4100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27615256,"num_objects":6743,"num_object_clones":0,"num_object_copies":13486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4130,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6743,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12348,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":446},{"shards":"2","objects":2614}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'14122","reported_seq":14196,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:52.734503+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:31:52.734503+0000","last_peered":"2024-09-17T15:31:52.734503+0000","last_clean":"2024-09-17T15:31:52.734503+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:31:52.734503+0000","last_undegraded":"2024-09-17T15:31:52.734503+0000","last_fullsized":"2024-09-17T15:31:52.734503+0000","mapping_epoch":33,"log_start":"41'4100","ondisk_log_start":"41'4100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28049408,"num_objects":6848,"num_object_clones":0,"num_object_copies":13696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14122,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14060","reported_seq":14194,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:57.351623+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:31:57.351623+0000","last_peered":"2024-09-17T15:31:57.351623+0000","last_clean":"2024-09-17T15:31:57.351623+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:31:57.351623+0000","last_undegraded":"2024-09-17T15:31:57.351623+0000","last_fullsized":"2024-09-17T15:31:57.351623+0000","mapping_epoch":32,"log_start":"41'4000","ondisk_log_start":"41'4000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27005411,"num_objects":6594,"num_object_clones":0,"num_object_copies":13188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6594,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11042,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14609","reported_seq":14879,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:57.568234+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:31:57.568234+0000","last_peered":"2024-09-17T15:31:57.568234+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:31:57.568234+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'4600","ondisk_log_start":"41'4600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27206115,"num_objects":6643,"num_object_clones":0,"num_object_copies":13286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6643,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11108,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2438},{"shards":"0,2","objects":287}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14266","reported_seq":14562,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:57.640370+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:31:57.640370+0000","last_peered":"2024-09-17T15:31:57.640370+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:31:57.640370+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'4200","ondisk_log_start":"41'4200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26633158,"num_objects":6504,"num_object_clones":0,"num_object_copies":13008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4097,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6504,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11035,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2407},{"shards":"0,2","objects":295}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14318","reported_seq":14451,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:57.649204+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:31:57.649204+0000","last_peered":"2024-09-17T15:31:57.649204+0000","last_clean":"2024-09-17T15:31:57.649204+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:31:57.649204+0000","last_undegraded":"2024-09-17T15:31:57.649204+0000","last_fullsized":"2024-09-17T15:31:57.649204+0000","mapping_epoch":32,"log_start":"41'4300","ondisk_log_start":"41'4300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27025408,"num_objects":6598,"num_object_clones":0,"num_object_copies":13196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11272,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.086967+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.086967+0000","last_peered":"2024-09-17T15:30:57.086967+0000","last_clean":"2024-09-17T15:30:57.086967+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.086967+0000","last_undegraded":"2024-09-17T15:30:57.086967+0000","last_fullsized":"2024-09-17T15:30:57.086967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14188","reported_seq":22475,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:56.918634+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:31:56.918634+0000","last_peered":"2024-09-17T15:31:56.918634+0000","last_clean":"2024-09-17T15:31:56.918634+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:31:56.918634+0000","last_undegraded":"2024-09-17T15:31:56.918634+0000","last_fullsized":"2024-09-17T15:31:56.918634+0000","mapping_epoch":42,"log_start":"41'4100","ondisk_log_start":"41'4100","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27152867,"num_objects":6630,"num_object_clones":0,"num_object_copies":13260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6630,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12188,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14002","reported_seq":14249,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:57.568243+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:31:57.568243+0000","last_peered":"2024-09-17T15:31:57.568243+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:31:57.568243+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"38'4000","ondisk_log_start":"38'4000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26787840,"num_objects":6540,"num_object_clones":0,"num_object_copies":13080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3956,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6540,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10598,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2586},{"shards":"0,2","objects":283}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13920","reported_seq":14122,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:52.466268+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:31:52.466268+0000","last_peered":"2024-09-17T15:31:52.466268+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:31:52.466268+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"37'3900","ondisk_log_start":"37'3900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27029987,"num_objects":6600,"num_object_clones":0,"num_object_copies":13200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4040,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6600,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11815,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":419},{"shards":"2","objects":2561}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14090","reported_seq":17427,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:52.711610+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:31:52.711610+0000","last_peered":"2024-09-17T15:31:52.711610+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:31:52.711610+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"37'4000","ondisk_log_start":"37'4000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27222016,"num_objects":6646,"num_object_clones":0,"num_object_copies":13292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2141,"num_objects_misplaced":392,"num_objects_unfound":0,"num_objects_dirty":6646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11852,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1606,"num_bytes_recovered":6467584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":392},{"shards":"2","objects":4113}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14350","reported_seq":14636,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:57.543754+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:31:57.543754+0000","last_peered":"2024-09-17T15:31:57.543754+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:31:57.543754+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'4300","ondisk_log_start":"41'4300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27099136,"num_objects":6616,"num_object_clones":0,"num_object_copies":13232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4066,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10898,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2551},{"shards":"0,2","objects":272}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14280","reported_seq":14533,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:57.594942+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:31:57.594942+0000","last_peered":"2024-09-17T15:31:57.594942+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:31:57.594942+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'4200","ondisk_log_start":"41'4200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26591232,"num_objects":6492,"num_object_clones":0,"num_object_copies":12984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10567,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2388},{"shards":"0,2","objects":267}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14237","reported_seq":14522,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:57.588509+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:31:57.588509+0000","last_peered":"2024-09-17T15:31:57.588509+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:31:57.588509+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'4200","ondisk_log_start":"41'4200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27234304,"num_objects":6649,"num_object_clones":0,"num_object_copies":13298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3990,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11024,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2661},{"shards":"0,2","objects":271}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14117","reported_seq":22496,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:56.954202+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:31:56.954202+0000","last_peered":"2024-09-17T15:31:56.954202+0000","last_clean":"2024-09-17T15:31:56.954202+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:31:56.954202+0000","last_undegraded":"2024-09-17T15:31:56.954202+0000","last_fullsized":"2024-09-17T15:31:56.954202+0000","mapping_epoch":44,"log_start":"38'4100","ondisk_log_start":"38'4100","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26686406,"num_objects":6517,"num_object_clones":0,"num_object_copies":13034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10559,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":434049782,"num_objects":105979,"num_object_clones":0,"num_object_copies":211958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38726,"num_objects_misplaced":392,"num_objects_unfound":0,"num_objects_dirty":105979,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":187008,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10342,"num_bytes_recovered":39965638,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1057865728,"data_stored":1057720290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160773,"ondisk_log_size":160773,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051666,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":668448,"kb_used_data":347984,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93703392,"statfs":{"total":96636764160,"available":95952273408,"internally_reserved":0,"allocated":356335616,"data_stored":356044519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":328134316},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,1,3,0,0,0,0,0,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724612,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":620452,"kb_used_data":304148,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93751388,"statfs":{"total":96636764160,"available":96001421312,"internally_reserved":0,"allocated":311447552,"data_stored":311123474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":323874475},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,0,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247895,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":762392,"kb_used_data":383668,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93609448,"statfs":{"total":96636764160,"available":95856074752,"internally_reserved":0,"allocated":392876032,"data_stored":392584930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":387772072},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,1,3,4,0,0,0,0,0,0,6],"upper_bound":8192},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":391340032,"data_stored":391303902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":310894592,"data_stored":310825486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":355631104,"data_stored":355590902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:31:58.462 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:31:59.001 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:31:59.003 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:31:59.024 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":265,"stamp":"2024-09-17T15:31:57.712108+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":434049801,"num_objects":105981,"num_object_clones":0,"num_object_copies":211962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38726,"num_objects_misplaced":392,"num_objects_unfound":0,"num_objects_dirty":105981,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":187010,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10343,"num_bytes_recovered":39965657,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160776,"ondisk_log_size":160776,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2051292,"kb_used_data":1035800,"kb_used_omap":13,"kb_used_meta":1015410,"kb_avail":281064228,"statfs":{"total":289910292480,"available":287809769472,"internally_reserved":0,"allocated":1060659200,"data_stored":1059752923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1039780863},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,7,7,5,10,2,0,0,0,0,0,7],"upper_bound":8192},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25296896,"num_objects":-6176,"num_object_clones":0,"num_object_copies":-12352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-61,"num_objects_misplaced":-272,"num_objects_unfound":0,"num_objects_dirty":-6176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6176,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":333,"num_bytes_recovered":1114112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004983"},"pg_stats":[{"pgid":"2.f","version":"45'14087","reported_seq":14324,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:57.669690+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:31:57.669690+0000","last_peered":"2024-09-17T15:31:57.669690+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:31:57.669690+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"38'4000","ondisk_log_start":"38'4000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26873856,"num_objects":6561,"num_object_clones":0,"num_object_copies":13122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3993,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14087,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2568},{"shards":"0,2","objects":307}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14136","reported_seq":14243,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:52.630798+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:31:52.630798+0000","last_peered":"2024-09-17T15:31:52.630798+0000","last_clean":"2024-09-17T15:31:52.630798+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:31:52.630798+0000","last_undegraded":"2024-09-17T15:31:52.630798+0000","last_fullsized":"2024-09-17T15:31:52.630798+0000","mapping_epoch":33,"log_start":"41'4100","ondisk_log_start":"41'4100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27837382,"num_objects":6798,"num_object_clones":0,"num_object_copies":13596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6798,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12493,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14191","reported_seq":14400,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:52.726250+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:31:52.726250+0000","last_peered":"2024-09-17T15:31:52.726250+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:31:52.726250+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"38'4100","ondisk_log_start":"38'4100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27615256,"num_objects":6743,"num_object_clones":0,"num_object_copies":13486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4130,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6743,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12348,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":446},{"shards":"2","objects":2614}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'14122","reported_seq":14196,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:52.734503+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:31:52.734503+0000","last_peered":"2024-09-17T15:31:52.734503+0000","last_clean":"2024-09-17T15:31:52.734503+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:31:52.734503+0000","last_undegraded":"2024-09-17T15:31:52.734503+0000","last_fullsized":"2024-09-17T15:31:52.734503+0000","mapping_epoch":33,"log_start":"41'4100","ondisk_log_start":"41'4100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28049408,"num_objects":6848,"num_object_clones":0,"num_object_copies":13696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14122,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14060","reported_seq":14194,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:57.351623+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:31:57.351623+0000","last_peered":"2024-09-17T15:31:57.351623+0000","last_clean":"2024-09-17T15:31:57.351623+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:31:57.351623+0000","last_undegraded":"2024-09-17T15:31:57.351623+0000","last_fullsized":"2024-09-17T15:31:57.351623+0000","mapping_epoch":32,"log_start":"41'4000","ondisk_log_start":"41'4000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27005411,"num_objects":6594,"num_object_clones":0,"num_object_copies":13188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6594,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11042,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14609","reported_seq":14879,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:57.568234+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:31:57.568234+0000","last_peered":"2024-09-17T15:31:57.568234+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:31:57.568234+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'4600","ondisk_log_start":"41'4600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27206115,"num_objects":6643,"num_object_clones":0,"num_object_copies":13286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6643,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11108,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2438},{"shards":"0,2","objects":287}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14266","reported_seq":14562,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:57.640370+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:31:57.640370+0000","last_peered":"2024-09-17T15:31:57.640370+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:31:57.640370+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'4200","ondisk_log_start":"41'4200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26633158,"num_objects":6504,"num_object_clones":0,"num_object_copies":13008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4097,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6504,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11035,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2407},{"shards":"0,2","objects":295}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14318","reported_seq":14451,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:57.649204+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:31:57.649204+0000","last_peered":"2024-09-17T15:31:57.649204+0000","last_clean":"2024-09-17T15:31:57.649204+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:31:57.649204+0000","last_undegraded":"2024-09-17T15:31:57.649204+0000","last_fullsized":"2024-09-17T15:31:57.649204+0000","mapping_epoch":32,"log_start":"41'4300","ondisk_log_start":"41'4300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27025408,"num_objects":6598,"num_object_clones":0,"num_object_copies":13196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11272,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.086967+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.086967+0000","last_peered":"2024-09-17T15:30:57.086967+0000","last_clean":"2024-09-17T15:30:57.086967+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.086967+0000","last_undegraded":"2024-09-17T15:30:57.086967+0000","last_fullsized":"2024-09-17T15:30:57.086967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14188","reported_seq":22475,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:56.918634+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:31:56.918634+0000","last_peered":"2024-09-17T15:31:56.918634+0000","last_clean":"2024-09-17T15:31:56.918634+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:31:56.918634+0000","last_undegraded":"2024-09-17T15:31:56.918634+0000","last_fullsized":"2024-09-17T15:31:56.918634+0000","mapping_epoch":42,"log_start":"41'4100","ondisk_log_start":"41'4100","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27152867,"num_objects":6630,"num_object_clones":0,"num_object_copies":13260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6630,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12188,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14002","reported_seq":14249,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:57.568243+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:31:57.568243+0000","last_peered":"2024-09-17T15:31:57.568243+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:31:57.568243+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"38'4000","ondisk_log_start":"38'4000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26787840,"num_objects":6540,"num_object_clones":0,"num_object_copies":13080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3956,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6540,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10598,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2586},{"shards":"0,2","objects":283}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13920","reported_seq":14122,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:52.466268+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:31:52.466268+0000","last_peered":"2024-09-17T15:31:52.466268+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:31:52.466268+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"37'3900","ondisk_log_start":"37'3900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27029987,"num_objects":6600,"num_object_clones":0,"num_object_copies":13200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4040,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6600,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11815,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":419},{"shards":"2","objects":2561}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14090","reported_seq":17427,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:52.711610+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:31:52.711610+0000","last_peered":"2024-09-17T15:31:52.711610+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:31:52.711610+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"37'4000","ondisk_log_start":"37'4000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27222016,"num_objects":6646,"num_object_clones":0,"num_object_copies":13292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2141,"num_objects_misplaced":392,"num_objects_unfound":0,"num_objects_dirty":6646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11852,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1606,"num_bytes_recovered":6467584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":392},{"shards":"2","objects":4113}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14350","reported_seq":14636,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:57.543754+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:31:57.543754+0000","last_peered":"2024-09-17T15:31:57.543754+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:31:57.543754+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'4300","ondisk_log_start":"41'4300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27099136,"num_objects":6616,"num_object_clones":0,"num_object_copies":13232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4066,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10898,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2551},{"shards":"0,2","objects":272}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14280","reported_seq":14533,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:57.594942+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:31:57.594942+0000","last_peered":"2024-09-17T15:31:57.594942+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:31:57.594942+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'4200","ondisk_log_start":"41'4200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26591232,"num_objects":6492,"num_object_clones":0,"num_object_copies":12984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10567,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2388},{"shards":"0,2","objects":267}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14237","reported_seq":14522,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:57.588509+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:31:57.588509+0000","last_peered":"2024-09-17T15:31:57.588509+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:31:57.588509+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'4200","ondisk_log_start":"41'4200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27234304,"num_objects":6649,"num_object_clones":0,"num_object_copies":13298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3990,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11024,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2661},{"shards":"0,2","objects":271}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14117","reported_seq":22496,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:56.954202+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:31:56.954202+0000","last_peered":"2024-09-17T15:31:56.954202+0000","last_clean":"2024-09-17T15:31:56.954202+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:31:56.954202+0000","last_undegraded":"2024-09-17T15:31:56.954202+0000","last_fullsized":"2024-09-17T15:31:56.954202+0000","mapping_epoch":44,"log_start":"38'4100","ondisk_log_start":"38'4100","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26686406,"num_objects":6517,"num_object_clones":0,"num_object_copies":13034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10559,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":434049782,"num_objects":105979,"num_object_clones":0,"num_object_copies":211958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38726,"num_objects_misplaced":392,"num_objects_unfound":0,"num_objects_dirty":105979,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":187008,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10342,"num_bytes_recovered":39965638,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1057865728,"data_stored":1057720290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160773,"ondisk_log_size":160773,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051666,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":668448,"kb_used_data":347984,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93703392,"statfs":{"total":96636764160,"available":95952273408,"internally_reserved":0,"allocated":356335616,"data_stored":356044519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":328134316},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,1,3,0,0,0,0,0,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724612,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":620452,"kb_used_data":304148,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93751388,"statfs":{"total":96636764160,"available":96001421312,"internally_reserved":0,"allocated":311447552,"data_stored":311123474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":323874475},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,0,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247895,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":762392,"kb_used_data":383668,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93609448,"statfs":{"total":96636764160,"available":95856074752,"internally_reserved":0,"allocated":392876032,"data_stored":392584930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":387772072},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,1,3,4,0,0,0,0,0,0,6],"upper_bound":8192},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":391340032,"data_stored":391303902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":310894592,"data_stored":310825486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":355631104,"data_stored":355590902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:31:59.025 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:31:59.025 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'14087", 'reported_seq': 14324, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:57.669690+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:31:57.669690+0000', 'last_peered': '2024-09-17T15:31:57.669690+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:31:57.669690+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "38'4000", 'ondisk_log_start': "38'4000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26873856, 'num_objects': 6561, 'num_object_clones': 0, 'num_object_copies': 13122, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3993, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6561, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14087, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2568}, {'shards': '0,2', 'objects': 307}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:59.026 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:31:59.026 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'14191", 'reported_seq': 14400, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:52.726250+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:31:52.726250+0000', 'last_peered': '2024-09-17T15:31:52.726250+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:31:52.726250+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "38'4100", 'ondisk_log_start': "38'4100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27615256, 'num_objects': 6743, 'num_object_clones': 0, 'num_object_copies': 13486, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4130, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6743, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12348, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 39, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 446}, {'shards': '2', 'objects': 2614}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:31:59.026 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:31:59.026 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'14609", 'reported_seq': 14879, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:57.568234+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:31:57.568234+0000', 'last_peered': '2024-09-17T15:31:57.568234+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:31:57.568234+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "41'4600", 'ondisk_log_start': "41'4600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27206115, 'num_objects': 6643, 'num_object_clones': 0, 'num_object_copies': 13286, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6643, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11108, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 65, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2438}, {'shards': '0,2', 'objects': 287}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:59.026 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:31:59.026 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'14266", 'reported_seq': 14562, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:57.640370+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:31:57.640370+0000', 'last_peered': '2024-09-17T15:31:57.640370+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:31:57.640370+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "41'4200", 'ondisk_log_start': "41'4200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26633158, 'num_objects': 6504, 'num_object_clones': 0, 'num_object_copies': 13008, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4097, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6504, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 11035, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2407}, {'shards': '0,2', 'objects': 295}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:59.026 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:31:59.026 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'14002", 'reported_seq': 14249, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:57.568243+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:31:57.568243+0000', 'last_peered': '2024-09-17T15:31:57.568243+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:31:57.568243+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "38'4000", 'ondisk_log_start': "38'4000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26787840, 'num_objects': 6540, 'num_object_clones': 0, 'num_object_copies': 13080, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3956, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6540, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10598, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2586}, {'shards': '0,2', 'objects': 283}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:59.026 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:31:59.026 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'13920", 'reported_seq': 14122, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:52.466268+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:31:52.466268+0000', 'last_peered': '2024-09-17T15:31:52.466268+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:31:52.466268+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "37'3900", 'ondisk_log_start': "37'3900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27029987, 'num_objects': 6600, 'num_object_clones': 0, 'num_object_copies': 13200, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4040, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6600, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11815, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 38, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 419}, {'shards': '2', 'objects': 2561}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:31:59.026 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:31:59.026 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'14090", 'reported_seq': 17427, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:52.711610+0000', 'last_change': '2024-09-17T15:30:55.086473+0000', 'last_active': '2024-09-17T15:31:52.711610+0000', 'last_peered': '2024-09-17T15:31:52.711610+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:31:52.711610+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "37'4000", 'ondisk_log_start': "37'4000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27222016, 'num_objects': 6646, 'num_object_clones': 0, 'num_object_copies': 13292, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2141, 'num_objects_misplaced': 392, 'num_objects_unfound': 0, 'num_objects_dirty': 6646, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11852, 'num_write_kb': 32532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1606, 'num_bytes_recovered': 6467584, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 392}, {'shards': '2', 'objects': 4113}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:31:59.027 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:31:59.027 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'14350", 'reported_seq': 14636, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:57.543754+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:31:57.543754+0000', 'last_peered': '2024-09-17T15:31:57.543754+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:31:57.543754+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "41'4300", 'ondisk_log_start': "41'4300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27099136, 'num_objects': 6616, 'num_object_clones': 0, 'num_object_copies': 13232, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4066, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6616, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10898, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 73, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2551}, {'shards': '0,2', 'objects': 272}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:59.027 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:31:59.027 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'14280", 'reported_seq': 14533, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:57.594942+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:31:57.594942+0000', 'last_peered': '2024-09-17T15:31:57.594942+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:31:57.594942+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "41'4200", 'ondisk_log_start': "41'4200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26591232, 'num_objects': 6492, 'num_object_clones': 0, 'num_object_copies': 12984, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4107, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6492, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10567, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2388}, {'shards': '0,2', 'objects': 267}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:59.027 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:31:59.027 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'14237", 'reported_seq': 14522, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:31:57.588509+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:31:57.588509+0000', 'last_peered': '2024-09-17T15:31:57.588509+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:31:57.588509+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "41'4200", 'ondisk_log_start': "41'4200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27234304, 'num_objects': 6649, 'num_object_clones': 0, 'num_object_copies': 13298, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3990, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6649, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11024, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2661}, {'shards': '0,2', 'objects': 271}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:31:59.027 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:31:59.699 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:31:59.699 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38726/211962 objects degraded (18.270%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":456,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":105981,"data_bytes":434049801,"bytes_used":2100523008,"bytes_avail":287809769472,"bytes_total":289910292480,"degraded_objects":38726,"degraded_total":211962,"degraded_ratio":0.18270255989281098,"misplaced_objects":392,"misplaced_total":211962,"misplaced_ratio":0.0018493880978665985,"recovering_objects_per_sec":27,"recovering_bytes_per_sec":92804,"recovering_keys_per_sec":0,"num_objects_recovered":333,"num_bytes_recovered":1114112,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":514},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-17T15:31:29.695933+0000","services":{}},"progress_events":{}} 2024-09-17T15:31:59.725 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:31:59.725 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:32:00.227 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:32:00.229 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:32:00.249 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":266,"stamp":"2024-09-17T15:31:59.712669+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":431141641,"num_objects":105271,"num_object_clones":0,"num_object_copies":210542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38726,"num_objects_misplaced":235,"num_objects_unfound":0,"num_objects_dirty":105271,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":187720,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10500,"num_bytes_recovered":40608729,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160786,"ondisk_log_size":160786,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2051000,"kb_used_data":1031344,"kb_used_omap":13,"kb_used_meta":1019570,"kb_avail":281064520,"statfs":{"total":289910292480,"available":287810068480,"internally_reserved":0,"allocated":1056096256,"data_stored":1055189979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1044040703},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,7,7,5,10,2,0,0,0,0,0,7],"upper_bound":8192},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21757952,"num_objects":-5312,"num_object_clones":0,"num_object_copies":-10624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-34,"num_objects_misplaced":-429,"num_objects_unfound":0,"num_objects_dirty":-5312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5312,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":463,"num_bytes_recovered":1757184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003668"},"pg_stats":[{"pgid":"2.f","version":"45'14087","reported_seq":14324,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:57.669690+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:31:57.669690+0000","last_peered":"2024-09-17T15:31:57.669690+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:31:57.669690+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"38'4000","ondisk_log_start":"38'4000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26873856,"num_objects":6561,"num_object_clones":0,"num_object_copies":13122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3993,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14087,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2568},{"shards":"0,2","objects":307}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14265","reported_seq":14372,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:57.720432+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:31:57.720432+0000","last_peered":"2024-09-17T15:31:57.720432+0000","last_clean":"2024-09-17T15:31:57.720432+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:31:57.720432+0000","last_undegraded":"2024-09-17T15:31:57.720432+0000","last_fullsized":"2024-09-17T15:31:57.720432+0000","mapping_epoch":33,"log_start":"41'4200","ondisk_log_start":"41'4200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27308998,"num_objects":6669,"num_object_clones":0,"num_object_copies":13338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6669,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12622,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14332","reported_seq":14541,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:57.651455+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:31:57.651455+0000","last_peered":"2024-09-17T15:31:57.651455+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:31:57.651455+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'4300","ondisk_log_start":"41'4300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27037720,"num_objects":6602,"num_object_clones":0,"num_object_copies":13204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4130,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6602,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12489,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":305},{"shards":"2","objects":2473}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'14242","reported_seq":14316,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:57.738038+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:31:57.738038+0000","last_peered":"2024-09-17T15:31:57.738038+0000","last_clean":"2024-09-17T15:31:57.738038+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:31:57.738038+0000","last_undegraded":"2024-09-17T15:31:57.738038+0000","last_fullsized":"2024-09-17T15:31:57.738038+0000","mapping_epoch":33,"log_start":"41'4200","ondisk_log_start":"41'4200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27557888,"num_objects":6728,"num_object_clones":0,"num_object_copies":13456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6728,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14242,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14060","reported_seq":14194,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:57.351623+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:31:57.351623+0000","last_peered":"2024-09-17T15:31:57.351623+0000","last_clean":"2024-09-17T15:31:57.351623+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:31:57.351623+0000","last_undegraded":"2024-09-17T15:31:57.351623+0000","last_fullsized":"2024-09-17T15:31:57.351623+0000","mapping_epoch":32,"log_start":"41'4000","ondisk_log_start":"41'4000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27005411,"num_objects":6594,"num_object_clones":0,"num_object_copies":13188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6594,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11042,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14609","reported_seq":14879,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:57.568234+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:31:57.568234+0000","last_peered":"2024-09-17T15:31:57.568234+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:31:57.568234+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'4600","ondisk_log_start":"41'4600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27206115,"num_objects":6643,"num_object_clones":0,"num_object_copies":13286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6643,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11108,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2438},{"shards":"0,2","objects":287}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14266","reported_seq":14562,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:57.640370+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:31:57.640370+0000","last_peered":"2024-09-17T15:31:57.640370+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:31:57.640370+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'4200","ondisk_log_start":"41'4200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26633158,"num_objects":6504,"num_object_clones":0,"num_object_copies":13008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4097,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6504,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11035,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2407},{"shards":"0,2","objects":295}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14318","reported_seq":14451,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:57.649204+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:31:57.649204+0000","last_peered":"2024-09-17T15:31:57.649204+0000","last_clean":"2024-09-17T15:31:57.649204+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:31:57.649204+0000","last_undegraded":"2024-09-17T15:31:57.649204+0000","last_fullsized":"2024-09-17T15:31:57.649204+0000","mapping_epoch":32,"log_start":"41'4300","ondisk_log_start":"41'4300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27025408,"num_objects":6598,"num_object_clones":0,"num_object_copies":13196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11272,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.086967+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.086967+0000","last_peered":"2024-09-17T15:30:57.086967+0000","last_clean":"2024-09-17T15:30:57.086967+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.086967+0000","last_undegraded":"2024-09-17T15:30:57.086967+0000","last_fullsized":"2024-09-17T15:30:57.086967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14188","reported_seq":22475,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:56.918634+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:31:56.918634+0000","last_peered":"2024-09-17T15:31:56.918634+0000","last_clean":"2024-09-17T15:31:56.918634+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:31:56.918634+0000","last_undegraded":"2024-09-17T15:31:56.918634+0000","last_fullsized":"2024-09-17T15:31:56.918634+0000","mapping_epoch":42,"log_start":"41'4100","ondisk_log_start":"41'4100","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27152867,"num_objects":6630,"num_object_clones":0,"num_object_copies":13260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6630,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12188,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14002","reported_seq":14249,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:57.568243+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:31:57.568243+0000","last_peered":"2024-09-17T15:31:57.568243+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:31:57.568243+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"38'4000","ondisk_log_start":"38'4000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26787840,"num_objects":6540,"num_object_clones":0,"num_object_copies":13080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3956,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6540,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10598,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2586},{"shards":"0,2","objects":283}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14079","reported_seq":14281,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:57.726194+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:31:57.726194+0000","last_peered":"2024-09-17T15:31:57.726194+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:31:57.726194+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"37'4000","ondisk_log_start":"37'4000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26378723,"num_objects":6441,"num_object_clones":0,"num_object_copies":12882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4040,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6441,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11974,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":260},{"shards":"2","objects":2402}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14251","reported_seq":17902,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:57.746426+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:31:57.746426+0000","last_peered":"2024-09-17T15:31:57.746426+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:31:57.746426+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"41'4200","ondisk_log_start":"41'4200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26562560,"num_objects":6485,"num_object_clones":0,"num_object_copies":12970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2141,"num_objects_misplaced":235,"num_objects_unfound":0,"num_objects_dirty":6485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12013,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1763,"num_bytes_recovered":7110656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":235},{"shards":"2","objects":4109}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14350","reported_seq":14636,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:57.543754+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:31:57.543754+0000","last_peered":"2024-09-17T15:31:57.543754+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:31:57.543754+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'4300","ondisk_log_start":"41'4300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27099136,"num_objects":6616,"num_object_clones":0,"num_object_copies":13232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4066,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10898,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2551},{"shards":"0,2","objects":272}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14280","reported_seq":14533,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:57.594942+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:31:57.594942+0000","last_peered":"2024-09-17T15:31:57.594942+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:31:57.594942+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'4200","ondisk_log_start":"41'4200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26591232,"num_objects":6492,"num_object_clones":0,"num_object_copies":12984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10567,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2388},{"shards":"0,2","objects":267}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14237","reported_seq":14522,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:31:57.588509+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:31:57.588509+0000","last_peered":"2024-09-17T15:31:57.588509+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:31:57.588509+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'4200","ondisk_log_start":"41'4200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27234304,"num_objects":6649,"num_object_clones":0,"num_object_copies":13298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3990,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11024,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2661},{"shards":"0,2","objects":271}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14117","reported_seq":22496,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:31:56.954202+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:31:56.954202+0000","last_peered":"2024-09-17T15:31:56.954202+0000","last_clean":"2024-09-17T15:31:56.954202+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:31:56.954202+0000","last_undegraded":"2024-09-17T15:31:56.954202+0000","last_fullsized":"2024-09-17T15:31:56.954202+0000","mapping_epoch":44,"log_start":"38'4100","ondisk_log_start":"38'4100","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26686406,"num_objects":6517,"num_object_clones":0,"num_object_copies":13034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10559,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":431141622,"num_objects":105269,"num_object_clones":0,"num_object_copies":210538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38726,"num_objects_misplaced":235,"num_objects_unfound":0,"num_objects_dirty":105269,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":187718,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10499,"num_bytes_recovered":40608710,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1053278208,"data_stored":1053132770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160783,"ondisk_log_size":160783,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051667,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":668156,"kb_used_data":343528,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93703684,"statfs":{"total":96636764160,"available":95952572416,"internally_reserved":0,"allocated":351772672,"data_stored":351481575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":332394156},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,1,3,0,0,0,0,0,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724612,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":620452,"kb_used_data":304148,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93751388,"statfs":{"total":96636764160,"available":96001421312,"internally_reserved":0,"allocated":311447552,"data_stored":311123474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":323874475},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,0,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247895,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":762392,"kb_used_data":383668,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93609448,"statfs":{"total":96636764160,"available":95856074752,"internally_reserved":0,"allocated":392876032,"data_stored":392584930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":387772072},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,1,3,4,0,0,0,0,0,0,6],"upper_bound":8192},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":391340032,"data_stored":391303902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":310894592,"data_stored":310825486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":351043584,"data_stored":351003382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:32:03.251 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:32:03.812 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:32:03.814 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:32:03.837 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":268,"stamp":"2024-09-17T15:32:03.713914+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":421901065,"num_objects":103015,"num_object_clones":0,"num_object_copies":206030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38680,"num_objects_misplaced":54,"num_objects_unfound":0,"num_objects_dirty":103015,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":189976,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10727,"num_bytes_recovered":41350105,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2055212,"kb_used_data":1018936,"kb_used_omap":13,"kb_used_meta":1036210,"kb_avail":281060308,"statfs":{"total":289910292480,"available":287805755392,"internally_reserved":0,"allocated":1043390464,"data_stored":1042484187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1061080063},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2,3,7,7,1,0,0,2,0,7],"upper_bound":4096},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28078080,"num_objects":-6855,"num_object_clones":0,"num_object_copies":-13710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-78,"num_objects_misplaced":-466,"num_objects_unfound":0,"num_objects_dirty":-6855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6855,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":544,"num_bytes_recovered":1908736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003941"},"pg_stats":[{"pgid":"2.f","version":"45'14229","reported_seq":14474,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:02.597882+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:32:02.597882+0000","last_peered":"2024-09-17T15:32:02.597882+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:32:02.597882+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'4200","ondisk_log_start":"41'4200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26292224,"num_objects":6419,"num_object_clones":0,"num_object_copies":12838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3989,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6419,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14229,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2430},{"shards":"0,2","objects":165}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14409","reported_seq":14516,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:02.704547+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:32:02.704547+0000","last_peered":"2024-09-17T15:32:02.704547+0000","last_clean":"2024-09-17T15:32:02.704547+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:32:02.704547+0000","last_undegraded":"2024-09-17T15:32:02.704547+0000","last_fullsized":"2024-09-17T15:32:02.704547+0000","mapping_epoch":33,"log_start":"41'4400","ondisk_log_start":"41'4400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26719174,"num_objects":6525,"num_object_clones":0,"num_object_copies":13050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6525,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12766,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14491","reported_seq":14708,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:02.542098+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:32:02.542098+0000","last_peered":"2024-09-17T15:32:02.542098+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:32:02.542098+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'4400","ondisk_log_start":"41'4400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26386456,"num_objects":6443,"num_object_clones":0,"num_object_copies":12886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6443,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12648,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":146},{"shards":"2","objects":2318}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'14416","reported_seq":14490,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:02.751547+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:32:02.751547+0000","last_peered":"2024-09-17T15:32:02.751547+0000","last_clean":"2024-09-17T15:32:02.751547+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:32:02.751547+0000","last_undegraded":"2024-09-17T15:32:02.751547+0000","last_fullsized":"2024-09-17T15:32:02.751547+0000","mapping_epoch":33,"log_start":"41'4400","ondisk_log_start":"41'4400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26845184,"num_objects":6554,"num_object_clones":0,"num_object_copies":13108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6554,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14416,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14196","reported_seq":14330,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:02.693330+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:32:02.693330+0000","last_peered":"2024-09-17T15:32:02.693330+0000","last_clean":"2024-09-17T15:32:02.693330+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:32:02.693330+0000","last_undegraded":"2024-09-17T15:32:02.693330+0000","last_fullsized":"2024-09-17T15:32:02.693330+0000","mapping_epoch":32,"log_start":"41'4100","ondisk_log_start":"41'4100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26448355,"num_objects":6458,"num_object_clones":0,"num_object_copies":12916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6458,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11178,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14748","reported_seq":15032,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:02.623130+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:32:02.623130+0000","last_peered":"2024-09-17T15:32:02.623130+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:32:02.623130+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'4700","ondisk_log_start":"41'4700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26636771,"num_objects":6504,"num_object_clones":0,"num_object_copies":13008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4199,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6504,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11247,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2306},{"shards":"0,2","objects":148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14414","reported_seq":14722,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:02.685445+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:32:02.685445+0000","last_peered":"2024-09-17T15:32:02.685445+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:32:02.685445+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'4400","ondisk_log_start":"41'4400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26026950,"num_objects":6356,"num_object_clones":0,"num_object_copies":12712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4092,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6356,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11183,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2264},{"shards":"0,2","objects":147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14451","reported_seq":14584,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:02.591436+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:32:02.591436+0000","last_peered":"2024-09-17T15:32:02.591436+0000","last_clean":"2024-09-17T15:32:02.591436+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:32:02.591436+0000","last_undegraded":"2024-09-17T15:32:02.591436+0000","last_fullsized":"2024-09-17T15:32:02.591436+0000","mapping_epoch":32,"log_start":"41'4400","ondisk_log_start":"41'4400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26480640,"num_objects":6465,"num_object_clones":0,"num_object_copies":12930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11405,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.086967+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.086967+0000","last_peered":"2024-09-17T15:30:57.086967+0000","last_clean":"2024-09-17T15:30:57.086967+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.086967+0000","last_undegraded":"2024-09-17T15:30:57.086967+0000","last_fullsized":"2024-09-17T15:30:57.086967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14333","reported_seq":22619,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:02.035823+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:32:02.035823+0000","last_peered":"2024-09-17T15:32:02.035823+0000","last_clean":"2024-09-17T15:32:02.035823+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:32:02.035823+0000","last_undegraded":"2024-09-17T15:32:02.035823+0000","last_fullsized":"2024-09-17T15:32:02.035823+0000","mapping_epoch":42,"log_start":"41'4300","ondisk_log_start":"41'4300","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26558947,"num_objects":6485,"num_object_clones":0,"num_object_copies":12970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6485,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12333,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14139","reported_seq":14400,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:02.573674+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:32:02.573674+0000","last_peered":"2024-09-17T15:32:02.573674+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:32:02.573674+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'4100","ondisk_log_start":"41'4100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26226688,"num_objects":6403,"num_object_clones":0,"num_object_copies":12806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3949,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6403,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10735,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2456},{"shards":"0,2","objects":146}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14219","reported_seq":14431,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:02.743283+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:32:02.743283+0000","last_peered":"2024-09-17T15:32:02.743283+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:32:02.743283+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'4200","ondisk_log_start":"41'4200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25805283,"num_objects":6301,"num_object_clones":0,"num_object_copies":12602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4035,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6301,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12114,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":120},{"shards":"2","objects":2267}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14382","reported_seq":18395,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:02.722022+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:32:02.722022+0000","last_peered":"2024-09-17T15:32:02.722022+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:32:02.722022+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"41'4300","ondisk_log_start":"41'4300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26025984,"num_objects":6354,"num_object_clones":0,"num_object_copies":12708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2141,"num_objects_misplaced":54,"num_objects_unfound":0,"num_objects_dirty":6354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12144,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1944,"num_bytes_recovered":7852032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":54},{"shards":"2","objects":4159}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14483","reported_seq":14772,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:02.676019+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:32:02.676019+0000","last_peered":"2024-09-17T15:32:02.676019+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:32:02.676019+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'4400","ondisk_log_start":"41'4400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26554368,"num_objects":6483,"num_object_clones":0,"num_object_copies":12966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4063,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11031,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2421},{"shards":"0,2","objects":139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14402","reported_seq":14667,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:02.608317+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:32:02.608317+0000","last_peered":"2024-09-17T15:32:02.608317+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:32:02.608317+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'4400","ondisk_log_start":"41'4400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26091520,"num_objects":6370,"num_object_clones":0,"num_object_copies":12740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4101,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10689,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2272},{"shards":"0,2","objects":145}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14370","reported_seq":14667,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:02.679465+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:32:02.679465+0000","last_peered":"2024-09-17T15:32:02.679465+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:32:02.679465+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'4300","ondisk_log_start":"41'4300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26689536,"num_objects":6516,"num_object_clones":0,"num_object_copies":13032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3985,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11157,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2533},{"shards":"0,2","objects":138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14257","reported_seq":22636,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:01.921396+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:32:01.921396+0000","last_peered":"2024-09-17T15:32:01.921396+0000","last_clean":"2024-09-17T15:32:01.921396+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:32:01.921396+0000","last_undegraded":"2024-09-17T15:32:01.921396+0000","last_fullsized":"2024-09-17T15:32:01.921396+0000","mapping_epoch":44,"log_start":"41'4200","ondisk_log_start":"41'4200","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26112966,"num_objects":6377,"num_object_clones":0,"num_object_copies":12754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10699,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":421901046,"num_objects":103013,"num_object_clones":0,"num_object_copies":206026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38680,"num_objects_misplaced":54,"num_objects_unfound":0,"num_objects_dirty":103013,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":189974,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10726,"num_bytes_recovered":41350086,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1040613376,"data_stored":1040467938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160739,"ondisk_log_size":160739,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051668,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":667572,"kb_used_data":338788,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93704268,"statfs":{"total":96636764160,"available":95953170432,"internally_reserved":0,"allocated":346918912,"data_stored":346627815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":336653996},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,3,1,0,0,0,2],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724613,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627780,"kb_used_data":303152,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93744060,"statfs":{"total":96636764160,"available":95993917440,"internally_reserved":0,"allocated":310427648,"data_stored":310103570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":332394155},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247896,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":759860,"kb_used_data":376996,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93611980,"statfs":{"total":96636764160,"available":95858667520,"internally_reserved":0,"allocated":386043904,"data_stored":385752802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":392031912},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,4,1,0,0,0,0,7],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":384430080,"data_stored":384393950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":309907456,"data_stored":309838350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":346275840,"data_stored":346235638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:32:03.839 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:32:04.374 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:32:04.375 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:32:04.397 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":268,"stamp":"2024-09-17T15:32:03.713914+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":421901065,"num_objects":103015,"num_object_clones":0,"num_object_copies":206030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38680,"num_objects_misplaced":54,"num_objects_unfound":0,"num_objects_dirty":103015,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":189976,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10727,"num_bytes_recovered":41350105,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2055212,"kb_used_data":1018936,"kb_used_omap":13,"kb_used_meta":1036210,"kb_avail":281060308,"statfs":{"total":289910292480,"available":287805755392,"internally_reserved":0,"allocated":1043390464,"data_stored":1042484187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1061080063},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2,3,7,7,1,0,0,2,0,7],"upper_bound":4096},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28078080,"num_objects":-6855,"num_object_clones":0,"num_object_copies":-13710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-78,"num_objects_misplaced":-466,"num_objects_unfound":0,"num_objects_dirty":-6855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6855,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":544,"num_bytes_recovered":1908736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003941"},"pg_stats":[{"pgid":"2.f","version":"45'14229","reported_seq":14474,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:02.597882+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:32:02.597882+0000","last_peered":"2024-09-17T15:32:02.597882+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:32:02.597882+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'4200","ondisk_log_start":"41'4200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26292224,"num_objects":6419,"num_object_clones":0,"num_object_copies":12838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3989,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6419,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14229,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2430},{"shards":"0,2","objects":165}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14409","reported_seq":14516,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:02.704547+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:32:02.704547+0000","last_peered":"2024-09-17T15:32:02.704547+0000","last_clean":"2024-09-17T15:32:02.704547+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:32:02.704547+0000","last_undegraded":"2024-09-17T15:32:02.704547+0000","last_fullsized":"2024-09-17T15:32:02.704547+0000","mapping_epoch":33,"log_start":"41'4400","ondisk_log_start":"41'4400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26719174,"num_objects":6525,"num_object_clones":0,"num_object_copies":13050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6525,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12766,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14491","reported_seq":14708,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:02.542098+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:32:02.542098+0000","last_peered":"2024-09-17T15:32:02.542098+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:32:02.542098+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'4400","ondisk_log_start":"41'4400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26386456,"num_objects":6443,"num_object_clones":0,"num_object_copies":12886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6443,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12648,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":146},{"shards":"2","objects":2318}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'14416","reported_seq":14490,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:02.751547+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:32:02.751547+0000","last_peered":"2024-09-17T15:32:02.751547+0000","last_clean":"2024-09-17T15:32:02.751547+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:32:02.751547+0000","last_undegraded":"2024-09-17T15:32:02.751547+0000","last_fullsized":"2024-09-17T15:32:02.751547+0000","mapping_epoch":33,"log_start":"41'4400","ondisk_log_start":"41'4400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26845184,"num_objects":6554,"num_object_clones":0,"num_object_copies":13108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6554,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14416,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14196","reported_seq":14330,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:02.693330+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:32:02.693330+0000","last_peered":"2024-09-17T15:32:02.693330+0000","last_clean":"2024-09-17T15:32:02.693330+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:32:02.693330+0000","last_undegraded":"2024-09-17T15:32:02.693330+0000","last_fullsized":"2024-09-17T15:32:02.693330+0000","mapping_epoch":32,"log_start":"41'4100","ondisk_log_start":"41'4100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26448355,"num_objects":6458,"num_object_clones":0,"num_object_copies":12916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6458,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11178,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14748","reported_seq":15032,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:02.623130+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:32:02.623130+0000","last_peered":"2024-09-17T15:32:02.623130+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:32:02.623130+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'4700","ondisk_log_start":"41'4700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26636771,"num_objects":6504,"num_object_clones":0,"num_object_copies":13008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4199,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6504,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11247,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2306},{"shards":"0,2","objects":148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14414","reported_seq":14722,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:02.685445+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:32:02.685445+0000","last_peered":"2024-09-17T15:32:02.685445+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:32:02.685445+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'4400","ondisk_log_start":"41'4400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26026950,"num_objects":6356,"num_object_clones":0,"num_object_copies":12712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4092,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6356,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11183,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2264},{"shards":"0,2","objects":147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14451","reported_seq":14584,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:02.591436+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:32:02.591436+0000","last_peered":"2024-09-17T15:32:02.591436+0000","last_clean":"2024-09-17T15:32:02.591436+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:32:02.591436+0000","last_undegraded":"2024-09-17T15:32:02.591436+0000","last_fullsized":"2024-09-17T15:32:02.591436+0000","mapping_epoch":32,"log_start":"41'4400","ondisk_log_start":"41'4400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26480640,"num_objects":6465,"num_object_clones":0,"num_object_copies":12930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11405,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.086967+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.086967+0000","last_peered":"2024-09-17T15:30:57.086967+0000","last_clean":"2024-09-17T15:30:57.086967+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.086967+0000","last_undegraded":"2024-09-17T15:30:57.086967+0000","last_fullsized":"2024-09-17T15:30:57.086967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14333","reported_seq":22619,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:02.035823+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:32:02.035823+0000","last_peered":"2024-09-17T15:32:02.035823+0000","last_clean":"2024-09-17T15:32:02.035823+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:32:02.035823+0000","last_undegraded":"2024-09-17T15:32:02.035823+0000","last_fullsized":"2024-09-17T15:32:02.035823+0000","mapping_epoch":42,"log_start":"41'4300","ondisk_log_start":"41'4300","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26558947,"num_objects":6485,"num_object_clones":0,"num_object_copies":12970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6485,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12333,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14139","reported_seq":14400,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:02.573674+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:32:02.573674+0000","last_peered":"2024-09-17T15:32:02.573674+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:32:02.573674+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'4100","ondisk_log_start":"41'4100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26226688,"num_objects":6403,"num_object_clones":0,"num_object_copies":12806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3949,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6403,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10735,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2456},{"shards":"0,2","objects":146}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14219","reported_seq":14431,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:02.743283+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:32:02.743283+0000","last_peered":"2024-09-17T15:32:02.743283+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:32:02.743283+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'4200","ondisk_log_start":"41'4200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25805283,"num_objects":6301,"num_object_clones":0,"num_object_copies":12602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4035,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6301,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12114,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":120},{"shards":"2","objects":2267}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14382","reported_seq":18395,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:02.722022+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:32:02.722022+0000","last_peered":"2024-09-17T15:32:02.722022+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:32:02.722022+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"41'4300","ondisk_log_start":"41'4300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26025984,"num_objects":6354,"num_object_clones":0,"num_object_copies":12708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2141,"num_objects_misplaced":54,"num_objects_unfound":0,"num_objects_dirty":6354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12144,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1944,"num_bytes_recovered":7852032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":54},{"shards":"2","objects":4159}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14483","reported_seq":14772,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:02.676019+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:32:02.676019+0000","last_peered":"2024-09-17T15:32:02.676019+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:32:02.676019+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'4400","ondisk_log_start":"41'4400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26554368,"num_objects":6483,"num_object_clones":0,"num_object_copies":12966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4063,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11031,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2421},{"shards":"0,2","objects":139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14402","reported_seq":14667,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:02.608317+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:32:02.608317+0000","last_peered":"2024-09-17T15:32:02.608317+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:32:02.608317+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'4400","ondisk_log_start":"41'4400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26091520,"num_objects":6370,"num_object_clones":0,"num_object_copies":12740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4101,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10689,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2272},{"shards":"0,2","objects":145}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14370","reported_seq":14667,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:02.679465+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:32:02.679465+0000","last_peered":"2024-09-17T15:32:02.679465+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:32:02.679465+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'4300","ondisk_log_start":"41'4300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26689536,"num_objects":6516,"num_object_clones":0,"num_object_copies":13032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3985,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11157,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2533},{"shards":"0,2","objects":138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14257","reported_seq":22636,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:01.921396+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:32:01.921396+0000","last_peered":"2024-09-17T15:32:01.921396+0000","last_clean":"2024-09-17T15:32:01.921396+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:32:01.921396+0000","last_undegraded":"2024-09-17T15:32:01.921396+0000","last_fullsized":"2024-09-17T15:32:01.921396+0000","mapping_epoch":44,"log_start":"41'4200","ondisk_log_start":"41'4200","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26112966,"num_objects":6377,"num_object_clones":0,"num_object_copies":12754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10699,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":421901046,"num_objects":103013,"num_object_clones":0,"num_object_copies":206026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38680,"num_objects_misplaced":54,"num_objects_unfound":0,"num_objects_dirty":103013,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":189974,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10726,"num_bytes_recovered":41350086,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1040613376,"data_stored":1040467938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160739,"ondisk_log_size":160739,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051668,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":667572,"kb_used_data":338788,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93704268,"statfs":{"total":96636764160,"available":95953170432,"internally_reserved":0,"allocated":346918912,"data_stored":346627815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":336653996},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,3,1,0,0,0,2],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724613,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627780,"kb_used_data":303152,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93744060,"statfs":{"total":96636764160,"available":95993917440,"internally_reserved":0,"allocated":310427648,"data_stored":310103570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":332394155},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247896,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":759860,"kb_used_data":376996,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93611980,"statfs":{"total":96636764160,"available":95858667520,"internally_reserved":0,"allocated":386043904,"data_stored":385752802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":392031912},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,4,1,0,0,0,0,7],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":384430080,"data_stored":384393950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":309907456,"data_stored":309838350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":346275840,"data_stored":346235638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:32:04.398 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:32:04.398 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'14229", 'reported_seq': 14474, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:02.597882+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:32:02.597882+0000', 'last_peered': '2024-09-17T15:32:02.597882+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:32:02.597882+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "41'4200", 'ondisk_log_start': "41'4200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26292224, 'num_objects': 6419, 'num_object_clones': 0, 'num_object_copies': 12838, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3989, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6419, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14229, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2430}, {'shards': '0,2', 'objects': 165}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:04.398 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:32:04.398 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'14491", 'reported_seq': 14708, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:02.542098+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:32:02.542098+0000', 'last_peered': '2024-09-17T15:32:02.542098+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:32:02.542098+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "41'4400", 'ondisk_log_start': "41'4400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26386456, 'num_objects': 6443, 'num_object_clones': 0, 'num_object_copies': 12886, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6443, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12648, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 146}, {'shards': '2', 'objects': 2318}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:32:04.398 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:32:04.398 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'14748", 'reported_seq': 15032, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:02.623130+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:32:02.623130+0000', 'last_peered': '2024-09-17T15:32:02.623130+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:32:02.623130+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "41'4700", 'ondisk_log_start': "41'4700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26636771, 'num_objects': 6504, 'num_object_clones': 0, 'num_object_copies': 13008, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4199, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6504, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11247, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 72, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2306}, {'shards': '0,2', 'objects': 148}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:04.399 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:32:04.399 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'14414", 'reported_seq': 14722, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:02.685445+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:32:02.685445+0000', 'last_peered': '2024-09-17T15:32:02.685445+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:32:02.685445+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "41'4400", 'ondisk_log_start': "41'4400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26026950, 'num_objects': 6356, 'num_object_clones': 0, 'num_object_copies': 12712, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4092, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6356, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 11183, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 82, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2264}, {'shards': '0,2', 'objects': 147}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:04.399 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:32:04.399 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'14139", 'reported_seq': 14400, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:02.573674+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:32:02.573674+0000', 'last_peered': '2024-09-17T15:32:02.573674+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:32:02.573674+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "41'4100", 'ondisk_log_start': "41'4100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26226688, 'num_objects': 6403, 'num_object_clones': 0, 'num_object_copies': 12806, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3949, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6403, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10735, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 63, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2456}, {'shards': '0,2', 'objects': 146}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:04.399 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:32:04.399 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'14219", 'reported_seq': 14431, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:02.743283+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:32:02.743283+0000', 'last_peered': '2024-09-17T15:32:02.743283+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:32:02.743283+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "41'4200", 'ondisk_log_start': "41'4200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25805283, 'num_objects': 6301, 'num_object_clones': 0, 'num_object_copies': 12602, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4035, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6301, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12114, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 120}, {'shards': '2', 'objects': 2267}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:32:04.399 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:32:04.399 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'14382", 'reported_seq': 18395, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:02.722022+0000', 'last_change': '2024-09-17T15:30:55.086473+0000', 'last_active': '2024-09-17T15:32:02.722022+0000', 'last_peered': '2024-09-17T15:32:02.722022+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:32:02.722022+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "41'4300", 'ondisk_log_start': "41'4300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26025984, 'num_objects': 6354, 'num_object_clones': 0, 'num_object_copies': 12708, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2141, 'num_objects_misplaced': 54, 'num_objects_unfound': 0, 'num_objects_dirty': 6354, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12144, 'num_write_kb': 32532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1944, 'num_bytes_recovered': 7852032, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 54}, {'shards': '2', 'objects': 4159}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:32:04.399 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:32:04.399 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'14483", 'reported_seq': 14772, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:02.676019+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:32:02.676019+0000', 'last_peered': '2024-09-17T15:32:02.676019+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:32:02.676019+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "41'4400", 'ondisk_log_start': "41'4400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26554368, 'num_objects': 6483, 'num_object_clones': 0, 'num_object_copies': 12966, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4063, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6483, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11031, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 76, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2421}, {'shards': '0,2', 'objects': 139}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:04.400 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:32:04.400 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'14402", 'reported_seq': 14667, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:02.608317+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:32:02.608317+0000', 'last_peered': '2024-09-17T15:32:02.608317+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:32:02.608317+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "41'4400", 'ondisk_log_start': "41'4400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26091520, 'num_objects': 6370, 'num_object_clones': 0, 'num_object_copies': 12740, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4101, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6370, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10689, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 59, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2272}, {'shards': '0,2', 'objects': 145}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:04.400 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:32:04.400 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'14370", 'reported_seq': 14667, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:02.679465+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:32:02.679465+0000', 'last_peered': '2024-09-17T15:32:02.679465+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:32:02.679465+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "41'4300", 'ondisk_log_start': "41'4300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26689536, 'num_objects': 6516, 'num_object_clones': 0, 'num_object_copies': 13032, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3985, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6516, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11157, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 80, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2533}, {'shards': '0,2', 'objects': 138}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:04.400 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:32:05.078 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:32:05.078 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38680/206030 objects degraded (18.774%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":462,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":103015,"data_bytes":421901065,"bytes_used":2104537088,"bytes_avail":287805755392,"bytes_total":289910292480,"degraded_objects":38680,"degraded_total":206030,"degraded_ratio":0.18773964956559724,"misplaced_objects":54,"misplaced_total":206030,"misplaced_ratio":0.00026209775275445326,"recovering_objects_per_sec":45,"recovering_bytes_per_sec":159009,"recovering_keys_per_sec":0,"num_objects_recovered":544,"num_bytes_recovered":1908736,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":571},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-17T15:31:29.695933+0000","services":{}},"progress_events":{}} 2024-09-17T15:32:05.104 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:32:05.104 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:32:05.638 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:32:05.639 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:32:05.662 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":268,"stamp":"2024-09-17T15:32:03.713914+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":421901065,"num_objects":103015,"num_object_clones":0,"num_object_copies":206030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38680,"num_objects_misplaced":54,"num_objects_unfound":0,"num_objects_dirty":103015,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":189976,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10727,"num_bytes_recovered":41350105,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2055212,"kb_used_data":1018936,"kb_used_omap":13,"kb_used_meta":1036210,"kb_avail":281060308,"statfs":{"total":289910292480,"available":287805755392,"internally_reserved":0,"allocated":1043390464,"data_stored":1042484187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1061080063},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2,3,7,7,1,0,0,2,0,7],"upper_bound":4096},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28078080,"num_objects":-6855,"num_object_clones":0,"num_object_copies":-13710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-78,"num_objects_misplaced":-466,"num_objects_unfound":0,"num_objects_dirty":-6855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6855,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":544,"num_bytes_recovered":1908736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003941"},"pg_stats":[{"pgid":"2.f","version":"45'14229","reported_seq":14474,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:02.597882+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:32:02.597882+0000","last_peered":"2024-09-17T15:32:02.597882+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:32:02.597882+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'4200","ondisk_log_start":"41'4200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26292224,"num_objects":6419,"num_object_clones":0,"num_object_copies":12838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3989,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6419,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14229,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2430},{"shards":"0,2","objects":165}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14409","reported_seq":14516,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:02.704547+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:32:02.704547+0000","last_peered":"2024-09-17T15:32:02.704547+0000","last_clean":"2024-09-17T15:32:02.704547+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:32:02.704547+0000","last_undegraded":"2024-09-17T15:32:02.704547+0000","last_fullsized":"2024-09-17T15:32:02.704547+0000","mapping_epoch":33,"log_start":"41'4400","ondisk_log_start":"41'4400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26719174,"num_objects":6525,"num_object_clones":0,"num_object_copies":13050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6525,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12766,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14491","reported_seq":14708,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:02.542098+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:32:02.542098+0000","last_peered":"2024-09-17T15:32:02.542098+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:32:02.542098+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'4400","ondisk_log_start":"41'4400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26386456,"num_objects":6443,"num_object_clones":0,"num_object_copies":12886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6443,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12648,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":146},{"shards":"2","objects":2318}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'14416","reported_seq":14490,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:02.751547+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:32:02.751547+0000","last_peered":"2024-09-17T15:32:02.751547+0000","last_clean":"2024-09-17T15:32:02.751547+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:32:02.751547+0000","last_undegraded":"2024-09-17T15:32:02.751547+0000","last_fullsized":"2024-09-17T15:32:02.751547+0000","mapping_epoch":33,"log_start":"41'4400","ondisk_log_start":"41'4400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26845184,"num_objects":6554,"num_object_clones":0,"num_object_copies":13108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6554,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14416,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14196","reported_seq":14330,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:02.693330+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:32:02.693330+0000","last_peered":"2024-09-17T15:32:02.693330+0000","last_clean":"2024-09-17T15:32:02.693330+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:32:02.693330+0000","last_undegraded":"2024-09-17T15:32:02.693330+0000","last_fullsized":"2024-09-17T15:32:02.693330+0000","mapping_epoch":32,"log_start":"41'4100","ondisk_log_start":"41'4100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26448355,"num_objects":6458,"num_object_clones":0,"num_object_copies":12916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6458,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11178,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14748","reported_seq":15032,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:02.623130+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:32:02.623130+0000","last_peered":"2024-09-17T15:32:02.623130+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:32:02.623130+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'4700","ondisk_log_start":"41'4700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26636771,"num_objects":6504,"num_object_clones":0,"num_object_copies":13008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4199,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6504,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11247,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2306},{"shards":"0,2","objects":148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14414","reported_seq":14722,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:02.685445+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:32:02.685445+0000","last_peered":"2024-09-17T15:32:02.685445+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:32:02.685445+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'4400","ondisk_log_start":"41'4400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26026950,"num_objects":6356,"num_object_clones":0,"num_object_copies":12712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4092,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6356,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11183,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2264},{"shards":"0,2","objects":147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14451","reported_seq":14584,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:02.591436+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:32:02.591436+0000","last_peered":"2024-09-17T15:32:02.591436+0000","last_clean":"2024-09-17T15:32:02.591436+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:32:02.591436+0000","last_undegraded":"2024-09-17T15:32:02.591436+0000","last_fullsized":"2024-09-17T15:32:02.591436+0000","mapping_epoch":32,"log_start":"41'4400","ondisk_log_start":"41'4400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26480640,"num_objects":6465,"num_object_clones":0,"num_object_copies":12930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11405,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.086967+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.086967+0000","last_peered":"2024-09-17T15:30:57.086967+0000","last_clean":"2024-09-17T15:30:57.086967+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.086967+0000","last_undegraded":"2024-09-17T15:30:57.086967+0000","last_fullsized":"2024-09-17T15:30:57.086967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14333","reported_seq":22619,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:02.035823+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:32:02.035823+0000","last_peered":"2024-09-17T15:32:02.035823+0000","last_clean":"2024-09-17T15:32:02.035823+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:32:02.035823+0000","last_undegraded":"2024-09-17T15:32:02.035823+0000","last_fullsized":"2024-09-17T15:32:02.035823+0000","mapping_epoch":42,"log_start":"41'4300","ondisk_log_start":"41'4300","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26558947,"num_objects":6485,"num_object_clones":0,"num_object_copies":12970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6485,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12333,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14139","reported_seq":14400,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:02.573674+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:32:02.573674+0000","last_peered":"2024-09-17T15:32:02.573674+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:32:02.573674+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'4100","ondisk_log_start":"41'4100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26226688,"num_objects":6403,"num_object_clones":0,"num_object_copies":12806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3949,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6403,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10735,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2456},{"shards":"0,2","objects":146}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14219","reported_seq":14431,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:02.743283+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:32:02.743283+0000","last_peered":"2024-09-17T15:32:02.743283+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:32:02.743283+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'4200","ondisk_log_start":"41'4200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25805283,"num_objects":6301,"num_object_clones":0,"num_object_copies":12602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4035,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6301,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12114,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":120},{"shards":"2","objects":2267}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14382","reported_seq":18395,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:02.722022+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:32:02.722022+0000","last_peered":"2024-09-17T15:32:02.722022+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:32:02.722022+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"41'4300","ondisk_log_start":"41'4300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26025984,"num_objects":6354,"num_object_clones":0,"num_object_copies":12708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2141,"num_objects_misplaced":54,"num_objects_unfound":0,"num_objects_dirty":6354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12144,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1944,"num_bytes_recovered":7852032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":54},{"shards":"2","objects":4159}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14483","reported_seq":14772,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:02.676019+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:32:02.676019+0000","last_peered":"2024-09-17T15:32:02.676019+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:32:02.676019+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'4400","ondisk_log_start":"41'4400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26554368,"num_objects":6483,"num_object_clones":0,"num_object_copies":12966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4063,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11031,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2421},{"shards":"0,2","objects":139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14402","reported_seq":14667,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:02.608317+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:32:02.608317+0000","last_peered":"2024-09-17T15:32:02.608317+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:32:02.608317+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'4400","ondisk_log_start":"41'4400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26091520,"num_objects":6370,"num_object_clones":0,"num_object_copies":12740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4101,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10689,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2272},{"shards":"0,2","objects":145}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14370","reported_seq":14667,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:02.679465+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:32:02.679465+0000","last_peered":"2024-09-17T15:32:02.679465+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:32:02.679465+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'4300","ondisk_log_start":"41'4300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26689536,"num_objects":6516,"num_object_clones":0,"num_object_copies":13032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3985,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11157,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2533},{"shards":"0,2","objects":138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14257","reported_seq":22636,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:01.921396+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:32:01.921396+0000","last_peered":"2024-09-17T15:32:01.921396+0000","last_clean":"2024-09-17T15:32:01.921396+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:32:01.921396+0000","last_undegraded":"2024-09-17T15:32:01.921396+0000","last_fullsized":"2024-09-17T15:32:01.921396+0000","mapping_epoch":44,"log_start":"41'4200","ondisk_log_start":"41'4200","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26112966,"num_objects":6377,"num_object_clones":0,"num_object_copies":12754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10699,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":421901046,"num_objects":103013,"num_object_clones":0,"num_object_copies":206026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38680,"num_objects_misplaced":54,"num_objects_unfound":0,"num_objects_dirty":103013,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":189974,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10726,"num_bytes_recovered":41350086,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1040613376,"data_stored":1040467938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160739,"ondisk_log_size":160739,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051668,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":667572,"kb_used_data":338788,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93704268,"statfs":{"total":96636764160,"available":95953170432,"internally_reserved":0,"allocated":346918912,"data_stored":346627815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":336653996},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,3,1,0,0,0,2],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724613,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627780,"kb_used_data":303152,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93744060,"statfs":{"total":96636764160,"available":95993917440,"internally_reserved":0,"allocated":310427648,"data_stored":310103570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":332394155},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247896,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":759860,"kb_used_data":376996,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93611980,"statfs":{"total":96636764160,"available":95858667520,"internally_reserved":0,"allocated":386043904,"data_stored":385752802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":392031912},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,4,1,0,0,0,0,7],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":384430080,"data_stored":384393950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":309907456,"data_stored":309838350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":346275840,"data_stored":346235638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:32:08.664 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:32:09.208 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:32:09.210 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:32:09.240 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":270,"stamp":"2024-09-17T15:32:07.715173+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":415413001,"num_objects":101431,"num_object_clones":0,"num_object_copies":202862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38656,"num_objects_misplaced":54,"num_objects_unfound":0,"num_objects_dirty":101431,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":191560,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10751,"num_bytes_recovered":41350105,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160826,"ondisk_log_size":160826,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2060904,"kb_used_data":1010744,"kb_used_omap":13,"kb_used_meta":1050098,"kb_avail":281054616,"statfs":{"total":289910292480,"available":287799926784,"internally_reserved":0,"allocated":1035001856,"data_stored":1034095579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1075301375},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,1,8,2,4,0,0,8],"upper_bound":1024},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24985600,"num_objects":-6100,"num_object_clones":0,"num_object_copies":-12200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-82,"num_objects_misplaced":-338,"num_objects_unfound":0,"num_objects_dirty":-6100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6100,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":420,"num_bytes_recovered":1384448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003675"},"pg_stats":[{"pgid":"2.f","version":"45'14395","reported_seq":14646,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:07.644424+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:32:07.644424+0000","last_peered":"2024-09-17T15:32:07.644424+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:32:07.644424+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'4300","ondisk_log_start":"41'4300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25612288,"num_objects":6253,"num_object_clones":0,"num_object_copies":12506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14395,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2267}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14409","reported_seq":14516,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:02.704547+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:32:02.704547+0000","last_peered":"2024-09-17T15:32:02.704547+0000","last_clean":"2024-09-17T15:32:02.704547+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:32:02.704547+0000","last_undegraded":"2024-09-17T15:32:02.704547+0000","last_fullsized":"2024-09-17T15:32:02.704547+0000","mapping_epoch":33,"log_start":"41'4400","ondisk_log_start":"41'4400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26719174,"num_objects":6525,"num_object_clones":0,"num_object_copies":13050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6525,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12766,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14491","reported_seq":14708,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:02.542098+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:32:02.542098+0000","last_peered":"2024-09-17T15:32:02.542098+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:32:02.542098+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'4400","ondisk_log_start":"41'4400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26386456,"num_objects":6443,"num_object_clones":0,"num_object_copies":12886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6443,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12648,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":146},{"shards":"2","objects":2318}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'14416","reported_seq":14490,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:02.751547+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:32:02.751547+0000","last_peered":"2024-09-17T15:32:02.751547+0000","last_clean":"2024-09-17T15:32:02.751547+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:32:02.751547+0000","last_undegraded":"2024-09-17T15:32:02.751547+0000","last_fullsized":"2024-09-17T15:32:02.751547+0000","mapping_epoch":33,"log_start":"41'4400","ondisk_log_start":"41'4400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26845184,"num_objects":6554,"num_object_clones":0,"num_object_copies":13108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6554,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14416,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14320","reported_seq":14454,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:07.421994+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:32:07.421994+0000","last_peered":"2024-09-17T15:32:07.421994+0000","last_clean":"2024-09-17T15:32:07.421994+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:32:07.421994+0000","last_undegraded":"2024-09-17T15:32:07.421994+0000","last_fullsized":"2024-09-17T15:32:07.421994+0000","mapping_epoch":32,"log_start":"41'4300","ondisk_log_start":"41'4300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25940451,"num_objects":6334,"num_object_clones":0,"num_object_copies":12668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6334,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11302,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14897","reported_seq":15193,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:07.676579+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:32:07.676579+0000","last_peered":"2024-09-17T15:32:07.676579+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:32:07.676579+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'4800","ondisk_log_start":"41'4800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26026467,"num_objects":6355,"num_object_clones":0,"num_object_copies":12710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6355,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11396,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2163},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14561","reported_seq":14873,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:07.677722+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:32:07.677722+0000","last_peered":"2024-09-17T15:32:07.677722+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:32:07.677722+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'4500","ondisk_log_start":"41'4500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25424838,"num_objects":6209,"num_object_clones":0,"num_object_copies":12418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4090,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6209,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11330,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2119},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14595","reported_seq":14728,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:07.453738+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:32:07.453738+0000","last_peered":"2024-09-17T15:32:07.453738+0000","last_clean":"2024-09-17T15:32:07.453738+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:32:07.453738+0000","last_undegraded":"2024-09-17T15:32:07.453738+0000","last_fullsized":"2024-09-17T15:32:07.453738+0000","mapping_epoch":32,"log_start":"41'4500","ondisk_log_start":"41'4500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25890816,"num_objects":6321,"num_object_clones":0,"num_object_copies":12642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6321,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11549,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.086967+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.086967+0000","last_peered":"2024-09-17T15:30:57.086967+0000","last_clean":"2024-09-17T15:30:57.086967+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.086967+0000","last_undegraded":"2024-09-17T15:30:57.086967+0000","last_fullsized":"2024-09-17T15:30:57.086967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14464","reported_seq":22751,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:07.039033+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:32:07.039033+0000","last_peered":"2024-09-17T15:32:07.039033+0000","last_clean":"2024-09-17T15:32:07.039033+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:32:07.039033+0000","last_undegraded":"2024-09-17T15:32:07.039033+0000","last_fullsized":"2024-09-17T15:32:07.039033+0000","mapping_epoch":42,"log_start":"41'4400","ondisk_log_start":"41'4400","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26022371,"num_objects":6354,"num_object_clones":0,"num_object_copies":12708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6354,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12464,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14286","reported_seq":14555,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:07.672313+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:32:07.672313+0000","last_peered":"2024-09-17T15:32:07.672313+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:32:07.672313+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'4200","ondisk_log_start":"41'4200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25624576,"num_objects":6256,"num_object_clones":0,"num_object_copies":12512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10882,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2313}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14219","reported_seq":14431,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:02.743283+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:32:02.743283+0000","last_peered":"2024-09-17T15:32:02.743283+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:32:02.743283+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'4200","ondisk_log_start":"41'4200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25805283,"num_objects":6301,"num_object_clones":0,"num_object_copies":12602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4035,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6301,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12114,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":120},{"shards":"2","objects":2267}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14382","reported_seq":18395,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:02.722022+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:32:02.722022+0000","last_peered":"2024-09-17T15:32:02.722022+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:32:02.722022+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"41'4300","ondisk_log_start":"41'4300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26025984,"num_objects":6354,"num_object_clones":0,"num_object_copies":12708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2141,"num_objects_misplaced":54,"num_objects_unfound":0,"num_objects_dirty":6354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12144,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1944,"num_bytes_recovered":7852032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":54},{"shards":"2","objects":4159}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14623","reported_seq":14921,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:07.628459+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:32:07.628459+0000","last_peered":"2024-09-17T15:32:07.628459+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:32:07.628459+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'4600","ondisk_log_start":"41'4600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25980928,"num_objects":6343,"num_object_clones":0,"num_object_copies":12686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4060,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11171,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":79,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2284}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14548","reported_seq":14819,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:07.614341+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:32:07.614341+0000","last_peered":"2024-09-17T15:32:07.614341+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:32:07.614341+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'4500","ondisk_log_start":"41'4500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25493504,"num_objects":6224,"num_object_clones":0,"num_object_copies":12448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10835,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2129}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14509","reported_seq":14812,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:07.657063+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:32:07.657063+0000","last_peered":"2024-09-17T15:32:07.657063+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:32:07.657063+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'4500","ondisk_log_start":"41'4500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26120192,"num_objects":6377,"num_object_clones":0,"num_object_copies":12754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11296,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2397}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14408","reported_seq":22787,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:07.025894+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:32:07.025894+0000","last_peered":"2024-09-17T15:32:07.025894+0000","last_clean":"2024-09-17T15:32:07.025894+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:32:07.025894+0000","last_undegraded":"2024-09-17T15:32:07.025894+0000","last_fullsized":"2024-09-17T15:32:07.025894+0000","mapping_epoch":44,"log_start":"41'4400","ondisk_log_start":"41'4400","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25494470,"num_objects":6226,"num_object_clones":0,"num_object_copies":12452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6226,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10850,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":415412982,"num_objects":101429,"num_object_clones":0,"num_object_copies":202858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38656,"num_objects_misplaced":54,"num_objects_unfound":0,"num_objects_dirty":101429,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":191558,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10750,"num_bytes_recovered":41350086,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1032253440,"data_stored":1032108002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160823,"ondisk_log_size":160823,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051668,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":667572,"kb_used_data":338788,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93704268,"statfs":{"total":96636764160,"available":95953170432,"internally_reserved":0,"allocated":346918912,"data_stored":346627815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":336653996},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,3,1,0,0,0,2],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724614,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":630760,"kb_used_data":301976,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93741080,"statfs":{"total":96636764160,"available":95990865920,"internally_reserved":0,"allocated":309223424,"data_stored":308899346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":336653995},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247897,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":762572,"kb_used_data":369980,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93609268,"statfs":{"total":96636764160,"available":95855890432,"internally_reserved":0,"allocated":378859520,"data_stored":378568418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":401993384},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,1,4,0,0,6],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":377389056,"data_stored":377352926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":308588544,"data_stored":308519438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":346275840,"data_stored":346235638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:32:09.242 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:32:09.785 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:32:09.786 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:32:09.810 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":271,"stamp":"2024-09-17T15:32:09.715871+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":412611337,"num_objects":100747,"num_object_clones":0,"num_object_copies":201494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38516,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100747,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":192244,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10945,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160810,"ondisk_log_size":160810,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2044116,"kb_used_data":1006436,"kb_used_omap":13,"kb_used_meta":1037618,"kb_avail":281071404,"statfs":{"total":289910292480,"available":287817117696,"internally_reserved":0,"allocated":1030590464,"data_stored":1029684187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1062521855},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,1,7,2,4,0,0,6],"upper_bound":1024},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21438464,"num_objects":-5234,"num_object_clones":0,"num_object_copies":-10468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-210,"num_objects_misplaced":-392,"num_objects_unfound":0,"num_objects_dirty":-5234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5234,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":602,"num_bytes_recovered":1605632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003762"},"pg_stats":[{"pgid":"2.f","version":"45'14395","reported_seq":14646,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:07.644424+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:32:07.644424+0000","last_peered":"2024-09-17T15:32:07.644424+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:32:07.644424+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'4300","ondisk_log_start":"41'4300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25612288,"num_objects":6253,"num_object_clones":0,"num_object_copies":12506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14395,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2267}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14543","reported_seq":14651,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:07.761147+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:32:07.761147+0000","last_peered":"2024-09-17T15:32:07.761147+0000","last_clean":"2024-09-17T15:32:07.761147+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:32:07.761147+0000","last_undegraded":"2024-09-17T15:32:07.761147+0000","last_fullsized":"2024-09-17T15:32:07.761147+0000","mapping_epoch":33,"log_start":"41'4500","ondisk_log_start":"41'4500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26170310,"num_objects":6391,"num_object_clones":0,"num_object_copies":12782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6391,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12900,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14645","reported_seq":14862,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:07.777487+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:32:07.777487+0000","last_peered":"2024-09-17T15:32:07.777487+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:32:07.777487+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'4600","ondisk_log_start":"41'4600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25755672,"num_objects":6289,"num_object_clones":0,"num_object_copies":12578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6289,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12802,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2164}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'14553","reported_seq":14628,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:07.790343+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:32:07.790343+0000","last_peered":"2024-09-17T15:32:07.790343+0000","last_clean":"2024-09-17T15:32:07.790343+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:32:07.790343+0000","last_undegraded":"2024-09-17T15:32:07.790343+0000","last_fullsized":"2024-09-17T15:32:07.790343+0000","mapping_epoch":33,"log_start":"41'4500","ondisk_log_start":"41'4500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26284032,"num_objects":6417,"num_object_clones":0,"num_object_copies":12834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6417,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14553,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14320","reported_seq":14454,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:07.421994+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:32:07.421994+0000","last_peered":"2024-09-17T15:32:07.421994+0000","last_clean":"2024-09-17T15:32:07.421994+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:32:07.421994+0000","last_undegraded":"2024-09-17T15:32:07.421994+0000","last_fullsized":"2024-09-17T15:32:07.421994+0000","mapping_epoch":32,"log_start":"41'4300","ondisk_log_start":"41'4300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25940451,"num_objects":6334,"num_object_clones":0,"num_object_copies":12668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6334,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11302,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14897","reported_seq":15193,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:07.676579+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:32:07.676579+0000","last_peered":"2024-09-17T15:32:07.676579+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:32:07.676579+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'4800","ondisk_log_start":"41'4800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26026467,"num_objects":6355,"num_object_clones":0,"num_object_copies":12710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6355,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11396,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2163},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14561","reported_seq":14873,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:07.677722+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:32:07.677722+0000","last_peered":"2024-09-17T15:32:07.677722+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:32:07.677722+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'4500","ondisk_log_start":"41'4500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25424838,"num_objects":6209,"num_object_clones":0,"num_object_copies":12418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4090,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6209,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11330,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2119},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14595","reported_seq":14728,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:07.453738+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:32:07.453738+0000","last_peered":"2024-09-17T15:32:07.453738+0000","last_clean":"2024-09-17T15:32:07.453738+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:32:07.453738+0000","last_undegraded":"2024-09-17T15:32:07.453738+0000","last_fullsized":"2024-09-17T15:32:07.453738+0000","mapping_epoch":32,"log_start":"41'4500","ondisk_log_start":"41'4500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25890816,"num_objects":6321,"num_object_clones":0,"num_object_copies":12642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6321,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11549,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.086967+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.086967+0000","last_peered":"2024-09-17T15:30:57.086967+0000","last_clean":"2024-09-17T15:30:57.086967+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.086967+0000","last_undegraded":"2024-09-17T15:30:57.086967+0000","last_fullsized":"2024-09-17T15:30:57.086967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14464","reported_seq":22751,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:07.039033+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:32:07.039033+0000","last_peered":"2024-09-17T15:32:07.039033+0000","last_clean":"2024-09-17T15:32:07.039033+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:32:07.039033+0000","last_undegraded":"2024-09-17T15:32:07.039033+0000","last_fullsized":"2024-09-17T15:32:07.039033+0000","mapping_epoch":42,"log_start":"41'4400","ondisk_log_start":"41'4400","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26022371,"num_objects":6354,"num_object_clones":0,"num_object_copies":12708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6354,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12464,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14286","reported_seq":14555,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:07.672313+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:32:07.672313+0000","last_peered":"2024-09-17T15:32:07.672313+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:32:07.672313+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'4200","ondisk_log_start":"41'4200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25624576,"num_objects":6256,"num_object_clones":0,"num_object_copies":12512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10882,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2313}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14344","reported_seq":14557,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:07.720374+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:32:07.720374+0000","last_peered":"2024-09-17T15:32:07.720374+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:32:07.720374+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'4300","ondisk_log_start":"41'4300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25293283,"num_objects":6176,"num_object_clones":0,"num_object_copies":12352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6176,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12239,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":2143}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14516","reported_seq":18915,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:07.804482+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:32:07.804482+0000","last_peered":"2024-09-17T15:32:07.804482+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:32:07.804482+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"41'4500","ondisk_log_start":"41'4500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25477120,"num_objects":6220,"num_object_clones":0,"num_object_copies":12440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2002,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12278,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2137,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4218}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14623","reported_seq":14921,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:07.628459+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:32:07.628459+0000","last_peered":"2024-09-17T15:32:07.628459+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:32:07.628459+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'4600","ondisk_log_start":"41'4600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25980928,"num_objects":6343,"num_object_clones":0,"num_object_copies":12686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4060,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11171,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":79,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2284}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14548","reported_seq":14819,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:07.614341+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:32:07.614341+0000","last_peered":"2024-09-17T15:32:07.614341+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:32:07.614341+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'4500","ondisk_log_start":"41'4500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25493504,"num_objects":6224,"num_object_clones":0,"num_object_copies":12448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10835,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2129}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14509","reported_seq":14812,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:07.657063+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:32:07.657063+0000","last_peered":"2024-09-17T15:32:07.657063+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:32:07.657063+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'4500","ondisk_log_start":"41'4500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26120192,"num_objects":6377,"num_object_clones":0,"num_object_copies":12754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11296,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2397}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14408","reported_seq":22787,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:07.025894+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:32:07.025894+0000","last_peered":"2024-09-17T15:32:07.025894+0000","last_clean":"2024-09-17T15:32:07.025894+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:32:07.025894+0000","last_undegraded":"2024-09-17T15:32:07.025894+0000","last_fullsized":"2024-09-17T15:32:07.025894+0000","mapping_epoch":44,"log_start":"41'4400","ondisk_log_start":"41'4400","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25494470,"num_objects":6226,"num_object_clones":0,"num_object_copies":12452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6226,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10850,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":412611318,"num_objects":100745,"num_object_clones":0,"num_object_copies":201490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38516,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100745,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":192242,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10944,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1027825664,"data_stored":1027680226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160807,"ondisk_log_size":160807,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051669,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":650784,"kb_used_data":334480,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93721056,"statfs":{"total":96636764160,"available":95970361344,"internally_reserved":0,"allocated":342507520,"data_stored":342216423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":323874476},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724614,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":630760,"kb_used_data":301976,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93741080,"statfs":{"total":96636764160,"available":95990865920,"internally_reserved":0,"allocated":309223424,"data_stored":308899346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":336653995},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247897,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":762572,"kb_used_data":369980,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93609268,"statfs":{"total":96636764160,"available":95855890432,"internally_reserved":0,"allocated":378859520,"data_stored":378568418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":401993384},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,1,4,0,0,6],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":377389056,"data_stored":377352926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":308588544,"data_stored":308519438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":341848064,"data_stored":341807862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:32:09.811 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:32:09.811 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'14395", 'reported_seq': 14646, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:07.644424+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:32:07.644424+0000', 'last_peered': '2024-09-17T15:32:07.644424+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:32:07.644424+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "41'4300", 'ondisk_log_start': "41'4300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25612288, 'num_objects': 6253, 'num_object_clones': 0, 'num_object_copies': 12506, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6253, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14395, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 84, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2267}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:09.812 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:32:09.812 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'14645", 'reported_seq': 14862, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:07.777487+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:32:07.777487+0000', 'last_peered': '2024-09-17T15:32:07.777487+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:32:07.777487+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "41'4600", 'ondisk_log_start': "41'4600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25755672, 'num_objects': 6289, 'num_object_clones': 0, 'num_object_copies': 12578, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6289, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12802, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:32:09.812 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:32:09.812 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'14897", 'reported_seq': 15193, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:07.676579+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:32:07.676579+0000', 'last_peered': '2024-09-17T15:32:07.676579+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:32:07.676579+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "41'4800", 'ondisk_log_start': "41'4800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26026467, 'num_objects': 6355, 'num_object_clones': 0, 'num_object_copies': 12710, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6355, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11396, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2163}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:09.812 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:32:09.812 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'14561", 'reported_seq': 14873, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:07.677722+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:32:07.677722+0000', 'last_peered': '2024-09-17T15:32:07.677722+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:32:07.677722+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "41'4500", 'ondisk_log_start': "41'4500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25424838, 'num_objects': 6209, 'num_object_clones': 0, 'num_object_copies': 12418, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4090, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6209, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 11330, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 84, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2119}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:09.812 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:32:09.812 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'14286", 'reported_seq': 14555, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:07.672313+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:32:07.672313+0000', 'last_peered': '2024-09-17T15:32:07.672313+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:32:07.672313+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "41'4200", 'ondisk_log_start': "41'4200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25624576, 'num_objects': 6256, 'num_object_clones': 0, 'num_object_copies': 12512, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6256, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10882, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2313}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:09.812 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:32:09.812 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'14344", 'reported_seq': 14557, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:07.720374+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:32:07.720374+0000', 'last_peered': '2024-09-17T15:32:07.720374+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:32:07.720374+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "41'4300", 'ondisk_log_start': "41'4300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25293283, 'num_objects': 6176, 'num_object_clones': 0, 'num_object_copies': 12352, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4034, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6176, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12239, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 2143}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:32:09.812 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:32:09.813 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'14516", 'reported_seq': 18915, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:07.804482+0000', 'last_change': '2024-09-17T15:30:55.086473+0000', 'last_active': '2024-09-17T15:32:07.804482+0000', 'last_peered': '2024-09-17T15:32:07.804482+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:32:07.804482+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "41'4500", 'ondisk_log_start': "41'4500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25477120, 'num_objects': 6220, 'num_object_clones': 0, 'num_object_copies': 12440, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2002, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6220, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12278, 'num_write_kb': 32532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2137, 'num_bytes_recovered': 8073216, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 4218}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:32:09.813 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:32:09.813 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'14623", 'reported_seq': 14921, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:07.628459+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:32:07.628459+0000', 'last_peered': '2024-09-17T15:32:07.628459+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:32:07.628459+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "41'4600", 'ondisk_log_start': "41'4600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25980928, 'num_objects': 6343, 'num_object_clones': 0, 'num_object_copies': 12686, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4060, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6343, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11171, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 79, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2284}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:09.813 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:32:09.813 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'14548", 'reported_seq': 14819, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:07.614341+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:32:07.614341+0000', 'last_peered': '2024-09-17T15:32:07.614341+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:32:07.614341+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "41'4500", 'ondisk_log_start': "41'4500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25493504, 'num_objects': 6224, 'num_object_clones': 0, 'num_object_copies': 12448, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4098, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6224, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10835, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 62, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2129}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:09.813 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:32:09.813 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'14509", 'reported_seq': 14812, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:07.657063+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:32:07.657063+0000', 'last_peered': '2024-09-17T15:32:07.657063+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:32:07.657063+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "41'4500", 'ondisk_log_start': "41'4500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26120192, 'num_objects': 6377, 'num_object_clones': 0, 'num_object_copies': 12754, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6377, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11296, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2397}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:09.813 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:32:10.529 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:32:10.529 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38516/201494 objects degraded (19.115%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":467,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":100747,"data_bytes":412611337,"bytes_used":2093174784,"bytes_avail":287817117696,"bytes_total":289910292480,"degraded_objects":38516,"degraded_total":201494,"degraded_ratio":0.19115209385887422,"recovering_objects_per_sec":50,"recovering_bytes_per_sec":133760,"recovering_keys_per_sec":0,"num_objects_recovered":602,"num_bytes_recovered":1605632,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":436},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-17T15:31:29.695933+0000","services":{}},"progress_events":{}} 2024-09-17T15:32:10.555 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:32:10.556 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:32:11.136 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:32:11.136 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:32:11.160 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":271,"stamp":"2024-09-17T15:32:09.715871+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":412611337,"num_objects":100747,"num_object_clones":0,"num_object_copies":201494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38516,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100747,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":192244,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10945,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160810,"ondisk_log_size":160810,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2044116,"kb_used_data":1006436,"kb_used_omap":13,"kb_used_meta":1037618,"kb_avail":281071404,"statfs":{"total":289910292480,"available":287817117696,"internally_reserved":0,"allocated":1030590464,"data_stored":1029684187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1062521855},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,1,7,2,4,0,0,6],"upper_bound":1024},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21438464,"num_objects":-5234,"num_object_clones":0,"num_object_copies":-10468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-210,"num_objects_misplaced":-392,"num_objects_unfound":0,"num_objects_dirty":-5234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5234,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":602,"num_bytes_recovered":1605632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003762"},"pg_stats":[{"pgid":"2.f","version":"45'14395","reported_seq":14646,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:07.644424+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:32:07.644424+0000","last_peered":"2024-09-17T15:32:07.644424+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:32:07.644424+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'4300","ondisk_log_start":"41'4300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25612288,"num_objects":6253,"num_object_clones":0,"num_object_copies":12506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14395,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2267}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14543","reported_seq":14651,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:07.761147+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:32:07.761147+0000","last_peered":"2024-09-17T15:32:07.761147+0000","last_clean":"2024-09-17T15:32:07.761147+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:32:07.761147+0000","last_undegraded":"2024-09-17T15:32:07.761147+0000","last_fullsized":"2024-09-17T15:32:07.761147+0000","mapping_epoch":33,"log_start":"41'4500","ondisk_log_start":"41'4500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26170310,"num_objects":6391,"num_object_clones":0,"num_object_copies":12782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6391,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12900,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14645","reported_seq":14862,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:07.777487+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:32:07.777487+0000","last_peered":"2024-09-17T15:32:07.777487+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:32:07.777487+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'4600","ondisk_log_start":"41'4600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25755672,"num_objects":6289,"num_object_clones":0,"num_object_copies":12578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6289,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12802,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2164}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'14553","reported_seq":14628,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:07.790343+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:32:07.790343+0000","last_peered":"2024-09-17T15:32:07.790343+0000","last_clean":"2024-09-17T15:32:07.790343+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:32:07.790343+0000","last_undegraded":"2024-09-17T15:32:07.790343+0000","last_fullsized":"2024-09-17T15:32:07.790343+0000","mapping_epoch":33,"log_start":"41'4500","ondisk_log_start":"41'4500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26284032,"num_objects":6417,"num_object_clones":0,"num_object_copies":12834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6417,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14553,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14320","reported_seq":14454,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:07.421994+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:32:07.421994+0000","last_peered":"2024-09-17T15:32:07.421994+0000","last_clean":"2024-09-17T15:32:07.421994+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:32:07.421994+0000","last_undegraded":"2024-09-17T15:32:07.421994+0000","last_fullsized":"2024-09-17T15:32:07.421994+0000","mapping_epoch":32,"log_start":"41'4300","ondisk_log_start":"41'4300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25940451,"num_objects":6334,"num_object_clones":0,"num_object_copies":12668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6334,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11302,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14897","reported_seq":15193,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:07.676579+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:32:07.676579+0000","last_peered":"2024-09-17T15:32:07.676579+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:32:07.676579+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'4800","ondisk_log_start":"41'4800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26026467,"num_objects":6355,"num_object_clones":0,"num_object_copies":12710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6355,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11396,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2163},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14561","reported_seq":14873,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:07.677722+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:32:07.677722+0000","last_peered":"2024-09-17T15:32:07.677722+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:32:07.677722+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'4500","ondisk_log_start":"41'4500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25424838,"num_objects":6209,"num_object_clones":0,"num_object_copies":12418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4090,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6209,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11330,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2119},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14595","reported_seq":14728,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:07.453738+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:32:07.453738+0000","last_peered":"2024-09-17T15:32:07.453738+0000","last_clean":"2024-09-17T15:32:07.453738+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:32:07.453738+0000","last_undegraded":"2024-09-17T15:32:07.453738+0000","last_fullsized":"2024-09-17T15:32:07.453738+0000","mapping_epoch":32,"log_start":"41'4500","ondisk_log_start":"41'4500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25890816,"num_objects":6321,"num_object_clones":0,"num_object_copies":12642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6321,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11549,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.086967+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.086967+0000","last_peered":"2024-09-17T15:30:57.086967+0000","last_clean":"2024-09-17T15:30:57.086967+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.086967+0000","last_undegraded":"2024-09-17T15:30:57.086967+0000","last_fullsized":"2024-09-17T15:30:57.086967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14464","reported_seq":22751,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:07.039033+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:32:07.039033+0000","last_peered":"2024-09-17T15:32:07.039033+0000","last_clean":"2024-09-17T15:32:07.039033+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:32:07.039033+0000","last_undegraded":"2024-09-17T15:32:07.039033+0000","last_fullsized":"2024-09-17T15:32:07.039033+0000","mapping_epoch":42,"log_start":"41'4400","ondisk_log_start":"41'4400","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26022371,"num_objects":6354,"num_object_clones":0,"num_object_copies":12708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6354,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12464,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14286","reported_seq":14555,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:07.672313+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:32:07.672313+0000","last_peered":"2024-09-17T15:32:07.672313+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:32:07.672313+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'4200","ondisk_log_start":"41'4200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25624576,"num_objects":6256,"num_object_clones":0,"num_object_copies":12512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10882,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2313}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14344","reported_seq":14557,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:07.720374+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:32:07.720374+0000","last_peered":"2024-09-17T15:32:07.720374+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:32:07.720374+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'4300","ondisk_log_start":"41'4300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25293283,"num_objects":6176,"num_object_clones":0,"num_object_copies":12352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6176,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12239,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":2143}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14516","reported_seq":18915,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:07.804482+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:32:07.804482+0000","last_peered":"2024-09-17T15:32:07.804482+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:32:07.804482+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"41'4500","ondisk_log_start":"41'4500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25477120,"num_objects":6220,"num_object_clones":0,"num_object_copies":12440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2002,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12278,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2137,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4218}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14623","reported_seq":14921,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:07.628459+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:32:07.628459+0000","last_peered":"2024-09-17T15:32:07.628459+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:32:07.628459+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'4600","ondisk_log_start":"41'4600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25980928,"num_objects":6343,"num_object_clones":0,"num_object_copies":12686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4060,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11171,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":79,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2284}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14548","reported_seq":14819,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:07.614341+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:32:07.614341+0000","last_peered":"2024-09-17T15:32:07.614341+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:32:07.614341+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'4500","ondisk_log_start":"41'4500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25493504,"num_objects":6224,"num_object_clones":0,"num_object_copies":12448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10835,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2129}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14509","reported_seq":14812,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:07.657063+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:32:07.657063+0000","last_peered":"2024-09-17T15:32:07.657063+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:32:07.657063+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'4500","ondisk_log_start":"41'4500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26120192,"num_objects":6377,"num_object_clones":0,"num_object_copies":12754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11296,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2397}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14408","reported_seq":22787,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:07.025894+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:32:07.025894+0000","last_peered":"2024-09-17T15:32:07.025894+0000","last_clean":"2024-09-17T15:32:07.025894+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:32:07.025894+0000","last_undegraded":"2024-09-17T15:32:07.025894+0000","last_fullsized":"2024-09-17T15:32:07.025894+0000","mapping_epoch":44,"log_start":"41'4400","ondisk_log_start":"41'4400","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25494470,"num_objects":6226,"num_object_clones":0,"num_object_copies":12452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6226,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10850,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":412611318,"num_objects":100745,"num_object_clones":0,"num_object_copies":201490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38516,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100745,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":192242,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10944,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1027825664,"data_stored":1027680226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160807,"ondisk_log_size":160807,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051669,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":650784,"kb_used_data":334480,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93721056,"statfs":{"total":96636764160,"available":95970361344,"internally_reserved":0,"allocated":342507520,"data_stored":342216423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":323874476},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724614,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":630760,"kb_used_data":301976,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93741080,"statfs":{"total":96636764160,"available":95990865920,"internally_reserved":0,"allocated":309223424,"data_stored":308899346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":336653995},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247897,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":762572,"kb_used_data":369980,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93609268,"statfs":{"total":96636764160,"available":95855890432,"internally_reserved":0,"allocated":378859520,"data_stored":378568418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":401993384},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,1,4,0,0,6],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":377389056,"data_stored":377352926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":308588544,"data_stored":308519438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":341848064,"data_stored":341807862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:32:14.163 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:32:14.757 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:32:14.758 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:32:14.787 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":273,"stamp":"2024-09-17T15:32:13.717290+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":402752265,"num_objects":98340,"num_object_clones":0,"num_object_copies":196680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38370,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98340,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":194651,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11091,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160817,"ondisk_log_size":160817,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2026520,"kb_used_data":988840,"kb_used_omap":13,"kb_used_meta":1037618,"kb_avail":281089000,"statfs":{"total":289910292480,"available":287835136000,"internally_reserved":0,"allocated":1012572160,"data_stored":1011665883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1062521855},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,5,4,10,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28389376,"num_objects":-6931,"num_object_clones":0,"num_object_copies":-13862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-356,"num_objects_misplaced":-235,"num_objects_unfound":0,"num_objects_dirty":-6931,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6931,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":591,"num_bytes_recovered":962560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003991"},"pg_stats":[{"pgid":"2.f","version":"45'14551","reported_seq":14804,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:12.674272+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:32:12.674272+0000","last_peered":"2024-09-17T15:32:12.674272+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:32:12.674272+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'4500","ondisk_log_start":"41'4500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24973312,"num_objects":6097,"num_object_clones":0,"num_object_copies":12194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6097,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14551,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2113}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14692","reported_seq":14799,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:12.803970+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:32:12.803970+0000","last_peered":"2024-09-17T15:32:12.803970+0000","last_clean":"2024-09-17T15:32:12.803970+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:32:12.803970+0000","last_undegraded":"2024-09-17T15:32:12.803970+0000","last_fullsized":"2024-09-17T15:32:12.803970+0000","mapping_epoch":33,"log_start":"41'4600","ondisk_log_start":"41'4600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25560006,"num_objects":6242,"num_object_clones":0,"num_object_copies":12484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6242,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13049,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14802","reported_seq":15019,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:12.793020+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:32:12.793020+0000","last_peered":"2024-09-17T15:32:12.793020+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:32:12.793020+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'4800","ondisk_log_start":"41'4800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25112600,"num_objects":6132,"num_object_clones":0,"num_object_copies":12264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6132,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12959,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2007}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'14730","reported_seq":14805,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:12.694500+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:32:12.694500+0000","last_peered":"2024-09-17T15:32:12.694500+0000","last_clean":"2024-09-17T15:32:12.694500+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:32:12.694500+0000","last_undegraded":"2024-09-17T15:32:12.694500+0000","last_fullsized":"2024-09-17T15:32:12.694500+0000","mapping_epoch":33,"log_start":"41'4700","ondisk_log_start":"41'4700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25559040,"num_objects":6240,"num_object_clones":0,"num_object_copies":12480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14730,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14458","reported_seq":14591,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:12.709355+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:32:12.709355+0000","last_peered":"2024-09-17T15:32:12.709355+0000","last_clean":"2024-09-17T15:32:12.709355+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:32:12.709355+0000","last_undegraded":"2024-09-17T15:32:12.709355+0000","last_fullsized":"2024-09-17T15:32:12.709355+0000","mapping_epoch":32,"log_start":"41'4400","ondisk_log_start":"41'4400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25375203,"num_objects":6196,"num_object_clones":0,"num_object_copies":12392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6196,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11440,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'15039","reported_seq":15335,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:12.694149+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:32:12.694149+0000","last_peered":"2024-09-17T15:32:12.694149+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:32:12.694149+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'5000","ondisk_log_start":"41'5000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25444835,"num_objects":6213,"num_object_clones":0,"num_object_copies":12426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6213,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11538,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2021},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14714","reported_seq":15028,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:12.695498+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:32:12.695498+0000","last_peered":"2024-09-17T15:32:12.695498+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:32:12.695498+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'4700","ondisk_log_start":"41'4700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24798150,"num_objects":6056,"num_object_clones":0,"num_object_copies":12112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6056,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11483,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1967},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14758","reported_seq":14891,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:12.717521+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:32:12.717521+0000","last_peered":"2024-09-17T15:32:12.717521+0000","last_clean":"2024-09-17T15:32:12.717521+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:32:12.717521+0000","last_undegraded":"2024-09-17T15:32:12.717521+0000","last_fullsized":"2024-09-17T15:32:12.717521+0000","mapping_epoch":32,"log_start":"41'4700","ondisk_log_start":"41'4700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25223168,"num_objects":6158,"num_object_clones":0,"num_object_copies":12316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11712,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.086967+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.086967+0000","last_peered":"2024-09-17T15:30:57.086967+0000","last_clean":"2024-09-17T15:30:57.086967+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.086967+0000","last_undegraded":"2024-09-17T15:30:57.086967+0000","last_fullsized":"2024-09-17T15:30:57.086967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14616","reported_seq":22901,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:12.084016+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:32:12.084016+0000","last_peered":"2024-09-17T15:32:12.084016+0000","last_clean":"2024-09-17T15:32:12.084016+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:32:12.084016+0000","last_undegraded":"2024-09-17T15:32:12.084016+0000","last_fullsized":"2024-09-17T15:32:12.084016+0000","mapping_epoch":42,"log_start":"41'4600","ondisk_log_start":"41'4600","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25399779,"num_objects":6202,"num_object_clones":0,"num_object_copies":12404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6202,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12616,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14434","reported_seq":14703,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:12.635466+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:32:12.635466+0000","last_peered":"2024-09-17T15:32:12.635466+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:32:12.635466+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'4400","ondisk_log_start":"41'4400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25018368,"num_objects":6108,"num_object_clones":0,"num_object_copies":12216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6108,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11030,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2165}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14479","reported_seq":14693,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:12.770212+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:32:12.770212+0000","last_peered":"2024-09-17T15:32:12.770212+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:32:12.770212+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'4400","ondisk_log_start":"41'4400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24740323,"num_objects":6041,"num_object_clones":0,"num_object_copies":12082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6041,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12374,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":2008}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14670","reported_seq":19349,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:12.809074+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:32:12.809074+0000","last_peered":"2024-09-17T15:32:12.809074+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:32:12.809074+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"41'4600","ondisk_log_start":"41'4600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24846336,"num_objects":6066,"num_object_clones":0,"num_object_copies":12132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1861,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12432,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2278,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4205}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14779","reported_seq":15078,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:12.723547+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:32:12.723547+0000","last_peered":"2024-09-17T15:32:12.723547+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:32:12.723547+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'4700","ondisk_log_start":"41'4700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25341952,"num_objects":6187,"num_object_clones":0,"num_object_copies":12374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6187,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11327,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14671","reported_seq":14942,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:12.713586+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:32:12.713586+0000","last_peered":"2024-09-17T15:32:12.713586+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:32:12.713586+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'4600","ondisk_log_start":"41'4600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24989696,"num_objects":6101,"num_object_clones":0,"num_object_copies":12202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10958,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2006}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14662","reported_seq":14965,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:12.684186+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:32:12.684186+0000","last_peered":"2024-09-17T15:32:12.684186+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:32:12.684186+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'4600","ondisk_log_start":"41'4600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25493504,"num_objects":6224,"num_object_clones":0,"num_object_copies":12448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11449,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2244}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14559","reported_seq":22938,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:12.098055+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:32:12.098055+0000","last_peered":"2024-09-17T15:32:12.098055+0000","last_clean":"2024-09-17T15:32:12.098055+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:32:12.098055+0000","last_undegraded":"2024-09-17T15:32:12.098055+0000","last_fullsized":"2024-09-17T15:32:12.098055+0000","mapping_epoch":44,"log_start":"41'4500","ondisk_log_start":"41'4500","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24875974,"num_objects":6075,"num_object_clones":0,"num_object_copies":12150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6075,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11001,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":402752246,"num_objects":98338,"num_object_clones":0,"num_object_copies":196676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38370,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98338,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":194649,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11090,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1008750592,"data_stored":1008605154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160814,"ondisk_log_size":160814,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051670,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":650080,"kb_used_data":329616,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93721760,"statfs":{"total":96636764160,"available":95971082240,"internally_reserved":0,"allocated":337526784,"data_stored":337235687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":328134316},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,2,0,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724615,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":616820,"kb_used_data":296356,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93755020,"statfs":{"total":96636764160,"available":96005140480,"internally_reserved":0,"allocated":303468544,"data_stored":303144466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":328134315},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,2,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247898,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":759620,"kb_used_data":362868,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93612220,"statfs":{"total":96636764160,"available":95858913280,"internally_reserved":0,"allocated":371576832,"data_stored":371285730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":406253224},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,7,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":369975296,"data_stored":369939166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":301932544,"data_stored":301863438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":336842752,"data_stored":336802550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:32:14.789 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:32:15.322 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:32:15.324 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:32:15.359 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":273,"stamp":"2024-09-17T15:32:13.717290+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":402752265,"num_objects":98340,"num_object_clones":0,"num_object_copies":196680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38370,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98340,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":194651,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11091,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160817,"ondisk_log_size":160817,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2026520,"kb_used_data":988840,"kb_used_omap":13,"kb_used_meta":1037618,"kb_avail":281089000,"statfs":{"total":289910292480,"available":287835136000,"internally_reserved":0,"allocated":1012572160,"data_stored":1011665883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1062521855},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,5,4,10,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28389376,"num_objects":-6931,"num_object_clones":0,"num_object_copies":-13862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-356,"num_objects_misplaced":-235,"num_objects_unfound":0,"num_objects_dirty":-6931,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6931,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":591,"num_bytes_recovered":962560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003991"},"pg_stats":[{"pgid":"2.f","version":"45'14551","reported_seq":14804,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:12.674272+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:32:12.674272+0000","last_peered":"2024-09-17T15:32:12.674272+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:32:12.674272+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'4500","ondisk_log_start":"41'4500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24973312,"num_objects":6097,"num_object_clones":0,"num_object_copies":12194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6097,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14551,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2113}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14692","reported_seq":14799,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:12.803970+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:32:12.803970+0000","last_peered":"2024-09-17T15:32:12.803970+0000","last_clean":"2024-09-17T15:32:12.803970+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:32:12.803970+0000","last_undegraded":"2024-09-17T15:32:12.803970+0000","last_fullsized":"2024-09-17T15:32:12.803970+0000","mapping_epoch":33,"log_start":"41'4600","ondisk_log_start":"41'4600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25560006,"num_objects":6242,"num_object_clones":0,"num_object_copies":12484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6242,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13049,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14802","reported_seq":15019,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:12.793020+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:32:12.793020+0000","last_peered":"2024-09-17T15:32:12.793020+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:32:12.793020+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'4800","ondisk_log_start":"41'4800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25112600,"num_objects":6132,"num_object_clones":0,"num_object_copies":12264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6132,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12959,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2007}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'14730","reported_seq":14805,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:12.694500+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:32:12.694500+0000","last_peered":"2024-09-17T15:32:12.694500+0000","last_clean":"2024-09-17T15:32:12.694500+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:32:12.694500+0000","last_undegraded":"2024-09-17T15:32:12.694500+0000","last_fullsized":"2024-09-17T15:32:12.694500+0000","mapping_epoch":33,"log_start":"41'4700","ondisk_log_start":"41'4700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25559040,"num_objects":6240,"num_object_clones":0,"num_object_copies":12480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14730,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14458","reported_seq":14591,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:12.709355+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:32:12.709355+0000","last_peered":"2024-09-17T15:32:12.709355+0000","last_clean":"2024-09-17T15:32:12.709355+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:32:12.709355+0000","last_undegraded":"2024-09-17T15:32:12.709355+0000","last_fullsized":"2024-09-17T15:32:12.709355+0000","mapping_epoch":32,"log_start":"41'4400","ondisk_log_start":"41'4400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25375203,"num_objects":6196,"num_object_clones":0,"num_object_copies":12392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6196,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11440,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'15039","reported_seq":15335,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:12.694149+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:32:12.694149+0000","last_peered":"2024-09-17T15:32:12.694149+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:32:12.694149+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'5000","ondisk_log_start":"41'5000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25444835,"num_objects":6213,"num_object_clones":0,"num_object_copies":12426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6213,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11538,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2021},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14714","reported_seq":15028,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:12.695498+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:32:12.695498+0000","last_peered":"2024-09-17T15:32:12.695498+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:32:12.695498+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'4700","ondisk_log_start":"41'4700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24798150,"num_objects":6056,"num_object_clones":0,"num_object_copies":12112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6056,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11483,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1967},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14758","reported_seq":14891,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:12.717521+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:32:12.717521+0000","last_peered":"2024-09-17T15:32:12.717521+0000","last_clean":"2024-09-17T15:32:12.717521+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:32:12.717521+0000","last_undegraded":"2024-09-17T15:32:12.717521+0000","last_fullsized":"2024-09-17T15:32:12.717521+0000","mapping_epoch":32,"log_start":"41'4700","ondisk_log_start":"41'4700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25223168,"num_objects":6158,"num_object_clones":0,"num_object_copies":12316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11712,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.086967+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.086967+0000","last_peered":"2024-09-17T15:30:57.086967+0000","last_clean":"2024-09-17T15:30:57.086967+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.086967+0000","last_undegraded":"2024-09-17T15:30:57.086967+0000","last_fullsized":"2024-09-17T15:30:57.086967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14616","reported_seq":22901,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:12.084016+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:32:12.084016+0000","last_peered":"2024-09-17T15:32:12.084016+0000","last_clean":"2024-09-17T15:32:12.084016+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:32:12.084016+0000","last_undegraded":"2024-09-17T15:32:12.084016+0000","last_fullsized":"2024-09-17T15:32:12.084016+0000","mapping_epoch":42,"log_start":"41'4600","ondisk_log_start":"41'4600","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25399779,"num_objects":6202,"num_object_clones":0,"num_object_copies":12404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6202,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12616,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14434","reported_seq":14703,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:12.635466+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:32:12.635466+0000","last_peered":"2024-09-17T15:32:12.635466+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:32:12.635466+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'4400","ondisk_log_start":"41'4400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25018368,"num_objects":6108,"num_object_clones":0,"num_object_copies":12216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6108,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11030,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2165}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14479","reported_seq":14693,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:12.770212+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:32:12.770212+0000","last_peered":"2024-09-17T15:32:12.770212+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:32:12.770212+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'4400","ondisk_log_start":"41'4400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24740323,"num_objects":6041,"num_object_clones":0,"num_object_copies":12082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6041,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12374,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":2008}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14670","reported_seq":19349,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:12.809074+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:32:12.809074+0000","last_peered":"2024-09-17T15:32:12.809074+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:32:12.809074+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"41'4600","ondisk_log_start":"41'4600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24846336,"num_objects":6066,"num_object_clones":0,"num_object_copies":12132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1861,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12432,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2278,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4205}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14779","reported_seq":15078,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:12.723547+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:32:12.723547+0000","last_peered":"2024-09-17T15:32:12.723547+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:32:12.723547+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'4700","ondisk_log_start":"41'4700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25341952,"num_objects":6187,"num_object_clones":0,"num_object_copies":12374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6187,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11327,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14671","reported_seq":14942,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:12.713586+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:32:12.713586+0000","last_peered":"2024-09-17T15:32:12.713586+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:32:12.713586+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'4600","ondisk_log_start":"41'4600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24989696,"num_objects":6101,"num_object_clones":0,"num_object_copies":12202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10958,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2006}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14662","reported_seq":14965,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:12.684186+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:32:12.684186+0000","last_peered":"2024-09-17T15:32:12.684186+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:32:12.684186+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'4600","ondisk_log_start":"41'4600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25493504,"num_objects":6224,"num_object_clones":0,"num_object_copies":12448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11449,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2244}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14559","reported_seq":22938,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:12.098055+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:32:12.098055+0000","last_peered":"2024-09-17T15:32:12.098055+0000","last_clean":"2024-09-17T15:32:12.098055+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:32:12.098055+0000","last_undegraded":"2024-09-17T15:32:12.098055+0000","last_fullsized":"2024-09-17T15:32:12.098055+0000","mapping_epoch":44,"log_start":"41'4500","ondisk_log_start":"41'4500","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24875974,"num_objects":6075,"num_object_clones":0,"num_object_copies":12150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6075,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11001,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":402752246,"num_objects":98338,"num_object_clones":0,"num_object_copies":196676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38370,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98338,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":194649,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11090,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1008750592,"data_stored":1008605154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160814,"ondisk_log_size":160814,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051670,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":650080,"kb_used_data":329616,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93721760,"statfs":{"total":96636764160,"available":95971082240,"internally_reserved":0,"allocated":337526784,"data_stored":337235687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":328134316},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,2,0,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724615,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":616820,"kb_used_data":296356,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93755020,"statfs":{"total":96636764160,"available":96005140480,"internally_reserved":0,"allocated":303468544,"data_stored":303144466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":328134315},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,2,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247898,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":759620,"kb_used_data":362868,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93612220,"statfs":{"total":96636764160,"available":95858913280,"internally_reserved":0,"allocated":371576832,"data_stored":371285730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":406253224},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,7,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":369975296,"data_stored":369939166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":301932544,"data_stored":301863438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":336842752,"data_stored":336802550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:32:15.360 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:32:15.360 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'14551", 'reported_seq': 14804, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:12.674272+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:32:12.674272+0000', 'last_peered': '2024-09-17T15:32:12.674272+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:32:12.674272+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "41'4500", 'ondisk_log_start': "41'4500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24973312, 'num_objects': 6097, 'num_object_clones': 0, 'num_object_copies': 12194, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6097, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14551, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2113}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:15.360 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:32:15.360 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'14802", 'reported_seq': 15019, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:12.793020+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:32:12.793020+0000', 'last_peered': '2024-09-17T15:32:12.793020+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:32:12.793020+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "41'4800", 'ondisk_log_start': "41'4800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25112600, 'num_objects': 6132, 'num_object_clones': 0, 'num_object_copies': 12264, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6132, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12959, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2007}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:32:15.360 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:32:15.361 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'15039", 'reported_seq': 15335, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:12.694149+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:32:12.694149+0000', 'last_peered': '2024-09-17T15:32:12.694149+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:32:12.694149+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "41'5000", 'ondisk_log_start': "41'5000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25444835, 'num_objects': 6213, 'num_object_clones': 0, 'num_object_copies': 12426, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6213, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11538, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2021}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:15.361 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:32:15.361 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'14714", 'reported_seq': 15028, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:12.695498+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:32:12.695498+0000', 'last_peered': '2024-09-17T15:32:12.695498+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:32:12.695498+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "41'4700", 'ondisk_log_start': "41'4700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24798150, 'num_objects': 6056, 'num_object_clones': 0, 'num_object_copies': 12112, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4089, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6056, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 11483, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 85, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1967}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:15.361 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:32:15.361 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'14434", 'reported_seq': 14703, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:12.635466+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:32:12.635466+0000', 'last_peered': '2024-09-17T15:32:12.635466+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:32:12.635466+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "41'4400", 'ondisk_log_start': "41'4400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25018368, 'num_objects': 6108, 'num_object_clones': 0, 'num_object_copies': 12216, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6108, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11030, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2165}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:15.361 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:32:15.361 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'14479", 'reported_seq': 14693, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:12.770212+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:32:12.770212+0000', 'last_peered': '2024-09-17T15:32:12.770212+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:32:12.770212+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "41'4400", 'ondisk_log_start': "41'4400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24740323, 'num_objects': 6041, 'num_object_clones': 0, 'num_object_copies': 12082, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4034, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6041, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12374, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 2008}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:32:15.361 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:32:15.361 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'14670", 'reported_seq': 19349, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:12.809074+0000', 'last_change': '2024-09-17T15:30:55.086473+0000', 'last_active': '2024-09-17T15:32:12.809074+0000', 'last_peered': '2024-09-17T15:32:12.809074+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:32:12.809074+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "41'4600", 'ondisk_log_start': "41'4600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24846336, 'num_objects': 6066, 'num_object_clones': 0, 'num_object_copies': 12132, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1861, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6066, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12432, 'num_write_kb': 32532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2278, 'num_bytes_recovered': 8073216, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 4205}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:32:15.361 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:32:15.362 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'14779", 'reported_seq': 15078, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:12.723547+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:32:12.723547+0000', 'last_peered': '2024-09-17T15:32:12.723547+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:32:12.723547+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "41'4700", 'ondisk_log_start': "41'4700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25341952, 'num_objects': 6187, 'num_object_clones': 0, 'num_object_copies': 12374, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6187, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11327, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2130}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:15.362 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:32:15.362 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'14671", 'reported_seq': 14942, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:12.713586+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:32:12.713586+0000', 'last_peered': '2024-09-17T15:32:12.713586+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:32:12.713586+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "41'4600", 'ondisk_log_start': "41'4600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24989696, 'num_objects': 6101, 'num_object_clones': 0, 'num_object_copies': 12202, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4098, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6101, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10958, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 62, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2006}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:15.362 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:32:15.362 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'14662", 'reported_seq': 14965, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:12.684186+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:32:12.684186+0000', 'last_peered': '2024-09-17T15:32:12.684186+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:32:12.684186+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "41'4600", 'ondisk_log_start': "41'4600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25493504, 'num_objects': 6224, 'num_object_clones': 0, 'num_object_copies': 12448, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6224, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11449, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2244}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:15.362 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:32:16.020 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:32:16.021 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38370/196680 objects degraded (19.509%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":473,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":98340,"data_bytes":402752265,"bytes_used":2075156480,"bytes_avail":287835136000,"bytes_total":289910292480,"degraded_objects":38370,"degraded_total":196680,"degraded_ratio":0.19508846857840145,"recovering_objects_per_sec":30,"recovering_bytes_per_sec":18425,"recovering_keys_per_sec":0,"num_objects_recovered":364,"num_bytes_recovered":221184,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":389},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-17T15:31:29.695933+0000","services":{}},"progress_events":{}} 2024-09-17T15:32:16.046 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:32:16.047 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:32:16.581 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:32:16.584 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:32:16.603 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":274,"stamp":"2024-09-17T15:32:15.717986+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":402752265,"num_objects":98340,"num_object_clones":0,"num_object_copies":196680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38370,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98340,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":194651,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11091,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160817,"ondisk_log_size":160817,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2026520,"kb_used_data":988840,"kb_used_omap":13,"kb_used_meta":1037618,"kb_avail":281089000,"statfs":{"total":289910292480,"available":287835136000,"internally_reserved":0,"allocated":1012572160,"data_stored":1011665883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1062521855},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,5,4,10,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19148800,"num_objects":-4675,"num_object_clones":0,"num_object_copies":-9350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-310,"num_objects_misplaced":-54,"num_objects_unfound":0,"num_objects_dirty":-4675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4675,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":364,"num_bytes_recovered":221184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004072"},"pg_stats":[{"pgid":"2.f","version":"45'14551","reported_seq":14804,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:12.674272+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:32:12.674272+0000","last_peered":"2024-09-17T15:32:12.674272+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:32:12.674272+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'4500","ondisk_log_start":"41'4500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24973312,"num_objects":6097,"num_object_clones":0,"num_object_copies":12194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6097,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14551,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2113}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14692","reported_seq":14799,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:12.803970+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:32:12.803970+0000","last_peered":"2024-09-17T15:32:12.803970+0000","last_clean":"2024-09-17T15:32:12.803970+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:32:12.803970+0000","last_undegraded":"2024-09-17T15:32:12.803970+0000","last_fullsized":"2024-09-17T15:32:12.803970+0000","mapping_epoch":33,"log_start":"41'4600","ondisk_log_start":"41'4600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25560006,"num_objects":6242,"num_object_clones":0,"num_object_copies":12484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6242,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13049,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14802","reported_seq":15019,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:12.793020+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:32:12.793020+0000","last_peered":"2024-09-17T15:32:12.793020+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:32:12.793020+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'4800","ondisk_log_start":"41'4800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25112600,"num_objects":6132,"num_object_clones":0,"num_object_copies":12264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6132,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12959,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2007}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'14730","reported_seq":14805,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:12.694500+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:32:12.694500+0000","last_peered":"2024-09-17T15:32:12.694500+0000","last_clean":"2024-09-17T15:32:12.694500+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:32:12.694500+0000","last_undegraded":"2024-09-17T15:32:12.694500+0000","last_fullsized":"2024-09-17T15:32:12.694500+0000","mapping_epoch":33,"log_start":"41'4700","ondisk_log_start":"41'4700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25559040,"num_objects":6240,"num_object_clones":0,"num_object_copies":12480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14730,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14458","reported_seq":14591,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:12.709355+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:32:12.709355+0000","last_peered":"2024-09-17T15:32:12.709355+0000","last_clean":"2024-09-17T15:32:12.709355+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:32:12.709355+0000","last_undegraded":"2024-09-17T15:32:12.709355+0000","last_fullsized":"2024-09-17T15:32:12.709355+0000","mapping_epoch":32,"log_start":"41'4400","ondisk_log_start":"41'4400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25375203,"num_objects":6196,"num_object_clones":0,"num_object_copies":12392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6196,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11440,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'15039","reported_seq":15335,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:12.694149+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:32:12.694149+0000","last_peered":"2024-09-17T15:32:12.694149+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:32:12.694149+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'5000","ondisk_log_start":"41'5000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25444835,"num_objects":6213,"num_object_clones":0,"num_object_copies":12426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6213,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11538,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2021},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14714","reported_seq":15028,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:12.695498+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:32:12.695498+0000","last_peered":"2024-09-17T15:32:12.695498+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:32:12.695498+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'4700","ondisk_log_start":"41'4700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24798150,"num_objects":6056,"num_object_clones":0,"num_object_copies":12112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6056,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11483,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1967},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14758","reported_seq":14891,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:12.717521+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:32:12.717521+0000","last_peered":"2024-09-17T15:32:12.717521+0000","last_clean":"2024-09-17T15:32:12.717521+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:32:12.717521+0000","last_undegraded":"2024-09-17T15:32:12.717521+0000","last_fullsized":"2024-09-17T15:32:12.717521+0000","mapping_epoch":32,"log_start":"41'4700","ondisk_log_start":"41'4700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25223168,"num_objects":6158,"num_object_clones":0,"num_object_copies":12316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11712,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.086967+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.086967+0000","last_peered":"2024-09-17T15:30:57.086967+0000","last_clean":"2024-09-17T15:30:57.086967+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.086967+0000","last_undegraded":"2024-09-17T15:30:57.086967+0000","last_fullsized":"2024-09-17T15:30:57.086967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14616","reported_seq":22901,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:12.084016+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:32:12.084016+0000","last_peered":"2024-09-17T15:32:12.084016+0000","last_clean":"2024-09-17T15:32:12.084016+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:32:12.084016+0000","last_undegraded":"2024-09-17T15:32:12.084016+0000","last_fullsized":"2024-09-17T15:32:12.084016+0000","mapping_epoch":42,"log_start":"41'4600","ondisk_log_start":"41'4600","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25399779,"num_objects":6202,"num_object_clones":0,"num_object_copies":12404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6202,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12616,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14434","reported_seq":14703,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:12.635466+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:32:12.635466+0000","last_peered":"2024-09-17T15:32:12.635466+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:32:12.635466+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'4400","ondisk_log_start":"41'4400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25018368,"num_objects":6108,"num_object_clones":0,"num_object_copies":12216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6108,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11030,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2165}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14479","reported_seq":14693,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:12.770212+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:32:12.770212+0000","last_peered":"2024-09-17T15:32:12.770212+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:32:12.770212+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'4400","ondisk_log_start":"41'4400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24740323,"num_objects":6041,"num_object_clones":0,"num_object_copies":12082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6041,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12374,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":2008}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14670","reported_seq":19349,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:12.809074+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:32:12.809074+0000","last_peered":"2024-09-17T15:32:12.809074+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:32:12.809074+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"41'4600","ondisk_log_start":"41'4600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24846336,"num_objects":6066,"num_object_clones":0,"num_object_copies":12132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1861,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12432,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2278,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4205}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14779","reported_seq":15078,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:12.723547+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:32:12.723547+0000","last_peered":"2024-09-17T15:32:12.723547+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:32:12.723547+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'4700","ondisk_log_start":"41'4700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25341952,"num_objects":6187,"num_object_clones":0,"num_object_copies":12374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6187,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11327,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14671","reported_seq":14942,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:12.713586+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:32:12.713586+0000","last_peered":"2024-09-17T15:32:12.713586+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:32:12.713586+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'4600","ondisk_log_start":"41'4600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24989696,"num_objects":6101,"num_object_clones":0,"num_object_copies":12202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10958,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2006}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14662","reported_seq":14965,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:12.684186+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:32:12.684186+0000","last_peered":"2024-09-17T15:32:12.684186+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:32:12.684186+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'4600","ondisk_log_start":"41'4600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25493504,"num_objects":6224,"num_object_clones":0,"num_object_copies":12448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11449,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2244}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14559","reported_seq":22938,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:12.098055+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:32:12.098055+0000","last_peered":"2024-09-17T15:32:12.098055+0000","last_clean":"2024-09-17T15:32:12.098055+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:32:12.098055+0000","last_undegraded":"2024-09-17T15:32:12.098055+0000","last_fullsized":"2024-09-17T15:32:12.098055+0000","mapping_epoch":44,"log_start":"41'4500","ondisk_log_start":"41'4500","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24875974,"num_objects":6075,"num_object_clones":0,"num_object_copies":12150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6075,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11001,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":402752246,"num_objects":98338,"num_object_clones":0,"num_object_copies":196676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38370,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98338,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":194649,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11090,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1008750592,"data_stored":1008605154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160814,"ondisk_log_size":160814,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051670,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":650080,"kb_used_data":329616,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93721760,"statfs":{"total":96636764160,"available":95971082240,"internally_reserved":0,"allocated":337526784,"data_stored":337235687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":328134316},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,2,0,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724615,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":616820,"kb_used_data":296356,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93755020,"statfs":{"total":96636764160,"available":96005140480,"internally_reserved":0,"allocated":303468544,"data_stored":303144466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":328134315},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,2,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247898,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":759620,"kb_used_data":362868,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93612220,"statfs":{"total":96636764160,"available":95858913280,"internally_reserved":0,"allocated":371576832,"data_stored":371285730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":406253224},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,7,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":369975296,"data_stored":369939166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":301932544,"data_stored":301863438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":336842752,"data_stored":336802550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:32:19.605 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:32:20.176 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:32:20.177 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:32:20.198 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":276,"stamp":"2024-09-17T15:32:19.719102+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":393122569,"num_objects":95989,"num_object_clones":0,"num_object_copies":191978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38224,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95989,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":197002,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11237,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160668,"ondisk_log_size":160668,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2046560,"kb_used_data":970020,"kb_used_omap":13,"kb_used_meta":1076466,"kb_avail":281068960,"statfs":{"total":289910292480,"available":287814615040,"internally_reserved":0,"allocated":993300480,"data_stored":992394203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1102302207},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,11,8,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22290432,"num_objects":-5442,"num_object_clones":0,"num_object_copies":-10884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-432,"num_objects_misplaced":-54,"num_objects_unfound":0,"num_objects_dirty":-5442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5442,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":486,"num_bytes_recovered":221184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003928"},"pg_stats":[{"pgid":"2.f","version":"45'14695","reported_seq":14950,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:17.711578+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:32:17.711578+0000","last_peered":"2024-09-17T15:32:17.711578+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:32:17.711578+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'4600","ondisk_log_start":"41'4600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24383488,"num_objects":5953,"num_object_clones":0,"num_object_copies":11906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14695,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1969}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14833","reported_seq":14941,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:17.819275+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:32:17.819275+0000","last_peered":"2024-09-17T15:32:17.819275+0000","last_clean":"2024-09-17T15:32:17.819275+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:32:17.819275+0000","last_undegraded":"2024-09-17T15:32:17.819275+0000","last_fullsized":"2024-09-17T15:32:17.819275+0000","mapping_epoch":33,"log_start":"41'4800","ondisk_log_start":"41'4800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24982470,"num_objects":6101,"num_object_clones":0,"num_object_copies":12202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6101,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13190,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14967","reported_seq":15184,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:17.775293+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:32:17.775293+0000","last_peered":"2024-09-17T15:32:17.775293+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:32:17.775293+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'4900","ondisk_log_start":"41'4900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24436760,"num_objects":5967,"num_object_clones":0,"num_object_copies":11934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5967,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13124,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1842}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'14881","reported_seq":14956,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:17.710660+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:32:17.710660+0000","last_peered":"2024-09-17T15:32:17.710660+0000","last_clean":"2024-09-17T15:32:17.710660+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:32:17.710660+0000","last_undegraded":"2024-09-17T15:32:17.710660+0000","last_fullsized":"2024-09-17T15:32:17.710660+0000","mapping_epoch":33,"log_start":"41'4800","ondisk_log_start":"41'4800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24940544,"num_objects":6089,"num_object_clones":0,"num_object_copies":12178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14881,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14608","reported_seq":14742,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:17.645481+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:32:17.645481+0000","last_peered":"2024-09-17T15:32:17.645481+0000","last_clean":"2024-09-17T15:32:17.645481+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:32:17.645481+0000","last_undegraded":"2024-09-17T15:32:17.645481+0000","last_fullsized":"2024-09-17T15:32:17.645481+0000","mapping_epoch":32,"log_start":"41'4600","ondisk_log_start":"41'4600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24760803,"num_objects":6046,"num_object_clones":0,"num_object_copies":12092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6046,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11590,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'15213","reported_seq":15508,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:17.743609+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:32:17.743609+0000","last_peered":"2024-09-17T15:32:17.743609+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:32:17.743609+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'5200","ondisk_log_start":"41'5200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24732131,"num_objects":6039,"num_object_clones":0,"num_object_copies":12078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6039,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11712,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1847},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14861","reported_seq":15175,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:17.697399+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:32:17.697399+0000","last_peered":"2024-09-17T15:32:17.697399+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:32:17.697399+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'4800","ondisk_log_start":"41'4800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24196038,"num_objects":5909,"num_object_clones":0,"num_object_copies":11818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5909,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11630,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1820},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14910","reported_seq":15043,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:17.692635+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:32:17.692635+0000","last_peered":"2024-09-17T15:32:17.692635+0000","last_clean":"2024-09-17T15:32:17.692635+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:32:17.692635+0000","last_undegraded":"2024-09-17T15:32:17.692635+0000","last_fullsized":"2024-09-17T15:32:17.692635+0000","mapping_epoch":32,"log_start":"41'4900","ondisk_log_start":"41'4900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24600576,"num_objects":6006,"num_object_clones":0,"num_object_copies":12012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11864,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.086967+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.086967+0000","last_peered":"2024-09-17T15:30:57.086967+0000","last_clean":"2024-09-17T15:30:57.086967+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.086967+0000","last_undegraded":"2024-09-17T15:30:57.086967+0000","last_fullsized":"2024-09-17T15:30:57.086967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14758","reported_seq":23045,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:17.103061+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:32:17.103061+0000","last_peered":"2024-09-17T15:32:17.103061+0000","last_clean":"2024-09-17T15:32:17.103061+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:32:17.103061+0000","last_undegraded":"2024-09-17T15:32:17.103061+0000","last_fullsized":"2024-09-17T15:32:17.103061+0000","mapping_epoch":42,"log_start":"41'4700","ondisk_log_start":"41'4700","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24818147,"num_objects":6060,"num_object_clones":0,"num_object_copies":12120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6060,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12758,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14567","reported_seq":14833,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:17.743250+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:32:17.743250+0000","last_peered":"2024-09-17T15:32:17.743250+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:32:17.743250+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'4500","ondisk_log_start":"41'4500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24473600,"num_objects":5975,"num_object_clones":0,"num_object_copies":11950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5975,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11163,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2032}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14602","reported_seq":14816,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:17.820670+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:32:17.820670+0000","last_peered":"2024-09-17T15:32:17.820670+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:32:17.820670+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'4600","ondisk_log_start":"41'4600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24236515,"num_objects":5918,"num_object_clones":0,"num_object_copies":11836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5918,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12497,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1885}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14789","reported_seq":19762,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:17.830830+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:32:17.830830+0000","last_peered":"2024-09-17T15:32:17.830830+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:32:17.830830+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"41'4700","ondisk_log_start":"41'4700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24358912,"num_objects":5947,"num_object_clones":0,"num_object_copies":11894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12551,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2424,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4232}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14919","reported_seq":15221,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:17.563843+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:32:17.563843+0000","last_peered":"2024-09-17T15:32:17.563843+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:32:17.563843+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'4900","ondisk_log_start":"41'4900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24768512,"num_objects":6047,"num_object_clones":0,"num_object_copies":12094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11467,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1990}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14817","reported_seq":15088,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:17.731895+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:32:17.731895+0000","last_peered":"2024-09-17T15:32:17.731895+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:32:17.731895+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'4800","ondisk_log_start":"41'4800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24391680,"num_objects":5955,"num_object_clones":0,"num_object_copies":11910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5955,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11104,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1860}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14838","reported_seq":15141,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:17.723089+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:32:17.723089+0000","last_peered":"2024-09-17T15:32:17.723089+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:32:17.723089+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'4800","ondisk_log_start":"41'4800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24772608,"num_objects":6048,"num_object_clones":0,"num_object_copies":12096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11625,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2068}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14707","reported_seq":23086,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:17.067069+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:32:17.067069+0000","last_peered":"2024-09-17T15:32:17.067069+0000","last_clean":"2024-09-17T15:32:17.067069+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:32:17.067069+0000","last_undegraded":"2024-09-17T15:32:17.067069+0000","last_fullsized":"2024-09-17T15:32:17.067069+0000","mapping_epoch":44,"log_start":"41'4700","ondisk_log_start":"41'4700","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24269766,"num_objects":5927,"num_object_clones":0,"num_object_copies":11854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11149,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":393122550,"num_objects":95987,"num_object_clones":0,"num_object_copies":191974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38224,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95987,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":197000,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11236,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":989483008,"data_stored":989337570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160665,"ondisk_log_size":160665,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051671,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":649692,"kb_used_data":325060,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93722148,"statfs":{"total":96636764160,"available":95971479552,"internally_reserved":0,"allocated":332861440,"data_stored":332570343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":332394156},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724616,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":636196,"kb_used_data":289360,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93735644,"statfs":{"total":96636764160,"available":95985299456,"internally_reserved":0,"allocated":296304640,"data_stored":295980562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247899,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":760672,"kb_used_data":355600,"kb_used_omap":4,"kb_used_meta":405051,"kb_avail":93611168,"statfs":{"total":96636764160,"available":95857836032,"internally_reserved":0,"allocated":364134400,"data_stored":363843298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":414772904},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,8,1,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":362598400,"data_stored":362562270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":294723584,"data_stored":294654478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":332161024,"data_stored":332120822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:32:20.199 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:32:20.746 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:32:20.749 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:32:20.775 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":276,"stamp":"2024-09-17T15:32:19.719102+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":393122569,"num_objects":95989,"num_object_clones":0,"num_object_copies":191978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38224,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95989,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":197002,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11237,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160668,"ondisk_log_size":160668,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2046560,"kb_used_data":970020,"kb_used_omap":13,"kb_used_meta":1076466,"kb_avail":281068960,"statfs":{"total":289910292480,"available":287814615040,"internally_reserved":0,"allocated":993300480,"data_stored":992394203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1102302207},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,11,8,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22290432,"num_objects":-5442,"num_object_clones":0,"num_object_copies":-10884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-432,"num_objects_misplaced":-54,"num_objects_unfound":0,"num_objects_dirty":-5442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5442,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":486,"num_bytes_recovered":221184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003928"},"pg_stats":[{"pgid":"2.f","version":"45'14695","reported_seq":14950,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:17.711578+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:32:17.711578+0000","last_peered":"2024-09-17T15:32:17.711578+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:32:17.711578+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'4600","ondisk_log_start":"41'4600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24383488,"num_objects":5953,"num_object_clones":0,"num_object_copies":11906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14695,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1969}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14833","reported_seq":14941,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:17.819275+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:32:17.819275+0000","last_peered":"2024-09-17T15:32:17.819275+0000","last_clean":"2024-09-17T15:32:17.819275+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:32:17.819275+0000","last_undegraded":"2024-09-17T15:32:17.819275+0000","last_fullsized":"2024-09-17T15:32:17.819275+0000","mapping_epoch":33,"log_start":"41'4800","ondisk_log_start":"41'4800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24982470,"num_objects":6101,"num_object_clones":0,"num_object_copies":12202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6101,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13190,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14967","reported_seq":15184,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:17.775293+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:32:17.775293+0000","last_peered":"2024-09-17T15:32:17.775293+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:32:17.775293+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'4900","ondisk_log_start":"41'4900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24436760,"num_objects":5967,"num_object_clones":0,"num_object_copies":11934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5967,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13124,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1842}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'14881","reported_seq":14956,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:17.710660+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:32:17.710660+0000","last_peered":"2024-09-17T15:32:17.710660+0000","last_clean":"2024-09-17T15:32:17.710660+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:32:17.710660+0000","last_undegraded":"2024-09-17T15:32:17.710660+0000","last_fullsized":"2024-09-17T15:32:17.710660+0000","mapping_epoch":33,"log_start":"41'4800","ondisk_log_start":"41'4800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24940544,"num_objects":6089,"num_object_clones":0,"num_object_copies":12178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14881,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14608","reported_seq":14742,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:17.645481+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:32:17.645481+0000","last_peered":"2024-09-17T15:32:17.645481+0000","last_clean":"2024-09-17T15:32:17.645481+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:32:17.645481+0000","last_undegraded":"2024-09-17T15:32:17.645481+0000","last_fullsized":"2024-09-17T15:32:17.645481+0000","mapping_epoch":32,"log_start":"41'4600","ondisk_log_start":"41'4600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24760803,"num_objects":6046,"num_object_clones":0,"num_object_copies":12092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6046,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11590,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'15213","reported_seq":15508,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:17.743609+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:32:17.743609+0000","last_peered":"2024-09-17T15:32:17.743609+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:32:17.743609+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'5200","ondisk_log_start":"41'5200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24732131,"num_objects":6039,"num_object_clones":0,"num_object_copies":12078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6039,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11712,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1847},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14861","reported_seq":15175,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:17.697399+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:32:17.697399+0000","last_peered":"2024-09-17T15:32:17.697399+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:32:17.697399+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'4800","ondisk_log_start":"41'4800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24196038,"num_objects":5909,"num_object_clones":0,"num_object_copies":11818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5909,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11630,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1820},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14910","reported_seq":15043,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:17.692635+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:32:17.692635+0000","last_peered":"2024-09-17T15:32:17.692635+0000","last_clean":"2024-09-17T15:32:17.692635+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:32:17.692635+0000","last_undegraded":"2024-09-17T15:32:17.692635+0000","last_fullsized":"2024-09-17T15:32:17.692635+0000","mapping_epoch":32,"log_start":"41'4900","ondisk_log_start":"41'4900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24600576,"num_objects":6006,"num_object_clones":0,"num_object_copies":12012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11864,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.086967+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.086967+0000","last_peered":"2024-09-17T15:30:57.086967+0000","last_clean":"2024-09-17T15:30:57.086967+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.086967+0000","last_undegraded":"2024-09-17T15:30:57.086967+0000","last_fullsized":"2024-09-17T15:30:57.086967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14758","reported_seq":23045,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:17.103061+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:32:17.103061+0000","last_peered":"2024-09-17T15:32:17.103061+0000","last_clean":"2024-09-17T15:32:17.103061+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:32:17.103061+0000","last_undegraded":"2024-09-17T15:32:17.103061+0000","last_fullsized":"2024-09-17T15:32:17.103061+0000","mapping_epoch":42,"log_start":"41'4700","ondisk_log_start":"41'4700","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24818147,"num_objects":6060,"num_object_clones":0,"num_object_copies":12120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6060,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12758,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14567","reported_seq":14833,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:17.743250+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:32:17.743250+0000","last_peered":"2024-09-17T15:32:17.743250+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:32:17.743250+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'4500","ondisk_log_start":"41'4500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24473600,"num_objects":5975,"num_object_clones":0,"num_object_copies":11950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5975,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11163,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2032}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14602","reported_seq":14816,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:17.820670+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:32:17.820670+0000","last_peered":"2024-09-17T15:32:17.820670+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:32:17.820670+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'4600","ondisk_log_start":"41'4600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24236515,"num_objects":5918,"num_object_clones":0,"num_object_copies":11836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5918,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12497,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1885}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14789","reported_seq":19762,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:17.830830+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:32:17.830830+0000","last_peered":"2024-09-17T15:32:17.830830+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:32:17.830830+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"41'4700","ondisk_log_start":"41'4700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24358912,"num_objects":5947,"num_object_clones":0,"num_object_copies":11894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12551,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2424,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4232}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14919","reported_seq":15221,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:17.563843+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:32:17.563843+0000","last_peered":"2024-09-17T15:32:17.563843+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:32:17.563843+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'4900","ondisk_log_start":"41'4900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24768512,"num_objects":6047,"num_object_clones":0,"num_object_copies":12094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11467,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1990}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14817","reported_seq":15088,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:17.731895+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:32:17.731895+0000","last_peered":"2024-09-17T15:32:17.731895+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:32:17.731895+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'4800","ondisk_log_start":"41'4800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24391680,"num_objects":5955,"num_object_clones":0,"num_object_copies":11910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5955,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11104,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1860}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14838","reported_seq":15141,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:17.723089+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:32:17.723089+0000","last_peered":"2024-09-17T15:32:17.723089+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:32:17.723089+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'4800","ondisk_log_start":"41'4800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24772608,"num_objects":6048,"num_object_clones":0,"num_object_copies":12096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11625,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2068}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14707","reported_seq":23086,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:17.067069+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:32:17.067069+0000","last_peered":"2024-09-17T15:32:17.067069+0000","last_clean":"2024-09-17T15:32:17.067069+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:32:17.067069+0000","last_undegraded":"2024-09-17T15:32:17.067069+0000","last_fullsized":"2024-09-17T15:32:17.067069+0000","mapping_epoch":44,"log_start":"41'4700","ondisk_log_start":"41'4700","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24269766,"num_objects":5927,"num_object_clones":0,"num_object_copies":11854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11149,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":393122550,"num_objects":95987,"num_object_clones":0,"num_object_copies":191974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38224,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95987,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":197000,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11236,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":989483008,"data_stored":989337570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160665,"ondisk_log_size":160665,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051671,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":649692,"kb_used_data":325060,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93722148,"statfs":{"total":96636764160,"available":95971479552,"internally_reserved":0,"allocated":332861440,"data_stored":332570343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":332394156},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724616,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":636196,"kb_used_data":289360,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93735644,"statfs":{"total":96636764160,"available":95985299456,"internally_reserved":0,"allocated":296304640,"data_stored":295980562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247899,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":760672,"kb_used_data":355600,"kb_used_omap":4,"kb_used_meta":405051,"kb_avail":93611168,"statfs":{"total":96636764160,"available":95857836032,"internally_reserved":0,"allocated":364134400,"data_stored":363843298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":414772904},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,8,1,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":362598400,"data_stored":362562270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":294723584,"data_stored":294654478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":332161024,"data_stored":332120822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:32:20.777 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:32:20.777 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'14695", 'reported_seq': 14950, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:17.711578+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:32:17.711578+0000', 'last_peered': '2024-09-17T15:32:17.711578+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:32:17.711578+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "41'4600", 'ondisk_log_start': "41'4600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24383488, 'num_objects': 5953, 'num_object_clones': 0, 'num_object_copies': 11906, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5953, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14695, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1969}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:20.777 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:32:20.777 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'14967", 'reported_seq': 15184, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:17.775293+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:32:17.775293+0000', 'last_peered': '2024-09-17T15:32:17.775293+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:32:17.775293+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "41'4900", 'ondisk_log_start': "41'4900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24436760, 'num_objects': 5967, 'num_object_clones': 0, 'num_object_copies': 11934, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5967, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13124, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1842}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:32:20.777 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:32:20.777 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'15213", 'reported_seq': 15508, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:17.743609+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:32:17.743609+0000', 'last_peered': '2024-09-17T15:32:17.743609+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:32:17.743609+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "41'5200", 'ondisk_log_start': "41'5200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24732131, 'num_objects': 6039, 'num_object_clones': 0, 'num_object_copies': 12078, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6039, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11712, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1847}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:20.778 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:32:20.778 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'14861", 'reported_seq': 15175, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:17.697399+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:32:17.697399+0000', 'last_peered': '2024-09-17T15:32:17.697399+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:32:17.697399+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "41'4800", 'ondisk_log_start': "41'4800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24196038, 'num_objects': 5909, 'num_object_clones': 0, 'num_object_copies': 11818, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4089, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5909, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 11630, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 85, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1820}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:20.778 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:32:20.778 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'14567", 'reported_seq': 14833, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:17.743250+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:32:17.743250+0000', 'last_peered': '2024-09-17T15:32:17.743250+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:32:17.743250+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "41'4500", 'ondisk_log_start': "41'4500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24473600, 'num_objects': 5975, 'num_object_clones': 0, 'num_object_copies': 11950, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5975, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11163, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2032}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:20.778 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:32:20.778 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'14602", 'reported_seq': 14816, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:17.820670+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:32:17.820670+0000', 'last_peered': '2024-09-17T15:32:17.820670+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:32:17.820670+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "41'4600", 'ondisk_log_start': "41'4600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24236515, 'num_objects': 5918, 'num_object_clones': 0, 'num_object_copies': 11836, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4034, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5918, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12497, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 1885}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:32:20.778 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:32:20.778 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'14789", 'reported_seq': 19762, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:17.830830+0000', 'last_change': '2024-09-17T15:30:55.086473+0000', 'last_active': '2024-09-17T15:32:17.830830+0000', 'last_peered': '2024-09-17T15:32:17.830830+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:32:17.830830+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "41'4700", 'ondisk_log_start': "41'4700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24358912, 'num_objects': 5947, 'num_object_clones': 0, 'num_object_copies': 11894, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1715, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5947, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12551, 'num_write_kb': 32532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2424, 'num_bytes_recovered': 8073216, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 4232}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:32:20.778 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:32:20.778 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'14919", 'reported_seq': 15221, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:17.563843+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:32:17.563843+0000', 'last_peered': '2024-09-17T15:32:17.563843+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:32:17.563843+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "41'4900", 'ondisk_log_start': "41'4900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24768512, 'num_objects': 6047, 'num_object_clones': 0, 'num_object_copies': 12094, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6047, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11467, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1990}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:20.779 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:32:20.779 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'14817", 'reported_seq': 15088, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:17.731895+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:32:17.731895+0000', 'last_peered': '2024-09-17T15:32:17.731895+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:32:17.731895+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "41'4800", 'ondisk_log_start': "41'4800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24391680, 'num_objects': 5955, 'num_object_clones': 0, 'num_object_copies': 11910, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4098, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5955, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11104, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 62, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1860}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:20.779 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:32:20.779 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'14838", 'reported_seq': 15141, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:17.723089+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:32:17.723089+0000', 'last_peered': '2024-09-17T15:32:17.723089+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:32:17.723089+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "41'4800", 'ondisk_log_start': "41'4800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24772608, 'num_objects': 6048, 'num_object_clones': 0, 'num_object_copies': 12096, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6048, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11625, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2068}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:20.779 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:32:21.491 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:32:21.491 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38224/191978 objects degraded (19.911%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":478,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":95989,"data_bytes":393122569,"bytes_used":2095677440,"bytes_avail":287814615040,"bytes_total":289910292480,"degraded_objects":38224,"degraded_total":191978,"degraded_ratio":0.19910614757941014,"recovering_objects_per_sec":40,"recovering_bytes_per_sec":18425,"recovering_keys_per_sec":0,"num_objects_recovered":486,"num_bytes_recovered":221184,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":453},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-17T15:31:29.695933+0000","services":{}},"progress_events":{}} 2024-09-17T15:32:21.517 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:32:21.518 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:32:22.112 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:32:22.114 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:32:22.134 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":277,"stamp":"2024-09-17T15:32:21.719584+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":393122569,"num_objects":95989,"num_object_clones":0,"num_object_copies":191978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38224,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95989,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":197002,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11237,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160668,"ondisk_log_size":160668,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2046560,"kb_used_data":970020,"kb_used_omap":13,"kb_used_meta":1076466,"kb_avail":281068960,"statfs":{"total":289910292480,"available":287814615040,"internally_reserved":0,"allocated":993300480,"data_stored":992394203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1102302207},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,11,8,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19488768,"num_objects":-4758,"num_object_clones":0,"num_object_copies":-9516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-292,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4758,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003712"},"pg_stats":[{"pgid":"2.f","version":"45'14695","reported_seq":14950,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:17.711578+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:32:17.711578+0000","last_peered":"2024-09-17T15:32:17.711578+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:32:17.711578+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'4600","ondisk_log_start":"41'4600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24383488,"num_objects":5953,"num_object_clones":0,"num_object_copies":11906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14695,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1969}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14833","reported_seq":14941,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:17.819275+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:32:17.819275+0000","last_peered":"2024-09-17T15:32:17.819275+0000","last_clean":"2024-09-17T15:32:17.819275+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:32:17.819275+0000","last_undegraded":"2024-09-17T15:32:17.819275+0000","last_fullsized":"2024-09-17T15:32:17.819275+0000","mapping_epoch":33,"log_start":"41'4800","ondisk_log_start":"41'4800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24982470,"num_objects":6101,"num_object_clones":0,"num_object_copies":12202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6101,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13190,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14967","reported_seq":15184,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:17.775293+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:32:17.775293+0000","last_peered":"2024-09-17T15:32:17.775293+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:32:17.775293+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'4900","ondisk_log_start":"41'4900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24436760,"num_objects":5967,"num_object_clones":0,"num_object_copies":11934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5967,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13124,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1842}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'14881","reported_seq":14956,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:17.710660+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:32:17.710660+0000","last_peered":"2024-09-17T15:32:17.710660+0000","last_clean":"2024-09-17T15:32:17.710660+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:32:17.710660+0000","last_undegraded":"2024-09-17T15:32:17.710660+0000","last_fullsized":"2024-09-17T15:32:17.710660+0000","mapping_epoch":33,"log_start":"41'4800","ondisk_log_start":"41'4800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24940544,"num_objects":6089,"num_object_clones":0,"num_object_copies":12178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14881,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14608","reported_seq":14742,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:17.645481+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:32:17.645481+0000","last_peered":"2024-09-17T15:32:17.645481+0000","last_clean":"2024-09-17T15:32:17.645481+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:32:17.645481+0000","last_undegraded":"2024-09-17T15:32:17.645481+0000","last_fullsized":"2024-09-17T15:32:17.645481+0000","mapping_epoch":32,"log_start":"41'4600","ondisk_log_start":"41'4600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24760803,"num_objects":6046,"num_object_clones":0,"num_object_copies":12092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6046,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11590,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'15213","reported_seq":15508,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:17.743609+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:32:17.743609+0000","last_peered":"2024-09-17T15:32:17.743609+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:32:17.743609+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'5200","ondisk_log_start":"41'5200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24732131,"num_objects":6039,"num_object_clones":0,"num_object_copies":12078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6039,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11712,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1847},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14861","reported_seq":15175,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:17.697399+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:32:17.697399+0000","last_peered":"2024-09-17T15:32:17.697399+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:32:17.697399+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'4800","ondisk_log_start":"41'4800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24196038,"num_objects":5909,"num_object_clones":0,"num_object_copies":11818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5909,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11630,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1820},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14910","reported_seq":15043,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:17.692635+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:32:17.692635+0000","last_peered":"2024-09-17T15:32:17.692635+0000","last_clean":"2024-09-17T15:32:17.692635+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:32:17.692635+0000","last_undegraded":"2024-09-17T15:32:17.692635+0000","last_fullsized":"2024-09-17T15:32:17.692635+0000","mapping_epoch":32,"log_start":"41'4900","ondisk_log_start":"41'4900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24600576,"num_objects":6006,"num_object_clones":0,"num_object_copies":12012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11864,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.086967+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.086967+0000","last_peered":"2024-09-17T15:30:57.086967+0000","last_clean":"2024-09-17T15:30:57.086967+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.086967+0000","last_undegraded":"2024-09-17T15:30:57.086967+0000","last_fullsized":"2024-09-17T15:30:57.086967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14758","reported_seq":23045,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:17.103061+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:32:17.103061+0000","last_peered":"2024-09-17T15:32:17.103061+0000","last_clean":"2024-09-17T15:32:17.103061+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:32:17.103061+0000","last_undegraded":"2024-09-17T15:32:17.103061+0000","last_fullsized":"2024-09-17T15:32:17.103061+0000","mapping_epoch":42,"log_start":"41'4700","ondisk_log_start":"41'4700","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24818147,"num_objects":6060,"num_object_clones":0,"num_object_copies":12120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6060,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12758,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14567","reported_seq":14833,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:17.743250+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:32:17.743250+0000","last_peered":"2024-09-17T15:32:17.743250+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:32:17.743250+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'4500","ondisk_log_start":"41'4500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24473600,"num_objects":5975,"num_object_clones":0,"num_object_copies":11950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5975,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11163,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2032}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14602","reported_seq":14816,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:17.820670+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:32:17.820670+0000","last_peered":"2024-09-17T15:32:17.820670+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:32:17.820670+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'4600","ondisk_log_start":"41'4600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24236515,"num_objects":5918,"num_object_clones":0,"num_object_copies":11836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5918,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12497,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1885}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14789","reported_seq":19762,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:17.830830+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:32:17.830830+0000","last_peered":"2024-09-17T15:32:17.830830+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:32:17.830830+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"41'4700","ondisk_log_start":"41'4700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24358912,"num_objects":5947,"num_object_clones":0,"num_object_copies":11894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12551,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2424,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4232}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14919","reported_seq":15221,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:17.563843+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:32:17.563843+0000","last_peered":"2024-09-17T15:32:17.563843+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:32:17.563843+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'4900","ondisk_log_start":"41'4900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24768512,"num_objects":6047,"num_object_clones":0,"num_object_copies":12094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11467,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1990}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14817","reported_seq":15088,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:17.731895+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:32:17.731895+0000","last_peered":"2024-09-17T15:32:17.731895+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:32:17.731895+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'4800","ondisk_log_start":"41'4800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24391680,"num_objects":5955,"num_object_clones":0,"num_object_copies":11910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5955,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11104,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1860}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14838","reported_seq":15141,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:17.723089+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:32:17.723089+0000","last_peered":"2024-09-17T15:32:17.723089+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:32:17.723089+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'4800","ondisk_log_start":"41'4800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24772608,"num_objects":6048,"num_object_clones":0,"num_object_copies":12096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11625,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2068}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14707","reported_seq":23086,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:17.067069+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:32:17.067069+0000","last_peered":"2024-09-17T15:32:17.067069+0000","last_clean":"2024-09-17T15:32:17.067069+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:32:17.067069+0000","last_undegraded":"2024-09-17T15:32:17.067069+0000","last_fullsized":"2024-09-17T15:32:17.067069+0000","mapping_epoch":44,"log_start":"41'4700","ondisk_log_start":"41'4700","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24269766,"num_objects":5927,"num_object_clones":0,"num_object_copies":11854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11149,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":393122550,"num_objects":95987,"num_object_clones":0,"num_object_copies":191974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38224,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95987,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":197000,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11236,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":989483008,"data_stored":989337570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160665,"ondisk_log_size":160665,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051671,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":649692,"kb_used_data":325060,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93722148,"statfs":{"total":96636764160,"available":95971479552,"internally_reserved":0,"allocated":332861440,"data_stored":332570343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":332394156},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724616,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":636196,"kb_used_data":289360,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93735644,"statfs":{"total":96636764160,"available":95985299456,"internally_reserved":0,"allocated":296304640,"data_stored":295980562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247899,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":760672,"kb_used_data":355600,"kb_used_omap":4,"kb_used_meta":405051,"kb_avail":93611168,"statfs":{"total":96636764160,"available":95857836032,"internally_reserved":0,"allocated":364134400,"data_stored":363843298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":414772904},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,8,1,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":362598400,"data_stored":362562270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":294723584,"data_stored":294654478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":332161024,"data_stored":332120822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:32:25.137 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:32:25.671 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:32:25.671 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:32:25.700 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":278,"stamp":"2024-09-17T15:32:23.720168+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":383275785,"num_objects":93585,"num_object_clones":0,"num_object_copies":187170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93585,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":199406,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11383,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160872,"ondisk_log_size":160872,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2006508,"kb_used_data":950776,"kb_used_omap":13,"kb_used_meta":1055666,"kb_avail":281109012,"statfs":{"total":289910292480,"available":287855628288,"internally_reserved":0,"allocated":973594624,"data_stored":972688347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1081003007},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,5,15,15],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29335552,"num_objects":-7162,"num_object_clones":0,"num_object_copies":-14324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7162,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":438,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003573"},"pg_stats":[{"pgid":"2.f","version":"45'14849","reported_seq":15101,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:22.774366+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:32:22.774366+0000","last_peered":"2024-09-17T15:32:22.774366+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:32:22.774366+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'4800","ondisk_log_start":"41'4800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23752704,"num_objects":5799,"num_object_clones":0,"num_object_copies":11598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5799,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14849,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1815}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14986","reported_seq":15094,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:22.730344+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:32:22.730344+0000","last_peered":"2024-09-17T15:32:22.730344+0000","last_clean":"2024-09-17T15:32:22.730344+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:32:22.730344+0000","last_undegraded":"2024-09-17T15:32:22.730344+0000","last_fullsized":"2024-09-17T15:32:22.730344+0000","mapping_epoch":33,"log_start":"41'4900","ondisk_log_start":"41'4900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24355782,"num_objects":5948,"num_object_clones":0,"num_object_copies":11896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5948,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13343,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'15118","reported_seq":15335,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:22.592410+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:32:22.592410+0000","last_peered":"2024-09-17T15:32:22.592410+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:32:22.592410+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'5100","ondisk_log_start":"41'5100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23818264,"num_objects":5816,"num_object_clones":0,"num_object_copies":11632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5816,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13275,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1691}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'15042","reported_seq":15117,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:22.797356+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:32:22.797356+0000","last_peered":"2024-09-17T15:32:22.797356+0000","last_clean":"2024-09-17T15:32:22.797356+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:32:22.797356+0000","last_undegraded":"2024-09-17T15:32:22.797356+0000","last_fullsized":"2024-09-17T15:32:22.797356+0000","mapping_epoch":33,"log_start":"41'5000","ondisk_log_start":"41'5000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24281088,"num_objects":5928,"num_object_clones":0,"num_object_copies":11856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5928,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15042,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14771","reported_seq":14905,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:22.714136+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:32:22.714136+0000","last_peered":"2024-09-17T15:32:22.714136+0000","last_clean":"2024-09-17T15:32:22.714136+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:32:22.714136+0000","last_undegraded":"2024-09-17T15:32:22.714136+0000","last_fullsized":"2024-09-17T15:32:22.714136+0000","mapping_epoch":32,"log_start":"41'4700","ondisk_log_start":"41'4700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24093155,"num_objects":5883,"num_object_clones":0,"num_object_copies":11766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5883,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11753,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'15359","reported_seq":15655,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:22.761181+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:32:22.761181+0000","last_peered":"2024-09-17T15:32:22.761181+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:32:22.761181+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'5300","ondisk_log_start":"41'5300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24134115,"num_objects":5893,"num_object_clones":0,"num_object_copies":11786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5893,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11858,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1701},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14990","reported_seq":15304,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:22.693227+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:32:22.693227+0000","last_peered":"2024-09-17T15:32:22.693227+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:32:22.693227+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'4900","ondisk_log_start":"41'4900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23667654,"num_objects":5780,"num_object_clones":0,"num_object_copies":11560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5780,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11759,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1691},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'15062","reported_seq":15195,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:22.721607+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:32:22.721607+0000","last_peered":"2024-09-17T15:32:22.721607+0000","last_clean":"2024-09-17T15:32:22.721607+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:32:22.721607+0000","last_undegraded":"2024-09-17T15:32:22.721607+0000","last_fullsized":"2024-09-17T15:32:22.721607+0000","mapping_epoch":32,"log_start":"41'5000","ondisk_log_start":"41'5000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23977984,"num_objects":5854,"num_object_clones":0,"num_object_copies":11708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12016,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087024+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.087024+0000","last_peered":"2024-09-17T15:30:57.087024+0000","last_clean":"2024-09-17T15:30:57.087024+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.087024+0000","last_undegraded":"2024-09-17T15:30:57.087024+0000","last_fullsized":"2024-09-17T15:30:57.087024+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14907","reported_seq":23194,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:22.113968+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:32:22.113968+0000","last_peered":"2024-09-17T15:32:22.113968+0000","last_clean":"2024-09-17T15:32:22.113968+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:32:22.113968+0000","last_undegraded":"2024-09-17T15:32:22.113968+0000","last_fullsized":"2024-09-17T15:32:22.113968+0000","mapping_epoch":42,"log_start":"41'4900","ondisk_log_start":"41'4900","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24207843,"num_objects":5911,"num_object_clones":0,"num_object_copies":11822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5911,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12907,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14716","reported_seq":14985,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:22.701012+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:32:22.701012+0000","last_peered":"2024-09-17T15:32:22.701012+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:32:22.701012+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'4700","ondisk_log_start":"41'4700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23863296,"num_objects":5826,"num_object_clones":0,"num_object_copies":11652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11312,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1883}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14757","reported_seq":14971,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:22.808642+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:32:22.808642+0000","last_peered":"2024-09-17T15:32:22.808642+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:32:22.808642+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'4700","ondisk_log_start":"41'4700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23601635,"num_objects":5763,"num_object_clones":0,"num_object_copies":11526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5763,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12652,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1730}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14929","reported_seq":20193,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:22.822101+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:32:22.822101+0000","last_peered":"2024-09-17T15:32:22.822101+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:32:22.822101+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"41'4900","ondisk_log_start":"41'4900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23785472,"num_objects":5807,"num_object_clones":0,"num_object_copies":11614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1569,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12691,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2570,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4238}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'15079","reported_seq":15381,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:22.704826+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:32:22.704826+0000","last_peered":"2024-09-17T15:32:22.704826+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:32:22.704826+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'5000","ondisk_log_start":"41'5000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24113152,"num_objects":5887,"num_object_clones":0,"num_object_copies":11774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11627,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1830}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14963","reported_seq":15234,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:22.696718+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:32:22.696718+0000","last_peered":"2024-09-17T15:32:22.696718+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:32:22.696718+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'4900","ondisk_log_start":"41'4900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23793664,"num_objects":5809,"num_object_clones":0,"num_object_copies":11618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5809,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11250,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1714}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14990","reported_seq":15293,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:22.750306+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:32:22.750306+0000","last_peered":"2024-09-17T15:32:22.750306+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:32:22.750306+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'4900","ondisk_log_start":"41'4900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24150016,"num_objects":5896,"num_object_clones":0,"num_object_copies":11792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11777,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14851","reported_seq":23230,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:22.135325+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:32:22.135325+0000","last_peered":"2024-09-17T15:32:22.135325+0000","last_clean":"2024-09-17T15:32:22.135325+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:32:22.135325+0000","last_undegraded":"2024-09-17T15:32:22.135325+0000","last_fullsized":"2024-09-17T15:32:22.135325+0000","mapping_epoch":44,"log_start":"41'4800","ondisk_log_start":"41'4800","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23679942,"num_objects":5783,"num_object_clones":0,"num_object_copies":11566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5783,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11293,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":383275766,"num_objects":93583,"num_object_clones":0,"num_object_copies":187166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93583,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":199404,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11382,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":969744384,"data_stored":969598946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160869,"ondisk_log_size":160869,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051672,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":648972,"kb_used_data":320184,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93722868,"statfs":{"total":96636764160,"available":95972216832,"internally_reserved":0,"allocated":327868416,"data_stored":327577319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":336653996},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724617,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":616484,"kb_used_data":282132,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93755356,"statfs":{"total":96636764160,"available":96005484544,"internally_reserved":0,"allocated":288903168,"data_stored":288579090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":342355627},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247900,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":741052,"kb_used_data":348460,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93630788,"statfs":{"total":96636764160,"available":95877926912,"internally_reserved":0,"allocated":356823040,"data_stored":356531938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":401993384},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,4,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":355160064,"data_stored":355123934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":287408128,"data_stored":287339022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":327176192,"data_stored":327135990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:32:25.702 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:32:26.248 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:32:26.253 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:32:26.274 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":279,"stamp":"2024-09-17T15:32:25.721039+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":383275785,"num_objects":93585,"num_object_clones":0,"num_object_copies":187170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93585,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":199406,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11383,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160872,"ondisk_log_size":160872,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2006508,"kb_used_data":950776,"kb_used_omap":13,"kb_used_meta":1055666,"kb_avail":281109012,"statfs":{"total":289910292480,"available":287855628288,"internally_reserved":0,"allocated":973594624,"data_stored":972688347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1081003007},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,5,15,15],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19476480,"num_objects":-4755,"num_object_clones":0,"num_object_copies":-9510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-292,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4755,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003749"},"pg_stats":[{"pgid":"2.f","version":"45'14849","reported_seq":15101,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:22.774366+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:32:22.774366+0000","last_peered":"2024-09-17T15:32:22.774366+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:32:22.774366+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'4800","ondisk_log_start":"41'4800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23752704,"num_objects":5799,"num_object_clones":0,"num_object_copies":11598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5799,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14849,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1815}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14986","reported_seq":15094,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:22.730344+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:32:22.730344+0000","last_peered":"2024-09-17T15:32:22.730344+0000","last_clean":"2024-09-17T15:32:22.730344+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:32:22.730344+0000","last_undegraded":"2024-09-17T15:32:22.730344+0000","last_fullsized":"2024-09-17T15:32:22.730344+0000","mapping_epoch":33,"log_start":"41'4900","ondisk_log_start":"41'4900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24355782,"num_objects":5948,"num_object_clones":0,"num_object_copies":11896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5948,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13343,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'15118","reported_seq":15335,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:22.592410+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:32:22.592410+0000","last_peered":"2024-09-17T15:32:22.592410+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:32:22.592410+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'5100","ondisk_log_start":"41'5100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23818264,"num_objects":5816,"num_object_clones":0,"num_object_copies":11632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5816,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13275,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1691}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'15042","reported_seq":15117,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:22.797356+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:32:22.797356+0000","last_peered":"2024-09-17T15:32:22.797356+0000","last_clean":"2024-09-17T15:32:22.797356+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:32:22.797356+0000","last_undegraded":"2024-09-17T15:32:22.797356+0000","last_fullsized":"2024-09-17T15:32:22.797356+0000","mapping_epoch":33,"log_start":"41'5000","ondisk_log_start":"41'5000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24281088,"num_objects":5928,"num_object_clones":0,"num_object_copies":11856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5928,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15042,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14771","reported_seq":14905,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:22.714136+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:32:22.714136+0000","last_peered":"2024-09-17T15:32:22.714136+0000","last_clean":"2024-09-17T15:32:22.714136+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:32:22.714136+0000","last_undegraded":"2024-09-17T15:32:22.714136+0000","last_fullsized":"2024-09-17T15:32:22.714136+0000","mapping_epoch":32,"log_start":"41'4700","ondisk_log_start":"41'4700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24093155,"num_objects":5883,"num_object_clones":0,"num_object_copies":11766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5883,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11753,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'15359","reported_seq":15655,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:22.761181+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:32:22.761181+0000","last_peered":"2024-09-17T15:32:22.761181+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:32:22.761181+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'5300","ondisk_log_start":"41'5300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24134115,"num_objects":5893,"num_object_clones":0,"num_object_copies":11786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5893,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11858,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1701},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14990","reported_seq":15304,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:22.693227+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:32:22.693227+0000","last_peered":"2024-09-17T15:32:22.693227+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:32:22.693227+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'4900","ondisk_log_start":"41'4900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23667654,"num_objects":5780,"num_object_clones":0,"num_object_copies":11560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5780,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11759,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1691},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'15062","reported_seq":15195,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:22.721607+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:32:22.721607+0000","last_peered":"2024-09-17T15:32:22.721607+0000","last_clean":"2024-09-17T15:32:22.721607+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:32:22.721607+0000","last_undegraded":"2024-09-17T15:32:22.721607+0000","last_fullsized":"2024-09-17T15:32:22.721607+0000","mapping_epoch":32,"log_start":"41'5000","ondisk_log_start":"41'5000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23977984,"num_objects":5854,"num_object_clones":0,"num_object_copies":11708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12016,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087024+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.087024+0000","last_peered":"2024-09-17T15:30:57.087024+0000","last_clean":"2024-09-17T15:30:57.087024+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.087024+0000","last_undegraded":"2024-09-17T15:30:57.087024+0000","last_fullsized":"2024-09-17T15:30:57.087024+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14907","reported_seq":23194,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:22.113968+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:32:22.113968+0000","last_peered":"2024-09-17T15:32:22.113968+0000","last_clean":"2024-09-17T15:32:22.113968+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:32:22.113968+0000","last_undegraded":"2024-09-17T15:32:22.113968+0000","last_fullsized":"2024-09-17T15:32:22.113968+0000","mapping_epoch":42,"log_start":"41'4900","ondisk_log_start":"41'4900","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24207843,"num_objects":5911,"num_object_clones":0,"num_object_copies":11822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5911,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12907,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14716","reported_seq":14985,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:22.701012+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:32:22.701012+0000","last_peered":"2024-09-17T15:32:22.701012+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:32:22.701012+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'4700","ondisk_log_start":"41'4700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23863296,"num_objects":5826,"num_object_clones":0,"num_object_copies":11652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11312,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1883}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14757","reported_seq":14971,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:22.808642+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:32:22.808642+0000","last_peered":"2024-09-17T15:32:22.808642+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:32:22.808642+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'4700","ondisk_log_start":"41'4700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23601635,"num_objects":5763,"num_object_clones":0,"num_object_copies":11526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5763,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12652,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1730}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14929","reported_seq":20193,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:22.822101+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:32:22.822101+0000","last_peered":"2024-09-17T15:32:22.822101+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:32:22.822101+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"41'4900","ondisk_log_start":"41'4900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23785472,"num_objects":5807,"num_object_clones":0,"num_object_copies":11614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1569,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12691,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2570,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4238}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'15079","reported_seq":15381,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:22.704826+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:32:22.704826+0000","last_peered":"2024-09-17T15:32:22.704826+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:32:22.704826+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'5000","ondisk_log_start":"41'5000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24113152,"num_objects":5887,"num_object_clones":0,"num_object_copies":11774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11627,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1830}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14963","reported_seq":15234,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:22.696718+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:32:22.696718+0000","last_peered":"2024-09-17T15:32:22.696718+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:32:22.696718+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'4900","ondisk_log_start":"41'4900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23793664,"num_objects":5809,"num_object_clones":0,"num_object_copies":11618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5809,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11250,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1714}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14990","reported_seq":15293,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:22.750306+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:32:22.750306+0000","last_peered":"2024-09-17T15:32:22.750306+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:32:22.750306+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'4900","ondisk_log_start":"41'4900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24150016,"num_objects":5896,"num_object_clones":0,"num_object_copies":11792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11777,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14851","reported_seq":23230,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:22.135325+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:32:22.135325+0000","last_peered":"2024-09-17T15:32:22.135325+0000","last_clean":"2024-09-17T15:32:22.135325+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:32:22.135325+0000","last_undegraded":"2024-09-17T15:32:22.135325+0000","last_fullsized":"2024-09-17T15:32:22.135325+0000","mapping_epoch":44,"log_start":"41'4800","ondisk_log_start":"41'4800","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23679942,"num_objects":5783,"num_object_clones":0,"num_object_copies":11566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5783,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11293,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":383275766,"num_objects":93583,"num_object_clones":0,"num_object_copies":187166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93583,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":199404,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11382,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":969744384,"data_stored":969598946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160869,"ondisk_log_size":160869,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051672,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":648972,"kb_used_data":320184,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93722868,"statfs":{"total":96636764160,"available":95972216832,"internally_reserved":0,"allocated":327868416,"data_stored":327577319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":336653996},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724617,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":616484,"kb_used_data":282132,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93755356,"statfs":{"total":96636764160,"available":96005484544,"internally_reserved":0,"allocated":288903168,"data_stored":288579090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":342355627},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247900,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":741052,"kb_used_data":348460,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93630788,"statfs":{"total":96636764160,"available":95877926912,"internally_reserved":0,"allocated":356823040,"data_stored":356531938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":401993384},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,4,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":355160064,"data_stored":355123934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":287408128,"data_stored":287339022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":327176192,"data_stored":327135990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:32:26.275 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:32:26.275 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'14849", 'reported_seq': 15101, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:22.774366+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:32:22.774366+0000', 'last_peered': '2024-09-17T15:32:22.774366+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:32:22.774366+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "41'4800", 'ondisk_log_start': "41'4800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23752704, 'num_objects': 5799, 'num_object_clones': 0, 'num_object_copies': 11598, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5799, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14849, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1815}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:26.275 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:32:26.275 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'15118", 'reported_seq': 15335, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:22.592410+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:32:22.592410+0000', 'last_peered': '2024-09-17T15:32:22.592410+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:32:22.592410+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "41'5100", 'ondisk_log_start': "41'5100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23818264, 'num_objects': 5816, 'num_object_clones': 0, 'num_object_copies': 11632, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5816, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13275, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1691}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:32:26.275 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:32:26.275 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'15359", 'reported_seq': 15655, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:22.761181+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:32:22.761181+0000', 'last_peered': '2024-09-17T15:32:22.761181+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:32:22.761181+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "41'5300", 'ondisk_log_start': "41'5300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24134115, 'num_objects': 5893, 'num_object_clones': 0, 'num_object_copies': 11786, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5893, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11858, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1701}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:26.276 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:32:26.276 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'14990", 'reported_seq': 15304, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:22.693227+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:32:22.693227+0000', 'last_peered': '2024-09-17T15:32:22.693227+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:32:22.693227+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "41'4900", 'ondisk_log_start': "41'4900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23667654, 'num_objects': 5780, 'num_object_clones': 0, 'num_object_copies': 11560, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4089, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5780, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 11759, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 85, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1691}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:26.276 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:32:26.276 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'14716", 'reported_seq': 14985, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:22.701012+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:32:22.701012+0000', 'last_peered': '2024-09-17T15:32:22.701012+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:32:22.701012+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "41'4700", 'ondisk_log_start': "41'4700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23863296, 'num_objects': 5826, 'num_object_clones': 0, 'num_object_copies': 11652, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5826, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11312, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1883}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:26.276 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:32:26.276 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'14757", 'reported_seq': 14971, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:22.808642+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:32:22.808642+0000', 'last_peered': '2024-09-17T15:32:22.808642+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:32:22.808642+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "41'4700", 'ondisk_log_start': "41'4700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23601635, 'num_objects': 5763, 'num_object_clones': 0, 'num_object_copies': 11526, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4034, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5763, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12652, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 1730}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:32:26.276 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:32:26.276 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'14929", 'reported_seq': 20193, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:22.822101+0000', 'last_change': '2024-09-17T15:30:55.086473+0000', 'last_active': '2024-09-17T15:32:22.822101+0000', 'last_peered': '2024-09-17T15:32:22.822101+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:32:22.822101+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "41'4900", 'ondisk_log_start': "41'4900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23785472, 'num_objects': 5807, 'num_object_clones': 0, 'num_object_copies': 11614, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1569, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5807, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12691, 'num_write_kb': 32532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2570, 'num_bytes_recovered': 8073216, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 4238}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:32:26.276 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:32:26.276 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'15079", 'reported_seq': 15381, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:22.704826+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:32:22.704826+0000', 'last_peered': '2024-09-17T15:32:22.704826+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:32:22.704826+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "41'5000", 'ondisk_log_start': "41'5000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24113152, 'num_objects': 5887, 'num_object_clones': 0, 'num_object_copies': 11774, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5887, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11627, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1830}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:26.277 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:32:26.277 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'14963", 'reported_seq': 15234, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:22.696718+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:32:22.696718+0000', 'last_peered': '2024-09-17T15:32:22.696718+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:32:22.696718+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "41'4900", 'ondisk_log_start': "41'4900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23793664, 'num_objects': 5809, 'num_object_clones': 0, 'num_object_copies': 11618, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4098, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5809, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11250, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 62, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1714}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:26.277 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:32:26.277 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'14990", 'reported_seq': 15293, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:22.750306+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:32:22.750306+0000', 'last_peered': '2024-09-17T15:32:22.750306+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:32:22.750306+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "41'4900", 'ondisk_log_start': "41'4900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24150016, 'num_objects': 5896, 'num_object_clones': 0, 'num_object_copies': 11792, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5896, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11777, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1916}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:26.277 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:32:26.962 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:32:26.962 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38078/187170 objects degraded (20.344%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":483,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":93585,"data_bytes":383275785,"bytes_used":2054664192,"bytes_avail":287855628288,"bytes_total":289910292480,"degraded_objects":38078,"degraded_total":187170,"degraded_ratio":0.20344072233798152,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":396},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-17T15:31:29.695933+0000","services":{}},"progress_events":{}} 2024-09-17T15:32:26.985 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:32:26.985 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:32:27.531 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:32:27.533 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:32:27.553 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":279,"stamp":"2024-09-17T15:32:25.721039+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":383275785,"num_objects":93585,"num_object_clones":0,"num_object_copies":187170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93585,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":199406,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11383,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160872,"ondisk_log_size":160872,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2006508,"kb_used_data":950776,"kb_used_omap":13,"kb_used_meta":1055666,"kb_avail":281109012,"statfs":{"total":289910292480,"available":287855628288,"internally_reserved":0,"allocated":973594624,"data_stored":972688347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1081003007},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,5,15,15],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19476480,"num_objects":-4755,"num_object_clones":0,"num_object_copies":-9510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-292,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4755,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003749"},"pg_stats":[{"pgid":"2.f","version":"45'14849","reported_seq":15101,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:22.774366+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:32:22.774366+0000","last_peered":"2024-09-17T15:32:22.774366+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:32:22.774366+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'4800","ondisk_log_start":"41'4800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23752704,"num_objects":5799,"num_object_clones":0,"num_object_copies":11598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5799,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14849,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1815}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14986","reported_seq":15094,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:22.730344+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:32:22.730344+0000","last_peered":"2024-09-17T15:32:22.730344+0000","last_clean":"2024-09-17T15:32:22.730344+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:32:22.730344+0000","last_undegraded":"2024-09-17T15:32:22.730344+0000","last_fullsized":"2024-09-17T15:32:22.730344+0000","mapping_epoch":33,"log_start":"41'4900","ondisk_log_start":"41'4900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24355782,"num_objects":5948,"num_object_clones":0,"num_object_copies":11896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5948,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13343,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'15118","reported_seq":15335,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:22.592410+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:32:22.592410+0000","last_peered":"2024-09-17T15:32:22.592410+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:32:22.592410+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'5100","ondisk_log_start":"41'5100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23818264,"num_objects":5816,"num_object_clones":0,"num_object_copies":11632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5816,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13275,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1691}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'15042","reported_seq":15117,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:22.797356+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:32:22.797356+0000","last_peered":"2024-09-17T15:32:22.797356+0000","last_clean":"2024-09-17T15:32:22.797356+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:32:22.797356+0000","last_undegraded":"2024-09-17T15:32:22.797356+0000","last_fullsized":"2024-09-17T15:32:22.797356+0000","mapping_epoch":33,"log_start":"41'5000","ondisk_log_start":"41'5000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24281088,"num_objects":5928,"num_object_clones":0,"num_object_copies":11856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5928,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15042,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14771","reported_seq":14905,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:22.714136+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:32:22.714136+0000","last_peered":"2024-09-17T15:32:22.714136+0000","last_clean":"2024-09-17T15:32:22.714136+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:32:22.714136+0000","last_undegraded":"2024-09-17T15:32:22.714136+0000","last_fullsized":"2024-09-17T15:32:22.714136+0000","mapping_epoch":32,"log_start":"41'4700","ondisk_log_start":"41'4700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24093155,"num_objects":5883,"num_object_clones":0,"num_object_copies":11766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5883,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11753,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'15359","reported_seq":15655,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:22.761181+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:32:22.761181+0000","last_peered":"2024-09-17T15:32:22.761181+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:32:22.761181+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'5300","ondisk_log_start":"41'5300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24134115,"num_objects":5893,"num_object_clones":0,"num_object_copies":11786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5893,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11858,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1701},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14990","reported_seq":15304,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:22.693227+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:32:22.693227+0000","last_peered":"2024-09-17T15:32:22.693227+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:32:22.693227+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'4900","ondisk_log_start":"41'4900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23667654,"num_objects":5780,"num_object_clones":0,"num_object_copies":11560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5780,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11759,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1691},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'15062","reported_seq":15195,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:22.721607+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:32:22.721607+0000","last_peered":"2024-09-17T15:32:22.721607+0000","last_clean":"2024-09-17T15:32:22.721607+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:32:22.721607+0000","last_undegraded":"2024-09-17T15:32:22.721607+0000","last_fullsized":"2024-09-17T15:32:22.721607+0000","mapping_epoch":32,"log_start":"41'5000","ondisk_log_start":"41'5000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23977984,"num_objects":5854,"num_object_clones":0,"num_object_copies":11708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12016,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087024+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.087024+0000","last_peered":"2024-09-17T15:30:57.087024+0000","last_clean":"2024-09-17T15:30:57.087024+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.087024+0000","last_undegraded":"2024-09-17T15:30:57.087024+0000","last_fullsized":"2024-09-17T15:30:57.087024+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14907","reported_seq":23194,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:22.113968+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:32:22.113968+0000","last_peered":"2024-09-17T15:32:22.113968+0000","last_clean":"2024-09-17T15:32:22.113968+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:32:22.113968+0000","last_undegraded":"2024-09-17T15:32:22.113968+0000","last_fullsized":"2024-09-17T15:32:22.113968+0000","mapping_epoch":42,"log_start":"41'4900","ondisk_log_start":"41'4900","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24207843,"num_objects":5911,"num_object_clones":0,"num_object_copies":11822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5911,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12907,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14716","reported_seq":14985,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:22.701012+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:32:22.701012+0000","last_peered":"2024-09-17T15:32:22.701012+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:32:22.701012+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'4700","ondisk_log_start":"41'4700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23863296,"num_objects":5826,"num_object_clones":0,"num_object_copies":11652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11312,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1883}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14757","reported_seq":14971,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:22.808642+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:32:22.808642+0000","last_peered":"2024-09-17T15:32:22.808642+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:32:22.808642+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'4700","ondisk_log_start":"41'4700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23601635,"num_objects":5763,"num_object_clones":0,"num_object_copies":11526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5763,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12652,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1730}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14929","reported_seq":20193,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:22.822101+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:32:22.822101+0000","last_peered":"2024-09-17T15:32:22.822101+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:32:22.822101+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"41'4900","ondisk_log_start":"41'4900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23785472,"num_objects":5807,"num_object_clones":0,"num_object_copies":11614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1569,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12691,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2570,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4238}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'15079","reported_seq":15381,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:22.704826+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:32:22.704826+0000","last_peered":"2024-09-17T15:32:22.704826+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:32:22.704826+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'5000","ondisk_log_start":"41'5000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24113152,"num_objects":5887,"num_object_clones":0,"num_object_copies":11774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11627,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1830}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14963","reported_seq":15234,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:22.696718+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:32:22.696718+0000","last_peered":"2024-09-17T15:32:22.696718+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:32:22.696718+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'4900","ondisk_log_start":"41'4900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23793664,"num_objects":5809,"num_object_clones":0,"num_object_copies":11618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5809,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11250,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1714}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14990","reported_seq":15293,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:22.750306+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:32:22.750306+0000","last_peered":"2024-09-17T15:32:22.750306+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:32:22.750306+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'4900","ondisk_log_start":"41'4900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24150016,"num_objects":5896,"num_object_clones":0,"num_object_copies":11792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11777,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14851","reported_seq":23230,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:22.135325+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:32:22.135325+0000","last_peered":"2024-09-17T15:32:22.135325+0000","last_clean":"2024-09-17T15:32:22.135325+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:32:22.135325+0000","last_undegraded":"2024-09-17T15:32:22.135325+0000","last_fullsized":"2024-09-17T15:32:22.135325+0000","mapping_epoch":44,"log_start":"41'4800","ondisk_log_start":"41'4800","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23679942,"num_objects":5783,"num_object_clones":0,"num_object_copies":11566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5783,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11293,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":383275766,"num_objects":93583,"num_object_clones":0,"num_object_copies":187166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93583,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":199404,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11382,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":969744384,"data_stored":969598946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160869,"ondisk_log_size":160869,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051672,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":648972,"kb_used_data":320184,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93722868,"statfs":{"total":96636764160,"available":95972216832,"internally_reserved":0,"allocated":327868416,"data_stored":327577319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":336653996},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724617,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":616484,"kb_used_data":282132,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93755356,"statfs":{"total":96636764160,"available":96005484544,"internally_reserved":0,"allocated":288903168,"data_stored":288579090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":342355627},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247900,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":741052,"kb_used_data":348460,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93630788,"statfs":{"total":96636764160,"available":95877926912,"internally_reserved":0,"allocated":356823040,"data_stored":356531938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":401993384},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,4,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":355160064,"data_stored":355123934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":287408128,"data_stored":287339022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":327176192,"data_stored":327135990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:32:30.555 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:32:31.125 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:32:31.125 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:32:31.161 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":281,"stamp":"2024-09-17T15:32:29.722200+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":373404425,"num_objects":91175,"num_object_clones":0,"num_object_copies":182350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37941,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91175,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":201816,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11520,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160682,"ondisk_log_size":160682,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1987124,"kb_used_data":931384,"kb_used_omap":13,"kb_used_meta":1055666,"kb_avail":281128396,"statfs":{"total":289910292480,"available":287875477504,"internally_reserved":0,"allocated":953737216,"data_stored":952830939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1081003007},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,8,4,6,9,6,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28160000,"num_objects":-6875,"num_object_clones":0,"num_object_copies":-13750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-429,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6875,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":429,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003571"},"pg_stats":[{"pgid":"2.f","version":"45'15000","reported_seq":15255,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:27.790474+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:32:27.790474+0000","last_peered":"2024-09-17T15:32:27.790474+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:32:27.790474+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'4900","ondisk_log_start":"41'4900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23134208,"num_objects":5648,"num_object_clones":0,"num_object_copies":11296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15000,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1664}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'15116","reported_seq":15224,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:27.775836+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:32:27.775836+0000","last_peered":"2024-09-17T15:32:27.775836+0000","last_clean":"2024-09-17T15:32:27.775836+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:32:27.775836+0000","last_undegraded":"2024-09-17T15:32:27.775836+0000","last_fullsized":"2024-09-17T15:32:27.775836+0000","mapping_epoch":33,"log_start":"41'5100","ondisk_log_start":"41'5100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23823302,"num_objects":5818,"num_object_clones":0,"num_object_copies":11636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5818,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13473,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'15264","reported_seq":15481,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:27.748977+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:32:27.748977+0000","last_peered":"2024-09-17T15:32:27.748977+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:32:27.748977+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'5200","ondisk_log_start":"41'5200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23220248,"num_objects":5670,"num_object_clones":0,"num_object_copies":11340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5670,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13421,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1545}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'15196","reported_seq":15270,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:27.841898+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:32:27.841898+0000","last_peered":"2024-09-17T15:32:27.841898+0000","last_clean":"2024-09-17T15:32:27.841898+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:32:27.841898+0000","last_undegraded":"2024-09-17T15:32:27.841898+0000","last_fullsized":"2024-09-17T15:32:27.841898+0000","mapping_epoch":33,"log_start":"41'5100","ondisk_log_start":"41'5100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23650304,"num_objects":5774,"num_object_clones":0,"num_object_copies":11548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15196,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14907","reported_seq":15041,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:27.763155+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:32:27.763155+0000","last_peered":"2024-09-17T15:32:27.763155+0000","last_clean":"2024-09-17T15:32:27.763155+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:32:27.763155+0000","last_undegraded":"2024-09-17T15:32:27.763155+0000","last_fullsized":"2024-09-17T15:32:27.763155+0000","mapping_epoch":32,"log_start":"41'4900","ondisk_log_start":"41'4900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23536099,"num_objects":5747,"num_object_clones":0,"num_object_copies":11494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5747,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11889,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'15505","reported_seq":15801,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:27.738603+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:32:27.738603+0000","last_peered":"2024-09-17T15:32:27.738603+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:32:27.738603+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'5500","ondisk_log_start":"41'5500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23536099,"num_objects":5747,"num_object_clones":0,"num_object_copies":11494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5747,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12004,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1555},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'15147","reported_seq":15461,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:27.725852+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:32:27.725852+0000","last_peered":"2024-09-17T15:32:27.725852+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:32:27.725852+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'5100","ondisk_log_start":"41'5100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23024582,"num_objects":5623,"num_object_clones":0,"num_object_copies":11246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5623,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11916,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1534},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'15207","reported_seq":15340,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:27.710327+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:32:27.710327+0000","last_peered":"2024-09-17T15:32:27.710327+0000","last_clean":"2024-09-17T15:32:27.710327+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:32:27.710327+0000","last_undegraded":"2024-09-17T15:32:27.710327+0000","last_fullsized":"2024-09-17T15:32:27.710327+0000","mapping_epoch":32,"log_start":"41'5200","ondisk_log_start":"41'5200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23384064,"num_objects":5709,"num_object_clones":0,"num_object_copies":11418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12161,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087024+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.087024+0000","last_peered":"2024-09-17T15:30:57.087024+0000","last_clean":"2024-09-17T15:30:57.087024+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.087024+0000","last_undegraded":"2024-09-17T15:30:57.087024+0000","last_fullsized":"2024-09-17T15:30:57.087024+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'15060","reported_seq":23347,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:27.014098+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:32:27.014098+0000","last_peered":"2024-09-17T15:32:27.014098+0000","last_clean":"2024-09-17T15:32:27.014098+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:32:27.014098+0000","last_undegraded":"2024-09-17T15:32:27.014098+0000","last_fullsized":"2024-09-17T15:32:27.014098+0000","mapping_epoch":42,"log_start":"41'5000","ondisk_log_start":"41'5000","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23581155,"num_objects":5758,"num_object_clones":0,"num_object_copies":11516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5758,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13060,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14871","reported_seq":15140,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:27.739874+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:32:27.739874+0000","last_peered":"2024-09-17T15:32:27.739874+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:32:27.739874+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'4800","ondisk_log_start":"41'4800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23228416,"num_objects":5671,"num_object_clones":0,"num_object_copies":11342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11467,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1728}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14922","reported_seq":15136,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:27.814253+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:32:27.814253+0000","last_peered":"2024-09-17T15:32:27.814253+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:32:27.814253+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'4900","ondisk_log_start":"41'4900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22925795,"num_objects":5598,"num_object_clones":0,"num_object_copies":11196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5598,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12817,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1565}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'15084","reported_seq":20623,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:27.841624+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:32:27.841624+0000","last_peered":"2024-09-17T15:32:27.841624+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:32:27.841624+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"41'5000","ondisk_log_start":"41'5000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23150592,"num_objects":5652,"num_object_clones":0,"num_object_copies":11304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1432,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12846,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2707,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4220}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'15251","reported_seq":15551,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:27.760369+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:32:27.760369+0000","last_peered":"2024-09-17T15:32:27.760369+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:32:27.760369+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'5200","ondisk_log_start":"41'5200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23408640,"num_objects":5715,"num_object_clones":0,"num_object_copies":11430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11799,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1658}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'15107","reported_seq":15378,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:27.708814+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:32:27.708814+0000","last_peered":"2024-09-17T15:32:27.708814+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:32:27.708814+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'5100","ondisk_log_start":"41'5100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23203840,"num_objects":5665,"num_object_clones":0,"num_object_copies":11330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11394,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1570}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'15132","reported_seq":15435,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:27.663884+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:32:27.663884+0000","last_peered":"2024-09-17T15:32:27.663884+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:32:27.663884+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'5100","ondisk_log_start":"41'5100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23568384,"num_objects":5754,"num_object_clones":0,"num_object_copies":11508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11919,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1774}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'15010","reported_seq":23389,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:27.177515+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:32:27.177515+0000","last_peered":"2024-09-17T15:32:27.177515+0000","last_clean":"2024-09-17T15:32:27.177515+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:32:27.177515+0000","last_undegraded":"2024-09-17T15:32:27.177515+0000","last_fullsized":"2024-09-17T15:32:27.177515+0000","mapping_epoch":44,"log_start":"41'5000","ondisk_log_start":"41'5000","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23028678,"num_objects":5624,"num_object_clones":0,"num_object_copies":11248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11452,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":373404406,"num_objects":91173,"num_object_clones":0,"num_object_copies":182346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37941,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91173,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":201814,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11519,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":950030336,"data_stored":949884898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160679,"ondisk_log_size":160679,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051673,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":631664,"kb_used_data":315356,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93740176,"statfs":{"total":96636764160,"available":95989940224,"internally_reserved":0,"allocated":322924544,"data_stored":322633447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":323874476},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,0,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724618,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":617476,"kb_used_data":274804,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93754364,"statfs":{"total":96636764160,"available":96004468736,"internally_reserved":0,"allocated":281399296,"data_stored":281075218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":350875307},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,3,4,0,4,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247901,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":737984,"kb_used_data":341224,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93633856,"statfs":{"total":96636764160,"available":95881068544,"internally_reserved":0,"allocated":349413376,"data_stored":349122274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":406253224},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,8,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":347881472,"data_stored":347845342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279834624,"data_stored":279765518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":322314240,"data_stored":322274038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:32:31.163 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:32:31.726 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:32:31.726 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:32:31.753 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":281,"stamp":"2024-09-17T15:32:29.722200+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":373404425,"num_objects":91175,"num_object_clones":0,"num_object_copies":182350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37941,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91175,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":201816,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11520,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160682,"ondisk_log_size":160682,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1987124,"kb_used_data":931384,"kb_used_omap":13,"kb_used_meta":1055666,"kb_avail":281128396,"statfs":{"total":289910292480,"available":287875477504,"internally_reserved":0,"allocated":953737216,"data_stored":952830939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1081003007},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,8,4,6,9,6,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28160000,"num_objects":-6875,"num_object_clones":0,"num_object_copies":-13750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-429,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6875,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":429,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003571"},"pg_stats":[{"pgid":"2.f","version":"45'15000","reported_seq":15255,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:27.790474+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:32:27.790474+0000","last_peered":"2024-09-17T15:32:27.790474+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:32:27.790474+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'4900","ondisk_log_start":"41'4900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23134208,"num_objects":5648,"num_object_clones":0,"num_object_copies":11296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15000,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1664}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'15116","reported_seq":15224,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:27.775836+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:32:27.775836+0000","last_peered":"2024-09-17T15:32:27.775836+0000","last_clean":"2024-09-17T15:32:27.775836+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:32:27.775836+0000","last_undegraded":"2024-09-17T15:32:27.775836+0000","last_fullsized":"2024-09-17T15:32:27.775836+0000","mapping_epoch":33,"log_start":"41'5100","ondisk_log_start":"41'5100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23823302,"num_objects":5818,"num_object_clones":0,"num_object_copies":11636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5818,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13473,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'15264","reported_seq":15481,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:27.748977+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:32:27.748977+0000","last_peered":"2024-09-17T15:32:27.748977+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:32:27.748977+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'5200","ondisk_log_start":"41'5200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23220248,"num_objects":5670,"num_object_clones":0,"num_object_copies":11340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5670,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13421,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1545}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'15196","reported_seq":15270,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:27.841898+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:32:27.841898+0000","last_peered":"2024-09-17T15:32:27.841898+0000","last_clean":"2024-09-17T15:32:27.841898+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:32:27.841898+0000","last_undegraded":"2024-09-17T15:32:27.841898+0000","last_fullsized":"2024-09-17T15:32:27.841898+0000","mapping_epoch":33,"log_start":"41'5100","ondisk_log_start":"41'5100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23650304,"num_objects":5774,"num_object_clones":0,"num_object_copies":11548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15196,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14907","reported_seq":15041,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:27.763155+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:32:27.763155+0000","last_peered":"2024-09-17T15:32:27.763155+0000","last_clean":"2024-09-17T15:32:27.763155+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:32:27.763155+0000","last_undegraded":"2024-09-17T15:32:27.763155+0000","last_fullsized":"2024-09-17T15:32:27.763155+0000","mapping_epoch":32,"log_start":"41'4900","ondisk_log_start":"41'4900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23536099,"num_objects":5747,"num_object_clones":0,"num_object_copies":11494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5747,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11889,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'15505","reported_seq":15801,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:27.738603+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:32:27.738603+0000","last_peered":"2024-09-17T15:32:27.738603+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:32:27.738603+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'5500","ondisk_log_start":"41'5500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23536099,"num_objects":5747,"num_object_clones":0,"num_object_copies":11494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5747,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12004,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1555},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'15147","reported_seq":15461,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:27.725852+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:32:27.725852+0000","last_peered":"2024-09-17T15:32:27.725852+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:32:27.725852+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'5100","ondisk_log_start":"41'5100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23024582,"num_objects":5623,"num_object_clones":0,"num_object_copies":11246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5623,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11916,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1534},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'15207","reported_seq":15340,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:27.710327+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:32:27.710327+0000","last_peered":"2024-09-17T15:32:27.710327+0000","last_clean":"2024-09-17T15:32:27.710327+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:32:27.710327+0000","last_undegraded":"2024-09-17T15:32:27.710327+0000","last_fullsized":"2024-09-17T15:32:27.710327+0000","mapping_epoch":32,"log_start":"41'5200","ondisk_log_start":"41'5200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23384064,"num_objects":5709,"num_object_clones":0,"num_object_copies":11418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12161,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087024+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.087024+0000","last_peered":"2024-09-17T15:30:57.087024+0000","last_clean":"2024-09-17T15:30:57.087024+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.087024+0000","last_undegraded":"2024-09-17T15:30:57.087024+0000","last_fullsized":"2024-09-17T15:30:57.087024+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'15060","reported_seq":23347,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:27.014098+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:32:27.014098+0000","last_peered":"2024-09-17T15:32:27.014098+0000","last_clean":"2024-09-17T15:32:27.014098+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:32:27.014098+0000","last_undegraded":"2024-09-17T15:32:27.014098+0000","last_fullsized":"2024-09-17T15:32:27.014098+0000","mapping_epoch":42,"log_start":"41'5000","ondisk_log_start":"41'5000","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23581155,"num_objects":5758,"num_object_clones":0,"num_object_copies":11516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5758,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13060,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14871","reported_seq":15140,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:27.739874+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:32:27.739874+0000","last_peered":"2024-09-17T15:32:27.739874+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:32:27.739874+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'4800","ondisk_log_start":"41'4800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23228416,"num_objects":5671,"num_object_clones":0,"num_object_copies":11342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11467,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1728}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14922","reported_seq":15136,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:27.814253+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:32:27.814253+0000","last_peered":"2024-09-17T15:32:27.814253+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:32:27.814253+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'4900","ondisk_log_start":"41'4900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22925795,"num_objects":5598,"num_object_clones":0,"num_object_copies":11196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5598,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12817,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1565}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'15084","reported_seq":20623,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:27.841624+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:32:27.841624+0000","last_peered":"2024-09-17T15:32:27.841624+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:32:27.841624+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"41'5000","ondisk_log_start":"41'5000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23150592,"num_objects":5652,"num_object_clones":0,"num_object_copies":11304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1432,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12846,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2707,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4220}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'15251","reported_seq":15551,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:27.760369+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:32:27.760369+0000","last_peered":"2024-09-17T15:32:27.760369+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:32:27.760369+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'5200","ondisk_log_start":"41'5200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23408640,"num_objects":5715,"num_object_clones":0,"num_object_copies":11430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11799,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1658}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'15107","reported_seq":15378,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:27.708814+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:32:27.708814+0000","last_peered":"2024-09-17T15:32:27.708814+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:32:27.708814+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'5100","ondisk_log_start":"41'5100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23203840,"num_objects":5665,"num_object_clones":0,"num_object_copies":11330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11394,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1570}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'15132","reported_seq":15435,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:27.663884+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:32:27.663884+0000","last_peered":"2024-09-17T15:32:27.663884+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:32:27.663884+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'5100","ondisk_log_start":"41'5100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23568384,"num_objects":5754,"num_object_clones":0,"num_object_copies":11508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11919,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1774}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'15010","reported_seq":23389,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:27.177515+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:32:27.177515+0000","last_peered":"2024-09-17T15:32:27.177515+0000","last_clean":"2024-09-17T15:32:27.177515+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:32:27.177515+0000","last_undegraded":"2024-09-17T15:32:27.177515+0000","last_fullsized":"2024-09-17T15:32:27.177515+0000","mapping_epoch":44,"log_start":"41'5000","ondisk_log_start":"41'5000","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23028678,"num_objects":5624,"num_object_clones":0,"num_object_copies":11248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11452,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":373404406,"num_objects":91173,"num_object_clones":0,"num_object_copies":182346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37941,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91173,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":201814,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11519,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":950030336,"data_stored":949884898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160679,"ondisk_log_size":160679,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051673,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":631664,"kb_used_data":315356,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93740176,"statfs":{"total":96636764160,"available":95989940224,"internally_reserved":0,"allocated":322924544,"data_stored":322633447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":323874476},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,0,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724618,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":617476,"kb_used_data":274804,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93754364,"statfs":{"total":96636764160,"available":96004468736,"internally_reserved":0,"allocated":281399296,"data_stored":281075218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":350875307},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,3,4,0,4,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247901,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":737984,"kb_used_data":341224,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93633856,"statfs":{"total":96636764160,"available":95881068544,"internally_reserved":0,"allocated":349413376,"data_stored":349122274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":406253224},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,8,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":347881472,"data_stored":347845342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279834624,"data_stored":279765518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":322314240,"data_stored":322274038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:32:31.754 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:32:31.754 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'15000", 'reported_seq': 15255, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:27.790474+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:32:27.790474+0000', 'last_peered': '2024-09-17T15:32:27.790474+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:32:27.790474+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "41'4900", 'ondisk_log_start': "41'4900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23134208, 'num_objects': 5648, 'num_object_clones': 0, 'num_object_copies': 11296, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5648, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15000, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1664}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:31.755 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:32:31.755 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'15264", 'reported_seq': 15481, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:27.748977+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:32:27.748977+0000', 'last_peered': '2024-09-17T15:32:27.748977+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:32:27.748977+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "41'5200", 'ondisk_log_start': "41'5200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23220248, 'num_objects': 5670, 'num_object_clones': 0, 'num_object_copies': 11340, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5670, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13421, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1545}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:32:31.755 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:32:31.755 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'15505", 'reported_seq': 15801, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:27.738603+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:32:27.738603+0000', 'last_peered': '2024-09-17T15:32:27.738603+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:32:27.738603+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "41'5500", 'ondisk_log_start': "41'5500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23536099, 'num_objects': 5747, 'num_object_clones': 0, 'num_object_copies': 11494, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5747, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12004, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1555}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:31.755 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:32:31.755 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'15147", 'reported_seq': 15461, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:27.725852+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:32:27.725852+0000', 'last_peered': '2024-09-17T15:32:27.725852+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:32:27.725852+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "41'5100", 'ondisk_log_start': "41'5100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23024582, 'num_objects': 5623, 'num_object_clones': 0, 'num_object_copies': 11246, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4089, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5623, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 11916, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 85, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1534}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:31.755 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:32:31.755 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'14871", 'reported_seq': 15140, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:27.739874+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:32:27.739874+0000', 'last_peered': '2024-09-17T15:32:27.739874+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:32:27.739874+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "41'4800", 'ondisk_log_start': "41'4800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23228416, 'num_objects': 5671, 'num_object_clones': 0, 'num_object_copies': 11342, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5671, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11467, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1728}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:31.755 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:32:31.756 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'14922", 'reported_seq': 15136, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:27.814253+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:32:27.814253+0000', 'last_peered': '2024-09-17T15:32:27.814253+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:32:27.814253+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "41'4900", 'ondisk_log_start': "41'4900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22925795, 'num_objects': 5598, 'num_object_clones': 0, 'num_object_copies': 11196, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4034, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5598, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12817, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 1565}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:32:31.756 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:32:31.756 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'15084", 'reported_seq': 20623, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:27.841624+0000', 'last_change': '2024-09-17T15:30:55.086473+0000', 'last_active': '2024-09-17T15:32:27.841624+0000', 'last_peered': '2024-09-17T15:32:27.841624+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:32:27.841624+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "41'5000", 'ondisk_log_start': "41'5000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23150592, 'num_objects': 5652, 'num_object_clones': 0, 'num_object_copies': 11304, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1432, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5652, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12846, 'num_write_kb': 32532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2707, 'num_bytes_recovered': 8073216, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 4220}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:32:31.756 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:32:31.756 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'15251", 'reported_seq': 15551, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:27.760369+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:32:27.760369+0000', 'last_peered': '2024-09-17T15:32:27.760369+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:32:27.760369+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "41'5200", 'ondisk_log_start': "41'5200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23408640, 'num_objects': 5715, 'num_object_clones': 0, 'num_object_copies': 11430, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5715, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11799, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1658}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:31.756 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:32:31.756 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'15107", 'reported_seq': 15378, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:27.708814+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:32:27.708814+0000', 'last_peered': '2024-09-17T15:32:27.708814+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:32:27.708814+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "41'5100", 'ondisk_log_start': "41'5100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23203840, 'num_objects': 5665, 'num_object_clones': 0, 'num_object_copies': 11330, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4098, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5665, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11394, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 62, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1570}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:31.756 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:32:31.756 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'15132", 'reported_seq': 15435, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:27.663884+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:32:27.663884+0000', 'last_peered': '2024-09-17T15:32:27.663884+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:32:27.663884+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "41'5100", 'ondisk_log_start': "41'5100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23568384, 'num_objects': 5754, 'num_object_clones': 0, 'num_object_copies': 11508, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5754, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11919, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1774}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:31.757 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:32:32.435 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:32:32.435 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37941/182350 objects degraded (20.807%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":489,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":91175,"data_bytes":373404425,"bytes_used":2034814976,"bytes_avail":287875477504,"bytes_total":289910292480,"degraded_objects":37941,"degraded_total":182350,"degraded_ratio":0.20806690430490815,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":283,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":401},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-17T15:31:29.695933+0000","services":{}},"progress_events":{}} 2024-09-17T15:32:32.463 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:32:32.463 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:32:33.011 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:32:33.012 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:32:33.043 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":282,"stamp":"2024-09-17T15:32:31.723271+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":373404425,"num_objects":91175,"num_object_clones":0,"num_object_copies":182350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37941,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91175,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":201816,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11520,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160682,"ondisk_log_size":160682,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1987124,"kb_used_data":931384,"kb_used_omap":13,"kb_used_meta":1055666,"kb_avail":281128396,"statfs":{"total":289910292480,"available":287875477504,"internally_reserved":0,"allocated":953737216,"data_stored":952830939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1081003007},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,8,4,6,9,6,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19718144,"num_objects":-4814,"num_object_clones":0,"num_object_copies":-9628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4814,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4814,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":283,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004169"},"pg_stats":[{"pgid":"2.f","version":"45'15000","reported_seq":15255,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:27.790474+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:32:27.790474+0000","last_peered":"2024-09-17T15:32:27.790474+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:32:27.790474+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'4900","ondisk_log_start":"41'4900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23134208,"num_objects":5648,"num_object_clones":0,"num_object_copies":11296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15000,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1664}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'15116","reported_seq":15224,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:27.775836+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:32:27.775836+0000","last_peered":"2024-09-17T15:32:27.775836+0000","last_clean":"2024-09-17T15:32:27.775836+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:32:27.775836+0000","last_undegraded":"2024-09-17T15:32:27.775836+0000","last_fullsized":"2024-09-17T15:32:27.775836+0000","mapping_epoch":33,"log_start":"41'5100","ondisk_log_start":"41'5100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23823302,"num_objects":5818,"num_object_clones":0,"num_object_copies":11636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5818,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13473,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'15264","reported_seq":15481,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:27.748977+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:32:27.748977+0000","last_peered":"2024-09-17T15:32:27.748977+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:32:27.748977+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'5200","ondisk_log_start":"41'5200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23220248,"num_objects":5670,"num_object_clones":0,"num_object_copies":11340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5670,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13421,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1545}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'15196","reported_seq":15270,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:27.841898+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:32:27.841898+0000","last_peered":"2024-09-17T15:32:27.841898+0000","last_clean":"2024-09-17T15:32:27.841898+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:32:27.841898+0000","last_undegraded":"2024-09-17T15:32:27.841898+0000","last_fullsized":"2024-09-17T15:32:27.841898+0000","mapping_epoch":33,"log_start":"41'5100","ondisk_log_start":"41'5100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23650304,"num_objects":5774,"num_object_clones":0,"num_object_copies":11548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15196,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14907","reported_seq":15041,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:27.763155+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:32:27.763155+0000","last_peered":"2024-09-17T15:32:27.763155+0000","last_clean":"2024-09-17T15:32:27.763155+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:32:27.763155+0000","last_undegraded":"2024-09-17T15:32:27.763155+0000","last_fullsized":"2024-09-17T15:32:27.763155+0000","mapping_epoch":32,"log_start":"41'4900","ondisk_log_start":"41'4900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23536099,"num_objects":5747,"num_object_clones":0,"num_object_copies":11494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5747,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11889,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'15505","reported_seq":15801,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:27.738603+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:32:27.738603+0000","last_peered":"2024-09-17T15:32:27.738603+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:32:27.738603+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'5500","ondisk_log_start":"41'5500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23536099,"num_objects":5747,"num_object_clones":0,"num_object_copies":11494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5747,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12004,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1555},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'15147","reported_seq":15461,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:27.725852+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:32:27.725852+0000","last_peered":"2024-09-17T15:32:27.725852+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:32:27.725852+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'5100","ondisk_log_start":"41'5100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23024582,"num_objects":5623,"num_object_clones":0,"num_object_copies":11246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5623,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11916,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1534},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'15207","reported_seq":15340,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:27.710327+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:32:27.710327+0000","last_peered":"2024-09-17T15:32:27.710327+0000","last_clean":"2024-09-17T15:32:27.710327+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:32:27.710327+0000","last_undegraded":"2024-09-17T15:32:27.710327+0000","last_fullsized":"2024-09-17T15:32:27.710327+0000","mapping_epoch":32,"log_start":"41'5200","ondisk_log_start":"41'5200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23384064,"num_objects":5709,"num_object_clones":0,"num_object_copies":11418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12161,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087024+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.087024+0000","last_peered":"2024-09-17T15:30:57.087024+0000","last_clean":"2024-09-17T15:30:57.087024+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.087024+0000","last_undegraded":"2024-09-17T15:30:57.087024+0000","last_fullsized":"2024-09-17T15:30:57.087024+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'15060","reported_seq":23347,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:27.014098+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:32:27.014098+0000","last_peered":"2024-09-17T15:32:27.014098+0000","last_clean":"2024-09-17T15:32:27.014098+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:32:27.014098+0000","last_undegraded":"2024-09-17T15:32:27.014098+0000","last_fullsized":"2024-09-17T15:32:27.014098+0000","mapping_epoch":42,"log_start":"41'5000","ondisk_log_start":"41'5000","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23581155,"num_objects":5758,"num_object_clones":0,"num_object_copies":11516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5758,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13060,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14871","reported_seq":15140,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:27.739874+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:32:27.739874+0000","last_peered":"2024-09-17T15:32:27.739874+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:32:27.739874+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'4800","ondisk_log_start":"41'4800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23228416,"num_objects":5671,"num_object_clones":0,"num_object_copies":11342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11467,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1728}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14922","reported_seq":15136,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:27.814253+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:32:27.814253+0000","last_peered":"2024-09-17T15:32:27.814253+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:32:27.814253+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'4900","ondisk_log_start":"41'4900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22925795,"num_objects":5598,"num_object_clones":0,"num_object_copies":11196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5598,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12817,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1565}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'15084","reported_seq":20623,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:27.841624+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:32:27.841624+0000","last_peered":"2024-09-17T15:32:27.841624+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:32:27.841624+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"41'5000","ondisk_log_start":"41'5000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23150592,"num_objects":5652,"num_object_clones":0,"num_object_copies":11304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1432,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12846,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2707,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4220}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'15251","reported_seq":15551,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:27.760369+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:32:27.760369+0000","last_peered":"2024-09-17T15:32:27.760369+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:32:27.760369+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'5200","ondisk_log_start":"41'5200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23408640,"num_objects":5715,"num_object_clones":0,"num_object_copies":11430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11799,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1658}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'15107","reported_seq":15378,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:27.708814+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:32:27.708814+0000","last_peered":"2024-09-17T15:32:27.708814+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:32:27.708814+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'5100","ondisk_log_start":"41'5100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23203840,"num_objects":5665,"num_object_clones":0,"num_object_copies":11330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11394,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1570}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'15132","reported_seq":15435,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:27.663884+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:32:27.663884+0000","last_peered":"2024-09-17T15:32:27.663884+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:32:27.663884+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'5100","ondisk_log_start":"41'5100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23568384,"num_objects":5754,"num_object_clones":0,"num_object_copies":11508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11919,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1774}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'15010","reported_seq":23389,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:27.177515+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:32:27.177515+0000","last_peered":"2024-09-17T15:32:27.177515+0000","last_clean":"2024-09-17T15:32:27.177515+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:32:27.177515+0000","last_undegraded":"2024-09-17T15:32:27.177515+0000","last_fullsized":"2024-09-17T15:32:27.177515+0000","mapping_epoch":44,"log_start":"41'5000","ondisk_log_start":"41'5000","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23028678,"num_objects":5624,"num_object_clones":0,"num_object_copies":11248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11452,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":373404406,"num_objects":91173,"num_object_clones":0,"num_object_copies":182346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37941,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91173,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":201814,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11519,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":950030336,"data_stored":949884898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160679,"ondisk_log_size":160679,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051673,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":631664,"kb_used_data":315356,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93740176,"statfs":{"total":96636764160,"available":95989940224,"internally_reserved":0,"allocated":322924544,"data_stored":322633447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":323874476},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,0,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724618,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":617476,"kb_used_data":274804,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93754364,"statfs":{"total":96636764160,"available":96004468736,"internally_reserved":0,"allocated":281399296,"data_stored":281075218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":350875307},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,3,4,0,4,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247901,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":737984,"kb_used_data":341224,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93633856,"statfs":{"total":96636764160,"available":95881068544,"internally_reserved":0,"allocated":349413376,"data_stored":349122274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":406253224},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,8,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":347881472,"data_stored":347845342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279834624,"data_stored":279765518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":322314240,"data_stored":322274038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:32:36.045 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:32:36.572 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:32:36.575 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:32:36.605 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":284,"stamp":"2024-09-17T15:32:35.724816+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":363463433,"num_objects":88748,"num_object_clones":0,"num_object_copies":177496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37783,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88748,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":204243,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11678,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160809,"ondisk_log_size":160809,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1966636,"kb_used_data":910904,"kb_used_omap":13,"kb_used_meta":1055666,"kb_avail":281148884,"statfs":{"total":289910292480,"available":287896457216,"internally_reserved":0,"allocated":932765696,"data_stored":931859419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1081003007},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,2,5,6,11,12,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19812352,"num_objects":-4837,"num_object_clones":0,"num_object_copies":-9674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-295,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4837,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":295,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004648"},"pg_stats":[{"pgid":"2.f","version":"45'15139","reported_seq":15394,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:32.871175+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:32:32.871175+0000","last_peered":"2024-09-17T15:32:32.871175+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:32:32.871175+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'5100","ondisk_log_start":"41'5100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22564864,"num_objects":5509,"num_object_clones":0,"num_object_copies":11018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15139,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1525}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'15266","reported_seq":15374,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:32.838164+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:32:32.838164+0000","last_peered":"2024-09-17T15:32:32.838164+0000","last_clean":"2024-09-17T15:32:32.838164+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:32:32.838164+0000","last_undegraded":"2024-09-17T15:32:32.838164+0000","last_fullsized":"2024-09-17T15:32:32.838164+0000","mapping_epoch":33,"log_start":"41'5200","ondisk_log_start":"41'5200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23208902,"num_objects":5668,"num_object_clones":0,"num_object_copies":11336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5668,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13623,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'15417","reported_seq":15634,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:32.550005+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:32:32.550005+0000","last_peered":"2024-09-17T15:32:32.550005+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:32:32.550005+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'5400","ondisk_log_start":"41'5400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22593560,"num_objects":5517,"num_object_clones":0,"num_object_copies":11034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5517,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13574,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1392}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'15336","reported_seq":15411,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:32.856492+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:32:32.856492+0000","last_peered":"2024-09-17T15:32:32.856492+0000","last_clean":"2024-09-17T15:32:32.856492+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:32:32.856492+0000","last_undegraded":"2024-09-17T15:32:32.856492+0000","last_fullsized":"2024-09-17T15:32:32.856492+0000","mapping_epoch":33,"log_start":"41'5300","ondisk_log_start":"41'5300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23076864,"num_objects":5634,"num_object_clones":0,"num_object_copies":11268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15336,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'15062","reported_seq":15196,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:32.845178+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:32:32.845178+0000","last_peered":"2024-09-17T15:32:32.845178+0000","last_clean":"2024-09-17T15:32:32.845178+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:32:32.845178+0000","last_undegraded":"2024-09-17T15:32:32.845178+0000","last_fullsized":"2024-09-17T15:32:32.845178+0000","mapping_epoch":32,"log_start":"41'5000","ondisk_log_start":"41'5000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22901219,"num_objects":5592,"num_object_clones":0,"num_object_copies":11184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5592,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12044,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'15670","reported_seq":15966,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:32.855807+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:32:32.855807+0000","last_peered":"2024-09-17T15:32:32.855807+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:32:32.855807+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'5600","ondisk_log_start":"41'5600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22860259,"num_objects":5582,"num_object_clones":0,"num_object_copies":11164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5582,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12169,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1390},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'15334","reported_seq":15648,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:32.826663+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:32:32.826663+0000","last_peered":"2024-09-17T15:32:32.826663+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:32:32.826663+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'5300","ondisk_log_start":"41'5300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22258630,"num_objects":5436,"num_object_clones":0,"num_object_copies":10872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5436,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12103,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1347},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'15361","reported_seq":15494,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:32.839229+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:32:32.839229+0000","last_peered":"2024-09-17T15:32:32.839229+0000","last_clean":"2024-09-17T15:32:32.839229+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:32:32.839229+0000","last_undegraded":"2024-09-17T15:32:32.839229+0000","last_fullsized":"2024-09-17T15:32:32.839229+0000","mapping_epoch":32,"log_start":"41'5300","ondisk_log_start":"41'5300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22753280,"num_objects":5555,"num_object_clones":0,"num_object_copies":11110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12315,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087024+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.087024+0000","last_peered":"2024-09-17T15:30:57.087024+0000","last_clean":"2024-09-17T15:30:57.087024+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.087024+0000","last_undegraded":"2024-09-17T15:30:57.087024+0000","last_fullsized":"2024-09-17T15:30:57.087024+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'15221","reported_seq":23508,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:32.248110+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:32:32.248110+0000","last_peered":"2024-09-17T15:32:32.248110+0000","last_clean":"2024-09-17T15:32:32.248110+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:32:32.248110+0000","last_undegraded":"2024-09-17T15:32:32.248110+0000","last_fullsized":"2024-09-17T15:32:32.248110+0000","mapping_epoch":42,"log_start":"41'5200","ondisk_log_start":"41'5200","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22921699,"num_objects":5597,"num_object_clones":0,"num_object_copies":11194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5597,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13221,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'15023","reported_seq":15293,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:32.818882+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:32:32.818882+0000","last_peered":"2024-09-17T15:32:32.818882+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:32:32.818882+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'5000","ondisk_log_start":"41'5000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22605824,"num_objects":5519,"num_object_clones":0,"num_object_copies":11038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11619,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1576}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'15056","reported_seq":15270,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:32.796566+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:32:32.796566+0000","last_peered":"2024-09-17T15:32:32.796566+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:32:32.796566+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'5000","ondisk_log_start":"41'5000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22376931,"num_objects":5464,"num_object_clones":0,"num_object_copies":10928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5464,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12951,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1431}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'15218","reported_seq":21072,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:32.854006+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:32:32.854006+0000","last_peered":"2024-09-17T15:32:32.854006+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:32:32.854006+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"41'5200","ondisk_log_start":"41'5200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22601728,"num_objects":5518,"num_object_clones":0,"num_object_copies":11036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1274,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12980,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2865,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4244}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'15398","reported_seq":15700,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:32.872140+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:32:32.872140+0000","last_peered":"2024-09-17T15:32:32.872140+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:32:32.872140+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'5300","ondisk_log_start":"41'5300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22806528,"num_objects":5568,"num_object_clones":0,"num_object_copies":11136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11946,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1511}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'15252","reported_seq":15521,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:32.892941+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:32:32.892941+0000","last_peered":"2024-09-17T15:32:32.892941+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:32:32.892941+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'5200","ondisk_log_start":"41'5200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22609920,"num_objects":5520,"num_object_clones":0,"num_object_copies":11040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11539,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1425}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'15297","reported_seq":15600,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:32.867154+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:32:32.867154+0000","last_peered":"2024-09-17T15:32:32.867154+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:32:32.867154+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'5200","ondisk_log_start":"41'5200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22892544,"num_objects":5589,"num_object_clones":0,"num_object_copies":11178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5589,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12084,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1609}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'15156","reported_seq":23535,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:32.204393+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:32:32.204393+0000","last_peered":"2024-09-17T15:32:32.204393+0000","last_clean":"2024-09-17T15:32:32.204393+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:32:32.204393+0000","last_undegraded":"2024-09-17T15:32:32.204393+0000","last_fullsized":"2024-09-17T15:32:32.204393+0000","mapping_epoch":44,"log_start":"41'5100","ondisk_log_start":"41'5100","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22430662,"num_objects":5478,"num_object_clones":0,"num_object_copies":10956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11598,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":363463414,"num_objects":88746,"num_object_clones":0,"num_object_copies":177492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37783,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88746,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":204241,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11677,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":930078720,"data_stored":929933282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160806,"ondisk_log_size":160806,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051674,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":631076,"kb_used_data":310612,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93740764,"statfs":{"total":96636764160,"available":95990542336,"internally_reserved":0,"allocated":318066688,"data_stored":317775591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":328134316},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724619,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":613264,"kb_used_data":266428,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93758576,"statfs":{"total":96636764160,"available":96008781824,"internally_reserved":0,"allocated":272822272,"data_stored":272498194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,4,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247902,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":722296,"kb_used_data":333864,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93649544,"statfs":{"total":96636764160,"available":95897133056,"internally_reserved":0,"allocated":341876736,"data_stored":341585634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":397733544},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,5,7,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":340267008,"data_stored":340230878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":272293888,"data_stored":272224782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":317517824,"data_stored":317477622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:32:36.607 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:32:37.179 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:32:37.180 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:32:37.206 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":284,"stamp":"2024-09-17T15:32:35.724816+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":363463433,"num_objects":88748,"num_object_clones":0,"num_object_copies":177496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37783,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88748,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":204243,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11678,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160809,"ondisk_log_size":160809,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1966636,"kb_used_data":910904,"kb_used_omap":13,"kb_used_meta":1055666,"kb_avail":281148884,"statfs":{"total":289910292480,"available":287896457216,"internally_reserved":0,"allocated":932765696,"data_stored":931859419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1081003007},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,2,5,6,11,12,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19812352,"num_objects":-4837,"num_object_clones":0,"num_object_copies":-9674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-295,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4837,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":295,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004648"},"pg_stats":[{"pgid":"2.f","version":"45'15139","reported_seq":15394,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:32.871175+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:32:32.871175+0000","last_peered":"2024-09-17T15:32:32.871175+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:32:32.871175+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'5100","ondisk_log_start":"41'5100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22564864,"num_objects":5509,"num_object_clones":0,"num_object_copies":11018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15139,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1525}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'15266","reported_seq":15374,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:32.838164+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:32:32.838164+0000","last_peered":"2024-09-17T15:32:32.838164+0000","last_clean":"2024-09-17T15:32:32.838164+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:32:32.838164+0000","last_undegraded":"2024-09-17T15:32:32.838164+0000","last_fullsized":"2024-09-17T15:32:32.838164+0000","mapping_epoch":33,"log_start":"41'5200","ondisk_log_start":"41'5200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23208902,"num_objects":5668,"num_object_clones":0,"num_object_copies":11336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5668,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13623,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'15417","reported_seq":15634,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:32.550005+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:32:32.550005+0000","last_peered":"2024-09-17T15:32:32.550005+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:32:32.550005+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'5400","ondisk_log_start":"41'5400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22593560,"num_objects":5517,"num_object_clones":0,"num_object_copies":11034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5517,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13574,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1392}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'15336","reported_seq":15411,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:32.856492+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:32:32.856492+0000","last_peered":"2024-09-17T15:32:32.856492+0000","last_clean":"2024-09-17T15:32:32.856492+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:32:32.856492+0000","last_undegraded":"2024-09-17T15:32:32.856492+0000","last_fullsized":"2024-09-17T15:32:32.856492+0000","mapping_epoch":33,"log_start":"41'5300","ondisk_log_start":"41'5300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23076864,"num_objects":5634,"num_object_clones":0,"num_object_copies":11268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15336,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'15062","reported_seq":15196,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:32.845178+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:32:32.845178+0000","last_peered":"2024-09-17T15:32:32.845178+0000","last_clean":"2024-09-17T15:32:32.845178+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:32:32.845178+0000","last_undegraded":"2024-09-17T15:32:32.845178+0000","last_fullsized":"2024-09-17T15:32:32.845178+0000","mapping_epoch":32,"log_start":"41'5000","ondisk_log_start":"41'5000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22901219,"num_objects":5592,"num_object_clones":0,"num_object_copies":11184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5592,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12044,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'15670","reported_seq":15966,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:32.855807+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:32:32.855807+0000","last_peered":"2024-09-17T15:32:32.855807+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:32:32.855807+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'5600","ondisk_log_start":"41'5600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22860259,"num_objects":5582,"num_object_clones":0,"num_object_copies":11164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5582,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12169,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1390},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'15334","reported_seq":15648,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:32.826663+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:32:32.826663+0000","last_peered":"2024-09-17T15:32:32.826663+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:32:32.826663+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'5300","ondisk_log_start":"41'5300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22258630,"num_objects":5436,"num_object_clones":0,"num_object_copies":10872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5436,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12103,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1347},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'15361","reported_seq":15494,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:32.839229+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:32:32.839229+0000","last_peered":"2024-09-17T15:32:32.839229+0000","last_clean":"2024-09-17T15:32:32.839229+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:32:32.839229+0000","last_undegraded":"2024-09-17T15:32:32.839229+0000","last_fullsized":"2024-09-17T15:32:32.839229+0000","mapping_epoch":32,"log_start":"41'5300","ondisk_log_start":"41'5300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22753280,"num_objects":5555,"num_object_clones":0,"num_object_copies":11110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12315,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087024+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.087024+0000","last_peered":"2024-09-17T15:30:57.087024+0000","last_clean":"2024-09-17T15:30:57.087024+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.087024+0000","last_undegraded":"2024-09-17T15:30:57.087024+0000","last_fullsized":"2024-09-17T15:30:57.087024+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'15221","reported_seq":23508,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:32.248110+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:32:32.248110+0000","last_peered":"2024-09-17T15:32:32.248110+0000","last_clean":"2024-09-17T15:32:32.248110+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:32:32.248110+0000","last_undegraded":"2024-09-17T15:32:32.248110+0000","last_fullsized":"2024-09-17T15:32:32.248110+0000","mapping_epoch":42,"log_start":"41'5200","ondisk_log_start":"41'5200","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22921699,"num_objects":5597,"num_object_clones":0,"num_object_copies":11194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5597,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13221,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'15023","reported_seq":15293,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:32.818882+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:32:32.818882+0000","last_peered":"2024-09-17T15:32:32.818882+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:32:32.818882+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'5000","ondisk_log_start":"41'5000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22605824,"num_objects":5519,"num_object_clones":0,"num_object_copies":11038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11619,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1576}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'15056","reported_seq":15270,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:32.796566+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:32:32.796566+0000","last_peered":"2024-09-17T15:32:32.796566+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:32:32.796566+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'5000","ondisk_log_start":"41'5000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22376931,"num_objects":5464,"num_object_clones":0,"num_object_copies":10928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5464,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12951,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1431}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":82,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087540+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087540+0000","last_peered":"2024-09-17T15:30:57.087540+0000","last_clean":"2024-09-17T15:30:57.087540+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087540+0000","last_undegraded":"2024-09-17T15:30:57.087540+0000","last_fullsized":"2024-09-17T15:30:57.087540+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'15218","reported_seq":21072,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:32.854006+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:32:32.854006+0000","last_peered":"2024-09-17T15:32:32.854006+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:32:32.854006+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"41'5200","ondisk_log_start":"41'5200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22601728,"num_objects":5518,"num_object_clones":0,"num_object_copies":11036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1274,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12980,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2865,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4244}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'15398","reported_seq":15700,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:32.872140+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:32:32.872140+0000","last_peered":"2024-09-17T15:32:32.872140+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:32:32.872140+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'5300","ondisk_log_start":"41'5300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22806528,"num_objects":5568,"num_object_clones":0,"num_object_copies":11136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11946,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1511}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087729+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087729+0000","last_peered":"2024-09-17T15:30:57.087729+0000","last_clean":"2024-09-17T15:30:57.087729+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087729+0000","last_undegraded":"2024-09-17T15:30:57.087729+0000","last_fullsized":"2024-09-17T15:30:57.087729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'15252","reported_seq":15521,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:32.892941+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:32:32.892941+0000","last_peered":"2024-09-17T15:32:32.892941+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:32:32.892941+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'5200","ondisk_log_start":"41'5200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22609920,"num_objects":5520,"num_object_clones":0,"num_object_copies":11040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11539,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1425}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'15297","reported_seq":15600,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:32.867154+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:32:32.867154+0000","last_peered":"2024-09-17T15:32:32.867154+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:32:32.867154+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'5200","ondisk_log_start":"41'5200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22892544,"num_objects":5589,"num_object_clones":0,"num_object_copies":11178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5589,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12084,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1609}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'15156","reported_seq":23535,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:32.204393+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:32:32.204393+0000","last_peered":"2024-09-17T15:32:32.204393+0000","last_clean":"2024-09-17T15:32:32.204393+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:32:32.204393+0000","last_undegraded":"2024-09-17T15:32:32.204393+0000","last_fullsized":"2024-09-17T15:32:32.204393+0000","mapping_epoch":44,"log_start":"41'5100","ondisk_log_start":"41'5100","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22430662,"num_objects":5478,"num_object_clones":0,"num_object_copies":10956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11598,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":363463414,"num_objects":88746,"num_object_clones":0,"num_object_copies":177492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37783,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88746,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":204241,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11677,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":930078720,"data_stored":929933282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160806,"ondisk_log_size":160806,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051674,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":631076,"kb_used_data":310612,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93740764,"statfs":{"total":96636764160,"available":95990542336,"internally_reserved":0,"allocated":318066688,"data_stored":317775591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":328134316},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724619,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":613264,"kb_used_data":266428,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93758576,"statfs":{"total":96636764160,"available":96008781824,"internally_reserved":0,"allocated":272822272,"data_stored":272498194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,4,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247902,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":722296,"kb_used_data":333864,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93649544,"statfs":{"total":96636764160,"available":95897133056,"internally_reserved":0,"allocated":341876736,"data_stored":341585634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":397733544},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,5,7,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":340267008,"data_stored":340230878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":272293888,"data_stored":272224782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":317517824,"data_stored":317477622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:32:37.207 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:32:37.207 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'15139", 'reported_seq': 15394, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:32.871175+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:32:32.871175+0000', 'last_peered': '2024-09-17T15:32:32.871175+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:32:32.871175+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "41'5100", 'ondisk_log_start': "41'5100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22564864, 'num_objects': 5509, 'num_object_clones': 0, 'num_object_copies': 11018, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5509, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15139, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1525}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:37.207 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:32:37.207 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'15417", 'reported_seq': 15634, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:32.550005+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:32:32.550005+0000', 'last_peered': '2024-09-17T15:32:32.550005+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:32:32.550005+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "41'5400", 'ondisk_log_start': "41'5400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22593560, 'num_objects': 5517, 'num_object_clones': 0, 'num_object_copies': 11034, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5517, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13574, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1392}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:32:37.207 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:32:37.208 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'15670", 'reported_seq': 15966, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:32.855807+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:32:32.855807+0000', 'last_peered': '2024-09-17T15:32:32.855807+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:32:32.855807+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "41'5600", 'ondisk_log_start': "41'5600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22860259, 'num_objects': 5582, 'num_object_clones': 0, 'num_object_copies': 11164, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5582, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12169, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1390}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:37.208 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:32:37.208 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'15334", 'reported_seq': 15648, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:32.826663+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:32:32.826663+0000', 'last_peered': '2024-09-17T15:32:32.826663+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:32:32.826663+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "41'5300", 'ondisk_log_start': "41'5300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22258630, 'num_objects': 5436, 'num_object_clones': 0, 'num_object_copies': 10872, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4089, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5436, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 12103, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 85, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1347}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:37.208 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:32:37.208 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'15023", 'reported_seq': 15293, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:32.818882+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:32:32.818882+0000', 'last_peered': '2024-09-17T15:32:32.818882+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:32:32.818882+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "41'5000", 'ondisk_log_start': "41'5000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22605824, 'num_objects': 5519, 'num_object_clones': 0, 'num_object_copies': 11038, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5519, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11619, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1576}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:37.208 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:32:37.208 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'15056", 'reported_seq': 15270, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:32.796566+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:32:32.796566+0000', 'last_peered': '2024-09-17T15:32:32.796566+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:32:32.796566+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "41'5000", 'ondisk_log_start': "41'5000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22376931, 'num_objects': 5464, 'num_object_clones': 0, 'num_object_copies': 10928, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4034, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5464, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12951, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 1431}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:32:37.208 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:32:37.208 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'15218", 'reported_seq': 21072, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:32.854006+0000', 'last_change': '2024-09-17T15:30:55.086473+0000', 'last_active': '2024-09-17T15:32:32.854006+0000', 'last_peered': '2024-09-17T15:32:32.854006+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:32:32.854006+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "41'5200", 'ondisk_log_start': "41'5200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22601728, 'num_objects': 5518, 'num_object_clones': 0, 'num_object_copies': 11036, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1274, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5518, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12980, 'num_write_kb': 32532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2865, 'num_bytes_recovered': 8073216, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 4244}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:32:37.208 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:32:37.208 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'15398", 'reported_seq': 15700, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:32.872140+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:32:32.872140+0000', 'last_peered': '2024-09-17T15:32:32.872140+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:32:32.872140+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "41'5300", 'ondisk_log_start': "41'5300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22806528, 'num_objects': 5568, 'num_object_clones': 0, 'num_object_copies': 11136, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5568, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11946, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1511}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:37.209 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:32:37.209 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'15252", 'reported_seq': 15521, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:32.892941+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:32:32.892941+0000', 'last_peered': '2024-09-17T15:32:32.892941+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:32:32.892941+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "41'5200", 'ondisk_log_start': "41'5200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22609920, 'num_objects': 5520, 'num_object_clones': 0, 'num_object_copies': 11040, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4098, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5520, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11539, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 62, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1425}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:37.209 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:32:37.209 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'15297", 'reported_seq': 15600, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:32.867154+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:32:32.867154+0000', 'last_peered': '2024-09-17T15:32:32.867154+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:32:32.867154+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "41'5200", 'ondisk_log_start': "41'5200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22892544, 'num_objects': 5589, 'num_object_clones': 0, 'num_object_copies': 11178, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5589, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12084, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1609}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:37.209 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:32:37.885 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:32:37.885 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37783/177496 objects degraded (21.287%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":494,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":88748,"data_bytes":363463433,"bytes_used":2013835264,"bytes_avail":287896457216,"bytes_total":289910292480,"degraded_objects":37783,"degraded_total":177496,"degraded_ratio":0.21286676882859332,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":295,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":402},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-17T15:31:29.695933+0000","services":{}},"progress_events":{}} 2024-09-17T15:32:37.909 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:32:37.909 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:32:38.428 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:32:38.429 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:32:38.453 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":285,"stamp":"2024-09-17T15:32:37.725380+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":362255113,"num_objects":88453,"num_object_clones":0,"num_object_copies":176906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37783,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88453,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":204538,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11678,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160804,"ondisk_log_size":160804,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1950992,"kb_used_data":903576,"kb_used_omap":13,"kb_used_meta":1047346,"kb_avail":281164528,"statfs":{"total":289910292480,"available":287912476672,"internally_reserved":0,"allocated":925261824,"data_stored":924355547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1072483327},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,6,8,11,17,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21020672,"num_objects":-5132,"num_object_clones":0,"num_object_copies":-10264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-295,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5132,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":295,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004340"},"pg_stats":[{"pgid":"2.f","version":"45'15139","reported_seq":15394,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:32.871175+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:32:32.871175+0000","last_peered":"2024-09-17T15:32:32.871175+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:32:32.871175+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'5100","ondisk_log_start":"41'5100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22564864,"num_objects":5509,"num_object_clones":0,"num_object_copies":11018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15139,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1525}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'15266","reported_seq":15374,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:32.838164+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:32:32.838164+0000","last_peered":"2024-09-17T15:32:32.838164+0000","last_clean":"2024-09-17T15:32:32.838164+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:32:32.838164+0000","last_undegraded":"2024-09-17T15:32:32.838164+0000","last_fullsized":"2024-09-17T15:32:32.838164+0000","mapping_epoch":33,"log_start":"41'5200","ondisk_log_start":"41'5200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23208902,"num_objects":5668,"num_object_clones":0,"num_object_copies":11336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5668,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13623,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'15417","reported_seq":15634,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:32.550005+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:32:32.550005+0000","last_peered":"2024-09-17T15:32:32.550005+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:32:32.550005+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'5400","ondisk_log_start":"41'5400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22593560,"num_objects":5517,"num_object_clones":0,"num_object_copies":11034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5517,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13574,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1392}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'15336","reported_seq":15411,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:32.856492+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:32:32.856492+0000","last_peered":"2024-09-17T15:32:32.856492+0000","last_clean":"2024-09-17T15:32:32.856492+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:32:32.856492+0000","last_undegraded":"2024-09-17T15:32:32.856492+0000","last_fullsized":"2024-09-17T15:32:32.856492+0000","mapping_epoch":33,"log_start":"41'5300","ondisk_log_start":"41'5300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23076864,"num_objects":5634,"num_object_clones":0,"num_object_copies":11268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15336,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'15062","reported_seq":15196,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:32.845178+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:32:32.845178+0000","last_peered":"2024-09-17T15:32:32.845178+0000","last_clean":"2024-09-17T15:32:32.845178+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:32:32.845178+0000","last_undegraded":"2024-09-17T15:32:32.845178+0000","last_fullsized":"2024-09-17T15:32:32.845178+0000","mapping_epoch":32,"log_start":"41'5000","ondisk_log_start":"41'5000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22901219,"num_objects":5592,"num_object_clones":0,"num_object_copies":11184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5592,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12044,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'15670","reported_seq":15966,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:32.855807+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:32:32.855807+0000","last_peered":"2024-09-17T15:32:32.855807+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:32:32.855807+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'5600","ondisk_log_start":"41'5600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22860259,"num_objects":5582,"num_object_clones":0,"num_object_copies":11164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5582,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12169,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1390},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'15334","reported_seq":15648,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:32.826663+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:32:32.826663+0000","last_peered":"2024-09-17T15:32:32.826663+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:32:32.826663+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'5300","ondisk_log_start":"41'5300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22258630,"num_objects":5436,"num_object_clones":0,"num_object_copies":10872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5436,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12103,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1347},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'15361","reported_seq":15494,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:32.839229+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:32:32.839229+0000","last_peered":"2024-09-17T15:32:32.839229+0000","last_clean":"2024-09-17T15:32:32.839229+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:32:32.839229+0000","last_undegraded":"2024-09-17T15:32:32.839229+0000","last_fullsized":"2024-09-17T15:32:32.839229+0000","mapping_epoch":32,"log_start":"41'5300","ondisk_log_start":"41'5300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22753280,"num_objects":5555,"num_object_clones":0,"num_object_copies":11110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12315,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087024+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.087024+0000","last_peered":"2024-09-17T15:30:57.087024+0000","last_clean":"2024-09-17T15:30:57.087024+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.087024+0000","last_undegraded":"2024-09-17T15:30:57.087024+0000","last_fullsized":"2024-09-17T15:30:57.087024+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'15360","reported_seq":23648,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:37.232037+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:32:37.232037+0000","last_peered":"2024-09-17T15:32:37.232037+0000","last_clean":"2024-09-17T15:32:37.232037+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:32:37.232037+0000","last_undegraded":"2024-09-17T15:32:37.232037+0000","last_fullsized":"2024-09-17T15:32:37.232037+0000","mapping_epoch":42,"log_start":"41'5300","ondisk_log_start":"41'5300","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22352355,"num_objects":5458,"num_object_clones":0,"num_object_copies":10916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5458,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13360,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'15023","reported_seq":15293,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:32.818882+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:32:32.818882+0000","last_peered":"2024-09-17T15:32:32.818882+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:32:32.818882+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'5000","ondisk_log_start":"41'5000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22605824,"num_objects":5519,"num_object_clones":0,"num_object_copies":11038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11619,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1576}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'15056","reported_seq":15270,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:32.796566+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:32:32.796566+0000","last_peered":"2024-09-17T15:32:32.796566+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:32:32.796566+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'5000","ondisk_log_start":"41'5000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22376931,"num_objects":5464,"num_object_clones":0,"num_object_copies":10928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5464,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12951,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1431}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":83,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087587+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087587+0000","last_peered":"2024-09-17T15:30:57.087587+0000","last_clean":"2024-09-17T15:30:57.087587+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087587+0000","last_undegraded":"2024-09-17T15:30:57.087587+0000","last_fullsized":"2024-09-17T15:30:57.087587+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'15218","reported_seq":21072,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:32.854006+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:32:32.854006+0000","last_peered":"2024-09-17T15:32:32.854006+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:32:32.854006+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"41'5200","ondisk_log_start":"41'5200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22601728,"num_objects":5518,"num_object_clones":0,"num_object_copies":11036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1274,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12980,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2865,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4244}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'15398","reported_seq":15700,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:32.872140+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:32:32.872140+0000","last_peered":"2024-09-17T15:32:32.872140+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:32:32.872140+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'5300","ondisk_log_start":"41'5300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22806528,"num_objects":5568,"num_object_clones":0,"num_object_copies":11136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11946,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1511}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'15252","reported_seq":15521,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:32.892941+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:32:32.892941+0000","last_peered":"2024-09-17T15:32:32.892941+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:32:32.892941+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'5200","ondisk_log_start":"41'5200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22609920,"num_objects":5520,"num_object_clones":0,"num_object_copies":11040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11539,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1425}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'15297","reported_seq":15600,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:32.867154+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:32:32.867154+0000","last_peered":"2024-09-17T15:32:32.867154+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:32:32.867154+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'5200","ondisk_log_start":"41'5200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22892544,"num_objects":5589,"num_object_clones":0,"num_object_copies":11178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5589,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12084,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1609}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'15312","reported_seq":23691,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:37.341095+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:32:37.341095+0000","last_peered":"2024-09-17T15:32:37.341095+0000","last_clean":"2024-09-17T15:32:37.341095+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:32:37.341095+0000","last_undegraded":"2024-09-17T15:32:37.341095+0000","last_fullsized":"2024-09-17T15:32:37.341095+0000","mapping_epoch":44,"log_start":"41'5300","ondisk_log_start":"41'5300","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21791686,"num_objects":5322,"num_object_clones":0,"num_object_copies":10644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11754,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":362255094,"num_objects":88451,"num_object_clones":0,"num_object_copies":176902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37783,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88451,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":204536,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11677,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":922488832,"data_stored":922343394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160801,"ondisk_log_size":160801,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051674,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":631076,"kb_used_data":310612,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93740764,"statfs":{"total":96636764160,"available":95990542336,"internally_reserved":0,"allocated":318066688,"data_stored":317775591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":328134316},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724620,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":597620,"kb_used_data":259100,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93774220,"statfs":{"total":96636764160,"available":96024801280,"internally_reserved":0,"allocated":265318400,"data_stored":264994322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":346615467},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,5,3,2,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247902,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":722296,"kb_used_data":333864,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93649544,"statfs":{"total":96636764160,"available":95897133056,"internally_reserved":0,"allocated":341876736,"data_stored":341585634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":397733544},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,5,7,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":340267008,"data_stored":340230878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":264704000,"data_stored":264634894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":317517824,"data_stored":317477622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:32:41.456 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:32:42.013 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:32:42.013 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:32:42.035 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":287,"stamp":"2024-09-17T15:32:41.730054+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":353493769,"num_objects":86314,"num_object_clones":0,"num_object_copies":172628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37625,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86314,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":206677,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11836,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160743,"ondisk_log_size":160743,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1947184,"kb_used_data":891444,"kb_used_omap":13,"kb_used_meta":1055666,"kb_avail":281168336,"statfs":{"total":289910292480,"available":287916376064,"internally_reserved":0,"allocated":912838656,"data_stored":911932379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1081003007},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,6,7,7,11,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19910656,"num_objects":-4861,"num_object_clones":0,"num_object_copies":-9722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-316,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4861,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":316,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007853"},"pg_stats":[{"pgid":"2.f","version":"45'15292","reported_seq":15547,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:37.907519+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:32:37.907519+0000","last_peered":"2024-09-17T15:32:37.907519+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:32:37.907519+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'5200","ondisk_log_start":"41'5200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21938176,"num_objects":5356,"num_object_clones":0,"num_object_copies":10712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15292,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1372}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'15419","reported_seq":15527,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:37.803492+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:32:37.803492+0000","last_peered":"2024-09-17T15:32:37.803492+0000","last_clean":"2024-09-17T15:32:37.803492+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:32:37.803492+0000","last_undegraded":"2024-09-17T15:32:37.803492+0000","last_fullsized":"2024-09-17T15:32:37.803492+0000","mapping_epoch":33,"log_start":"41'5400","ondisk_log_start":"41'5400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22582214,"num_objects":5515,"num_object_clones":0,"num_object_copies":11030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5515,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13776,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'15574","reported_seq":15792,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:37.846887+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:32:37.846887+0000","last_peered":"2024-09-17T15:32:37.846887+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:32:37.846887+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'5500","ondisk_log_start":"41'5500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21950488,"num_objects":5360,"num_object_clones":0,"num_object_copies":10720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5360,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13731,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1235}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'15488","reported_seq":15563,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:37.857697+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:32:37.857697+0000","last_peered":"2024-09-17T15:32:37.857697+0000","last_clean":"2024-09-17T15:32:37.857697+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:32:37.857697+0000","last_undegraded":"2024-09-17T15:32:37.857697+0000","last_fullsized":"2024-09-17T15:32:37.857697+0000","mapping_epoch":33,"log_start":"41'5400","ondisk_log_start":"41'5400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22454272,"num_objects":5482,"num_object_clones":0,"num_object_copies":10964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15488,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'15207","reported_seq":15341,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:37.916136+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:32:37.916136+0000","last_peered":"2024-09-17T15:32:37.916136+0000","last_clean":"2024-09-17T15:32:37.916136+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:32:37.916136+0000","last_undegraded":"2024-09-17T15:32:37.916136+0000","last_fullsized":"2024-09-17T15:32:37.916136+0000","mapping_epoch":32,"log_start":"41'5200","ondisk_log_start":"41'5200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22307299,"num_objects":5447,"num_object_clones":0,"num_object_copies":10894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5447,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12189,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'15827","reported_seq":16119,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:37.900580+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:32:37.900580+0000","last_peered":"2024-09-17T15:32:37.900580+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:32:37.900580+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'5800","ondisk_log_start":"41'5800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22217187,"num_objects":5425,"num_object_clones":0,"num_object_copies":10850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5425,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12326,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1233},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'15486","reported_seq":15800,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:37.834494+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:32:37.834494+0000","last_peered":"2024-09-17T15:32:37.834494+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:32:37.834494+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'5400","ondisk_log_start":"41'5400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21636038,"num_objects":5284,"num_object_clones":0,"num_object_copies":10568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5284,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12255,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1195},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'15505","reported_seq":15638,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:37.868234+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:32:37.868234+0000","last_peered":"2024-09-17T15:32:37.868234+0000","last_clean":"2024-09-17T15:32:37.868234+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:32:37.868234+0000","last_undegraded":"2024-09-17T15:32:37.868234+0000","last_fullsized":"2024-09-17T15:32:37.868234+0000","mapping_epoch":32,"log_start":"41'5500","ondisk_log_start":"41'5500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22163456,"num_objects":5411,"num_object_clones":0,"num_object_copies":10822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12459,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087024+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.087024+0000","last_peered":"2024-09-17T15:30:57.087024+0000","last_clean":"2024-09-17T15:30:57.087024+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.087024+0000","last_undegraded":"2024-09-17T15:30:57.087024+0000","last_fullsized":"2024-09-17T15:30:57.087024+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'15360","reported_seq":23648,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:37.232037+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:32:37.232037+0000","last_peered":"2024-09-17T15:32:37.232037+0000","last_clean":"2024-09-17T15:32:37.232037+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:32:37.232037+0000","last_undegraded":"2024-09-17T15:32:37.232037+0000","last_fullsized":"2024-09-17T15:32:37.232037+0000","mapping_epoch":42,"log_start":"41'5300","ondisk_log_start":"41'5300","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22352355,"num_objects":5458,"num_object_clones":0,"num_object_copies":10916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5458,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13360,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'15152","reported_seq":15421,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:37.895244+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:32:37.895244+0000","last_peered":"2024-09-17T15:32:37.895244+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:32:37.895244+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'5100","ondisk_log_start":"41'5100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22077440,"num_objects":5390,"num_object_clones":0,"num_object_copies":10780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11748,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1447}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'15212","reported_seq":15426,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:37.795874+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:32:37.795874+0000","last_peered":"2024-09-17T15:32:37.795874+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:32:37.795874+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'5200","ondisk_log_start":"41'5200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21737955,"num_objects":5308,"num_object_clones":0,"num_object_copies":10616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5308,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13107,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1275}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":83,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087587+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087587+0000","last_peered":"2024-09-17T15:30:57.087587+0000","last_clean":"2024-09-17T15:30:57.087587+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087587+0000","last_undegraded":"2024-09-17T15:30:57.087587+0000","last_fullsized":"2024-09-17T15:30:57.087587+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'15366","reported_seq":21537,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:37.856617+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:32:37.856617+0000","last_peered":"2024-09-17T15:32:37.856617+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:32:37.856617+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"41'5300","ondisk_log_start":"41'5300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21995520,"num_objects":5370,"num_object_clones":0,"num_object_copies":10740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13128,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3023,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4254}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'15581","reported_seq":15883,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:37.907541+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:32:37.907541+0000","last_peered":"2024-09-17T15:32:37.907541+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:32:37.907541+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'5500","ondisk_log_start":"41'5500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22056960,"num_objects":5385,"num_object_clones":0,"num_object_copies":10770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12129,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1328}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'15408","reported_seq":15678,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:37.856504+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:32:37.856504+0000","last_peered":"2024-09-17T15:32:37.856504+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:32:37.856504+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'5400","ondisk_log_start":"41'5400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21970944,"num_objects":5364,"num_object_clones":0,"num_object_copies":10728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11695,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1269}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'15451","reported_seq":15754,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:37.834300+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:32:37.834300+0000","last_peered":"2024-09-17T15:32:37.834300+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:32:37.834300+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'5400","ondisk_log_start":"41'5400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22261760,"num_objects":5435,"num_object_clones":0,"num_object_copies":10870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12238,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1455}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'15312","reported_seq":23691,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:37.341095+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:32:37.341095+0000","last_peered":"2024-09-17T15:32:37.341095+0000","last_clean":"2024-09-17T15:32:37.341095+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:32:37.341095+0000","last_undegraded":"2024-09-17T15:32:37.341095+0000","last_fullsized":"2024-09-17T15:32:37.341095+0000","mapping_epoch":44,"log_start":"41'5300","ondisk_log_start":"41'5300","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21791686,"num_objects":5322,"num_object_clones":0,"num_object_copies":10644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11754,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":353493750,"num_objects":86312,"num_object_clones":0,"num_object_copies":172624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37625,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86312,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":206675,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11835,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":910102528,"data_stored":909957090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160740,"ondisk_log_size":160740,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051675,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":630456,"kb_used_data":305832,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93741384,"statfs":{"total":96636764160,"available":95991177216,"internally_reserved":0,"allocated":313171968,"data_stored":312880871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":332394156},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724620,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":597620,"kb_used_data":259100,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93774220,"statfs":{"total":96636764160,"available":96024801280,"internally_reserved":0,"allocated":265318400,"data_stored":264994322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":346615467},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,5,3,2,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247903,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":719108,"kb_used_data":326512,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93652732,"statfs":{"total":96636764160,"available":95900397568,"internally_reserved":0,"allocated":334348288,"data_stored":334057186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":401993384},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,1,0,4,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":332742656,"data_stored":332706526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":264704000,"data_stored":264634894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":312655872,"data_stored":312615670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:32:42.036 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:32:42.587 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:32:42.587 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:32:42.611 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":287,"stamp":"2024-09-17T15:32:41.730054+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":353493769,"num_objects":86314,"num_object_clones":0,"num_object_copies":172628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37625,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86314,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":206677,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11836,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160743,"ondisk_log_size":160743,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1947184,"kb_used_data":891444,"kb_used_omap":13,"kb_used_meta":1055666,"kb_avail":281168336,"statfs":{"total":289910292480,"available":287916376064,"internally_reserved":0,"allocated":912838656,"data_stored":911932379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1081003007},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,6,7,7,11,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19910656,"num_objects":-4861,"num_object_clones":0,"num_object_copies":-9722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-316,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4861,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":316,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007853"},"pg_stats":[{"pgid":"2.f","version":"45'15292","reported_seq":15547,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:37.907519+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:32:37.907519+0000","last_peered":"2024-09-17T15:32:37.907519+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:32:37.907519+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'5200","ondisk_log_start":"41'5200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21938176,"num_objects":5356,"num_object_clones":0,"num_object_copies":10712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15292,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1372}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'15419","reported_seq":15527,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:37.803492+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:32:37.803492+0000","last_peered":"2024-09-17T15:32:37.803492+0000","last_clean":"2024-09-17T15:32:37.803492+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:32:37.803492+0000","last_undegraded":"2024-09-17T15:32:37.803492+0000","last_fullsized":"2024-09-17T15:32:37.803492+0000","mapping_epoch":33,"log_start":"41'5400","ondisk_log_start":"41'5400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22582214,"num_objects":5515,"num_object_clones":0,"num_object_copies":11030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5515,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13776,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'15574","reported_seq":15792,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:37.846887+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:32:37.846887+0000","last_peered":"2024-09-17T15:32:37.846887+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:32:37.846887+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'5500","ondisk_log_start":"41'5500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21950488,"num_objects":5360,"num_object_clones":0,"num_object_copies":10720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5360,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13731,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1235}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'15488","reported_seq":15563,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:37.857697+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:32:37.857697+0000","last_peered":"2024-09-17T15:32:37.857697+0000","last_clean":"2024-09-17T15:32:37.857697+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:32:37.857697+0000","last_undegraded":"2024-09-17T15:32:37.857697+0000","last_fullsized":"2024-09-17T15:32:37.857697+0000","mapping_epoch":33,"log_start":"41'5400","ondisk_log_start":"41'5400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22454272,"num_objects":5482,"num_object_clones":0,"num_object_copies":10964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15488,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'15207","reported_seq":15341,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:37.916136+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:32:37.916136+0000","last_peered":"2024-09-17T15:32:37.916136+0000","last_clean":"2024-09-17T15:32:37.916136+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:32:37.916136+0000","last_undegraded":"2024-09-17T15:32:37.916136+0000","last_fullsized":"2024-09-17T15:32:37.916136+0000","mapping_epoch":32,"log_start":"41'5200","ondisk_log_start":"41'5200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22307299,"num_objects":5447,"num_object_clones":0,"num_object_copies":10894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5447,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12189,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'15827","reported_seq":16119,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:37.900580+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:32:37.900580+0000","last_peered":"2024-09-17T15:32:37.900580+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:32:37.900580+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'5800","ondisk_log_start":"41'5800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22217187,"num_objects":5425,"num_object_clones":0,"num_object_copies":10850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5425,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12326,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1233},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'15486","reported_seq":15800,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:37.834494+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:32:37.834494+0000","last_peered":"2024-09-17T15:32:37.834494+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:32:37.834494+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'5400","ondisk_log_start":"41'5400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21636038,"num_objects":5284,"num_object_clones":0,"num_object_copies":10568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5284,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12255,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1195},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'15505","reported_seq":15638,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:37.868234+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:32:37.868234+0000","last_peered":"2024-09-17T15:32:37.868234+0000","last_clean":"2024-09-17T15:32:37.868234+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:32:37.868234+0000","last_undegraded":"2024-09-17T15:32:37.868234+0000","last_fullsized":"2024-09-17T15:32:37.868234+0000","mapping_epoch":32,"log_start":"41'5500","ondisk_log_start":"41'5500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22163456,"num_objects":5411,"num_object_clones":0,"num_object_copies":10822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12459,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087024+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.087024+0000","last_peered":"2024-09-17T15:30:57.087024+0000","last_clean":"2024-09-17T15:30:57.087024+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.087024+0000","last_undegraded":"2024-09-17T15:30:57.087024+0000","last_fullsized":"2024-09-17T15:30:57.087024+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'15360","reported_seq":23648,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:37.232037+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:32:37.232037+0000","last_peered":"2024-09-17T15:32:37.232037+0000","last_clean":"2024-09-17T15:32:37.232037+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:32:37.232037+0000","last_undegraded":"2024-09-17T15:32:37.232037+0000","last_fullsized":"2024-09-17T15:32:37.232037+0000","mapping_epoch":42,"log_start":"41'5300","ondisk_log_start":"41'5300","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22352355,"num_objects":5458,"num_object_clones":0,"num_object_copies":10916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5458,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13360,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'15152","reported_seq":15421,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:37.895244+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:32:37.895244+0000","last_peered":"2024-09-17T15:32:37.895244+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:32:37.895244+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'5100","ondisk_log_start":"41'5100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22077440,"num_objects":5390,"num_object_clones":0,"num_object_copies":10780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11748,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1447}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'15212","reported_seq":15426,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:37.795874+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:32:37.795874+0000","last_peered":"2024-09-17T15:32:37.795874+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:32:37.795874+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'5200","ondisk_log_start":"41'5200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21737955,"num_objects":5308,"num_object_clones":0,"num_object_copies":10616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5308,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13107,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1275}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":83,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087587+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087587+0000","last_peered":"2024-09-17T15:30:57.087587+0000","last_clean":"2024-09-17T15:30:57.087587+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087587+0000","last_undegraded":"2024-09-17T15:30:57.087587+0000","last_fullsized":"2024-09-17T15:30:57.087587+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'15366","reported_seq":21537,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:37.856617+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:32:37.856617+0000","last_peered":"2024-09-17T15:32:37.856617+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:32:37.856617+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"41'5300","ondisk_log_start":"41'5300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21995520,"num_objects":5370,"num_object_clones":0,"num_object_copies":10740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13128,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3023,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4254}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'15581","reported_seq":15883,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:37.907541+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:32:37.907541+0000","last_peered":"2024-09-17T15:32:37.907541+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:32:37.907541+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'5500","ondisk_log_start":"41'5500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22056960,"num_objects":5385,"num_object_clones":0,"num_object_copies":10770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12129,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1328}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'15408","reported_seq":15678,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:37.856504+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:32:37.856504+0000","last_peered":"2024-09-17T15:32:37.856504+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:32:37.856504+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'5400","ondisk_log_start":"41'5400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21970944,"num_objects":5364,"num_object_clones":0,"num_object_copies":10728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11695,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1269}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'15451","reported_seq":15754,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:37.834300+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:32:37.834300+0000","last_peered":"2024-09-17T15:32:37.834300+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:32:37.834300+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'5400","ondisk_log_start":"41'5400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22261760,"num_objects":5435,"num_object_clones":0,"num_object_copies":10870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12238,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1455}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'15312","reported_seq":23691,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:37.341095+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:32:37.341095+0000","last_peered":"2024-09-17T15:32:37.341095+0000","last_clean":"2024-09-17T15:32:37.341095+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:32:37.341095+0000","last_undegraded":"2024-09-17T15:32:37.341095+0000","last_fullsized":"2024-09-17T15:32:37.341095+0000","mapping_epoch":44,"log_start":"41'5300","ondisk_log_start":"41'5300","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21791686,"num_objects":5322,"num_object_clones":0,"num_object_copies":10644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11754,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":353493750,"num_objects":86312,"num_object_clones":0,"num_object_copies":172624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37625,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86312,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":206675,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11835,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":910102528,"data_stored":909957090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160740,"ondisk_log_size":160740,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051675,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":630456,"kb_used_data":305832,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93741384,"statfs":{"total":96636764160,"available":95991177216,"internally_reserved":0,"allocated":313171968,"data_stored":312880871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":332394156},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724620,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":597620,"kb_used_data":259100,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93774220,"statfs":{"total":96636764160,"available":96024801280,"internally_reserved":0,"allocated":265318400,"data_stored":264994322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":346615467},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,5,3,2,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247903,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":719108,"kb_used_data":326512,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93652732,"statfs":{"total":96636764160,"available":95900397568,"internally_reserved":0,"allocated":334348288,"data_stored":334057186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":401993384},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,1,0,4,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":332742656,"data_stored":332706526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":264704000,"data_stored":264634894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":312655872,"data_stored":312615670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:32:42.613 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:32:42.613 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'15292", 'reported_seq': 15547, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:37.907519+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:32:37.907519+0000', 'last_peered': '2024-09-17T15:32:37.907519+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:32:37.907519+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "41'5200", 'ondisk_log_start': "41'5200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21938176, 'num_objects': 5356, 'num_object_clones': 0, 'num_object_copies': 10712, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5356, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15292, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1372}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:42.613 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:32:42.613 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'15574", 'reported_seq': 15792, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:37.846887+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:32:37.846887+0000', 'last_peered': '2024-09-17T15:32:37.846887+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:32:37.846887+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "41'5500", 'ondisk_log_start': "41'5500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21950488, 'num_objects': 5360, 'num_object_clones': 0, 'num_object_copies': 10720, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5360, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13731, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1235}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:32:42.613 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:32:42.613 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'15827", 'reported_seq': 16119, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:37.900580+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:32:37.900580+0000', 'last_peered': '2024-09-17T15:32:37.900580+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:32:37.900580+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "41'5800", 'ondisk_log_start': "41'5800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22217187, 'num_objects': 5425, 'num_object_clones': 0, 'num_object_copies': 10850, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5425, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12326, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1233}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:42.613 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:32:42.613 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'15486", 'reported_seq': 15800, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:37.834494+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:32:37.834494+0000', 'last_peered': '2024-09-17T15:32:37.834494+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:32:37.834494+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "41'5400", 'ondisk_log_start': "41'5400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21636038, 'num_objects': 5284, 'num_object_clones': 0, 'num_object_copies': 10568, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4089, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5284, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 12255, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 85, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1195}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:42.613 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:32:42.613 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'15152", 'reported_seq': 15421, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:37.895244+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:32:37.895244+0000', 'last_peered': '2024-09-17T15:32:37.895244+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:32:37.895244+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "41'5100", 'ondisk_log_start': "41'5100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22077440, 'num_objects': 5390, 'num_object_clones': 0, 'num_object_copies': 10780, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5390, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11748, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1447}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:42.614 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:32:42.614 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'15212", 'reported_seq': 15426, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:37.795874+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:32:37.795874+0000', 'last_peered': '2024-09-17T15:32:37.795874+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:32:37.795874+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "41'5200", 'ondisk_log_start': "41'5200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21737955, 'num_objects': 5308, 'num_object_clones': 0, 'num_object_copies': 10616, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4034, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5308, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13107, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 1275}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:32:42.614 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:32:42.614 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'15366", 'reported_seq': 21537, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:37.856617+0000', 'last_change': '2024-09-17T15:30:55.086473+0000', 'last_active': '2024-09-17T15:32:37.856617+0000', 'last_peered': '2024-09-17T15:32:37.856617+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:32:37.856617+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "41'5300", 'ondisk_log_start': "41'5300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21995520, 'num_objects': 5370, 'num_object_clones': 0, 'num_object_copies': 10740, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1116, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5370, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13128, 'num_write_kb': 32532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3023, 'num_bytes_recovered': 8073216, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 4254}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:32:42.614 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:32:42.614 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'15581", 'reported_seq': 15883, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:37.907541+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:32:37.907541+0000', 'last_peered': '2024-09-17T15:32:37.907541+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:32:37.907541+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "41'5500", 'ondisk_log_start': "41'5500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22056960, 'num_objects': 5385, 'num_object_clones': 0, 'num_object_copies': 10770, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5385, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12129, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1328}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:42.614 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:32:42.614 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'15408", 'reported_seq': 15678, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:37.856504+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:32:37.856504+0000', 'last_peered': '2024-09-17T15:32:37.856504+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:32:37.856504+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "41'5400", 'ondisk_log_start': "41'5400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21970944, 'num_objects': 5364, 'num_object_clones': 0, 'num_object_copies': 10728, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4098, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5364, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11695, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 62, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1269}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:42.614 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:32:42.614 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'15451", 'reported_seq': 15754, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:37.834300+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:32:37.834300+0000', 'last_peered': '2024-09-17T15:32:37.834300+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:32:37.834300+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "41'5400", 'ondisk_log_start': "41'5400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22261760, 'num_objects': 5435, 'num_object_clones': 0, 'num_object_copies': 10870, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5435, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12238, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1455}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:42.615 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:32:43.295 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:32:43.295 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37625/172628 objects degraded (21.795%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":500,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":86314,"data_bytes":353493769,"bytes_used":1993916416,"bytes_avail":287916376064,"bytes_total":289910292480,"degraded_objects":37625,"degraded_total":172628,"degraded_ratio":0.21795421368491785,"recovering_objects_per_sec":26,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":316,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":404},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-17T15:31:29.695933+0000","services":{}},"progress_events":{}} 2024-09-17T15:32:43.315 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:32:43.316 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:32:43.851 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:32:43.852 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:32:43.874 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2024-09-17T15:32:43.730716+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":343372553,"num_objects":83843,"num_object_clones":0,"num_object_copies":167686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37486,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83843,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":209148,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11975,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160714,"ondisk_log_size":160714,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1944040,"kb_used_data":871664,"kb_used_omap":13,"kb_used_meta":1072306,"kb_avail":281171480,"statfs":{"total":289910292480,"available":287919595520,"internally_reserved":0,"allocated":892583936,"data_stored":891677659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1098042367},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,3,5,8,8,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-30031872,"num_objects":-7332,"num_object_clones":0,"num_object_copies":-14664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-455,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7332,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":455,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007445"},"pg_stats":[{"pgid":"2.f","version":"45'15430","reported_seq":15685,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:42.869017+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:32:42.869017+0000","last_peered":"2024-09-17T15:32:42.869017+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:32:42.869017+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'5400","ondisk_log_start":"41'5400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21372928,"num_objects":5218,"num_object_clones":0,"num_object_copies":10436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15430,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1234}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'15582","reported_seq":15690,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:42.857281+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:32:42.857281+0000","last_peered":"2024-09-17T15:32:42.857281+0000","last_clean":"2024-09-17T15:32:42.857281+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:32:42.857281+0000","last_undegraded":"2024-09-17T15:32:42.857281+0000","last_fullsized":"2024-09-17T15:32:42.857281+0000","mapping_epoch":33,"log_start":"41'5500","ondisk_log_start":"41'5500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21914566,"num_objects":5352,"num_object_clones":0,"num_object_copies":10704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5352,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13939,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'15724","reported_seq":15942,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:42.800174+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:32:42.800174+0000","last_peered":"2024-09-17T15:32:42.800174+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:32:42.800174+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'5700","ondisk_log_start":"41'5700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21336088,"num_objects":5210,"num_object_clones":0,"num_object_copies":10420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5210,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13881,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1085}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'15646","reported_seq":15721,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:42.724155+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:32:42.724155+0000","last_peered":"2024-09-17T15:32:42.724155+0000","last_clean":"2024-09-17T15:32:42.724155+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:32:42.724155+0000","last_undegraded":"2024-09-17T15:32:42.724155+0000","last_fullsized":"2024-09-17T15:32:42.724155+0000","mapping_epoch":33,"log_start":"41'5600","ondisk_log_start":"41'5600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21807104,"num_objects":5324,"num_object_clones":0,"num_object_copies":10648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15646,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'15362","reported_seq":15496,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:42.932466+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:32:42.932466+0000","last_peered":"2024-09-17T15:32:42.932466+0000","last_clean":"2024-09-17T15:32:42.932466+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:32:42.932466+0000","last_undegraded":"2024-09-17T15:32:42.932466+0000","last_fullsized":"2024-09-17T15:32:42.932466+0000","mapping_epoch":32,"log_start":"41'5300","ondisk_log_start":"41'5300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21672419,"num_objects":5292,"num_object_clones":0,"num_object_copies":10584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5292,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12344,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'15981","reported_seq":16277,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:42.872019+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:32:42.872019+0000","last_peered":"2024-09-17T15:32:42.872019+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:32:42.872019+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'5900","ondisk_log_start":"41'5900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21586403,"num_objects":5271,"num_object_clones":0,"num_object_copies":10542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5271,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12480,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1079},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'15626","reported_seq":15937,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:42.791628+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:32:42.791628+0000","last_peered":"2024-09-17T15:32:42.791628+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:32:42.791628+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'5600","ondisk_log_start":"41'5600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21062598,"num_objects":5144,"num_object_clones":0,"num_object_copies":10288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5144,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12395,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1055},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'15671","reported_seq":15804,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:42.962909+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:32:42.962909+0000","last_peered":"2024-09-17T15:32:42.962909+0000","last_clean":"2024-09-17T15:32:42.962909+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:32:42.962909+0000","last_undegraded":"2024-09-17T15:32:42.962909+0000","last_fullsized":"2024-09-17T15:32:42.962909+0000","mapping_epoch":32,"log_start":"41'5600","ondisk_log_start":"41'5600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21483520,"num_objects":5245,"num_object_clones":0,"num_object_copies":10490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12625,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087024+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.087024+0000","last_peered":"2024-09-17T15:30:57.087024+0000","last_clean":"2024-09-17T15:30:57.087024+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.087024+0000","last_undegraded":"2024-09-17T15:30:57.087024+0000","last_fullsized":"2024-09-17T15:30:57.087024+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'15503","reported_seq":23789,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:42.313667+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:32:42.313667+0000","last_peered":"2024-09-17T15:32:42.313667+0000","last_clean":"2024-09-17T15:32:42.313667+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:32:42.313667+0000","last_undegraded":"2024-09-17T15:32:42.313667+0000","last_fullsized":"2024-09-17T15:32:42.313667+0000","mapping_epoch":42,"log_start":"41'5500","ondisk_log_start":"41'5500","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21766627,"num_objects":5315,"num_object_clones":0,"num_object_copies":10630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5315,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13503,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'15309","reported_seq":15579,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:42.895259+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:32:42.895259+0000","last_peered":"2024-09-17T15:32:42.895259+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:32:42.895259+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'5300","ondisk_log_start":"41'5300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21434368,"num_objects":5233,"num_object_clones":0,"num_object_copies":10466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11905,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1290}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'15349","reported_seq":15563,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:42.831385+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:32:42.831385+0000","last_peered":"2024-09-17T15:32:42.831385+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:32:42.831385+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'5300","ondisk_log_start":"41'5300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21176803,"num_objects":5171,"num_object_clones":0,"num_object_copies":10342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5171,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13244,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1138}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":83,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087587+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087587+0000","last_peered":"2024-09-17T15:30:57.087587+0000","last_clean":"2024-09-17T15:30:57.087587+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087587+0000","last_undegraded":"2024-09-17T15:30:57.087587+0000","last_fullsized":"2024-09-17T15:30:57.087587+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'15528","reported_seq":21977,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:42.868445+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:32:42.868445+0000","last_peered":"2024-09-17T15:32:42.868445+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:32:42.868445+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"41'5500","ondisk_log_start":"41'5500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21331968,"num_objects":5208,"num_object_clones":0,"num_object_copies":10416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":977,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13290,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3162,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4231}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'15737","reported_seq":16040,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:42.937981+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:32:42.937981+0000","last_peered":"2024-09-17T15:32:42.937981+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:32:42.937981+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'5700","ondisk_log_start":"41'5700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21417984,"num_objects":5229,"num_object_clones":0,"num_object_copies":10458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12285,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1172}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'15574","reported_seq":15840,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:42.746295+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:32:42.746295+0000","last_peered":"2024-09-17T15:32:42.746295+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:32:42.746295+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'5500","ondisk_log_start":"41'5500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21291008,"num_objects":5198,"num_object_clones":0,"num_object_copies":10396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5198,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11861,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1103}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'15622","reported_seq":15924,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:42.954212+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:32:42.954212+0000","last_peered":"2024-09-17T15:32:42.954212+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:32:42.954212+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'5600","ondisk_log_start":"41'5600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21561344,"num_objects":5264,"num_object_clones":0,"num_object_copies":10528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12409,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1284}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087804+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087804+0000","last_peered":"2024-09-17T15:30:57.087804+0000","last_clean":"2024-09-17T15:30:57.087804+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087804+0000","last_undegraded":"2024-09-17T15:30:57.087804+0000","last_fullsized":"2024-09-17T15:30:57.087804+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'15467","reported_seq":23845,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:42.346564+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:32:42.346564+0000","last_peered":"2024-09-17T15:32:42.346564+0000","last_clean":"2024-09-17T15:32:42.346564+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:32:42.346564+0000","last_undegraded":"2024-09-17T15:32:42.346564+0000","last_fullsized":"2024-09-17T15:32:42.346564+0000","mapping_epoch":44,"log_start":"41'5400","ondisk_log_start":"41'5400","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21156806,"num_objects":5167,"num_object_clones":0,"num_object_copies":10334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11909,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":343372534,"num_objects":83841,"num_object_clones":0,"num_object_copies":167682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37486,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83841,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":209146,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11974,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":889876480,"data_stored":889731042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160711,"ondisk_log_size":160711,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051676,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":629624,"kb_used_data":300832,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93742216,"statfs":{"total":96636764160,"available":95992029184,"internally_reserved":0,"allocated":308051968,"data_stored":307760871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":336653996},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,1,4,3,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724621,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594528,"kb_used_data":251856,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93777312,"statfs":{"total":96636764160,"available":96027967488,"internally_reserved":0,"allocated":257900544,"data_stored":257576466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":350875307},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,2,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247904,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":719888,"kb_used_data":318976,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93651952,"statfs":{"total":96636764160,"available":95899598848,"internally_reserved":0,"allocated":326631424,"data_stored":326340322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":410513064},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,2,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":325083136,"data_stored":325047006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":257187840,"data_stored":257118734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":307605504,"data_stored":307565302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:32:46.877 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:32:47.445 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:32:47.447 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:32:47.469 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-09-17T15:32:45.731396+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":343372553,"num_objects":83843,"num_object_clones":0,"num_object_copies":167686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37486,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83843,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":209148,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11975,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160714,"ondisk_log_size":160714,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1944040,"kb_used_data":871664,"kb_used_omap":13,"kb_used_meta":1072306,"kb_avail":281171480,"statfs":{"total":289910292480,"available":287919595520,"internally_reserved":0,"allocated":892583936,"data_stored":891677659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1098042367},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,3,5,8,8,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20090880,"num_objects":-4905,"num_object_clones":0,"num_object_copies":-9810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-297,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4905,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":297,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007398"},"pg_stats":[{"pgid":"2.f","version":"45'15430","reported_seq":15685,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:42.869017+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:32:42.869017+0000","last_peered":"2024-09-17T15:32:42.869017+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:32:42.869017+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'5400","ondisk_log_start":"41'5400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21372928,"num_objects":5218,"num_object_clones":0,"num_object_copies":10436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15430,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1234}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'15582","reported_seq":15690,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:42.857281+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:32:42.857281+0000","last_peered":"2024-09-17T15:32:42.857281+0000","last_clean":"2024-09-17T15:32:42.857281+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:32:42.857281+0000","last_undegraded":"2024-09-17T15:32:42.857281+0000","last_fullsized":"2024-09-17T15:32:42.857281+0000","mapping_epoch":33,"log_start":"41'5500","ondisk_log_start":"41'5500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21914566,"num_objects":5352,"num_object_clones":0,"num_object_copies":10704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5352,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13939,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'15724","reported_seq":15942,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:42.800174+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:32:42.800174+0000","last_peered":"2024-09-17T15:32:42.800174+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:32:42.800174+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'5700","ondisk_log_start":"41'5700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21336088,"num_objects":5210,"num_object_clones":0,"num_object_copies":10420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5210,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13881,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1085}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'15646","reported_seq":15721,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:42.724155+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:32:42.724155+0000","last_peered":"2024-09-17T15:32:42.724155+0000","last_clean":"2024-09-17T15:32:42.724155+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:32:42.724155+0000","last_undegraded":"2024-09-17T15:32:42.724155+0000","last_fullsized":"2024-09-17T15:32:42.724155+0000","mapping_epoch":33,"log_start":"41'5600","ondisk_log_start":"41'5600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21807104,"num_objects":5324,"num_object_clones":0,"num_object_copies":10648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15646,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'15362","reported_seq":15496,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:42.932466+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:32:42.932466+0000","last_peered":"2024-09-17T15:32:42.932466+0000","last_clean":"2024-09-17T15:32:42.932466+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:32:42.932466+0000","last_undegraded":"2024-09-17T15:32:42.932466+0000","last_fullsized":"2024-09-17T15:32:42.932466+0000","mapping_epoch":32,"log_start":"41'5300","ondisk_log_start":"41'5300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21672419,"num_objects":5292,"num_object_clones":0,"num_object_copies":10584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5292,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12344,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'15981","reported_seq":16277,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:42.872019+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:32:42.872019+0000","last_peered":"2024-09-17T15:32:42.872019+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:32:42.872019+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'5900","ondisk_log_start":"41'5900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21586403,"num_objects":5271,"num_object_clones":0,"num_object_copies":10542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5271,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12480,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1079},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'15626","reported_seq":15937,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:42.791628+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:32:42.791628+0000","last_peered":"2024-09-17T15:32:42.791628+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:32:42.791628+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'5600","ondisk_log_start":"41'5600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21062598,"num_objects":5144,"num_object_clones":0,"num_object_copies":10288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5144,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12395,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1055},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'15671","reported_seq":15804,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:42.962909+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:32:42.962909+0000","last_peered":"2024-09-17T15:32:42.962909+0000","last_clean":"2024-09-17T15:32:42.962909+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:32:42.962909+0000","last_undegraded":"2024-09-17T15:32:42.962909+0000","last_fullsized":"2024-09-17T15:32:42.962909+0000","mapping_epoch":32,"log_start":"41'5600","ondisk_log_start":"41'5600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21483520,"num_objects":5245,"num_object_clones":0,"num_object_copies":10490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12625,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087024+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.087024+0000","last_peered":"2024-09-17T15:30:57.087024+0000","last_clean":"2024-09-17T15:30:57.087024+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.087024+0000","last_undegraded":"2024-09-17T15:30:57.087024+0000","last_fullsized":"2024-09-17T15:30:57.087024+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'15503","reported_seq":23789,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:42.313667+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:32:42.313667+0000","last_peered":"2024-09-17T15:32:42.313667+0000","last_clean":"2024-09-17T15:32:42.313667+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:32:42.313667+0000","last_undegraded":"2024-09-17T15:32:42.313667+0000","last_fullsized":"2024-09-17T15:32:42.313667+0000","mapping_epoch":42,"log_start":"41'5500","ondisk_log_start":"41'5500","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21766627,"num_objects":5315,"num_object_clones":0,"num_object_copies":10630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5315,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13503,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'15309","reported_seq":15579,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:42.895259+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:32:42.895259+0000","last_peered":"2024-09-17T15:32:42.895259+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:32:42.895259+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'5300","ondisk_log_start":"41'5300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21434368,"num_objects":5233,"num_object_clones":0,"num_object_copies":10466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11905,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1290}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'15349","reported_seq":15563,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:42.831385+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:32:42.831385+0000","last_peered":"2024-09-17T15:32:42.831385+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:32:42.831385+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'5300","ondisk_log_start":"41'5300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21176803,"num_objects":5171,"num_object_clones":0,"num_object_copies":10342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5171,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13244,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1138}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":83,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087587+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087587+0000","last_peered":"2024-09-17T15:30:57.087587+0000","last_clean":"2024-09-17T15:30:57.087587+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087587+0000","last_undegraded":"2024-09-17T15:30:57.087587+0000","last_fullsized":"2024-09-17T15:30:57.087587+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'15528","reported_seq":21977,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:42.868445+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:32:42.868445+0000","last_peered":"2024-09-17T15:32:42.868445+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:32:42.868445+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"41'5500","ondisk_log_start":"41'5500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21331968,"num_objects":5208,"num_object_clones":0,"num_object_copies":10416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":977,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13290,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3162,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4231}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'15737","reported_seq":16040,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:42.937981+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:32:42.937981+0000","last_peered":"2024-09-17T15:32:42.937981+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:32:42.937981+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'5700","ondisk_log_start":"41'5700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21417984,"num_objects":5229,"num_object_clones":0,"num_object_copies":10458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12285,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1172}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'15574","reported_seq":15840,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:42.746295+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:32:42.746295+0000","last_peered":"2024-09-17T15:32:42.746295+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:32:42.746295+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'5500","ondisk_log_start":"41'5500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21291008,"num_objects":5198,"num_object_clones":0,"num_object_copies":10396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5198,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11861,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1103}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'15622","reported_seq":15924,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:42.954212+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:32:42.954212+0000","last_peered":"2024-09-17T15:32:42.954212+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:32:42.954212+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'5600","ondisk_log_start":"41'5600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21561344,"num_objects":5264,"num_object_clones":0,"num_object_copies":10528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12409,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1284}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087804+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087804+0000","last_peered":"2024-09-17T15:30:57.087804+0000","last_clean":"2024-09-17T15:30:57.087804+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087804+0000","last_undegraded":"2024-09-17T15:30:57.087804+0000","last_fullsized":"2024-09-17T15:30:57.087804+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'15467","reported_seq":23845,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:42.346564+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:32:42.346564+0000","last_peered":"2024-09-17T15:32:42.346564+0000","last_clean":"2024-09-17T15:32:42.346564+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:32:42.346564+0000","last_undegraded":"2024-09-17T15:32:42.346564+0000","last_fullsized":"2024-09-17T15:32:42.346564+0000","mapping_epoch":44,"log_start":"41'5400","ondisk_log_start":"41'5400","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21156806,"num_objects":5167,"num_object_clones":0,"num_object_copies":10334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11909,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":343372534,"num_objects":83841,"num_object_clones":0,"num_object_copies":167682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37486,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83841,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":209146,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11974,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":889876480,"data_stored":889731042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160711,"ondisk_log_size":160711,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051676,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":629624,"kb_used_data":300832,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93742216,"statfs":{"total":96636764160,"available":95992029184,"internally_reserved":0,"allocated":308051968,"data_stored":307760871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":336653996},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,1,4,3,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724621,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594528,"kb_used_data":251856,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93777312,"statfs":{"total":96636764160,"available":96027967488,"internally_reserved":0,"allocated":257900544,"data_stored":257576466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":350875307},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,2,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247904,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":719888,"kb_used_data":318976,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93651952,"statfs":{"total":96636764160,"available":95899598848,"internally_reserved":0,"allocated":326631424,"data_stored":326340322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":410513064},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,2,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":325083136,"data_stored":325047006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":257187840,"data_stored":257118734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":307605504,"data_stored":307565302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:32:47.470 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:32:48.006 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:32:48.009 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:32:48.035 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":290,"stamp":"2024-09-17T15:32:47.732231+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":342078217,"num_objects":83527,"num_object_clones":0,"num_object_copies":167054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37486,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83527,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":209464,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11975,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160730,"ondisk_log_size":160730,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1928420,"kb_used_data":864352,"kb_used_omap":13,"kb_used_meta":1063986,"kb_avail":281187100,"statfs":{"total":289910292480,"available":287935590400,"internally_reserved":0,"allocated":885096448,"data_stored":884190171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1089522687},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,3,3,8,11,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21385216,"num_objects":-5221,"num_object_clones":0,"num_object_copies":-10442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-297,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5221,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":297,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007415"},"pg_stats":[{"pgid":"2.f","version":"45'15430","reported_seq":15685,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:42.869017+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:32:42.869017+0000","last_peered":"2024-09-17T15:32:42.869017+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:32:42.869017+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'5400","ondisk_log_start":"41'5400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21372928,"num_objects":5218,"num_object_clones":0,"num_object_copies":10436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15430,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1234}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'15582","reported_seq":15690,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:42.857281+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:32:42.857281+0000","last_peered":"2024-09-17T15:32:42.857281+0000","last_clean":"2024-09-17T15:32:42.857281+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:32:42.857281+0000","last_undegraded":"2024-09-17T15:32:42.857281+0000","last_fullsized":"2024-09-17T15:32:42.857281+0000","mapping_epoch":33,"log_start":"41'5500","ondisk_log_start":"41'5500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21914566,"num_objects":5352,"num_object_clones":0,"num_object_copies":10704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5352,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13939,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'15724","reported_seq":15942,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:42.800174+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:32:42.800174+0000","last_peered":"2024-09-17T15:32:42.800174+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:32:42.800174+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'5700","ondisk_log_start":"41'5700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21336088,"num_objects":5210,"num_object_clones":0,"num_object_copies":10420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5210,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13881,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1085}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'15646","reported_seq":15721,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:42.724155+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:32:42.724155+0000","last_peered":"2024-09-17T15:32:42.724155+0000","last_clean":"2024-09-17T15:32:42.724155+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:32:42.724155+0000","last_undegraded":"2024-09-17T15:32:42.724155+0000","last_fullsized":"2024-09-17T15:32:42.724155+0000","mapping_epoch":33,"log_start":"41'5600","ondisk_log_start":"41'5600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21807104,"num_objects":5324,"num_object_clones":0,"num_object_copies":10648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15646,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'15362","reported_seq":15496,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:42.932466+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:32:42.932466+0000","last_peered":"2024-09-17T15:32:42.932466+0000","last_clean":"2024-09-17T15:32:42.932466+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:32:42.932466+0000","last_undegraded":"2024-09-17T15:32:42.932466+0000","last_fullsized":"2024-09-17T15:32:42.932466+0000","mapping_epoch":32,"log_start":"41'5300","ondisk_log_start":"41'5300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21672419,"num_objects":5292,"num_object_clones":0,"num_object_copies":10584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5292,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12344,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'15981","reported_seq":16277,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:42.872019+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:32:42.872019+0000","last_peered":"2024-09-17T15:32:42.872019+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:32:42.872019+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'5900","ondisk_log_start":"41'5900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21586403,"num_objects":5271,"num_object_clones":0,"num_object_copies":10542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5271,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12480,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1079},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'15626","reported_seq":15937,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:42.791628+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:32:42.791628+0000","last_peered":"2024-09-17T15:32:42.791628+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:32:42.791628+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'5600","ondisk_log_start":"41'5600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21062598,"num_objects":5144,"num_object_clones":0,"num_object_copies":10288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5144,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12395,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1055},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'15671","reported_seq":15804,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:42.962909+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:32:42.962909+0000","last_peered":"2024-09-17T15:32:42.962909+0000","last_clean":"2024-09-17T15:32:42.962909+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:32:42.962909+0000","last_undegraded":"2024-09-17T15:32:42.962909+0000","last_fullsized":"2024-09-17T15:32:42.962909+0000","mapping_epoch":32,"log_start":"41'5600","ondisk_log_start":"41'5600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21483520,"num_objects":5245,"num_object_clones":0,"num_object_copies":10490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12625,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087024+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.087024+0000","last_peered":"2024-09-17T15:30:57.087024+0000","last_clean":"2024-09-17T15:30:57.087024+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.087024+0000","last_undegraded":"2024-09-17T15:30:57.087024+0000","last_fullsized":"2024-09-17T15:30:57.087024+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'15669","reported_seq":23957,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:47.339291+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:32:47.339291+0000","last_peered":"2024-09-17T15:32:47.339291+0000","last_clean":"2024-09-17T15:32:47.339291+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:32:47.339291+0000","last_undegraded":"2024-09-17T15:32:47.339291+0000","last_fullsized":"2024-09-17T15:32:47.339291+0000","mapping_epoch":42,"log_start":"41'5600","ondisk_log_start":"41'5600","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21086691,"num_objects":5149,"num_object_clones":0,"num_object_copies":10298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5149,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13669,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'15309","reported_seq":15579,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:42.895259+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:32:42.895259+0000","last_peered":"2024-09-17T15:32:42.895259+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:32:42.895259+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'5300","ondisk_log_start":"41'5300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21434368,"num_objects":5233,"num_object_clones":0,"num_object_copies":10466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11905,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1290}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'15349","reported_seq":15563,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:42.831385+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:32:42.831385+0000","last_peered":"2024-09-17T15:32:42.831385+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:32:42.831385+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'5300","ondisk_log_start":"41'5300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21176803,"num_objects":5171,"num_object_clones":0,"num_object_copies":10342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5171,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13244,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1138}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":83,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087587+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087587+0000","last_peered":"2024-09-17T15:30:57.087587+0000","last_clean":"2024-09-17T15:30:57.087587+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087587+0000","last_undegraded":"2024-09-17T15:30:57.087587+0000","last_fullsized":"2024-09-17T15:30:57.087587+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'15528","reported_seq":21977,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:42.868445+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:32:42.868445+0000","last_peered":"2024-09-17T15:32:42.868445+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:32:42.868445+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"41'5500","ondisk_log_start":"41'5500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21331968,"num_objects":5208,"num_object_clones":0,"num_object_copies":10416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":977,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13290,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3162,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4231}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'15737","reported_seq":16040,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:42.937981+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:32:42.937981+0000","last_peered":"2024-09-17T15:32:42.937981+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:32:42.937981+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'5700","ondisk_log_start":"41'5700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21417984,"num_objects":5229,"num_object_clones":0,"num_object_copies":10458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12285,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1172}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'15574","reported_seq":15840,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:42.746295+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:32:42.746295+0000","last_peered":"2024-09-17T15:32:42.746295+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:32:42.746295+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'5500","ondisk_log_start":"41'5500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21291008,"num_objects":5198,"num_object_clones":0,"num_object_copies":10396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5198,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11861,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1103}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'15622","reported_seq":15924,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:42.954212+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:32:42.954212+0000","last_peered":"2024-09-17T15:32:42.954212+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:32:42.954212+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'5600","ondisk_log_start":"41'5600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21561344,"num_objects":5264,"num_object_clones":0,"num_object_copies":10528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12409,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1284}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087804+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087804+0000","last_peered":"2024-09-17T15:30:57.087804+0000","last_clean":"2024-09-17T15:30:57.087804+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087804+0000","last_undegraded":"2024-09-17T15:30:57.087804+0000","last_fullsized":"2024-09-17T15:30:57.087804+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'15617","reported_seq":23996,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:47.443760+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:32:47.443760+0000","last_peered":"2024-09-17T15:32:47.443760+0000","last_clean":"2024-09-17T15:32:47.443760+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:32:47.443760+0000","last_undegraded":"2024-09-17T15:32:47.443760+0000","last_fullsized":"2024-09-17T15:32:47.443760+0000","mapping_epoch":44,"log_start":"41'5600","ondisk_log_start":"41'5600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20542406,"num_objects":5017,"num_object_clones":0,"num_object_copies":10034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12059,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":342078198,"num_objects":83525,"num_object_clones":0,"num_object_copies":167050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37486,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83525,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":209462,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11974,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":882274304,"data_stored":882128866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160727,"ondisk_log_size":160727,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051676,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":629624,"kb_used_data":300832,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93742216,"statfs":{"total":96636764160,"available":95992029184,"internally_reserved":0,"allocated":308051968,"data_stored":307760871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":336653996},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,1,4,3,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724622,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":578908,"kb_used_data":244544,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93792932,"statfs":{"total":96636764160,"available":96043962368,"internally_reserved":0,"allocated":250413056,"data_stored":250088978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":342355627},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247904,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":719888,"kb_used_data":318976,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93651952,"statfs":{"total":96636764160,"available":95899598848,"internally_reserved":0,"allocated":326631424,"data_stored":326340322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":410513064},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,2,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":325083136,"data_stored":325047006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":249585664,"data_stored":249516558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":307605504,"data_stored":307565302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:32:48.036 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:32:48.037 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'15430", 'reported_seq': 15685, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:42.869017+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:32:42.869017+0000', 'last_peered': '2024-09-17T15:32:42.869017+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:32:42.869017+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "41'5400", 'ondisk_log_start': "41'5400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21372928, 'num_objects': 5218, 'num_object_clones': 0, 'num_object_copies': 10436, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5218, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15430, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1234}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:48.037 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:32:48.037 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'15724", 'reported_seq': 15942, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:42.800174+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:32:42.800174+0000', 'last_peered': '2024-09-17T15:32:42.800174+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:32:42.800174+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "41'5700", 'ondisk_log_start': "41'5700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21336088, 'num_objects': 5210, 'num_object_clones': 0, 'num_object_copies': 10420, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5210, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13881, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1085}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:32:48.037 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:32:48.037 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'15981", 'reported_seq': 16277, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:42.872019+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:32:42.872019+0000', 'last_peered': '2024-09-17T15:32:42.872019+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:32:42.872019+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "41'5900", 'ondisk_log_start': "41'5900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21586403, 'num_objects': 5271, 'num_object_clones': 0, 'num_object_copies': 10542, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5271, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12480, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1079}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:48.037 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:32:48.037 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'15626", 'reported_seq': 15937, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:42.791628+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:32:42.791628+0000', 'last_peered': '2024-09-17T15:32:42.791628+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:32:42.791628+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "41'5600", 'ondisk_log_start': "41'5600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21062598, 'num_objects': 5144, 'num_object_clones': 0, 'num_object_copies': 10288, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4089, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5144, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 12395, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 85, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1055}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:48.037 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:32:48.038 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'15309", 'reported_seq': 15579, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:42.895259+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:32:42.895259+0000', 'last_peered': '2024-09-17T15:32:42.895259+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:32:42.895259+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "41'5300", 'ondisk_log_start': "41'5300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21434368, 'num_objects': 5233, 'num_object_clones': 0, 'num_object_copies': 10466, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5233, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11905, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1290}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:48.038 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:32:48.038 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'15349", 'reported_seq': 15563, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:42.831385+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:32:42.831385+0000', 'last_peered': '2024-09-17T15:32:42.831385+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:32:42.831385+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "41'5300", 'ondisk_log_start': "41'5300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21176803, 'num_objects': 5171, 'num_object_clones': 0, 'num_object_copies': 10342, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4034, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5171, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13244, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 1138}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:32:48.038 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:32:48.038 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'15528", 'reported_seq': 21977, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:42.868445+0000', 'last_change': '2024-09-17T15:30:55.086473+0000', 'last_active': '2024-09-17T15:32:42.868445+0000', 'last_peered': '2024-09-17T15:32:42.868445+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:32:42.868445+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "41'5500", 'ondisk_log_start': "41'5500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21331968, 'num_objects': 5208, 'num_object_clones': 0, 'num_object_copies': 10416, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 977, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5208, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13290, 'num_write_kb': 32532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3162, 'num_bytes_recovered': 8073216, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 4231}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:32:48.038 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:32:48.038 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'15737", 'reported_seq': 16040, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:42.937981+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:32:42.937981+0000', 'last_peered': '2024-09-17T15:32:42.937981+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:32:42.937981+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "41'5700", 'ondisk_log_start': "41'5700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21417984, 'num_objects': 5229, 'num_object_clones': 0, 'num_object_copies': 10458, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5229, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12285, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1172}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:48.038 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:32:48.038 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'15574", 'reported_seq': 15840, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:42.746295+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:32:42.746295+0000', 'last_peered': '2024-09-17T15:32:42.746295+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:32:42.746295+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "41'5500", 'ondisk_log_start': "41'5500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21291008, 'num_objects': 5198, 'num_object_clones': 0, 'num_object_copies': 10396, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4098, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5198, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11861, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 62, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1103}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:48.038 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:32:48.038 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'15622", 'reported_seq': 15924, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:42.954212+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:32:42.954212+0000', 'last_peered': '2024-09-17T15:32:42.954212+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:32:42.954212+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "41'5600", 'ondisk_log_start': "41'5600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21561344, 'num_objects': 5264, 'num_object_clones': 0, 'num_object_copies': 10528, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5264, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12409, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1284}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:48.039 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:32:48.720 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:32:48.721 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37486/167054 objects degraded (22.439%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":505,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":83527,"data_bytes":342078217,"bytes_used":1974702080,"bytes_avail":287935590400,"bytes_total":289910292480,"degraded_objects":37486,"degraded_total":167054,"degraded_ratio":0.22439450716534773,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":297,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":434},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-17T15:31:29.695933+0000","services":{}},"progress_events":{}} 2024-09-17T15:32:48.747 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:32:48.747 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:32:49.303 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:32:49.303 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:32:49.330 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":290,"stamp":"2024-09-17T15:32:47.732231+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":342078217,"num_objects":83527,"num_object_clones":0,"num_object_copies":167054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37486,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83527,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":209464,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11975,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160730,"ondisk_log_size":160730,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1928420,"kb_used_data":864352,"kb_used_omap":13,"kb_used_meta":1063986,"kb_avail":281187100,"statfs":{"total":289910292480,"available":287935590400,"internally_reserved":0,"allocated":885096448,"data_stored":884190171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1089522687},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,3,3,8,11,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21385216,"num_objects":-5221,"num_object_clones":0,"num_object_copies":-10442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-297,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5221,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":297,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007415"},"pg_stats":[{"pgid":"2.f","version":"45'15430","reported_seq":15685,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:42.869017+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:32:42.869017+0000","last_peered":"2024-09-17T15:32:42.869017+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:32:42.869017+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'5400","ondisk_log_start":"41'5400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21372928,"num_objects":5218,"num_object_clones":0,"num_object_copies":10436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15430,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1234}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'15582","reported_seq":15690,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:42.857281+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:32:42.857281+0000","last_peered":"2024-09-17T15:32:42.857281+0000","last_clean":"2024-09-17T15:32:42.857281+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:32:42.857281+0000","last_undegraded":"2024-09-17T15:32:42.857281+0000","last_fullsized":"2024-09-17T15:32:42.857281+0000","mapping_epoch":33,"log_start":"41'5500","ondisk_log_start":"41'5500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21914566,"num_objects":5352,"num_object_clones":0,"num_object_copies":10704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5352,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13939,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'15724","reported_seq":15942,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:42.800174+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:32:42.800174+0000","last_peered":"2024-09-17T15:32:42.800174+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:32:42.800174+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'5700","ondisk_log_start":"41'5700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21336088,"num_objects":5210,"num_object_clones":0,"num_object_copies":10420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5210,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13881,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1085}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'15646","reported_seq":15721,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:42.724155+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:32:42.724155+0000","last_peered":"2024-09-17T15:32:42.724155+0000","last_clean":"2024-09-17T15:32:42.724155+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:32:42.724155+0000","last_undegraded":"2024-09-17T15:32:42.724155+0000","last_fullsized":"2024-09-17T15:32:42.724155+0000","mapping_epoch":33,"log_start":"41'5600","ondisk_log_start":"41'5600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21807104,"num_objects":5324,"num_object_clones":0,"num_object_copies":10648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15646,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'15362","reported_seq":15496,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:42.932466+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:32:42.932466+0000","last_peered":"2024-09-17T15:32:42.932466+0000","last_clean":"2024-09-17T15:32:42.932466+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:32:42.932466+0000","last_undegraded":"2024-09-17T15:32:42.932466+0000","last_fullsized":"2024-09-17T15:32:42.932466+0000","mapping_epoch":32,"log_start":"41'5300","ondisk_log_start":"41'5300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21672419,"num_objects":5292,"num_object_clones":0,"num_object_copies":10584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5292,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12344,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'15981","reported_seq":16277,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:42.872019+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:32:42.872019+0000","last_peered":"2024-09-17T15:32:42.872019+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:32:42.872019+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'5900","ondisk_log_start":"41'5900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21586403,"num_objects":5271,"num_object_clones":0,"num_object_copies":10542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5271,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12480,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1079},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'15626","reported_seq":15937,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:42.791628+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:32:42.791628+0000","last_peered":"2024-09-17T15:32:42.791628+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:32:42.791628+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'5600","ondisk_log_start":"41'5600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21062598,"num_objects":5144,"num_object_clones":0,"num_object_copies":10288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5144,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12395,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1055},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'15671","reported_seq":15804,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:42.962909+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:32:42.962909+0000","last_peered":"2024-09-17T15:32:42.962909+0000","last_clean":"2024-09-17T15:32:42.962909+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:32:42.962909+0000","last_undegraded":"2024-09-17T15:32:42.962909+0000","last_fullsized":"2024-09-17T15:32:42.962909+0000","mapping_epoch":32,"log_start":"41'5600","ondisk_log_start":"41'5600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21483520,"num_objects":5245,"num_object_clones":0,"num_object_copies":10490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12625,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087024+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.087024+0000","last_peered":"2024-09-17T15:30:57.087024+0000","last_clean":"2024-09-17T15:30:57.087024+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.087024+0000","last_undegraded":"2024-09-17T15:30:57.087024+0000","last_fullsized":"2024-09-17T15:30:57.087024+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'15669","reported_seq":23957,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:47.339291+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:32:47.339291+0000","last_peered":"2024-09-17T15:32:47.339291+0000","last_clean":"2024-09-17T15:32:47.339291+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:32:47.339291+0000","last_undegraded":"2024-09-17T15:32:47.339291+0000","last_fullsized":"2024-09-17T15:32:47.339291+0000","mapping_epoch":42,"log_start":"41'5600","ondisk_log_start":"41'5600","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21086691,"num_objects":5149,"num_object_clones":0,"num_object_copies":10298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5149,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13669,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'15309","reported_seq":15579,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:42.895259+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:32:42.895259+0000","last_peered":"2024-09-17T15:32:42.895259+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:32:42.895259+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'5300","ondisk_log_start":"41'5300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21434368,"num_objects":5233,"num_object_clones":0,"num_object_copies":10466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11905,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1290}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'15349","reported_seq":15563,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:42.831385+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:32:42.831385+0000","last_peered":"2024-09-17T15:32:42.831385+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:32:42.831385+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'5300","ondisk_log_start":"41'5300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21176803,"num_objects":5171,"num_object_clones":0,"num_object_copies":10342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5171,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13244,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1138}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":83,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087587+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087587+0000","last_peered":"2024-09-17T15:30:57.087587+0000","last_clean":"2024-09-17T15:30:57.087587+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087587+0000","last_undegraded":"2024-09-17T15:30:57.087587+0000","last_fullsized":"2024-09-17T15:30:57.087587+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'15528","reported_seq":21977,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:42.868445+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:32:42.868445+0000","last_peered":"2024-09-17T15:32:42.868445+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:32:42.868445+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"41'5500","ondisk_log_start":"41'5500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21331968,"num_objects":5208,"num_object_clones":0,"num_object_copies":10416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":977,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13290,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3162,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4231}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'15737","reported_seq":16040,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:42.937981+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:32:42.937981+0000","last_peered":"2024-09-17T15:32:42.937981+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:32:42.937981+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'5700","ondisk_log_start":"41'5700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21417984,"num_objects":5229,"num_object_clones":0,"num_object_copies":10458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12285,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1172}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'15574","reported_seq":15840,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:42.746295+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:32:42.746295+0000","last_peered":"2024-09-17T15:32:42.746295+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:32:42.746295+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'5500","ondisk_log_start":"41'5500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21291008,"num_objects":5198,"num_object_clones":0,"num_object_copies":10396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5198,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11861,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1103}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'15622","reported_seq":15924,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:42.954212+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:32:42.954212+0000","last_peered":"2024-09-17T15:32:42.954212+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:32:42.954212+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'5600","ondisk_log_start":"41'5600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21561344,"num_objects":5264,"num_object_clones":0,"num_object_copies":10528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12409,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1284}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087804+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087804+0000","last_peered":"2024-09-17T15:30:57.087804+0000","last_clean":"2024-09-17T15:30:57.087804+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087804+0000","last_undegraded":"2024-09-17T15:30:57.087804+0000","last_fullsized":"2024-09-17T15:30:57.087804+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'15617","reported_seq":23996,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:47.443760+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:32:47.443760+0000","last_peered":"2024-09-17T15:32:47.443760+0000","last_clean":"2024-09-17T15:32:47.443760+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:32:47.443760+0000","last_undegraded":"2024-09-17T15:32:47.443760+0000","last_fullsized":"2024-09-17T15:32:47.443760+0000","mapping_epoch":44,"log_start":"41'5600","ondisk_log_start":"41'5600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20542406,"num_objects":5017,"num_object_clones":0,"num_object_copies":10034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12059,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":342078198,"num_objects":83525,"num_object_clones":0,"num_object_copies":167050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37486,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83525,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":209462,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11974,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":882274304,"data_stored":882128866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160727,"ondisk_log_size":160727,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051676,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":629624,"kb_used_data":300832,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93742216,"statfs":{"total":96636764160,"available":95992029184,"internally_reserved":0,"allocated":308051968,"data_stored":307760871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":336653996},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,1,4,3,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724622,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":578908,"kb_used_data":244544,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93792932,"statfs":{"total":96636764160,"available":96043962368,"internally_reserved":0,"allocated":250413056,"data_stored":250088978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":342355627},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247904,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":719888,"kb_used_data":318976,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93651952,"statfs":{"total":96636764160,"available":95899598848,"internally_reserved":0,"allocated":326631424,"data_stored":326340322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":410513064},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,2,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":325083136,"data_stored":325047006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":249585664,"data_stored":249516558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":307605504,"data_stored":307565302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:32:52.333 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:32:52.751 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:32:52.752 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:32:52.781 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2024-09-17T15:32:51.733900+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":333263625,"num_objects":81375,"num_object_clones":0,"num_object_copies":162750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37330,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81375,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":211616,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12131,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160782,"ondisk_log_size":160782,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1891188,"kb_used_data":852088,"kb_used_omap":13,"kb_used_meta":1039026,"kb_avail":281224332,"statfs":{"total":289910292480,"available":287973715968,"internally_reserved":0,"allocated":872538112,"data_stored":871631835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1063963647},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,3,5,14,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20230144,"num_objects":-4939,"num_object_clones":0,"num_object_copies":-9878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-295,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4939,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4939,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":295,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004618"},"pg_stats":[{"pgid":"2.f","version":"45'15594","reported_seq":15849,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:47.784017+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:32:47.784017+0000","last_peered":"2024-09-17T15:32:47.784017+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:32:47.784017+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'5500","ondisk_log_start":"41'5500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20701184,"num_objects":5054,"num_object_clones":0,"num_object_copies":10108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5054,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15594,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'15739","reported_seq":15847,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:47.766129+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:32:47.766129+0000","last_peered":"2024-09-17T15:32:47.766129+0000","last_clean":"2024-09-17T15:32:47.766129+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:32:47.766129+0000","last_undegraded":"2024-09-17T15:32:47.766129+0000","last_fullsized":"2024-09-17T15:32:47.766129+0000","mapping_epoch":33,"log_start":"41'5700","ondisk_log_start":"41'5700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21271494,"num_objects":5195,"num_object_clones":0,"num_object_copies":10390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5195,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14096,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'15870","reported_seq":16088,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:47.816697+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:32:47.816697+0000","last_peered":"2024-09-17T15:32:47.816697+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:32:47.816697+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'5800","ondisk_log_start":"41'5800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20738072,"num_objects":5064,"num_object_clones":0,"num_object_copies":10128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5064,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14027,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":939}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'15815","reported_seq":15890,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:47.830060+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:32:47.830060+0000","last_peered":"2024-09-17T15:32:47.830060+0000","last_clean":"2024-09-17T15:32:47.830060+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:32:47.830060+0000","last_undegraded":"2024-09-17T15:32:47.830060+0000","last_fullsized":"2024-09-17T15:32:47.830060+0000","mapping_epoch":33,"log_start":"41'5800","ondisk_log_start":"41'5800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21114880,"num_objects":5155,"num_object_clones":0,"num_object_copies":10310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15815,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'15516","reported_seq":15650,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:47.940213+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:32:47.940213+0000","last_peered":"2024-09-17T15:32:47.940213+0000","last_clean":"2024-09-17T15:32:47.940213+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:32:47.940213+0000","last_undegraded":"2024-09-17T15:32:47.940213+0000","last_fullsized":"2024-09-17T15:32:47.940213+0000","mapping_epoch":32,"log_start":"41'5500","ondisk_log_start":"41'5500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21041635,"num_objects":5138,"num_object_clones":0,"num_object_copies":10276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5138,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12498,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'16137","reported_seq":16433,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:47.853825+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:32:47.853825+0000","last_peered":"2024-09-17T15:32:47.853825+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:32:47.853825+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'6100","ondisk_log_start":"41'6100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20947427,"num_objects":5115,"num_object_clones":0,"num_object_copies":10230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5115,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12636,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":923},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'15782","reported_seq":16095,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:47.932714+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:32:47.932714+0000","last_peered":"2024-09-17T15:32:47.932714+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:32:47.932714+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'5700","ondisk_log_start":"41'5700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20423622,"num_objects":4988,"num_object_clones":0,"num_object_copies":9976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4988,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12551,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":899},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'15826","reported_seq":15959,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:47.917183+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:32:47.917183+0000","last_peered":"2024-09-17T15:32:47.917183+0000","last_clean":"2024-09-17T15:32:47.917183+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:32:47.917183+0000","last_undegraded":"2024-09-17T15:32:47.917183+0000","last_fullsized":"2024-09-17T15:32:47.917183+0000","mapping_epoch":32,"log_start":"41'5800","ondisk_log_start":"41'5800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20848640,"num_objects":5090,"num_object_clones":0,"num_object_copies":10180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12780,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087024+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.087024+0000","last_peered":"2024-09-17T15:30:57.087024+0000","last_clean":"2024-09-17T15:30:57.087024+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.087024+0000","last_undegraded":"2024-09-17T15:30:57.087024+0000","last_fullsized":"2024-09-17T15:30:57.087024+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'15669","reported_seq":23957,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:47.339291+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:32:47.339291+0000","last_peered":"2024-09-17T15:32:47.339291+0000","last_clean":"2024-09-17T15:32:47.339291+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:32:47.339291+0000","last_undegraded":"2024-09-17T15:32:47.339291+0000","last_fullsized":"2024-09-17T15:32:47.339291+0000","mapping_epoch":42,"log_start":"41'5600","ondisk_log_start":"41'5600","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21086691,"num_objects":5149,"num_object_clones":0,"num_object_copies":10298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5149,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13669,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'15467","reported_seq":15737,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:47.917979+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:32:47.917979+0000","last_peered":"2024-09-17T15:32:47.917979+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:32:47.917979+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'5400","ondisk_log_start":"41'5400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20787200,"num_objects":5075,"num_object_clones":0,"num_object_copies":10150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5075,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12063,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'15510","reported_seq":15722,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:47.886457+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:32:47.886457+0000","last_peered":"2024-09-17T15:32:47.886457+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:32:47.886457+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'5500","ondisk_log_start":"41'5500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20517347,"num_objects":5010,"num_object_clones":0,"num_object_copies":10020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5010,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13405,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":977}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":83,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087587+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087587+0000","last_peered":"2024-09-17T15:30:57.087587+0000","last_clean":"2024-09-17T15:30:57.087587+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087587+0000","last_undegraded":"2024-09-17T15:30:57.087587+0000","last_fullsized":"2024-09-17T15:30:57.087587+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'15688","reported_seq":22449,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:47.885253+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:32:47.885253+0000","last_peered":"2024-09-17T15:32:47.885253+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:32:47.885253+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"41'5600","ondisk_log_start":"41'5600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20676608,"num_objects":5048,"num_object_clones":0,"num_object_copies":10096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":821,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13450,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3318,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4227}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'15868","reported_seq":16171,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:47.837993+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:32:47.837993+0000","last_peered":"2024-09-17T15:32:47.837993+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:32:47.837993+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'5800","ondisk_log_start":"41'5800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20881408,"num_objects":5098,"num_object_clones":0,"num_object_copies":10196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5098,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12416,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1041}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'15710","reported_seq":15981,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:47.953524+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:32:47.953524+0000","last_peered":"2024-09-17T15:32:47.953524+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:32:47.953524+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'5700","ondisk_log_start":"41'5700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20733952,"num_objects":5062,"num_object_clones":0,"num_object_copies":10124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11997,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":967}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'15771","reported_seq":16072,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:47.986918+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:32:47.986918+0000","last_peered":"2024-09-17T15:32:47.986918+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:32:47.986918+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'5700","ondisk_log_start":"41'5700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20951040,"num_objects":5115,"num_object_clones":0,"num_object_copies":10230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12558,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1135}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087804+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087804+0000","last_peered":"2024-09-17T15:30:57.087804+0000","last_clean":"2024-09-17T15:30:57.087804+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087804+0000","last_undegraded":"2024-09-17T15:30:57.087804+0000","last_fullsized":"2024-09-17T15:30:57.087804+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'15617","reported_seq":23996,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:47.443760+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:32:47.443760+0000","last_peered":"2024-09-17T15:32:47.443760+0000","last_clean":"2024-09-17T15:32:47.443760+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:32:47.443760+0000","last_undegraded":"2024-09-17T15:32:47.443760+0000","last_fullsized":"2024-09-17T15:32:47.443760+0000","mapping_epoch":44,"log_start":"41'5600","ondisk_log_start":"41'5600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20542406,"num_objects":5017,"num_object_clones":0,"num_object_copies":10034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12059,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":333263606,"num_objects":81373,"num_object_clones":0,"num_object_copies":162746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37330,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81373,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":211614,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12130,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":869564416,"data_stored":869418978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160779,"ondisk_log_size":160779,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051677,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":612224,"kb_used_data":295920,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93759616,"statfs":{"total":96636764160,"available":96009846784,"internally_reserved":0,"allocated":303022080,"data_stored":302730983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":323874476},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,1,0,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724622,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":578908,"kb_used_data":244544,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93792932,"statfs":{"total":96636764160,"available":96043962368,"internally_reserved":0,"allocated":250413056,"data_stored":250088978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":342355627},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247905,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":700056,"kb_used_data":311624,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93671784,"statfs":{"total":96636764160,"available":95919906816,"internally_reserved":0,"allocated":319102976,"data_stored":318811874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":397733544},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2,1,8,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":317554688,"data_stored":317518558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":249585664,"data_stored":249516558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":302424064,"data_stored":302383862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:32:52.782 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:32:53.352 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:32:53.352 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:32:53.376 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2024-09-17T15:32:51.733900+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":333263625,"num_objects":81375,"num_object_clones":0,"num_object_copies":162750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37330,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81375,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":211616,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12131,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160782,"ondisk_log_size":160782,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1891188,"kb_used_data":852088,"kb_used_omap":13,"kb_used_meta":1039026,"kb_avail":281224332,"statfs":{"total":289910292480,"available":287973715968,"internally_reserved":0,"allocated":872538112,"data_stored":871631835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1063963647},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,3,5,14,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20230144,"num_objects":-4939,"num_object_clones":0,"num_object_copies":-9878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-295,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4939,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4939,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":295,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004618"},"pg_stats":[{"pgid":"2.f","version":"45'15594","reported_seq":15849,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:47.784017+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:32:47.784017+0000","last_peered":"2024-09-17T15:32:47.784017+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:32:47.784017+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'5500","ondisk_log_start":"41'5500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20701184,"num_objects":5054,"num_object_clones":0,"num_object_copies":10108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5054,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15594,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'15739","reported_seq":15847,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:47.766129+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:32:47.766129+0000","last_peered":"2024-09-17T15:32:47.766129+0000","last_clean":"2024-09-17T15:32:47.766129+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:32:47.766129+0000","last_undegraded":"2024-09-17T15:32:47.766129+0000","last_fullsized":"2024-09-17T15:32:47.766129+0000","mapping_epoch":33,"log_start":"41'5700","ondisk_log_start":"41'5700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21271494,"num_objects":5195,"num_object_clones":0,"num_object_copies":10390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5195,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14096,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'15870","reported_seq":16088,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:47.816697+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:32:47.816697+0000","last_peered":"2024-09-17T15:32:47.816697+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:32:47.816697+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'5800","ondisk_log_start":"41'5800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20738072,"num_objects":5064,"num_object_clones":0,"num_object_copies":10128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5064,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14027,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":939}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'15815","reported_seq":15890,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:47.830060+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:32:47.830060+0000","last_peered":"2024-09-17T15:32:47.830060+0000","last_clean":"2024-09-17T15:32:47.830060+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:32:47.830060+0000","last_undegraded":"2024-09-17T15:32:47.830060+0000","last_fullsized":"2024-09-17T15:32:47.830060+0000","mapping_epoch":33,"log_start":"41'5800","ondisk_log_start":"41'5800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21114880,"num_objects":5155,"num_object_clones":0,"num_object_copies":10310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15815,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'15516","reported_seq":15650,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:47.940213+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:32:47.940213+0000","last_peered":"2024-09-17T15:32:47.940213+0000","last_clean":"2024-09-17T15:32:47.940213+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:32:47.940213+0000","last_undegraded":"2024-09-17T15:32:47.940213+0000","last_fullsized":"2024-09-17T15:32:47.940213+0000","mapping_epoch":32,"log_start":"41'5500","ondisk_log_start":"41'5500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21041635,"num_objects":5138,"num_object_clones":0,"num_object_copies":10276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5138,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12498,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'16137","reported_seq":16433,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:47.853825+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:32:47.853825+0000","last_peered":"2024-09-17T15:32:47.853825+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:32:47.853825+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'6100","ondisk_log_start":"41'6100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20947427,"num_objects":5115,"num_object_clones":0,"num_object_copies":10230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5115,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12636,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":923},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'15782","reported_seq":16095,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:47.932714+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:32:47.932714+0000","last_peered":"2024-09-17T15:32:47.932714+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:32:47.932714+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'5700","ondisk_log_start":"41'5700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20423622,"num_objects":4988,"num_object_clones":0,"num_object_copies":9976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4988,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12551,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":899},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'15826","reported_seq":15959,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:47.917183+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:32:47.917183+0000","last_peered":"2024-09-17T15:32:47.917183+0000","last_clean":"2024-09-17T15:32:47.917183+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:32:47.917183+0000","last_undegraded":"2024-09-17T15:32:47.917183+0000","last_fullsized":"2024-09-17T15:32:47.917183+0000","mapping_epoch":32,"log_start":"41'5800","ondisk_log_start":"41'5800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20848640,"num_objects":5090,"num_object_clones":0,"num_object_copies":10180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12780,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087024+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.087024+0000","last_peered":"2024-09-17T15:30:57.087024+0000","last_clean":"2024-09-17T15:30:57.087024+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.087024+0000","last_undegraded":"2024-09-17T15:30:57.087024+0000","last_fullsized":"2024-09-17T15:30:57.087024+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'15669","reported_seq":23957,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:47.339291+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:32:47.339291+0000","last_peered":"2024-09-17T15:32:47.339291+0000","last_clean":"2024-09-17T15:32:47.339291+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:32:47.339291+0000","last_undegraded":"2024-09-17T15:32:47.339291+0000","last_fullsized":"2024-09-17T15:32:47.339291+0000","mapping_epoch":42,"log_start":"41'5600","ondisk_log_start":"41'5600","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21086691,"num_objects":5149,"num_object_clones":0,"num_object_copies":10298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5149,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13669,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'15467","reported_seq":15737,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:47.917979+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:32:47.917979+0000","last_peered":"2024-09-17T15:32:47.917979+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:32:47.917979+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'5400","ondisk_log_start":"41'5400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20787200,"num_objects":5075,"num_object_clones":0,"num_object_copies":10150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5075,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12063,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'15510","reported_seq":15722,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:47.886457+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:32:47.886457+0000","last_peered":"2024-09-17T15:32:47.886457+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:32:47.886457+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'5500","ondisk_log_start":"41'5500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20517347,"num_objects":5010,"num_object_clones":0,"num_object_copies":10020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5010,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13405,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":977}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":83,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087587+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087587+0000","last_peered":"2024-09-17T15:30:57.087587+0000","last_clean":"2024-09-17T15:30:57.087587+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087587+0000","last_undegraded":"2024-09-17T15:30:57.087587+0000","last_fullsized":"2024-09-17T15:30:57.087587+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'15688","reported_seq":22449,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:47.885253+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:32:47.885253+0000","last_peered":"2024-09-17T15:32:47.885253+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:32:47.885253+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"41'5600","ondisk_log_start":"41'5600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20676608,"num_objects":5048,"num_object_clones":0,"num_object_copies":10096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":821,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13450,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3318,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4227}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'15868","reported_seq":16171,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:47.837993+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:32:47.837993+0000","last_peered":"2024-09-17T15:32:47.837993+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:32:47.837993+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'5800","ondisk_log_start":"41'5800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20881408,"num_objects":5098,"num_object_clones":0,"num_object_copies":10196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5098,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12416,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1041}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'15710","reported_seq":15981,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:47.953524+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:32:47.953524+0000","last_peered":"2024-09-17T15:32:47.953524+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:32:47.953524+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'5700","ondisk_log_start":"41'5700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20733952,"num_objects":5062,"num_object_clones":0,"num_object_copies":10124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11997,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":967}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'15771","reported_seq":16072,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:47.986918+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:32:47.986918+0000","last_peered":"2024-09-17T15:32:47.986918+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:32:47.986918+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'5700","ondisk_log_start":"41'5700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20951040,"num_objects":5115,"num_object_clones":0,"num_object_copies":10230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12558,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1135}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087804+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087804+0000","last_peered":"2024-09-17T15:30:57.087804+0000","last_clean":"2024-09-17T15:30:57.087804+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087804+0000","last_undegraded":"2024-09-17T15:30:57.087804+0000","last_fullsized":"2024-09-17T15:30:57.087804+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088829+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088829+0000","last_peered":"2024-09-17T15:30:57.088829+0000","last_clean":"2024-09-17T15:30:57.088829+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088829+0000","last_undegraded":"2024-09-17T15:30:57.088829+0000","last_fullsized":"2024-09-17T15:30:57.088829+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'15617","reported_seq":23996,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:47.443760+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:32:47.443760+0000","last_peered":"2024-09-17T15:32:47.443760+0000","last_clean":"2024-09-17T15:32:47.443760+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:32:47.443760+0000","last_undegraded":"2024-09-17T15:32:47.443760+0000","last_fullsized":"2024-09-17T15:32:47.443760+0000","mapping_epoch":44,"log_start":"41'5600","ondisk_log_start":"41'5600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20542406,"num_objects":5017,"num_object_clones":0,"num_object_copies":10034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12059,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":333263606,"num_objects":81373,"num_object_clones":0,"num_object_copies":162746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37330,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81373,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":211614,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12130,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":869564416,"data_stored":869418978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160779,"ondisk_log_size":160779,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051677,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":612224,"kb_used_data":295920,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93759616,"statfs":{"total":96636764160,"available":96009846784,"internally_reserved":0,"allocated":303022080,"data_stored":302730983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":323874476},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,1,0,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724622,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":578908,"kb_used_data":244544,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93792932,"statfs":{"total":96636764160,"available":96043962368,"internally_reserved":0,"allocated":250413056,"data_stored":250088978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":342355627},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247905,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":700056,"kb_used_data":311624,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93671784,"statfs":{"total":96636764160,"available":95919906816,"internally_reserved":0,"allocated":319102976,"data_stored":318811874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":397733544},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2,1,8,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":317554688,"data_stored":317518558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":249585664,"data_stored":249516558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":302424064,"data_stored":302383862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:32:53.377 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:32:53.377 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'15594", 'reported_seq': 15849, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:47.784017+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:32:47.784017+0000', 'last_peered': '2024-09-17T15:32:47.784017+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:32:47.784017+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "41'5500", 'ondisk_log_start': "41'5500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20701184, 'num_objects': 5054, 'num_object_clones': 0, 'num_object_copies': 10108, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5054, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15594, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1070}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:53.377 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:32:53.377 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'15870", 'reported_seq': 16088, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:47.816697+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:32:47.816697+0000', 'last_peered': '2024-09-17T15:32:47.816697+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:32:47.816697+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "41'5800", 'ondisk_log_start': "41'5800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20738072, 'num_objects': 5064, 'num_object_clones': 0, 'num_object_copies': 10128, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5064, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14027, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 939}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:32:53.378 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:32:53.378 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'16137", 'reported_seq': 16433, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:47.853825+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:32:47.853825+0000', 'last_peered': '2024-09-17T15:32:47.853825+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:32:47.853825+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "41'6100", 'ondisk_log_start': "41'6100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20947427, 'num_objects': 5115, 'num_object_clones': 0, 'num_object_copies': 10230, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5115, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12636, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 923}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:53.378 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:32:53.378 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'15782", 'reported_seq': 16095, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:47.932714+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:32:47.932714+0000', 'last_peered': '2024-09-17T15:32:47.932714+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:32:47.932714+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "41'5700", 'ondisk_log_start': "41'5700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20423622, 'num_objects': 4988, 'num_object_clones': 0, 'num_object_copies': 9976, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4089, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4988, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 12551, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 85, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 899}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:53.378 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:32:53.378 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'15467", 'reported_seq': 15737, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:47.917979+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:32:47.917979+0000', 'last_peered': '2024-09-17T15:32:47.917979+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:32:47.917979+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "41'5400", 'ondisk_log_start': "41'5400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20787200, 'num_objects': 5075, 'num_object_clones': 0, 'num_object_copies': 10150, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5075, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12063, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:53.378 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:32:53.378 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'15510", 'reported_seq': 15722, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:47.886457+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:32:47.886457+0000', 'last_peered': '2024-09-17T15:32:47.886457+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:32:47.886457+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "41'5500", 'ondisk_log_start': "41'5500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20517347, 'num_objects': 5010, 'num_object_clones': 0, 'num_object_copies': 10020, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4034, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5010, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13405, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 977}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:32:53.378 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:32:53.378 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'15688", 'reported_seq': 22449, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:47.885253+0000', 'last_change': '2024-09-17T15:30:55.086473+0000', 'last_active': '2024-09-17T15:32:47.885253+0000', 'last_peered': '2024-09-17T15:32:47.885253+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:32:47.885253+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "41'5600", 'ondisk_log_start': "41'5600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20676608, 'num_objects': 5048, 'num_object_clones': 0, 'num_object_copies': 10096, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 821, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5048, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13450, 'num_write_kb': 32532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3318, 'num_bytes_recovered': 8073216, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 4227}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:32:53.378 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:32:53.379 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'15868", 'reported_seq': 16171, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:47.837993+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:32:47.837993+0000', 'last_peered': '2024-09-17T15:32:47.837993+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:32:47.837993+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "41'5800", 'ondisk_log_start': "41'5800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20881408, 'num_objects': 5098, 'num_object_clones': 0, 'num_object_copies': 10196, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5098, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12416, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1041}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:53.379 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:32:53.379 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'15710", 'reported_seq': 15981, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:47.953524+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:32:47.953524+0000', 'last_peered': '2024-09-17T15:32:47.953524+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:32:47.953524+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "41'5700", 'ondisk_log_start': "41'5700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20733952, 'num_objects': 5062, 'num_object_clones': 0, 'num_object_copies': 10124, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4098, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5062, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11997, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 62, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 967}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:53.379 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:32:53.379 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'15771", 'reported_seq': 16072, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:47.986918+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:32:47.986918+0000', 'last_peered': '2024-09-17T15:32:47.986918+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:32:47.986918+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "41'5700", 'ondisk_log_start': "41'5700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20951040, 'num_objects': 5115, 'num_object_clones': 0, 'num_object_copies': 10230, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5115, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12558, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1135}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:53.379 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:32:54.054 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:32:54.054 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37330/162750 objects degraded (22.937%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":511,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":81375,"data_bytes":333263625,"bytes_used":1936576512,"bytes_avail":287973715968,"bytes_total":289910292480,"degraded_objects":37330,"degraded_total":162750,"degraded_ratio":0.22937019969278033,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":295,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":411},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-17T15:31:29.695933+0000","services":{}},"progress_events":{}} 2024-09-17T15:32:54.076 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:32:54.076 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:32:54.615 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:32:54.618 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:32:54.645 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-09-17T15:32:53.734615+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":323568393,"num_objects":79008,"num_object_clones":0,"num_object_copies":158016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37143,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79008,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":213983,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12318,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160749,"ondisk_log_size":160749,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1892532,"kb_used_data":832640,"kb_used_omap":13,"kb_used_meta":1059826,"kb_avail":281222988,"statfs":{"total":289910292480,"available":287972339712,"internally_reserved":0,"allocated":852623360,"data_stored":851717083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1085262847},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,12,6,12,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29925376,"num_objects":-7306,"num_object_clones":0,"num_object_copies":-14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-482,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":482,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004561"},"pg_stats":[{"pgid":"2.f","version":"45'15748","reported_seq":16003,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:52.930304+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:32:52.930304+0000","last_peered":"2024-09-17T15:32:52.930304+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:32:52.930304+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'5700","ondisk_log_start":"41'5700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20070400,"num_objects":4900,"num_object_clones":0,"num_object_copies":9800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15748,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":916}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'15901","reported_seq":16008,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:52.882125+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:32:52.882125+0000","last_peered":"2024-09-17T15:32:52.882125+0000","last_clean":"2024-09-17T15:32:52.882125+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:32:52.882125+0000","last_undegraded":"2024-09-17T15:32:52.882125+0000","last_fullsized":"2024-09-17T15:32:52.882125+0000","mapping_epoch":33,"log_start":"41'5900","ondisk_log_start":"41'5900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20607942,"num_objects":5033,"num_object_clones":0,"num_object_copies":10066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5033,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14258,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'15999","reported_seq":16216,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:52.888002+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:32:52.888002+0000","last_peered":"2024-09-17T15:32:52.888002+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:32:52.888002+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'5900","ondisk_log_start":"41'5900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20209688,"num_objects":4935,"num_object_clones":0,"num_object_copies":9870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4935,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14156,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":810}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'15971","reported_seq":16046,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:52.874492+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:32:52.874492+0000","last_peered":"2024-09-17T15:32:52.874492+0000","last_clean":"2024-09-17T15:32:52.874492+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:32:52.874492+0000","last_undegraded":"2024-09-17T15:32:52.874492+0000","last_fullsized":"2024-09-17T15:32:52.874492+0000","mapping_epoch":33,"log_start":"41'5900","ondisk_log_start":"41'5900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20475904,"num_objects":4999,"num_object_clones":0,"num_object_copies":9998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15971,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'15653","reported_seq":15788,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:52.953371+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:32:52.953371+0000","last_peered":"2024-09-17T15:32:52.953371+0000","last_clean":"2024-09-17T15:32:52.953371+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:32:52.953371+0000","last_undegraded":"2024-09-17T15:32:52.953371+0000","last_fullsized":"2024-09-17T15:32:52.953371+0000","mapping_epoch":32,"log_start":"41'5600","ondisk_log_start":"41'5600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20480483,"num_objects":5001,"num_object_clones":0,"num_object_copies":10002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5001,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'16259","reported_seq":16555,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:52.943708+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:32:52.943708+0000","last_peered":"2024-09-17T15:32:52.943708+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:32:52.943708+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'6200","ondisk_log_start":"41'6200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20447715,"num_objects":4993,"num_object_clones":0,"num_object_copies":9986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4993,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12758,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":801},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'15925","reported_seq":16239,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:52.780119+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:32:52.780119+0000","last_peered":"2024-09-17T15:32:52.780119+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:32:52.780119+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'5900","ondisk_log_start":"41'5900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19837894,"num_objects":4845,"num_object_clones":0,"num_object_copies":9690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4845,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12694,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":756},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'15964","reported_seq":16097,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:52.991437+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:32:52.991437+0000","last_peered":"2024-09-17T15:32:52.991437+0000","last_clean":"2024-09-17T15:32:52.991437+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:32:52.991437+0000","last_undegraded":"2024-09-17T15:32:52.991437+0000","last_fullsized":"2024-09-17T15:32:52.991437+0000","mapping_epoch":32,"log_start":"41'5900","ondisk_log_start":"41'5900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20283392,"num_objects":4952,"num_object_clones":0,"num_object_copies":9904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12918,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087024+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.087024+0000","last_peered":"2024-09-17T15:30:57.087024+0000","last_clean":"2024-09-17T15:30:57.087024+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.087024+0000","last_undegraded":"2024-09-17T15:30:57.087024+0000","last_fullsized":"2024-09-17T15:30:57.087024+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'15801","reported_seq":24089,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:52.211913+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:32:52.211913+0000","last_peered":"2024-09-17T15:32:52.211913+0000","last_clean":"2024-09-17T15:32:52.211913+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:32:52.211913+0000","last_undegraded":"2024-09-17T15:32:52.211913+0000","last_fullsized":"2024-09-17T15:32:52.211913+0000","mapping_epoch":42,"log_start":"41'5800","ondisk_log_start":"41'5800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20546019,"num_objects":5017,"num_object_clones":0,"num_object_copies":10034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5017,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13801,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'15609","reported_seq":15879,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:52.811510+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:32:52.811510+0000","last_peered":"2024-09-17T15:32:52.811510+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:32:52.811510+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'5600","ondisk_log_start":"41'5600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20205568,"num_objects":4933,"num_object_clones":0,"num_object_copies":9866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4933,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12205,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":990}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'15672","reported_seq":15886,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:52.908631+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:32:52.908631+0000","last_peered":"2024-09-17T15:32:52.908631+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:32:52.908631+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'5600","ondisk_log_start":"41'5600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19853795,"num_objects":4848,"num_object_clones":0,"num_object_copies":9696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4848,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13567,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":815}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":83,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087587+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087587+0000","last_peered":"2024-09-17T15:30:57.087587+0000","last_clean":"2024-09-17T15:30:57.087587+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087587+0000","last_undegraded":"2024-09-17T15:30:57.087587+0000","last_fullsized":"2024-09-17T15:30:57.087587+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'15833","reported_seq":22968,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:52.889871+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:32:52.889871+0000","last_peered":"2024-09-17T15:32:52.889871+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:32:52.889871+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"41'5800","ondisk_log_start":"41'5800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20082688,"num_objects":4903,"num_object_clones":0,"num_object_copies":9806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":634,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13595,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3505,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4269}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'16025","reported_seq":16328,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:52.914648+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:32:52.914648+0000","last_peered":"2024-09-17T15:32:52.914648+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:32:52.914648+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'6000","ondisk_log_start":"41'6000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20238336,"num_objects":4941,"num_object_clones":0,"num_object_copies":9882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12573,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":884}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'15867","reported_seq":16138,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:52.922428+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:32:52.922428+0000","last_peered":"2024-09-17T15:32:52.922428+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:32:52.922428+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'5800","ondisk_log_start":"41'5800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20090880,"num_objects":4905,"num_object_clones":0,"num_object_copies":9810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12154,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":810}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'15934","reported_seq":16235,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:52.889458+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:32:52.889458+0000","last_peered":"2024-09-17T15:32:52.889458+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:32:52.889458+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'5900","ondisk_log_start":"41'5900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20283392,"num_objects":4952,"num_object_clones":0,"num_object_copies":9904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12721,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":972}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087804+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087804+0000","last_peered":"2024-09-17T15:30:57.087804+0000","last_clean":"2024-09-17T15:30:57.087804+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087804+0000","last_undegraded":"2024-09-17T15:30:57.087804+0000","last_fullsized":"2024-09-17T15:30:57.087804+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088875+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088875+0000","last_peered":"2024-09-17T15:30:57.088875+0000","last_clean":"2024-09-17T15:30:57.088875+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088875+0000","last_undegraded":"2024-09-17T15:30:57.088875+0000","last_fullsized":"2024-09-17T15:30:57.088875+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'15785","reported_seq":24164,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:52.269949+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:32:52.269949+0000","last_peered":"2024-09-17T15:32:52.269949+0000","last_clean":"2024-09-17T15:32:52.269949+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:32:52.269949+0000","last_undegraded":"2024-09-17T15:32:52.269949+0000","last_fullsized":"2024-09-17T15:32:52.269949+0000","mapping_epoch":44,"log_start":"41'5700","ondisk_log_start":"41'5700","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19854278,"num_objects":4849,"num_object_clones":0,"num_object_copies":9698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4849,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12227,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":323568374,"num_objects":79006,"num_object_clones":0,"num_object_copies":158012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37143,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79006,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":213981,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12317,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":850149376,"data_stored":850003938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160746,"ondisk_log_size":160746,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051678,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":611744,"kb_used_data":291276,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93760096,"statfs":{"total":96636764160,"available":96010338304,"internally_reserved":0,"allocated":298266624,"data_stored":297975527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":328134316},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,0,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724623,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":579824,"kb_used_data":237152,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93792016,"statfs":{"total":96636764160,"available":96043024384,"internally_reserved":0,"allocated":242843648,"data_stored":242519570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":350875307},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,4,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247906,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":700964,"kb_used_data":304212,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93670876,"statfs":{"total":96636764160,"available":95918977024,"internally_reserved":0,"allocated":311513088,"data_stored":311221986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":406253224},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,6,2,5,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":310173696,"data_stored":310137566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":242298880,"data_stored":242229774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":297676800,"data_stored":297636598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:32:57.647 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:32:58.244 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:32:58.245 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:32:58.271 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2024-09-17T15:32:57.736217+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":322257673,"num_objects":78688,"num_object_clones":0,"num_object_copies":157376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37143,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78688,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":214303,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12318,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160769,"ondisk_log_size":160769,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1907660,"kb_used_data":825552,"kb_used_omap":13,"kb_used_meta":1082034,"kb_avail":281207860,"statfs":{"total":289910292480,"available":287956848640,"internally_reserved":0,"allocated":845365248,"data_stored":844458971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1108003839},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,10,5,10,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21114880,"num_objects":-5155,"num_object_clones":0,"num_object_copies":-10310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5155,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":343,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004820"},"pg_stats":[{"pgid":"2.f","version":"45'15748","reported_seq":16003,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:52.930304+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:32:52.930304+0000","last_peered":"2024-09-17T15:32:52.930304+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:32:52.930304+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'5700","ondisk_log_start":"41'5700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20070400,"num_objects":4900,"num_object_clones":0,"num_object_copies":9800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15748,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":916}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'15901","reported_seq":16008,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:52.882125+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:32:52.882125+0000","last_peered":"2024-09-17T15:32:52.882125+0000","last_clean":"2024-09-17T15:32:52.882125+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:32:52.882125+0000","last_undegraded":"2024-09-17T15:32:52.882125+0000","last_fullsized":"2024-09-17T15:32:52.882125+0000","mapping_epoch":33,"log_start":"41'5900","ondisk_log_start":"41'5900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20607942,"num_objects":5033,"num_object_clones":0,"num_object_copies":10066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5033,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14258,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'15999","reported_seq":16216,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:52.888002+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:32:52.888002+0000","last_peered":"2024-09-17T15:32:52.888002+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:32:52.888002+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'5900","ondisk_log_start":"41'5900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20209688,"num_objects":4935,"num_object_clones":0,"num_object_copies":9870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4935,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14156,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":810}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'15971","reported_seq":16046,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:52.874492+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:32:52.874492+0000","last_peered":"2024-09-17T15:32:52.874492+0000","last_clean":"2024-09-17T15:32:52.874492+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:32:52.874492+0000","last_undegraded":"2024-09-17T15:32:52.874492+0000","last_fullsized":"2024-09-17T15:32:52.874492+0000","mapping_epoch":33,"log_start":"41'5900","ondisk_log_start":"41'5900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20475904,"num_objects":4999,"num_object_clones":0,"num_object_copies":9998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15971,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'15653","reported_seq":15788,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:52.953371+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:32:52.953371+0000","last_peered":"2024-09-17T15:32:52.953371+0000","last_clean":"2024-09-17T15:32:52.953371+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:32:52.953371+0000","last_undegraded":"2024-09-17T15:32:52.953371+0000","last_fullsized":"2024-09-17T15:32:52.953371+0000","mapping_epoch":32,"log_start":"41'5600","ondisk_log_start":"41'5600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20480483,"num_objects":5001,"num_object_clones":0,"num_object_copies":10002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5001,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'16259","reported_seq":16555,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:52.943708+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:32:52.943708+0000","last_peered":"2024-09-17T15:32:52.943708+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:32:52.943708+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'6200","ondisk_log_start":"41'6200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20447715,"num_objects":4993,"num_object_clones":0,"num_object_copies":9986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4993,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12758,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":801},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'15925","reported_seq":16239,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:52.780119+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:32:52.780119+0000","last_peered":"2024-09-17T15:32:52.780119+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:32:52.780119+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'5900","ondisk_log_start":"41'5900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19837894,"num_objects":4845,"num_object_clones":0,"num_object_copies":9690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4845,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12694,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":756},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'15964","reported_seq":16097,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:52.991437+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:32:52.991437+0000","last_peered":"2024-09-17T15:32:52.991437+0000","last_clean":"2024-09-17T15:32:52.991437+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:32:52.991437+0000","last_undegraded":"2024-09-17T15:32:52.991437+0000","last_fullsized":"2024-09-17T15:32:52.991437+0000","mapping_epoch":32,"log_start":"41'5900","ondisk_log_start":"41'5900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20283392,"num_objects":4952,"num_object_clones":0,"num_object_copies":9904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12918,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087024+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.087024+0000","last_peered":"2024-09-17T15:30:57.087024+0000","last_clean":"2024-09-17T15:30:57.087024+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.087024+0000","last_undegraded":"2024-09-17T15:30:57.087024+0000","last_fullsized":"2024-09-17T15:30:57.087024+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'15972","reported_seq":24260,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:57.427756+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:32:57.427756+0000","last_peered":"2024-09-17T15:32:57.427756+0000","last_clean":"2024-09-17T15:32:57.427756+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:32:57.427756+0000","last_undegraded":"2024-09-17T15:32:57.427756+0000","last_fullsized":"2024-09-17T15:32:57.427756+0000","mapping_epoch":42,"log_start":"41'5900","ondisk_log_start":"41'5900","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19845603,"num_objects":4846,"num_object_clones":0,"num_object_copies":9692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4846,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13972,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'15609","reported_seq":15879,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:52.811510+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:32:52.811510+0000","last_peered":"2024-09-17T15:32:52.811510+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:32:52.811510+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'5600","ondisk_log_start":"41'5600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20205568,"num_objects":4933,"num_object_clones":0,"num_object_copies":9866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4933,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12205,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":990}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'15672","reported_seq":15886,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:52.908631+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:32:52.908631+0000","last_peered":"2024-09-17T15:32:52.908631+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:32:52.908631+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'5600","ondisk_log_start":"41'5600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19853795,"num_objects":4848,"num_object_clones":0,"num_object_copies":9696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4848,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13567,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":815}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":83,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087587+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087587+0000","last_peered":"2024-09-17T15:30:57.087587+0000","last_clean":"2024-09-17T15:30:57.087587+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087587+0000","last_undegraded":"2024-09-17T15:30:57.087587+0000","last_fullsized":"2024-09-17T15:30:57.087587+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'15833","reported_seq":22968,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:52.889871+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:32:52.889871+0000","last_peered":"2024-09-17T15:32:52.889871+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:32:52.889871+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"41'5800","ondisk_log_start":"41'5800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20082688,"num_objects":4903,"num_object_clones":0,"num_object_copies":9806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":634,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13595,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3505,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4269}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'16025","reported_seq":16328,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:52.914648+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:32:52.914648+0000","last_peered":"2024-09-17T15:32:52.914648+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:32:52.914648+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'6000","ondisk_log_start":"41'6000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20238336,"num_objects":4941,"num_object_clones":0,"num_object_copies":9882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12573,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":884}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'15867","reported_seq":16138,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:52.922428+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:32:52.922428+0000","last_peered":"2024-09-17T15:32:52.922428+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:32:52.922428+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'5800","ondisk_log_start":"41'5800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20090880,"num_objects":4905,"num_object_clones":0,"num_object_copies":9810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12154,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":810}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'15934","reported_seq":16235,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:52.889458+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:32:52.889458+0000","last_peered":"2024-09-17T15:32:52.889458+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:32:52.889458+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'5900","ondisk_log_start":"41'5900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20283392,"num_objects":4952,"num_object_clones":0,"num_object_copies":9904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12721,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":972}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087804+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087804+0000","last_peered":"2024-09-17T15:30:57.087804+0000","last_clean":"2024-09-17T15:30:57.087804+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087804+0000","last_undegraded":"2024-09-17T15:30:57.087804+0000","last_fullsized":"2024-09-17T15:30:57.087804+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088875+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088875+0000","last_peered":"2024-09-17T15:30:57.088875+0000","last_clean":"2024-09-17T15:30:57.088875+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088875+0000","last_undegraded":"2024-09-17T15:30:57.088875+0000","last_fullsized":"2024-09-17T15:30:57.088875+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'15934","reported_seq":24313,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:57.338179+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:32:57.338179+0000","last_peered":"2024-09-17T15:32:57.338179+0000","last_clean":"2024-09-17T15:32:57.338179+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:32:57.338179+0000","last_undegraded":"2024-09-17T15:32:57.338179+0000","last_fullsized":"2024-09-17T15:32:57.338179+0000","mapping_epoch":44,"log_start":"41'5900","ondisk_log_start":"41'5900","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19243974,"num_objects":4700,"num_object_clones":0,"num_object_copies":9400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12376,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":322257654,"num_objects":78686,"num_object_clones":0,"num_object_copies":157372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37143,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78686,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":214301,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12317,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":842698752,"data_stored":842553314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160766,"ondisk_log_size":160766,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051678,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":611744,"kb_used_data":291276,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93760096,"statfs":{"total":96636764160,"available":96010338304,"internally_reserved":0,"allocated":298266624,"data_stored":297975527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":328134316},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,0,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724624,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594952,"kb_used_data":230064,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93776888,"statfs":{"total":96636764160,"available":96027533312,"internally_reserved":0,"allocated":235585536,"data_stored":235261458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":373616299},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,3,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247906,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":700964,"kb_used_data":304212,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93670876,"statfs":{"total":96636764160,"available":95918977024,"internally_reserved":0,"allocated":311513088,"data_stored":311221986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":406253224},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,6,2,5,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":310173696,"data_stored":310137566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":234848256,"data_stored":234779150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":297676800,"data_stored":297636598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:32:58.272 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:32:58.841 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:32:58.843 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:32:58.870 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2024-09-17T15:32:57.736217+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":322257673,"num_objects":78688,"num_object_clones":0,"num_object_copies":157376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37143,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78688,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":214303,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12318,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160769,"ondisk_log_size":160769,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1907660,"kb_used_data":825552,"kb_used_omap":13,"kb_used_meta":1082034,"kb_avail":281207860,"statfs":{"total":289910292480,"available":287956848640,"internally_reserved":0,"allocated":845365248,"data_stored":844458971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1108003839},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,10,5,10,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21114880,"num_objects":-5155,"num_object_clones":0,"num_object_copies":-10310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5155,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":343,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004820"},"pg_stats":[{"pgid":"2.f","version":"45'15748","reported_seq":16003,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:52.930304+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:32:52.930304+0000","last_peered":"2024-09-17T15:32:52.930304+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:32:52.930304+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'5700","ondisk_log_start":"41'5700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20070400,"num_objects":4900,"num_object_clones":0,"num_object_copies":9800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15748,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":916}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'15901","reported_seq":16008,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:52.882125+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:32:52.882125+0000","last_peered":"2024-09-17T15:32:52.882125+0000","last_clean":"2024-09-17T15:32:52.882125+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:32:52.882125+0000","last_undegraded":"2024-09-17T15:32:52.882125+0000","last_fullsized":"2024-09-17T15:32:52.882125+0000","mapping_epoch":33,"log_start":"41'5900","ondisk_log_start":"41'5900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20607942,"num_objects":5033,"num_object_clones":0,"num_object_copies":10066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5033,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14258,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'15999","reported_seq":16216,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:52.888002+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:32:52.888002+0000","last_peered":"2024-09-17T15:32:52.888002+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:32:52.888002+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'5900","ondisk_log_start":"41'5900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20209688,"num_objects":4935,"num_object_clones":0,"num_object_copies":9870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4935,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14156,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":810}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'15971","reported_seq":16046,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:52.874492+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:32:52.874492+0000","last_peered":"2024-09-17T15:32:52.874492+0000","last_clean":"2024-09-17T15:32:52.874492+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:32:52.874492+0000","last_undegraded":"2024-09-17T15:32:52.874492+0000","last_fullsized":"2024-09-17T15:32:52.874492+0000","mapping_epoch":33,"log_start":"41'5900","ondisk_log_start":"41'5900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20475904,"num_objects":4999,"num_object_clones":0,"num_object_copies":9998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15971,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'15653","reported_seq":15788,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:52.953371+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:32:52.953371+0000","last_peered":"2024-09-17T15:32:52.953371+0000","last_clean":"2024-09-17T15:32:52.953371+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:32:52.953371+0000","last_undegraded":"2024-09-17T15:32:52.953371+0000","last_fullsized":"2024-09-17T15:32:52.953371+0000","mapping_epoch":32,"log_start":"41'5600","ondisk_log_start":"41'5600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20480483,"num_objects":5001,"num_object_clones":0,"num_object_copies":10002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5001,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'16259","reported_seq":16555,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:52.943708+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:32:52.943708+0000","last_peered":"2024-09-17T15:32:52.943708+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:32:52.943708+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'6200","ondisk_log_start":"41'6200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20447715,"num_objects":4993,"num_object_clones":0,"num_object_copies":9986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4993,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12758,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":801},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'15925","reported_seq":16239,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:52.780119+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:32:52.780119+0000","last_peered":"2024-09-17T15:32:52.780119+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:32:52.780119+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'5900","ondisk_log_start":"41'5900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19837894,"num_objects":4845,"num_object_clones":0,"num_object_copies":9690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4845,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12694,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":756},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'15964","reported_seq":16097,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:52.991437+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:32:52.991437+0000","last_peered":"2024-09-17T15:32:52.991437+0000","last_clean":"2024-09-17T15:32:52.991437+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:32:52.991437+0000","last_undegraded":"2024-09-17T15:32:52.991437+0000","last_fullsized":"2024-09-17T15:32:52.991437+0000","mapping_epoch":32,"log_start":"41'5900","ondisk_log_start":"41'5900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20283392,"num_objects":4952,"num_object_clones":0,"num_object_copies":9904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12918,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087024+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.087024+0000","last_peered":"2024-09-17T15:30:57.087024+0000","last_clean":"2024-09-17T15:30:57.087024+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.087024+0000","last_undegraded":"2024-09-17T15:30:57.087024+0000","last_fullsized":"2024-09-17T15:30:57.087024+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'15972","reported_seq":24260,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:57.427756+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:32:57.427756+0000","last_peered":"2024-09-17T15:32:57.427756+0000","last_clean":"2024-09-17T15:32:57.427756+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:32:57.427756+0000","last_undegraded":"2024-09-17T15:32:57.427756+0000","last_fullsized":"2024-09-17T15:32:57.427756+0000","mapping_epoch":42,"log_start":"41'5900","ondisk_log_start":"41'5900","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19845603,"num_objects":4846,"num_object_clones":0,"num_object_copies":9692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4846,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13972,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'15609","reported_seq":15879,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:52.811510+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:32:52.811510+0000","last_peered":"2024-09-17T15:32:52.811510+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:32:52.811510+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'5600","ondisk_log_start":"41'5600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20205568,"num_objects":4933,"num_object_clones":0,"num_object_copies":9866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4933,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12205,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":990}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'15672","reported_seq":15886,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:52.908631+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:32:52.908631+0000","last_peered":"2024-09-17T15:32:52.908631+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:32:52.908631+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'5600","ondisk_log_start":"41'5600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19853795,"num_objects":4848,"num_object_clones":0,"num_object_copies":9696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4848,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13567,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":815}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":83,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087587+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087587+0000","last_peered":"2024-09-17T15:30:57.087587+0000","last_clean":"2024-09-17T15:30:57.087587+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087587+0000","last_undegraded":"2024-09-17T15:30:57.087587+0000","last_fullsized":"2024-09-17T15:30:57.087587+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'15833","reported_seq":22968,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:52.889871+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:32:52.889871+0000","last_peered":"2024-09-17T15:32:52.889871+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:32:52.889871+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"41'5800","ondisk_log_start":"41'5800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20082688,"num_objects":4903,"num_object_clones":0,"num_object_copies":9806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":634,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13595,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3505,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4269}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'16025","reported_seq":16328,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:52.914648+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:32:52.914648+0000","last_peered":"2024-09-17T15:32:52.914648+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:32:52.914648+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'6000","ondisk_log_start":"41'6000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20238336,"num_objects":4941,"num_object_clones":0,"num_object_copies":9882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12573,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":884}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'15867","reported_seq":16138,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:52.922428+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:32:52.922428+0000","last_peered":"2024-09-17T15:32:52.922428+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:32:52.922428+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'5800","ondisk_log_start":"41'5800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20090880,"num_objects":4905,"num_object_clones":0,"num_object_copies":9810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12154,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":810}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'15934","reported_seq":16235,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:52.889458+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:32:52.889458+0000","last_peered":"2024-09-17T15:32:52.889458+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:32:52.889458+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'5900","ondisk_log_start":"41'5900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20283392,"num_objects":4952,"num_object_clones":0,"num_object_copies":9904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12721,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":972}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087804+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087804+0000","last_peered":"2024-09-17T15:30:57.087804+0000","last_clean":"2024-09-17T15:30:57.087804+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087804+0000","last_undegraded":"2024-09-17T15:30:57.087804+0000","last_fullsized":"2024-09-17T15:30:57.087804+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088875+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088875+0000","last_peered":"2024-09-17T15:30:57.088875+0000","last_clean":"2024-09-17T15:30:57.088875+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088875+0000","last_undegraded":"2024-09-17T15:30:57.088875+0000","last_fullsized":"2024-09-17T15:30:57.088875+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'15934","reported_seq":24313,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:57.338179+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:32:57.338179+0000","last_peered":"2024-09-17T15:32:57.338179+0000","last_clean":"2024-09-17T15:32:57.338179+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:32:57.338179+0000","last_undegraded":"2024-09-17T15:32:57.338179+0000","last_fullsized":"2024-09-17T15:32:57.338179+0000","mapping_epoch":44,"log_start":"41'5900","ondisk_log_start":"41'5900","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19243974,"num_objects":4700,"num_object_clones":0,"num_object_copies":9400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12376,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":322257654,"num_objects":78686,"num_object_clones":0,"num_object_copies":157372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37143,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78686,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":214301,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12317,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":842698752,"data_stored":842553314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160766,"ondisk_log_size":160766,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051678,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":611744,"kb_used_data":291276,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93760096,"statfs":{"total":96636764160,"available":96010338304,"internally_reserved":0,"allocated":298266624,"data_stored":297975527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":328134316},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,0,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724624,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594952,"kb_used_data":230064,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93776888,"statfs":{"total":96636764160,"available":96027533312,"internally_reserved":0,"allocated":235585536,"data_stored":235261458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":373616299},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,3,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247906,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":700964,"kb_used_data":304212,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93670876,"statfs":{"total":96636764160,"available":95918977024,"internally_reserved":0,"allocated":311513088,"data_stored":311221986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":406253224},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,6,2,5,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":310173696,"data_stored":310137566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":234848256,"data_stored":234779150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":297676800,"data_stored":297636598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:32:58.871 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:32:58.872 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'15748", 'reported_seq': 16003, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:52.930304+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:32:52.930304+0000', 'last_peered': '2024-09-17T15:32:52.930304+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:32:52.930304+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "41'5700", 'ondisk_log_start': "41'5700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20070400, 'num_objects': 4900, 'num_object_clones': 0, 'num_object_copies': 9800, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4900, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15748, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 916}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:58.872 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:32:58.872 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'15999", 'reported_seq': 16216, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:52.888002+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:32:52.888002+0000', 'last_peered': '2024-09-17T15:32:52.888002+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:32:52.888002+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "41'5900", 'ondisk_log_start': "41'5900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20209688, 'num_objects': 4935, 'num_object_clones': 0, 'num_object_copies': 9870, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4935, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14156, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 810}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:32:58.872 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:32:58.872 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'16259", 'reported_seq': 16555, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:52.943708+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:32:52.943708+0000', 'last_peered': '2024-09-17T15:32:52.943708+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:32:52.943708+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "41'6200", 'ondisk_log_start': "41'6200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20447715, 'num_objects': 4993, 'num_object_clones': 0, 'num_object_copies': 9986, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4993, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12758, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 801}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:58.872 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:32:58.872 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'15925", 'reported_seq': 16239, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:52.780119+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:32:52.780119+0000', 'last_peered': '2024-09-17T15:32:52.780119+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:32:52.780119+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "41'5900", 'ondisk_log_start': "41'5900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19837894, 'num_objects': 4845, 'num_object_clones': 0, 'num_object_copies': 9690, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4089, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4845, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 12694, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 85, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 756}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:58.872 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:32:58.872 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'15609", 'reported_seq': 15879, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:52.811510+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:32:52.811510+0000', 'last_peered': '2024-09-17T15:32:52.811510+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:32:52.811510+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "41'5600", 'ondisk_log_start': "41'5600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20205568, 'num_objects': 4933, 'num_object_clones': 0, 'num_object_copies': 9866, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4933, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12205, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 990}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:58.872 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:32:58.873 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'15672", 'reported_seq': 15886, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:52.908631+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:32:52.908631+0000', 'last_peered': '2024-09-17T15:32:52.908631+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:32:52.908631+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "41'5600", 'ondisk_log_start': "41'5600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19853795, 'num_objects': 4848, 'num_object_clones': 0, 'num_object_copies': 9696, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4034, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4848, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13567, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 815}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:32:58.873 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:32:58.873 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'15833", 'reported_seq': 22968, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:52.889871+0000', 'last_change': '2024-09-17T15:30:55.086473+0000', 'last_active': '2024-09-17T15:32:52.889871+0000', 'last_peered': '2024-09-17T15:32:52.889871+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:32:52.889871+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "41'5800", 'ondisk_log_start': "41'5800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20082688, 'num_objects': 4903, 'num_object_clones': 0, 'num_object_copies': 9806, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 634, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4903, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13595, 'num_write_kb': 32532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3505, 'num_bytes_recovered': 8073216, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 4269}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:32:58.873 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:32:58.873 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'16025", 'reported_seq': 16328, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:52.914648+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:32:52.914648+0000', 'last_peered': '2024-09-17T15:32:52.914648+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:32:52.914648+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "41'6000", 'ondisk_log_start': "41'6000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20238336, 'num_objects': 4941, 'num_object_clones': 0, 'num_object_copies': 9882, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4941, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12573, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 884}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:58.873 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:32:58.873 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'15867", 'reported_seq': 16138, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:52.922428+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:32:52.922428+0000', 'last_peered': '2024-09-17T15:32:52.922428+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:32:52.922428+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "41'5800", 'ondisk_log_start': "41'5800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20090880, 'num_objects': 4905, 'num_object_clones': 0, 'num_object_copies': 9810, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4098, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4905, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12154, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 62, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 810}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:58.873 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:32:58.873 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'15934", 'reported_seq': 16235, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:32:52.889458+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:32:52.889458+0000', 'last_peered': '2024-09-17T15:32:52.889458+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:32:52.889458+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "41'5900", 'ondisk_log_start': "41'5900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20283392, 'num_objects': 4952, 'num_object_clones': 0, 'num_object_copies': 9904, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4952, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12721, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 972}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:32:58.874 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:32:59.525 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:32:59.526 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37143/157376 objects degraded (23.601%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":516,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":78688,"data_bytes":322257673,"bytes_used":1953443840,"bytes_avail":287956848640,"bytes_total":289910292480,"degraded_objects":37143,"degraded_total":157376,"degraded_ratio":0.23601438592923954,"recovering_objects_per_sec":28,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":343,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":429},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-17T15:31:29.695933+0000","services":{}},"progress_events":{}} 2024-09-17T15:32:59.546 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:32:59.546 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:33:00.119 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:33:00.119 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:33:00.146 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2024-09-17T15:32:59.737062+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":313496329,"num_objects":76549,"num_object_clones":0,"num_object_copies":153098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36989,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76549,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":216442,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12472,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160808,"ondisk_log_size":160808,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1903840,"kb_used_data":813408,"kb_used_omap":13,"kb_used_meta":1090354,"kb_avail":281211680,"statfs":{"total":289910292480,"available":287960760320,"internally_reserved":0,"allocated":832929792,"data_stored":832023515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1116523519},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,6,15,12,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28581888,"num_objects":-6978,"num_object_clones":0,"num_object_copies":-13956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-497,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6978,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":497,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004830"},"pg_stats":[{"pgid":"2.f","version":"45'15881","reported_seq":16136,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:57.894036+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:32:57.894036+0000","last_peered":"2024-09-17T15:32:57.894036+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:32:57.894036+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'5800","ondisk_log_start":"41'5800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19525632,"num_objects":4767,"num_object_clones":0,"num_object_copies":9534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15881,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":783}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'16031","reported_seq":16138,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:57.839889+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:32:57.839889+0000","last_peered":"2024-09-17T15:32:57.839889+0000","last_clean":"2024-09-17T15:32:57.839889+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:32:57.839889+0000","last_undegraded":"2024-09-17T15:32:57.839889+0000","last_fullsized":"2024-09-17T15:32:57.839889+0000","mapping_epoch":33,"log_start":"41'6000","ondisk_log_start":"41'6000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20075462,"num_objects":4903,"num_object_clones":0,"num_object_copies":9806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4903,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14388,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'16134","reported_seq":16352,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:57.843775+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:32:57.843775+0000","last_peered":"2024-09-17T15:32:57.843775+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:32:57.843775+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'6100","ondisk_log_start":"41'6100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19656728,"num_objects":4800,"num_object_clones":0,"num_object_copies":9600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4800,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14291,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":675}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'16128","reported_seq":16203,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:57.838354+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:32:57.838354+0000","last_peered":"2024-09-17T15:32:57.838354+0000","last_clean":"2024-09-17T15:32:57.838354+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:32:57.838354+0000","last_undegraded":"2024-09-17T15:32:57.838354+0000","last_fullsized":"2024-09-17T15:32:57.838354+0000","mapping_epoch":33,"log_start":"41'6100","ondisk_log_start":"41'6100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19832832,"num_objects":4842,"num_object_clones":0,"num_object_copies":9684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16128,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'15836","reported_seq":15970,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:57.966241+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:32:57.966241+0000","last_peered":"2024-09-17T15:32:57.966241+0000","last_clean":"2024-09-17T15:32:57.966241+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:32:57.966241+0000","last_undegraded":"2024-09-17T15:32:57.966241+0000","last_fullsized":"2024-09-17T15:32:57.966241+0000","mapping_epoch":32,"log_start":"41'5800","ondisk_log_start":"41'5800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19730915,"num_objects":4818,"num_object_clones":0,"num_object_copies":9636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4818,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12818,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'16402","reported_seq":16693,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:58.018545+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:32:58.018545+0000","last_peered":"2024-09-17T15:32:58.018545+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:32:58.018545+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'6400","ondisk_log_start":"41'6400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19861987,"num_objects":4850,"num_object_clones":0,"num_object_copies":9700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4850,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12901,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":658},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'16076","reported_seq":16390,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:57.993341+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:32:57.993341+0000","last_peered":"2024-09-17T15:32:57.993341+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:32:57.993341+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'6000","ondisk_log_start":"41'6000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19219398,"num_objects":4694,"num_object_clones":0,"num_object_copies":9388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4694,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12845,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":605},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'16110","reported_seq":16243,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:57.997729+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:32:57.997729+0000","last_peered":"2024-09-17T15:32:57.997729+0000","last_clean":"2024-09-17T15:32:57.997729+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:32:57.997729+0000","last_undegraded":"2024-09-17T15:32:57.997729+0000","last_fullsized":"2024-09-17T15:32:57.997729+0000","mapping_epoch":32,"log_start":"41'6100","ondisk_log_start":"41'6100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19685376,"num_objects":4806,"num_object_clones":0,"num_object_copies":9612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13064,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087024+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.087024+0000","last_peered":"2024-09-17T15:30:57.087024+0000","last_clean":"2024-09-17T15:30:57.087024+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.087024+0000","last_undegraded":"2024-09-17T15:30:57.087024+0000","last_fullsized":"2024-09-17T15:30:57.087024+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'15972","reported_seq":24260,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:57.427756+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:32:57.427756+0000","last_peered":"2024-09-17T15:32:57.427756+0000","last_clean":"2024-09-17T15:32:57.427756+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:32:57.427756+0000","last_undegraded":"2024-09-17T15:32:57.427756+0000","last_fullsized":"2024-09-17T15:32:57.427756+0000","mapping_epoch":42,"log_start":"41'5900","ondisk_log_start":"41'5900","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19845603,"num_objects":4846,"num_object_clones":0,"num_object_copies":9692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4846,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13972,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'15789","reported_seq":16058,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:57.898424+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:32:57.898424+0000","last_peered":"2024-09-17T15:32:57.898424+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:32:57.898424+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'5700","ondisk_log_start":"41'5700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19468288,"num_objects":4753,"num_object_clones":0,"num_object_copies":9506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12385,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":810}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'15807","reported_seq":16021,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:57.857014+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:32:57.857014+0000","last_peered":"2024-09-17T15:32:57.857014+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:32:57.857014+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'5800","ondisk_log_start":"41'5800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19300835,"num_objects":4713,"num_object_clones":0,"num_object_copies":9426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4713,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13702,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":680}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":83,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087587+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087587+0000","last_peered":"2024-09-17T15:30:57.087587+0000","last_clean":"2024-09-17T15:30:57.087587+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087587+0000","last_undegraded":"2024-09-17T15:30:57.087587+0000","last_fullsized":"2024-09-17T15:30:57.087587+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'15998","reported_seq":23441,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:57.918963+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:32:57.918963+0000","last_peered":"2024-09-17T15:32:57.918963+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:32:57.918963+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"41'5900","ondisk_log_start":"41'5900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19406848,"num_objects":4738,"num_object_clones":0,"num_object_copies":9476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":480,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4738,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13760,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3659,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4258}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'16171","reported_seq":16474,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:58.036192+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:32:58.036192+0000","last_peered":"2024-09-17T15:32:58.036192+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:32:58.036192+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'6100","ondisk_log_start":"41'6100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19640320,"num_objects":4795,"num_object_clones":0,"num_object_copies":9590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12719,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":738}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'16036","reported_seq":16307,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:57.925500+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:32:57.925500+0000","last_peered":"2024-09-17T15:32:57.925500+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:32:57.925500+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'6000","ondisk_log_start":"41'6000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19398656,"num_objects":4736,"num_object_clones":0,"num_object_copies":9472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12323,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":641}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'16100","reported_seq":16403,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:57.980271+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:32:57.980271+0000","last_peered":"2024-09-17T15:32:57.980271+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:32:57.980271+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'6000","ondisk_log_start":"41'6000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19603456,"num_objects":4786,"num_object_clones":0,"num_object_copies":9572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12887,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":806}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087804+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087804+0000","last_peered":"2024-09-17T15:30:57.087804+0000","last_clean":"2024-09-17T15:30:57.087804+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087804+0000","last_undegraded":"2024-09-17T15:30:57.087804+0000","last_fullsized":"2024-09-17T15:30:57.087804+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088875+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088875+0000","last_peered":"2024-09-17T15:30:57.088875+0000","last_clean":"2024-09-17T15:30:57.088875+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088875+0000","last_undegraded":"2024-09-17T15:30:57.088875+0000","last_fullsized":"2024-09-17T15:30:57.088875+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'15934","reported_seq":24313,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:57.338179+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:32:57.338179+0000","last_peered":"2024-09-17T15:32:57.338179+0000","last_clean":"2024-09-17T15:32:57.338179+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:32:57.338179+0000","last_undegraded":"2024-09-17T15:32:57.338179+0000","last_fullsized":"2024-09-17T15:32:57.338179+0000","mapping_epoch":44,"log_start":"41'5900","ondisk_log_start":"41'5900","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19243974,"num_objects":4700,"num_object_clones":0,"num_object_copies":9400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12376,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":313496310,"num_objects":76547,"num_object_clones":0,"num_object_copies":153094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36989,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76547,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":216440,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12471,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":830029824,"data_stored":829884386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160805,"ondisk_log_size":160805,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051679,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610888,"kb_used_data":286260,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93760952,"statfs":{"total":96636764160,"available":96011214848,"internally_reserved":0,"allocated":293130240,"data_stored":292839143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":332394156},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724624,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594952,"kb_used_data":230064,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93776888,"statfs":{"total":96636764160,"available":96027533312,"internally_reserved":0,"allocated":235585536,"data_stored":235261458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":373616299},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,3,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247907,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":698000,"kb_used_data":297084,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93673840,"statfs":{"total":96636764160,"available":95922012160,"internally_reserved":0,"allocated":304214016,"data_stored":303922914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":410513064},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,3,5,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":302542848,"data_stored":302506718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":234848256,"data_stored":234779150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":292638720,"data_stored":292598518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:33:03.149 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:33:03.724 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:33:03.726 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:33:03.748 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":297,"stamp":"2024-09-17T15:33:01.737789+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":313496329,"num_objects":76549,"num_object_clones":0,"num_object_copies":153098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36989,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76549,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":216442,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12472,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160808,"ondisk_log_size":160808,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1903840,"kb_used_data":813408,"kb_used_omap":13,"kb_used_meta":1090354,"kb_avail":281211680,"statfs":{"total":289910292480,"available":287960760320,"internally_reserved":0,"allocated":832929792,"data_stored":832023515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1116523519},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,6,15,12,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19767296,"num_objects":-4826,"num_object_clones":0,"num_object_copies":-9652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4826,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":341,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004728"},"pg_stats":[{"pgid":"2.f","version":"45'15881","reported_seq":16136,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:57.894036+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:32:57.894036+0000","last_peered":"2024-09-17T15:32:57.894036+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:32:57.894036+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'5800","ondisk_log_start":"41'5800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19525632,"num_objects":4767,"num_object_clones":0,"num_object_copies":9534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15881,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":783}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'16031","reported_seq":16138,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:57.839889+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:32:57.839889+0000","last_peered":"2024-09-17T15:32:57.839889+0000","last_clean":"2024-09-17T15:32:57.839889+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:32:57.839889+0000","last_undegraded":"2024-09-17T15:32:57.839889+0000","last_fullsized":"2024-09-17T15:32:57.839889+0000","mapping_epoch":33,"log_start":"41'6000","ondisk_log_start":"41'6000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20075462,"num_objects":4903,"num_object_clones":0,"num_object_copies":9806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4903,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14388,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'16134","reported_seq":16352,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:57.843775+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:32:57.843775+0000","last_peered":"2024-09-17T15:32:57.843775+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:32:57.843775+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'6100","ondisk_log_start":"41'6100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19656728,"num_objects":4800,"num_object_clones":0,"num_object_copies":9600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4800,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14291,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":675}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'16128","reported_seq":16203,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:57.838354+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:32:57.838354+0000","last_peered":"2024-09-17T15:32:57.838354+0000","last_clean":"2024-09-17T15:32:57.838354+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:32:57.838354+0000","last_undegraded":"2024-09-17T15:32:57.838354+0000","last_fullsized":"2024-09-17T15:32:57.838354+0000","mapping_epoch":33,"log_start":"41'6100","ondisk_log_start":"41'6100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19832832,"num_objects":4842,"num_object_clones":0,"num_object_copies":9684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16128,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'15836","reported_seq":15970,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:57.966241+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:32:57.966241+0000","last_peered":"2024-09-17T15:32:57.966241+0000","last_clean":"2024-09-17T15:32:57.966241+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:32:57.966241+0000","last_undegraded":"2024-09-17T15:32:57.966241+0000","last_fullsized":"2024-09-17T15:32:57.966241+0000","mapping_epoch":32,"log_start":"41'5800","ondisk_log_start":"41'5800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19730915,"num_objects":4818,"num_object_clones":0,"num_object_copies":9636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4818,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12818,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'16402","reported_seq":16693,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:58.018545+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:32:58.018545+0000","last_peered":"2024-09-17T15:32:58.018545+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:32:58.018545+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'6400","ondisk_log_start":"41'6400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19861987,"num_objects":4850,"num_object_clones":0,"num_object_copies":9700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4850,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12901,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":658},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'16076","reported_seq":16390,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:57.993341+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:32:57.993341+0000","last_peered":"2024-09-17T15:32:57.993341+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:32:57.993341+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'6000","ondisk_log_start":"41'6000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19219398,"num_objects":4694,"num_object_clones":0,"num_object_copies":9388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4694,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12845,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":605},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'16110","reported_seq":16243,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:57.997729+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:32:57.997729+0000","last_peered":"2024-09-17T15:32:57.997729+0000","last_clean":"2024-09-17T15:32:57.997729+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:32:57.997729+0000","last_undegraded":"2024-09-17T15:32:57.997729+0000","last_fullsized":"2024-09-17T15:32:57.997729+0000","mapping_epoch":32,"log_start":"41'6100","ondisk_log_start":"41'6100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19685376,"num_objects":4806,"num_object_clones":0,"num_object_copies":9612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13064,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087024+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.087024+0000","last_peered":"2024-09-17T15:30:57.087024+0000","last_clean":"2024-09-17T15:30:57.087024+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.087024+0000","last_undegraded":"2024-09-17T15:30:57.087024+0000","last_fullsized":"2024-09-17T15:30:57.087024+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'15972","reported_seq":24260,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:57.427756+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:32:57.427756+0000","last_peered":"2024-09-17T15:32:57.427756+0000","last_clean":"2024-09-17T15:32:57.427756+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:32:57.427756+0000","last_undegraded":"2024-09-17T15:32:57.427756+0000","last_fullsized":"2024-09-17T15:32:57.427756+0000","mapping_epoch":42,"log_start":"41'5900","ondisk_log_start":"41'5900","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19845603,"num_objects":4846,"num_object_clones":0,"num_object_copies":9692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4846,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13972,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'15789","reported_seq":16058,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:57.898424+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:32:57.898424+0000","last_peered":"2024-09-17T15:32:57.898424+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:32:57.898424+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'5700","ondisk_log_start":"41'5700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19468288,"num_objects":4753,"num_object_clones":0,"num_object_copies":9506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12385,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":810}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'15807","reported_seq":16021,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:57.857014+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:32:57.857014+0000","last_peered":"2024-09-17T15:32:57.857014+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:32:57.857014+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'5800","ondisk_log_start":"41'5800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19300835,"num_objects":4713,"num_object_clones":0,"num_object_copies":9426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4713,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13702,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":680}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":83,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087587+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087587+0000","last_peered":"2024-09-17T15:30:57.087587+0000","last_clean":"2024-09-17T15:30:57.087587+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087587+0000","last_undegraded":"2024-09-17T15:30:57.087587+0000","last_fullsized":"2024-09-17T15:30:57.087587+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'15998","reported_seq":23441,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:57.918963+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:32:57.918963+0000","last_peered":"2024-09-17T15:32:57.918963+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:32:57.918963+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"41'5900","ondisk_log_start":"41'5900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19406848,"num_objects":4738,"num_object_clones":0,"num_object_copies":9476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":480,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4738,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13760,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3659,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4258}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'16171","reported_seq":16474,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:58.036192+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:32:58.036192+0000","last_peered":"2024-09-17T15:32:58.036192+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:32:58.036192+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'6100","ondisk_log_start":"41'6100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19640320,"num_objects":4795,"num_object_clones":0,"num_object_copies":9590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12719,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":738}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'16036","reported_seq":16307,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:57.925500+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:32:57.925500+0000","last_peered":"2024-09-17T15:32:57.925500+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:32:57.925500+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'6000","ondisk_log_start":"41'6000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19398656,"num_objects":4736,"num_object_clones":0,"num_object_copies":9472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12323,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":641}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'16100","reported_seq":16403,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:32:57.980271+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:32:57.980271+0000","last_peered":"2024-09-17T15:32:57.980271+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:32:57.980271+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'6000","ondisk_log_start":"41'6000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19603456,"num_objects":4786,"num_object_clones":0,"num_object_copies":9572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12887,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":806}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087804+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087804+0000","last_peered":"2024-09-17T15:30:57.087804+0000","last_clean":"2024-09-17T15:30:57.087804+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087804+0000","last_undegraded":"2024-09-17T15:30:57.087804+0000","last_fullsized":"2024-09-17T15:30:57.087804+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088875+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088875+0000","last_peered":"2024-09-17T15:30:57.088875+0000","last_clean":"2024-09-17T15:30:57.088875+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088875+0000","last_undegraded":"2024-09-17T15:30:57.088875+0000","last_fullsized":"2024-09-17T15:30:57.088875+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'15934","reported_seq":24313,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:32:57.338179+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:32:57.338179+0000","last_peered":"2024-09-17T15:32:57.338179+0000","last_clean":"2024-09-17T15:32:57.338179+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:32:57.338179+0000","last_undegraded":"2024-09-17T15:32:57.338179+0000","last_fullsized":"2024-09-17T15:32:57.338179+0000","mapping_epoch":44,"log_start":"41'5900","ondisk_log_start":"41'5900","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19243974,"num_objects":4700,"num_object_clones":0,"num_object_copies":9400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12376,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":313496310,"num_objects":76547,"num_object_clones":0,"num_object_copies":153094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36989,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76547,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":216440,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12471,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":830029824,"data_stored":829884386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160805,"ondisk_log_size":160805,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051679,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610888,"kb_used_data":286260,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93760952,"statfs":{"total":96636764160,"available":96011214848,"internally_reserved":0,"allocated":293130240,"data_stored":292839143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":332394156},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724624,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594952,"kb_used_data":230064,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93776888,"statfs":{"total":96636764160,"available":96027533312,"internally_reserved":0,"allocated":235585536,"data_stored":235261458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":373616299},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,3,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247907,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":698000,"kb_used_data":297084,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93673840,"statfs":{"total":96636764160,"available":95922012160,"internally_reserved":0,"allocated":304214016,"data_stored":303922914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":410513064},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,3,5,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":302542848,"data_stored":302506718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":234848256,"data_stored":234779150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":292638720,"data_stored":292598518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:33:03.750 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:33:04.320 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:33:04.321 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:33:04.344 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":298,"stamp":"2024-09-17T15:33:03.738272+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":303461129,"num_objects":74099,"num_object_clones":0,"num_object_copies":148198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36795,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74099,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":218892,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12666,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160858,"ondisk_log_size":160858,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1885820,"kb_used_data":793984,"kb_used_omap":13,"kb_used_meta":1091762,"kb_avail":281229700,"statfs":{"total":289910292480,"available":287979212800,"internally_reserved":0,"allocated":813039616,"data_stored":812133339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1117965311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,7,10,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29802496,"num_objects":-7276,"num_object_clones":0,"num_object_copies":-14552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7276,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7276,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004372"},"pg_stats":[{"pgid":"2.f","version":"45'16053","reported_seq":16303,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:02.971684+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:33:02.971684+0000","last_peered":"2024-09-17T15:33:02.971684+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:33:02.971684+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'6000","ondisk_log_start":"41'6000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18821120,"num_objects":4595,"num_object_clones":0,"num_object_copies":9190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16053,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":611}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'16186","reported_seq":16294,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:33:02.935500+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:33:02.935500+0000","last_peered":"2024-09-17T15:33:02.935500+0000","last_clean":"2024-09-17T15:33:02.935500+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:33:02.935500+0000","last_undegraded":"2024-09-17T15:33:02.935500+0000","last_fullsized":"2024-09-17T15:33:02.935500+0000","mapping_epoch":33,"log_start":"41'6100","ondisk_log_start":"41'6100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19440582,"num_objects":4748,"num_object_clones":0,"num_object_copies":9496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4748,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14543,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'16294","reported_seq":16512,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:02.896014+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:33:02.896014+0000","last_peered":"2024-09-17T15:33:02.896014+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:33:02.896014+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'6200","ondisk_log_start":"41'6200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19001368,"num_objects":4640,"num_object_clones":0,"num_object_copies":9280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4640,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14451,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":515}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'16293","reported_seq":16367,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:33:02.880206+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:33:02.880206+0000","last_peered":"2024-09-17T15:33:02.880206+0000","last_clean":"2024-09-17T15:33:02.880206+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:33:02.880206+0000","last_undegraded":"2024-09-17T15:33:02.880206+0000","last_fullsized":"2024-09-17T15:33:02.880206+0000","mapping_epoch":33,"log_start":"41'6200","ondisk_log_start":"41'6200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19156992,"num_objects":4677,"num_object_clones":0,"num_object_copies":9354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16293,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'16002","reported_seq":16137,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:33:03.070526+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:33:03.070526+0000","last_peered":"2024-09-17T15:33:03.070526+0000","last_clean":"2024-09-17T15:33:03.070526+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:33:03.070526+0000","last_undegraded":"2024-09-17T15:33:03.070526+0000","last_fullsized":"2024-09-17T15:33:03.070526+0000","mapping_epoch":32,"log_start":"41'6000","ondisk_log_start":"41'6000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19050979,"num_objects":4652,"num_object_clones":0,"num_object_copies":9304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4652,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12984,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'16534","reported_seq":16830,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:03.002282+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:33:03.002282+0000","last_peered":"2024-09-17T15:33:03.002282+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:33:03.002282+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'6500","ondisk_log_start":"41'6500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19321315,"num_objects":4718,"num_object_clones":0,"num_object_copies":9436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4718,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13033,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":526},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'16238","reported_seq":16552,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:03.049164+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:33:03.049164+0000","last_peered":"2024-09-17T15:33:03.049164+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:33:03.049164+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'6200","ondisk_log_start":"41'6200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18555846,"num_objects":4532,"num_object_clones":0,"num_object_copies":9064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4532,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13007,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":443},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'16265","reported_seq":16398,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:33:03.075799+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:33:03.075799+0000","last_peered":"2024-09-17T15:33:03.075799+0000","last_clean":"2024-09-17T15:33:03.075799+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:33:03.075799+0000","last_undegraded":"2024-09-17T15:33:03.075799+0000","last_fullsized":"2024-09-17T15:33:03.075799+0000","mapping_epoch":32,"log_start":"41'6200","ondisk_log_start":"41'6200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19050496,"num_objects":4651,"num_object_clones":0,"num_object_copies":9302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13219,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087024+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.087024+0000","last_peered":"2024-09-17T15:30:57.087024+0000","last_clean":"2024-09-17T15:30:57.087024+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.087024+0000","last_undegraded":"2024-09-17T15:30:57.087024+0000","last_fullsized":"2024-09-17T15:30:57.087024+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'16104","reported_seq":24392,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:33:02.487371+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:33:02.487371+0000","last_peered":"2024-09-17T15:33:02.487371+0000","last_clean":"2024-09-17T15:33:02.487371+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:33:02.487371+0000","last_undegraded":"2024-09-17T15:33:02.487371+0000","last_fullsized":"2024-09-17T15:33:02.487371+0000","mapping_epoch":42,"log_start":"41'6100","ondisk_log_start":"41'6100","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19304931,"num_objects":4714,"num_object_clones":0,"num_object_copies":9428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4714,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14104,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'15911","reported_seq":16181,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:02.966593+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:33:02.966593+0000","last_peered":"2024-09-17T15:33:02.966593+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:33:02.966593+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'5900","ondisk_log_start":"41'5900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18968576,"num_objects":4631,"num_object_clones":0,"num_object_copies":9262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12507,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":688}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'15977","reported_seq":16188,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:02.912437+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:33:02.912437+0000","last_peered":"2024-09-17T15:33:02.912437+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:33:02.912437+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'5900","ondisk_log_start":"41'5900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18604515,"num_objects":4543,"num_object_clones":0,"num_object_copies":9086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4543,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13872,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":510}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":83,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087587+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087587+0000","last_peered":"2024-09-17T15:30:57.087587+0000","last_clean":"2024-09-17T15:30:57.087587+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087587+0000","last_undegraded":"2024-09-17T15:30:57.087587+0000","last_fullsized":"2024-09-17T15:30:57.087587+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'16143","reported_seq":23974,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:02.945785+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:33:02.945785+0000","last_peered":"2024-09-17T15:33:02.945785+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:33:02.945785+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"41'6100","ondisk_log_start":"41'6100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18812928,"num_objects":4593,"num_object_clones":0,"num_object_copies":9186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":286,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13905,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3853,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4307}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'16342","reported_seq":16641,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:03.050554+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:33:03.050554+0000","last_peered":"2024-09-17T15:33:03.050554+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:33:03.050554+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'6300","ondisk_log_start":"41'6300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18939904,"num_objects":4624,"num_object_clones":0,"num_object_copies":9248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12890,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":567}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'16188","reported_seq":16459,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:02.974035+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:33:02.974035+0000","last_peered":"2024-09-17T15:33:02.974035+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:33:02.974035+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'6100","ondisk_log_start":"41'6100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18776064,"num_objects":4584,"num_object_clones":0,"num_object_copies":9168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12475,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":489}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'16249","reported_seq":16551,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:03.013979+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:33:03.013979+0000","last_peered":"2024-09-17T15:33:03.013979+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:33:03.013979+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'6200","ondisk_log_start":"41'6200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18993152,"num_objects":4637,"num_object_clones":0,"num_object_copies":9274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13036,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":657}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087804+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087804+0000","last_peered":"2024-09-17T15:30:57.087804+0000","last_clean":"2024-09-17T15:30:57.087804+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087804+0000","last_undegraded":"2024-09-17T15:30:57.087804+0000","last_fullsized":"2024-09-17T15:30:57.087804+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088875+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088875+0000","last_peered":"2024-09-17T15:30:57.088875+0000","last_clean":"2024-09-17T15:30:57.088875+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088875+0000","last_undegraded":"2024-09-17T15:30:57.088875+0000","last_fullsized":"2024-09-17T15:30:57.088875+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'16076","reported_seq":24456,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:33:02.513117+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:33:02.513117+0000","last_peered":"2024-09-17T15:33:02.513117+0000","last_clean":"2024-09-17T15:33:02.513117+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:33:02.513117+0000","last_undegraded":"2024-09-17T15:33:02.513117+0000","last_fullsized":"2024-09-17T15:33:02.513117+0000","mapping_epoch":44,"log_start":"41'6000","ondisk_log_start":"41'6000","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18662342,"num_objects":4558,"num_object_clones":0,"num_object_copies":9116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12518,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":303461110,"num_objects":74097,"num_object_clones":0,"num_object_copies":148194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36795,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74097,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":218890,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12665,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":810119168,"data_stored":809973730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051680,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610160,"kb_used_data":281376,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93761680,"statfs":{"total":96636764160,"available":96011960320,"internally_reserved":0,"allocated":288129024,"data_stored":287837927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":336653996},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724625,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":579372,"kb_used_data":222800,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93792468,"statfs":{"total":96636764160,"available":96043487232,"internally_reserved":0,"allocated":228147200,"data_stored":227823122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":365096619},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,5,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247908,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":696288,"kb_used_data":289808,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93675552,"statfs":{"total":96636764160,"available":95923765248,"internally_reserved":0,"allocated":296763392,"data_stored":296472290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":416214696},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,5,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":295034880,"data_stored":294998750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":227524608,"data_stored":227455502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":287559680,"data_stored":287519478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:33:04.345 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:33:04.345 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'16053", 'reported_seq': 16303, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:02.971684+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:33:02.971684+0000', 'last_peered': '2024-09-17T15:33:02.971684+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:33:02.971684+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "41'6000", 'ondisk_log_start': "41'6000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18821120, 'num_objects': 4595, 'num_object_clones': 0, 'num_object_copies': 9190, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4595, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16053, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 611}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:04.346 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:33:04.346 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'16294", 'reported_seq': 16512, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:02.896014+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:33:02.896014+0000', 'last_peered': '2024-09-17T15:33:02.896014+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:33:02.896014+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "41'6200", 'ondisk_log_start': "41'6200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19001368, 'num_objects': 4640, 'num_object_clones': 0, 'num_object_copies': 9280, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4640, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14451, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 515}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:33:04.346 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:33:04.346 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'16534", 'reported_seq': 16830, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:03.002282+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:33:03.002282+0000', 'last_peered': '2024-09-17T15:33:03.002282+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:33:03.002282+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "41'6500", 'ondisk_log_start': "41'6500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19321315, 'num_objects': 4718, 'num_object_clones': 0, 'num_object_copies': 9436, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4718, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13033, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 526}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:04.346 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:33:04.346 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'16238", 'reported_seq': 16552, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:03.049164+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:33:03.049164+0000', 'last_peered': '2024-09-17T15:33:03.049164+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:33:03.049164+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "41'6200", 'ondisk_log_start': "41'6200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18555846, 'num_objects': 4532, 'num_object_clones': 0, 'num_object_copies': 9064, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4089, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4532, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 13007, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 85, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 443}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:04.346 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:33:04.346 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'15911", 'reported_seq': 16181, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:02.966593+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:33:02.966593+0000', 'last_peered': '2024-09-17T15:33:02.966593+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:33:02.966593+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "41'5900", 'ondisk_log_start': "41'5900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18968576, 'num_objects': 4631, 'num_object_clones': 0, 'num_object_copies': 9262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4631, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12507, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 688}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:04.346 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:33:04.346 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'15977", 'reported_seq': 16188, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:02.912437+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:33:02.912437+0000', 'last_peered': '2024-09-17T15:33:02.912437+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:33:02.912437+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "41'5900", 'ondisk_log_start': "41'5900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18604515, 'num_objects': 4543, 'num_object_clones': 0, 'num_object_copies': 9086, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4034, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4543, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13872, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 510}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:33:04.346 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:33:04.346 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'16143", 'reported_seq': 23974, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:02.945785+0000', 'last_change': '2024-09-17T15:30:55.086473+0000', 'last_active': '2024-09-17T15:33:02.945785+0000', 'last_peered': '2024-09-17T15:33:02.945785+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:33:02.945785+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "41'6100", 'ondisk_log_start': "41'6100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18812928, 'num_objects': 4593, 'num_object_clones': 0, 'num_object_copies': 9186, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 286, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4593, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13905, 'num_write_kb': 32532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3853, 'num_bytes_recovered': 8073216, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 4307}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:33:04.347 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:33:04.347 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'16342", 'reported_seq': 16641, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:03.050554+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:33:03.050554+0000', 'last_peered': '2024-09-17T15:33:03.050554+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:33:03.050554+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "41'6300", 'ondisk_log_start': "41'6300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18939904, 'num_objects': 4624, 'num_object_clones': 0, 'num_object_copies': 9248, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4624, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12890, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 567}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:04.347 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:33:04.347 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'16188", 'reported_seq': 16459, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:02.974035+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:33:02.974035+0000', 'last_peered': '2024-09-17T15:33:02.974035+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:33:02.974035+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "41'6100", 'ondisk_log_start': "41'6100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18776064, 'num_objects': 4584, 'num_object_clones': 0, 'num_object_copies': 9168, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4098, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4584, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12475, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 62, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 489}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:04.347 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:33:04.347 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'16249", 'reported_seq': 16551, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:03.013979+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:33:03.013979+0000', 'last_peered': '2024-09-17T15:33:03.013979+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:33:03.013979+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "41'6200", 'ondisk_log_start': "41'6200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18993152, 'num_objects': 4637, 'num_object_clones': 0, 'num_object_copies': 9274, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4637, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13036, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 657}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:04.347 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:33:05.006 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:33:05.006 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36795/148198 objects degraded (24.828%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":522,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":74099,"data_bytes":303461129,"bytes_used":1931079680,"bytes_avail":287979212800,"bytes_total":289910292480,"degraded_objects":36795,"degraded_total":148198,"degraded_ratio":0.24828270287048407,"recovering_objects_per_sec":44,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":535,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":606},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-17T15:31:29.695933+0000","services":{}},"progress_events":{}} 2024-09-17T15:33:05.031 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:33:05.032 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:33:05.576 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:33:05.579 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:33:05.605 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":298,"stamp":"2024-09-17T15:33:03.738272+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":303461129,"num_objects":74099,"num_object_clones":0,"num_object_copies":148198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36795,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74099,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":218892,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12666,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160858,"ondisk_log_size":160858,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1885820,"kb_used_data":793984,"kb_used_omap":13,"kb_used_meta":1091762,"kb_avail":281229700,"statfs":{"total":289910292480,"available":287979212800,"internally_reserved":0,"allocated":813039616,"data_stored":812133339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1117965311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,7,10,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29802496,"num_objects":-7276,"num_object_clones":0,"num_object_copies":-14552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7276,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7276,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004372"},"pg_stats":[{"pgid":"2.f","version":"45'16053","reported_seq":16303,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:02.971684+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:33:02.971684+0000","last_peered":"2024-09-17T15:33:02.971684+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:33:02.971684+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'6000","ondisk_log_start":"41'6000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18821120,"num_objects":4595,"num_object_clones":0,"num_object_copies":9190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16053,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":611}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'16186","reported_seq":16294,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:33:02.935500+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:33:02.935500+0000","last_peered":"2024-09-17T15:33:02.935500+0000","last_clean":"2024-09-17T15:33:02.935500+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:33:02.935500+0000","last_undegraded":"2024-09-17T15:33:02.935500+0000","last_fullsized":"2024-09-17T15:33:02.935500+0000","mapping_epoch":33,"log_start":"41'6100","ondisk_log_start":"41'6100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19440582,"num_objects":4748,"num_object_clones":0,"num_object_copies":9496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4748,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14543,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'16294","reported_seq":16512,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:02.896014+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:33:02.896014+0000","last_peered":"2024-09-17T15:33:02.896014+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:33:02.896014+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'6200","ondisk_log_start":"41'6200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19001368,"num_objects":4640,"num_object_clones":0,"num_object_copies":9280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4640,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14451,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":515}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'16293","reported_seq":16367,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:33:02.880206+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:33:02.880206+0000","last_peered":"2024-09-17T15:33:02.880206+0000","last_clean":"2024-09-17T15:33:02.880206+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:33:02.880206+0000","last_undegraded":"2024-09-17T15:33:02.880206+0000","last_fullsized":"2024-09-17T15:33:02.880206+0000","mapping_epoch":33,"log_start":"41'6200","ondisk_log_start":"41'6200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19156992,"num_objects":4677,"num_object_clones":0,"num_object_copies":9354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16293,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'16002","reported_seq":16137,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:33:03.070526+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:33:03.070526+0000","last_peered":"2024-09-17T15:33:03.070526+0000","last_clean":"2024-09-17T15:33:03.070526+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:33:03.070526+0000","last_undegraded":"2024-09-17T15:33:03.070526+0000","last_fullsized":"2024-09-17T15:33:03.070526+0000","mapping_epoch":32,"log_start":"41'6000","ondisk_log_start":"41'6000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19050979,"num_objects":4652,"num_object_clones":0,"num_object_copies":9304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4652,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12984,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'16534","reported_seq":16830,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:03.002282+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:33:03.002282+0000","last_peered":"2024-09-17T15:33:03.002282+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:33:03.002282+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'6500","ondisk_log_start":"41'6500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19321315,"num_objects":4718,"num_object_clones":0,"num_object_copies":9436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4718,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13033,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":526},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'16238","reported_seq":16552,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:03.049164+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:33:03.049164+0000","last_peered":"2024-09-17T15:33:03.049164+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:33:03.049164+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'6200","ondisk_log_start":"41'6200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18555846,"num_objects":4532,"num_object_clones":0,"num_object_copies":9064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4532,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13007,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":443},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'16265","reported_seq":16398,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:33:03.075799+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:33:03.075799+0000","last_peered":"2024-09-17T15:33:03.075799+0000","last_clean":"2024-09-17T15:33:03.075799+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:33:03.075799+0000","last_undegraded":"2024-09-17T15:33:03.075799+0000","last_fullsized":"2024-09-17T15:33:03.075799+0000","mapping_epoch":32,"log_start":"41'6200","ondisk_log_start":"41'6200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19050496,"num_objects":4651,"num_object_clones":0,"num_object_copies":9302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13219,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087024+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.087024+0000","last_peered":"2024-09-17T15:30:57.087024+0000","last_clean":"2024-09-17T15:30:57.087024+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.087024+0000","last_undegraded":"2024-09-17T15:30:57.087024+0000","last_fullsized":"2024-09-17T15:30:57.087024+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'16104","reported_seq":24392,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:33:02.487371+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:33:02.487371+0000","last_peered":"2024-09-17T15:33:02.487371+0000","last_clean":"2024-09-17T15:33:02.487371+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:33:02.487371+0000","last_undegraded":"2024-09-17T15:33:02.487371+0000","last_fullsized":"2024-09-17T15:33:02.487371+0000","mapping_epoch":42,"log_start":"41'6100","ondisk_log_start":"41'6100","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19304931,"num_objects":4714,"num_object_clones":0,"num_object_copies":9428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4714,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14104,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'15911","reported_seq":16181,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:02.966593+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:33:02.966593+0000","last_peered":"2024-09-17T15:33:02.966593+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:33:02.966593+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'5900","ondisk_log_start":"41'5900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18968576,"num_objects":4631,"num_object_clones":0,"num_object_copies":9262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12507,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":688}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'15977","reported_seq":16188,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:02.912437+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:33:02.912437+0000","last_peered":"2024-09-17T15:33:02.912437+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:33:02.912437+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'5900","ondisk_log_start":"41'5900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18604515,"num_objects":4543,"num_object_clones":0,"num_object_copies":9086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4543,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13872,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":510}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":83,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087587+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087587+0000","last_peered":"2024-09-17T15:30:57.087587+0000","last_clean":"2024-09-17T15:30:57.087587+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087587+0000","last_undegraded":"2024-09-17T15:30:57.087587+0000","last_fullsized":"2024-09-17T15:30:57.087587+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'16143","reported_seq":23974,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:02.945785+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:33:02.945785+0000","last_peered":"2024-09-17T15:33:02.945785+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:33:02.945785+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"41'6100","ondisk_log_start":"41'6100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18812928,"num_objects":4593,"num_object_clones":0,"num_object_copies":9186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":286,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13905,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3853,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4307}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'16342","reported_seq":16641,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:03.050554+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:33:03.050554+0000","last_peered":"2024-09-17T15:33:03.050554+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:33:03.050554+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'6300","ondisk_log_start":"41'6300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18939904,"num_objects":4624,"num_object_clones":0,"num_object_copies":9248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12890,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":567}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'16188","reported_seq":16459,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:02.974035+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:33:02.974035+0000","last_peered":"2024-09-17T15:33:02.974035+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:33:02.974035+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'6100","ondisk_log_start":"41'6100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18776064,"num_objects":4584,"num_object_clones":0,"num_object_copies":9168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12475,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":489}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'16249","reported_seq":16551,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:03.013979+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:33:03.013979+0000","last_peered":"2024-09-17T15:33:03.013979+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:33:03.013979+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'6200","ondisk_log_start":"41'6200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18993152,"num_objects":4637,"num_object_clones":0,"num_object_copies":9274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13036,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":657}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087804+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087804+0000","last_peered":"2024-09-17T15:30:57.087804+0000","last_clean":"2024-09-17T15:30:57.087804+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087804+0000","last_undegraded":"2024-09-17T15:30:57.087804+0000","last_fullsized":"2024-09-17T15:30:57.087804+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088875+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088875+0000","last_peered":"2024-09-17T15:30:57.088875+0000","last_clean":"2024-09-17T15:30:57.088875+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088875+0000","last_undegraded":"2024-09-17T15:30:57.088875+0000","last_fullsized":"2024-09-17T15:30:57.088875+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'16076","reported_seq":24456,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:33:02.513117+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:33:02.513117+0000","last_peered":"2024-09-17T15:33:02.513117+0000","last_clean":"2024-09-17T15:33:02.513117+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:33:02.513117+0000","last_undegraded":"2024-09-17T15:33:02.513117+0000","last_fullsized":"2024-09-17T15:33:02.513117+0000","mapping_epoch":44,"log_start":"41'6000","ondisk_log_start":"41'6000","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18662342,"num_objects":4558,"num_object_clones":0,"num_object_copies":9116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12518,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":303461110,"num_objects":74097,"num_object_clones":0,"num_object_copies":148194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36795,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74097,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":218890,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12665,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":810119168,"data_stored":809973730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051680,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610160,"kb_used_data":281376,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93761680,"statfs":{"total":96636764160,"available":96011960320,"internally_reserved":0,"allocated":288129024,"data_stored":287837927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":336653996},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724625,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":579372,"kb_used_data":222800,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93792468,"statfs":{"total":96636764160,"available":96043487232,"internally_reserved":0,"allocated":228147200,"data_stored":227823122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":365096619},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,5,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247908,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":696288,"kb_used_data":289808,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93675552,"statfs":{"total":96636764160,"available":95923765248,"internally_reserved":0,"allocated":296763392,"data_stored":296472290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":416214696},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,5,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":295034880,"data_stored":294998750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":227524608,"data_stored":227455502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":287559680,"data_stored":287519478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:33:08.608 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:33:09.194 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:33:09.196 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:33:09.219 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":300,"stamp":"2024-09-17T15:33:07.739677+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":302265097,"num_objects":73807,"num_object_clones":0,"num_object_copies":147614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36795,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73807,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":219184,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12666,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160850,"ondisk_log_size":160850,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1882840,"kb_used_data":786852,"kb_used_omap":13,"kb_used_meta":1095922,"kb_avail":281232680,"statfs":{"total":289910292480,"available":287982264320,"internally_reserved":0,"allocated":805736448,"data_stored":804830171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1122225151},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,2,7,11,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21303296,"num_objects":-5201,"num_object_clones":0,"num_object_copies":-10402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-348,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5201,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":348,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004171"},"pg_stats":[{"pgid":"2.f","version":"45'16053","reported_seq":16303,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:02.971684+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:33:02.971684+0000","last_peered":"2024-09-17T15:33:02.971684+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:33:02.971684+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'6000","ondisk_log_start":"41'6000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18821120,"num_objects":4595,"num_object_clones":0,"num_object_copies":9190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16053,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":611}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'16186","reported_seq":16294,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:33:02.935500+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:33:02.935500+0000","last_peered":"2024-09-17T15:33:02.935500+0000","last_clean":"2024-09-17T15:33:02.935500+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:33:02.935500+0000","last_undegraded":"2024-09-17T15:33:02.935500+0000","last_fullsized":"2024-09-17T15:33:02.935500+0000","mapping_epoch":33,"log_start":"41'6100","ondisk_log_start":"41'6100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19440582,"num_objects":4748,"num_object_clones":0,"num_object_copies":9496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4748,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14543,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'16294","reported_seq":16512,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:02.896014+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:33:02.896014+0000","last_peered":"2024-09-17T15:33:02.896014+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:33:02.896014+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'6200","ondisk_log_start":"41'6200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19001368,"num_objects":4640,"num_object_clones":0,"num_object_copies":9280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4640,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14451,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":515}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'16293","reported_seq":16367,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:33:02.880206+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:33:02.880206+0000","last_peered":"2024-09-17T15:33:02.880206+0000","last_clean":"2024-09-17T15:33:02.880206+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:33:02.880206+0000","last_undegraded":"2024-09-17T15:33:02.880206+0000","last_fullsized":"2024-09-17T15:33:02.880206+0000","mapping_epoch":33,"log_start":"41'6200","ondisk_log_start":"41'6200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19156992,"num_objects":4677,"num_object_clones":0,"num_object_copies":9354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16293,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'16002","reported_seq":16137,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:33:03.070526+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:33:03.070526+0000","last_peered":"2024-09-17T15:33:03.070526+0000","last_clean":"2024-09-17T15:33:03.070526+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:33:03.070526+0000","last_undegraded":"2024-09-17T15:33:03.070526+0000","last_fullsized":"2024-09-17T15:33:03.070526+0000","mapping_epoch":32,"log_start":"41'6000","ondisk_log_start":"41'6000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19050979,"num_objects":4652,"num_object_clones":0,"num_object_copies":9304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4652,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12984,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'16534","reported_seq":16830,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:03.002282+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:33:03.002282+0000","last_peered":"2024-09-17T15:33:03.002282+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:33:03.002282+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'6500","ondisk_log_start":"41'6500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19321315,"num_objects":4718,"num_object_clones":0,"num_object_copies":9436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4718,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13033,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":526},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'16238","reported_seq":16552,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:03.049164+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:33:03.049164+0000","last_peered":"2024-09-17T15:33:03.049164+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:33:03.049164+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'6200","ondisk_log_start":"41'6200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18555846,"num_objects":4532,"num_object_clones":0,"num_object_copies":9064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4532,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13007,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":443},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'16265","reported_seq":16398,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:33:03.075799+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:33:03.075799+0000","last_peered":"2024-09-17T15:33:03.075799+0000","last_clean":"2024-09-17T15:33:03.075799+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:33:03.075799+0000","last_undegraded":"2024-09-17T15:33:03.075799+0000","last_fullsized":"2024-09-17T15:33:03.075799+0000","mapping_epoch":32,"log_start":"41'6200","ondisk_log_start":"41'6200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19050496,"num_objects":4651,"num_object_clones":0,"num_object_copies":9302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13219,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087024+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.087024+0000","last_peered":"2024-09-17T15:30:57.087024+0000","last_clean":"2024-09-17T15:30:57.087024+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.087024+0000","last_undegraded":"2024-09-17T15:30:57.087024+0000","last_fullsized":"2024-09-17T15:30:57.087024+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'16256","reported_seq":24544,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:33:07.529595+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:33:07.529595+0000","last_peered":"2024-09-17T15:33:07.529595+0000","last_clean":"2024-09-17T15:33:07.529595+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:33:07.529595+0000","last_undegraded":"2024-09-17T15:33:07.529595+0000","last_fullsized":"2024-09-17T15:33:07.529595+0000","mapping_epoch":42,"log_start":"41'6200","ondisk_log_start":"41'6200","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18682339,"num_objects":4562,"num_object_clones":0,"num_object_copies":9124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4562,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14256,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'15911","reported_seq":16181,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:02.966593+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:33:02.966593+0000","last_peered":"2024-09-17T15:33:02.966593+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:33:02.966593+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'5900","ondisk_log_start":"41'5900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18968576,"num_objects":4631,"num_object_clones":0,"num_object_copies":9262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12507,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":688}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'15977","reported_seq":16188,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:02.912437+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:33:02.912437+0000","last_peered":"2024-09-17T15:33:02.912437+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:33:02.912437+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'5900","ondisk_log_start":"41'5900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18604515,"num_objects":4543,"num_object_clones":0,"num_object_copies":9086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4543,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13872,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":510}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":83,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087587+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087587+0000","last_peered":"2024-09-17T15:30:57.087587+0000","last_clean":"2024-09-17T15:30:57.087587+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087587+0000","last_undegraded":"2024-09-17T15:30:57.087587+0000","last_fullsized":"2024-09-17T15:30:57.087587+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'16143","reported_seq":23974,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:02.945785+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:33:02.945785+0000","last_peered":"2024-09-17T15:33:02.945785+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:33:02.945785+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"41'6100","ondisk_log_start":"41'6100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18812928,"num_objects":4593,"num_object_clones":0,"num_object_copies":9186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":286,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13905,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3853,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4307}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'16342","reported_seq":16641,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:03.050554+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:33:03.050554+0000","last_peered":"2024-09-17T15:33:03.050554+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:33:03.050554+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'6300","ondisk_log_start":"41'6300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18939904,"num_objects":4624,"num_object_clones":0,"num_object_copies":9248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12890,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":567}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'16188","reported_seq":16459,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:02.974035+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:33:02.974035+0000","last_peered":"2024-09-17T15:33:02.974035+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:33:02.974035+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'6100","ondisk_log_start":"41'6100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18776064,"num_objects":4584,"num_object_clones":0,"num_object_copies":9168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12475,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":489}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'16249","reported_seq":16551,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:03.013979+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:33:03.013979+0000","last_peered":"2024-09-17T15:33:03.013979+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:33:03.013979+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'6200","ondisk_log_start":"41'6200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18993152,"num_objects":4637,"num_object_clones":0,"num_object_copies":9274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13036,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":657}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087804+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087804+0000","last_peered":"2024-09-17T15:30:57.087804+0000","last_clean":"2024-09-17T15:30:57.087804+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087804+0000","last_undegraded":"2024-09-17T15:30:57.087804+0000","last_fullsized":"2024-09-17T15:30:57.087804+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088875+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088875+0000","last_peered":"2024-09-17T15:30:57.088875+0000","last_clean":"2024-09-17T15:30:57.088875+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088875+0000","last_undegraded":"2024-09-17T15:30:57.088875+0000","last_fullsized":"2024-09-17T15:30:57.088875+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'16216","reported_seq":24596,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:33:07.548353+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:33:07.548353+0000","last_peered":"2024-09-17T15:33:07.548353+0000","last_clean":"2024-09-17T15:33:07.548353+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:33:07.548353+0000","last_undegraded":"2024-09-17T15:33:07.548353+0000","last_fullsized":"2024-09-17T15:33:07.548353+0000","mapping_epoch":44,"log_start":"41'6200","ondisk_log_start":"41'6200","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18088902,"num_objects":4418,"num_object_clones":0,"num_object_copies":8836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12658,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":302265078,"num_objects":73805,"num_object_clones":0,"num_object_copies":147610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36795,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73805,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":219182,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12665,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802766848,"data_stored":802621410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160847,"ondisk_log_size":160847,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051680,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610160,"kb_used_data":281376,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93761680,"statfs":{"total":96636764160,"available":96011960320,"internally_reserved":0,"allocated":288129024,"data_stored":287837927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":336653996},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724626,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576392,"kb_used_data":215668,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93795448,"statfs":{"total":96636764160,"available":96046538752,"internally_reserved":0,"allocated":220844032,"data_stored":220519954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":369356459},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,3,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247908,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":696288,"kb_used_data":289808,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93675552,"statfs":{"total":96636764160,"available":95923765248,"internally_reserved":0,"allocated":296763392,"data_stored":296472290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":416214696},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,5,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":295034880,"data_stored":294998750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":220172288,"data_stored":220103182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":287559680,"data_stored":287519478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:33:09.220 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:33:09.776 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:33:09.778 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:33:09.801 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2024-09-17T15:33:09.740102+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":293810953,"num_objects":71743,"num_object_clones":0,"num_object_copies":143486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36580,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71743,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":221248,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12881,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160714,"ondisk_log_size":160714,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1867096,"kb_used_data":775268,"kb_used_omap":13,"kb_used_meta":1091762,"kb_avail":281248424,"statfs":{"total":289910292480,"available":287998386176,"internally_reserved":0,"allocated":793874432,"data_stored":792968155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1117965311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,1,8,10,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28446720,"num_objects":-6945,"num_object_clones":0,"num_object_copies":-13890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6945,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":563,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003885"},"pg_stats":[{"pgid":"2.f","version":"45'16202","reported_seq":16457,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:08.076486+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:33:08.076486+0000","last_peered":"2024-09-17T15:33:08.076486+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:33:08.076486+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'6200","ondisk_log_start":"41'6200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18210816,"num_objects":4446,"num_object_clones":0,"num_object_copies":8892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16202,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":462}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'16339","reported_seq":16447,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:33:07.940366+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:33:07.940366+0000","last_peered":"2024-09-17T15:33:07.940366+0000","last_clean":"2024-09-17T15:33:07.940366+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:33:07.940366+0000","last_undegraded":"2024-09-17T15:33:07.940366+0000","last_fullsized":"2024-09-17T15:33:07.940366+0000","mapping_epoch":33,"log_start":"41'6300","ondisk_log_start":"41'6300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18813894,"num_objects":4595,"num_object_clones":0,"num_object_copies":9190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4595,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14696,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'16430","reported_seq":16648,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:07.877800+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:33:07.877800+0000","last_peered":"2024-09-17T15:33:07.877800+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:33:07.877800+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'6400","ondisk_log_start":"41'6400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18444312,"num_objects":4504,"num_object_clones":0,"num_object_copies":9008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4504,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14587,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":379}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'16428","reported_seq":16503,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:33:07.901030+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:33:07.901030+0000","last_peered":"2024-09-17T15:33:07.901030+0000","last_clean":"2024-09-17T15:33:07.901030+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:33:07.901030+0000","last_undegraded":"2024-09-17T15:33:07.901030+0000","last_fullsized":"2024-09-17T15:33:07.901030+0000","mapping_epoch":33,"log_start":"41'6400","ondisk_log_start":"41'6400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18604032,"num_objects":4542,"num_object_clones":0,"num_object_copies":9084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16428,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'16155","reported_seq":16290,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:33:08.069867+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:33:08.069867+0000","last_peered":"2024-09-17T15:33:08.069867+0000","last_clean":"2024-09-17T15:33:08.069867+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:33:08.069867+0000","last_undegraded":"2024-09-17T15:33:08.069867+0000","last_fullsized":"2024-09-17T15:33:08.069867+0000","mapping_epoch":32,"log_start":"41'6100","ondisk_log_start":"41'6100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18424291,"num_objects":4499,"num_object_clones":0,"num_object_copies":8998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4499,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13137,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'16679","reported_seq":16974,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:08.068671+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:33:08.068671+0000","last_peered":"2024-09-17T15:33:08.068671+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:33:08.068671+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'6600","ondisk_log_start":"41'6600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18727395,"num_objects":4573,"num_object_clones":0,"num_object_copies":9146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4573,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13178,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":381},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'16387","reported_seq":16701,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:08.055843+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:33:08.055843+0000","last_peered":"2024-09-17T15:33:08.055843+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:33:08.055843+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'6300","ondisk_log_start":"41'6300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17945542,"num_objects":4383,"num_object_clones":0,"num_object_copies":8766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4383,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13156,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":294},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'16389","reported_seq":16522,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:33:08.042275+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:33:08.042275+0000","last_peered":"2024-09-17T15:33:08.042275+0000","last_clean":"2024-09-17T15:33:08.042275+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:33:08.042275+0000","last_undegraded":"2024-09-17T15:33:08.042275+0000","last_fullsized":"2024-09-17T15:33:08.042275+0000","mapping_epoch":32,"log_start":"41'6300","ondisk_log_start":"41'6300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18542592,"num_objects":4527,"num_object_clones":0,"num_object_copies":9054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4527,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13343,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087024+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.087024+0000","last_peered":"2024-09-17T15:30:57.087024+0000","last_clean":"2024-09-17T15:30:57.087024+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.087024+0000","last_undegraded":"2024-09-17T15:30:57.087024+0000","last_fullsized":"2024-09-17T15:30:57.087024+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'16256","reported_seq":24544,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:33:07.529595+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:33:07.529595+0000","last_peered":"2024-09-17T15:33:07.529595+0000","last_clean":"2024-09-17T15:33:07.529595+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:33:07.529595+0000","last_undegraded":"2024-09-17T15:33:07.529595+0000","last_fullsized":"2024-09-17T15:33:07.529595+0000","mapping_epoch":42,"log_start":"41'6200","ondisk_log_start":"41'6200","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18682339,"num_objects":4562,"num_object_clones":0,"num_object_copies":9124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4562,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14256,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'16065","reported_seq":16334,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:08.021340+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:33:08.021340+0000","last_peered":"2024-09-17T15:33:08.021340+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:33:08.021340+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'6000","ondisk_log_start":"41'6000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18337792,"num_objects":4477,"num_object_clones":0,"num_object_copies":8954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12661,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":534}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'16130","reported_seq":16344,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:07.898733+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:33:07.898733+0000","last_peered":"2024-09-17T15:33:07.898733+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:33:07.898733+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'6100","ondisk_log_start":"41'6100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17977827,"num_objects":4390,"num_object_clones":0,"num_object_copies":8780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4390,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14025,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":357}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":83,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087587+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087587+0000","last_peered":"2024-09-17T15:30:57.087587+0000","last_clean":"2024-09-17T15:30:57.087587+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087587+0000","last_undegraded":"2024-09-17T15:30:57.087587+0000","last_fullsized":"2024-09-17T15:30:57.087587+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'16290","reported_seq":24551,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:07.947909+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:33:07.947909+0000","last_peered":"2024-09-17T15:33:07.947909+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:33:07.947909+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"41'6200","ondisk_log_start":"41'6200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18210816,"num_objects":4446,"num_object_clones":0,"num_object_copies":8892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":71,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14052,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4068,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4375}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'16508","reported_seq":16811,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:08.069937+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:33:08.069937+0000","last_peered":"2024-09-17T15:33:08.069937+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:33:08.069937+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'6500","ondisk_log_start":"41'6500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18259968,"num_objects":4458,"num_object_clones":0,"num_object_copies":8916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13056,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":401}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'16324","reported_seq":16593,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:08.091680+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:33:08.091680+0000","last_peered":"2024-09-17T15:33:08.091680+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:33:08.091680+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'6300","ondisk_log_start":"41'6300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18219008,"num_objects":4448,"num_object_clones":0,"num_object_copies":8896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12611,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":353}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'16413","reported_seq":16714,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:08.092797+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:33:08.092797+0000","last_peered":"2024-09-17T15:33:08.092797+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:33:08.092797+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'6400","ondisk_log_start":"41'6400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18321408,"num_objects":4473,"num_object_clones":0,"num_object_copies":8946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13200,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":493}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087804+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087804+0000","last_peered":"2024-09-17T15:30:57.087804+0000","last_clean":"2024-09-17T15:30:57.087804+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087804+0000","last_undegraded":"2024-09-17T15:30:57.087804+0000","last_fullsized":"2024-09-17T15:30:57.087804+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088875+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088875+0000","last_peered":"2024-09-17T15:30:57.088875+0000","last_clean":"2024-09-17T15:30:57.088875+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088875+0000","last_undegraded":"2024-09-17T15:30:57.088875+0000","last_fullsized":"2024-09-17T15:30:57.088875+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'16216","reported_seq":24596,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:33:07.548353+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:33:07.548353+0000","last_peered":"2024-09-17T15:33:07.548353+0000","last_clean":"2024-09-17T15:33:07.548353+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:33:07.548353+0000","last_undegraded":"2024-09-17T15:33:07.548353+0000","last_fullsized":"2024-09-17T15:33:07.548353+0000","mapping_epoch":44,"log_start":"41'6200","ondisk_log_start":"41'6200","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18088902,"num_objects":4418,"num_object_clones":0,"num_object_copies":8836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12658,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":293810934,"num_objects":71741,"num_object_clones":0,"num_object_copies":143482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36580,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71741,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":221246,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12880,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":790769664,"data_stored":790624226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160711,"ondisk_log_size":160711,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051681,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":593068,"kb_used_data":276764,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93778772,"statfs":{"total":96636764160,"available":96029462528,"internally_reserved":0,"allocated":283406336,"data_stored":283115239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":323874476},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724626,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576392,"kb_used_data":215668,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93795448,"statfs":{"total":96636764160,"available":96046538752,"internally_reserved":0,"allocated":220844032,"data_stored":220519954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":369356459},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,3,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247909,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":697636,"kb_used_data":282836,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93674204,"statfs":{"total":96636764160,"available":95922384896,"internally_reserved":0,"allocated":289624064,"data_stored":289332962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":424734376},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":287784960,"data_stored":287748830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":220172288,"data_stored":220103182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":282812416,"data_stored":282772214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:33:09.803 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:33:09.803 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'16202", 'reported_seq': 16457, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:08.076486+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:33:08.076486+0000', 'last_peered': '2024-09-17T15:33:08.076486+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:33:08.076486+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "41'6200", 'ondisk_log_start': "41'6200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18210816, 'num_objects': 4446, 'num_object_clones': 0, 'num_object_copies': 8892, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4446, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16202, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 462}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:09.803 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:33:09.803 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'16430", 'reported_seq': 16648, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:07.877800+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:33:07.877800+0000', 'last_peered': '2024-09-17T15:33:07.877800+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:33:07.877800+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "41'6400", 'ondisk_log_start': "41'6400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18444312, 'num_objects': 4504, 'num_object_clones': 0, 'num_object_copies': 9008, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4504, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14587, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 379}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:33:09.803 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:33:09.803 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'16679", 'reported_seq': 16974, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:08.068671+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:33:08.068671+0000', 'last_peered': '2024-09-17T15:33:08.068671+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:33:08.068671+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "41'6600", 'ondisk_log_start': "41'6600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18727395, 'num_objects': 4573, 'num_object_clones': 0, 'num_object_copies': 9146, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4573, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13178, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 381}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:09.804 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:33:09.804 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'16387", 'reported_seq': 16701, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:08.055843+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:33:08.055843+0000', 'last_peered': '2024-09-17T15:33:08.055843+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:33:08.055843+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "41'6300", 'ondisk_log_start': "41'6300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17945542, 'num_objects': 4383, 'num_object_clones': 0, 'num_object_copies': 8766, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4089, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4383, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 13156, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 85, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 294}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:09.804 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:33:09.804 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'16065", 'reported_seq': 16334, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:08.021340+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:33:08.021340+0000', 'last_peered': '2024-09-17T15:33:08.021340+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:33:08.021340+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "41'6000", 'ondisk_log_start': "41'6000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18337792, 'num_objects': 4477, 'num_object_clones': 0, 'num_object_copies': 8954, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4477, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12661, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 534}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:09.804 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:33:09.804 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'16130", 'reported_seq': 16344, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:07.898733+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:33:07.898733+0000', 'last_peered': '2024-09-17T15:33:07.898733+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:33:07.898733+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "41'6100", 'ondisk_log_start': "41'6100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17977827, 'num_objects': 4390, 'num_object_clones': 0, 'num_object_copies': 8780, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4034, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4390, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14025, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 357}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:33:09.804 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-17T15:33:09.804 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'16290", 'reported_seq': 24551, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:07.947909+0000', 'last_change': '2024-09-17T15:30:55.086473+0000', 'last_active': '2024-09-17T15:33:07.947909+0000', 'last_peered': '2024-09-17T15:33:07.947909+0000', 'last_clean': '2024-09-17T15:26:20.164921+0000', 'last_became_active': '2024-09-17T15:26:22.576469+0000', 'last_became_peered': '2024-09-17T15:26:22.576469+0000', 'last_unstale': '2024-09-17T15:33:07.947909+0000', 'last_undegraded': '2024-09-17T15:26:22.232025+0000', 'last_fullsized': '2024-09-17T15:26:22.189170+0000', 'mapping_epoch': 40, 'log_start': "41'6200", 'ondisk_log_start': "41'6200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2275", 'last_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:56.130304+0000', 'objects_scrubbed': 2232, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 749, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18210816, 'num_objects': 4446, 'num_object_clones': 0, 'num_object_copies': 8892, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 71, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4446, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14052, 'num_write_kb': 32532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4068, 'num_bytes_recovered': 8073216, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 4375}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:33:09.804 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:33:09.804 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'16508", 'reported_seq': 16811, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:08.069937+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:33:08.069937+0000', 'last_peered': '2024-09-17T15:33:08.069937+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:33:08.069937+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "41'6500", 'ondisk_log_start': "41'6500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18259968, 'num_objects': 4458, 'num_object_clones': 0, 'num_object_copies': 8916, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4458, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13056, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 401}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:09.805 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:33:09.805 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'16324", 'reported_seq': 16593, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:08.091680+0000', 'last_change': '2024-09-17T15:26:22.682099+0000', 'last_active': '2024-09-17T15:33:08.091680+0000', 'last_peered': '2024-09-17T15:33:08.091680+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:33:08.091680+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "41'6300", 'ondisk_log_start': "41'6300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18219008, 'num_objects': 4448, 'num_object_clones': 0, 'num_object_copies': 8896, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4098, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4448, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12611, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 62, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 353}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:09.805 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:33:09.805 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'16413", 'reported_seq': 16714, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:08.092797+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:33:08.092797+0000', 'last_peered': '2024-09-17T15:33:08.092797+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:33:08.092797+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "41'6400", 'ondisk_log_start': "41'6400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18321408, 'num_objects': 4473, 'num_object_clones': 0, 'num_object_copies': 8946, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4473, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13200, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 493}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:09.805 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:33:10.485 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:33:10.485 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36580/143486 objects degraded (25.494%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":527,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":71743,"data_bytes":293810953,"bytes_used":1911906304,"bytes_avail":287998386176,"bytes_total":289910292480,"degraded_objects":36580,"degraded_total":143486,"degraded_ratio":0.25493776396303475,"recovering_objects_per_sec":46,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":563,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":578},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:33:10.513 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:33:10.513 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:33:11.021 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:33:11.021 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:33:11.036 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2024-09-17T15:33:10.227532+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":293810953,"num_objects":71743,"num_object_clones":0,"num_object_copies":143486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36580,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71743,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":221248,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12881,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160714,"ondisk_log_size":160714,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1867096,"kb_used_data":775268,"kb_used_omap":13,"kb_used_meta":1091762,"kb_avail":281248424,"statfs":{"total":289910292480,"available":287998386176,"internally_reserved":0,"allocated":793874432,"data_stored":792968155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1117965311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,1,8,10,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19685376,"num_objects":-4806,"num_object_clones":0,"num_object_copies":-9612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-409,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4806,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":409,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.490470"},"pg_stats":[{"pgid":"2.f","version":"45'16202","reported_seq":16457,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:08.076486+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:33:08.076486+0000","last_peered":"2024-09-17T15:33:08.076486+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:33:08.076486+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'6200","ondisk_log_start":"41'6200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18210816,"num_objects":4446,"num_object_clones":0,"num_object_copies":8892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16202,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":462}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'16339","reported_seq":16447,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:33:07.940366+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:33:07.940366+0000","last_peered":"2024-09-17T15:33:07.940366+0000","last_clean":"2024-09-17T15:33:07.940366+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:33:07.940366+0000","last_undegraded":"2024-09-17T15:33:07.940366+0000","last_fullsized":"2024-09-17T15:33:07.940366+0000","mapping_epoch":33,"log_start":"41'6300","ondisk_log_start":"41'6300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18813894,"num_objects":4595,"num_object_clones":0,"num_object_copies":9190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4595,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14696,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'16430","reported_seq":16648,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:07.877800+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:33:07.877800+0000","last_peered":"2024-09-17T15:33:07.877800+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:33:07.877800+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'6400","ondisk_log_start":"41'6400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18444312,"num_objects":4504,"num_object_clones":0,"num_object_copies":9008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4504,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14587,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":379}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'16428","reported_seq":16503,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:33:07.901030+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:33:07.901030+0000","last_peered":"2024-09-17T15:33:07.901030+0000","last_clean":"2024-09-17T15:33:07.901030+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:33:07.901030+0000","last_undegraded":"2024-09-17T15:33:07.901030+0000","last_fullsized":"2024-09-17T15:33:07.901030+0000","mapping_epoch":33,"log_start":"41'6400","ondisk_log_start":"41'6400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18604032,"num_objects":4542,"num_object_clones":0,"num_object_copies":9084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16428,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'16155","reported_seq":16290,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:33:08.069867+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:33:08.069867+0000","last_peered":"2024-09-17T15:33:08.069867+0000","last_clean":"2024-09-17T15:33:08.069867+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:33:08.069867+0000","last_undegraded":"2024-09-17T15:33:08.069867+0000","last_fullsized":"2024-09-17T15:33:08.069867+0000","mapping_epoch":32,"log_start":"41'6100","ondisk_log_start":"41'6100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18424291,"num_objects":4499,"num_object_clones":0,"num_object_copies":8998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4499,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13137,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'16679","reported_seq":16974,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:08.068671+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:33:08.068671+0000","last_peered":"2024-09-17T15:33:08.068671+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:33:08.068671+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'6600","ondisk_log_start":"41'6600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18727395,"num_objects":4573,"num_object_clones":0,"num_object_copies":9146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4573,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13178,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":381},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'16387","reported_seq":16701,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:08.055843+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:33:08.055843+0000","last_peered":"2024-09-17T15:33:08.055843+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:33:08.055843+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'6300","ondisk_log_start":"41'6300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17945542,"num_objects":4383,"num_object_clones":0,"num_object_copies":8766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4383,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13156,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":294},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'16389","reported_seq":16522,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:33:08.042275+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:33:08.042275+0000","last_peered":"2024-09-17T15:33:08.042275+0000","last_clean":"2024-09-17T15:33:08.042275+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:33:08.042275+0000","last_undegraded":"2024-09-17T15:33:08.042275+0000","last_fullsized":"2024-09-17T15:33:08.042275+0000","mapping_epoch":32,"log_start":"41'6300","ondisk_log_start":"41'6300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18542592,"num_objects":4527,"num_object_clones":0,"num_object_copies":9054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4527,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13343,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087024+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:30:57.087024+0000","last_peered":"2024-09-17T15:30:57.087024+0000","last_clean":"2024-09-17T15:30:57.087024+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:30:57.087024+0000","last_undegraded":"2024-09-17T15:30:57.087024+0000","last_fullsized":"2024-09-17T15:30:57.087024+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'16256","reported_seq":24544,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:33:07.529595+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:33:07.529595+0000","last_peered":"2024-09-17T15:33:07.529595+0000","last_clean":"2024-09-17T15:33:07.529595+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:33:07.529595+0000","last_undegraded":"2024-09-17T15:33:07.529595+0000","last_fullsized":"2024-09-17T15:33:07.529595+0000","mapping_epoch":42,"log_start":"41'6200","ondisk_log_start":"41'6200","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18682339,"num_objects":4562,"num_object_clones":0,"num_object_copies":9124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4562,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14256,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088545+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:30:57.088545+0000","last_peered":"2024-09-17T15:30:57.088545+0000","last_clean":"2024-09-17T15:30:57.088545+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:30:57.088545+0000","last_undegraded":"2024-09-17T15:30:57.088545+0000","last_fullsized":"2024-09-17T15:30:57.088545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'16065","reported_seq":16334,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:08.021340+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:33:08.021340+0000","last_peered":"2024-09-17T15:33:08.021340+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:33:08.021340+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'6000","ondisk_log_start":"41'6000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18337792,"num_objects":4477,"num_object_clones":0,"num_object_copies":8954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12661,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":534}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":108,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.092584+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:30:57.092584+0000","last_peered":"2024-09-17T15:30:57.092584+0000","last_clean":"2024-09-17T15:30:57.092584+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:30:57.092584+0000","last_undegraded":"2024-09-17T15:30:57.092584+0000","last_fullsized":"2024-09-17T15:30:57.092584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'16130","reported_seq":16344,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:07.898733+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:33:07.898733+0000","last_peered":"2024-09-17T15:33:07.898733+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:33:07.898733+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'6100","ondisk_log_start":"41'6100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17977827,"num_objects":4390,"num_object_clones":0,"num_object_copies":8780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4390,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14025,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":357}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":83,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087587+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:30:57.087587+0000","last_peered":"2024-09-17T15:30:57.087587+0000","last_clean":"2024-09-17T15:30:57.087587+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:30:57.087587+0000","last_undegraded":"2024-09-17T15:30:57.087587+0000","last_fullsized":"2024-09-17T15:30:57.087587+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'16290","reported_seq":24551,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:07.947909+0000","last_change":"2024-09-17T15:30:55.086473+0000","last_active":"2024-09-17T15:33:07.947909+0000","last_peered":"2024-09-17T15:33:07.947909+0000","last_clean":"2024-09-17T15:26:20.164921+0000","last_became_active":"2024-09-17T15:26:22.576469+0000","last_became_peered":"2024-09-17T15:26:22.576469+0000","last_unstale":"2024-09-17T15:33:07.947909+0000","last_undegraded":"2024-09-17T15:26:22.232025+0000","last_fullsized":"2024-09-17T15:26:22.189170+0000","mapping_epoch":40,"log_start":"41'6200","ondisk_log_start":"41'6200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18210816,"num_objects":4446,"num_object_clones":0,"num_object_copies":8892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":71,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14052,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4068,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4375}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'16508","reported_seq":16811,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:08.069937+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:33:08.069937+0000","last_peered":"2024-09-17T15:33:08.069937+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:33:08.069937+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'6500","ondisk_log_start":"41'6500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18259968,"num_objects":4458,"num_object_clones":0,"num_object_copies":8916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13056,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":401}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087765+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:30:57.087765+0000","last_peered":"2024-09-17T15:30:57.087765+0000","last_clean":"2024-09-17T15:30:57.087765+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:30:57.087765+0000","last_undegraded":"2024-09-17T15:30:57.087765+0000","last_fullsized":"2024-09-17T15:30:57.087765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'16324","reported_seq":16593,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:08.091680+0000","last_change":"2024-09-17T15:26:22.682099+0000","last_active":"2024-09-17T15:33:08.091680+0000","last_peered":"2024-09-17T15:33:08.091680+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:33:08.091680+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'6300","ondisk_log_start":"41'6300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18219008,"num_objects":4448,"num_object_clones":0,"num_object_copies":8896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12611,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":353}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088891+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:30:57.088891+0000","last_peered":"2024-09-17T15:30:57.088891+0000","last_clean":"2024-09-17T15:30:57.088891+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:30:57.088891+0000","last_undegraded":"2024-09-17T15:30:57.088891+0000","last_fullsized":"2024-09-17T15:30:57.088891+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'16413","reported_seq":16714,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:08.092797+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:33:08.092797+0000","last_peered":"2024-09-17T15:33:08.092797+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:33:08.092797+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'6400","ondisk_log_start":"41'6400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18321408,"num_objects":4473,"num_object_clones":0,"num_object_copies":8946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13200,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":493}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.087804+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:30:57.087804+0000","last_peered":"2024-09-17T15:30:57.087804+0000","last_clean":"2024-09-17T15:30:57.087804+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:30:57.087804+0000","last_undegraded":"2024-09-17T15:30:57.087804+0000","last_fullsized":"2024-09-17T15:30:57.087804+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:30:57.088875+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:30:57.088875+0000","last_peered":"2024-09-17T15:30:57.088875+0000","last_clean":"2024-09-17T15:30:57.088875+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:30:57.088875+0000","last_undegraded":"2024-09-17T15:30:57.088875+0000","last_fullsized":"2024-09-17T15:30:57.088875+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'16216","reported_seq":24596,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T15:33:07.548353+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:33:07.548353+0000","last_peered":"2024-09-17T15:33:07.548353+0000","last_clean":"2024-09-17T15:33:07.548353+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:33:07.548353+0000","last_undegraded":"2024-09-17T15:33:07.548353+0000","last_fullsized":"2024-09-17T15:33:07.548353+0000","mapping_epoch":44,"log_start":"41'6200","ondisk_log_start":"41'6200","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18088902,"num_objects":4418,"num_object_clones":0,"num_object_copies":8836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12658,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":293810934,"num_objects":71741,"num_object_clones":0,"num_object_copies":143482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36580,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71741,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":221246,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12880,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":790769664,"data_stored":790624226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160711,"ondisk_log_size":160711,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051681,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":593068,"kb_used_data":276764,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93778772,"statfs":{"total":96636764160,"available":96029462528,"internally_reserved":0,"allocated":283406336,"data_stored":283115239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":323874476},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724626,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576392,"kb_used_data":215668,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93795448,"statfs":{"total":96636764160,"available":96046538752,"internally_reserved":0,"allocated":220844032,"data_stored":220519954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":369356459},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,3,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247909,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":697636,"kb_used_data":282836,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93674204,"statfs":{"total":96636764160,"available":95922384896,"internally_reserved":0,"allocated":289624064,"data_stored":289332962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":424734376},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":287784960,"data_stored":287748830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":220172288,"data_stored":220103182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":282812416,"data_stored":282772214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:33:14.038 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:33:14.579 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:33:14.580 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:33:14.609 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":305,"stamp":"2024-09-17T15:33:13.741275+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":284242697,"num_objects":69407,"num_object_clones":0,"num_object_copies":138814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36418,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69407,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":223584,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13043,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160750,"ondisk_log_size":160750,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1866428,"kb_used_data":756536,"kb_used_omap":13,"kb_used_meta":1109810,"kb_avail":281249092,"statfs":{"total":289910292480,"available":287999070208,"internally_reserved":0,"allocated":774692864,"data_stored":773765923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1136446463},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,6,12,6,9,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19218432,"num_objects":-4692,"num_object_clones":0,"num_object_copies":-9384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-377,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4692,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":377,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002253"},"pg_stats":[{"pgid":"2.f","version":"47'16348","reported_seq":16606,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:13.100459+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:33:13.100459+0000","last_peered":"2024-09-17T15:33:13.100459+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:33:13.100459+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'6300","ondisk_log_start":"41'6300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17612800,"num_objects":4300,"num_object_clones":0,"num_object_copies":8600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16348,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":316}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16500","reported_seq":16611,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:12.960536+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:33:12.960536+0000","last_peered":"2024-09-17T15:33:12.960536+0000","last_clean":"2024-09-17T15:33:12.960536+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:33:12.960536+0000","last_undegraded":"2024-09-17T15:33:12.960536+0000","last_fullsized":"2024-09-17T15:33:12.960536+0000","mapping_epoch":33,"log_start":"41'6400","ondisk_log_start":"41'6400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18154438,"num_objects":4434,"num_object_clones":0,"num_object_copies":8868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4434,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14857,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16565","reported_seq":16787,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:12.921509+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:33:12.921509+0000","last_peered":"2024-09-17T15:33:12.921509+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:33:12.921509+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'6500","ondisk_log_start":"41'6500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17891352,"num_objects":4369,"num_object_clones":0,"num_object_copies":8738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4369,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14722,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":244}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16567","reported_seq":16646,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:12.902634+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:33:12.902634+0000","last_peered":"2024-09-17T15:33:12.902634+0000","last_clean":"2024-09-17T15:33:12.902634+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:33:12.902634+0000","last_undegraded":"2024-09-17T15:33:12.902634+0000","last_fullsized":"2024-09-17T15:33:12.902634+0000","mapping_epoch":33,"log_start":"41'6500","ondisk_log_start":"41'6500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18034688,"num_objects":4403,"num_object_clones":0,"num_object_copies":8806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4403,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16567,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16320","reported_seq":16459,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:13.092068+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:33:13.092068+0000","last_peered":"2024-09-17T15:33:13.092068+0000","last_clean":"2024-09-17T15:33:13.092068+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:33:13.092068+0000","last_undegraded":"2024-09-17T15:33:13.092068+0000","last_fullsized":"2024-09-17T15:33:13.092068+0000","mapping_epoch":32,"log_start":"41'6300","ondisk_log_start":"41'6300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17748451,"num_objects":4334,"num_object_clones":0,"num_object_copies":8668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4334,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13302,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16829","reported_seq":17128,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:13.085067+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:33:13.085067+0000","last_peered":"2024-09-17T15:33:13.085067+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:33:13.085067+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'6800","ondisk_log_start":"41'6800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18112995,"num_objects":4423,"num_object_clones":0,"num_object_copies":8846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4423,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13328,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":231},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16549","reported_seq":16867,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:13.074818+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:33:13.074818+0000","last_peered":"2024-09-17T15:33:13.074818+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:33:13.074818+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'6500","ondisk_log_start":"41'6500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17281990,"num_objects":4221,"num_object_clones":0,"num_object_copies":8442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4221,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13318,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":132},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16516","reported_seq":16653,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:13.048733+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:33:13.048733+0000","last_peered":"2024-09-17T15:33:13.048733+0000","last_clean":"2024-09-17T15:33:13.048733+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:33:13.048733+0000","last_undegraded":"2024-09-17T15:33:13.048733+0000","last_fullsized":"2024-09-17T15:33:13.048733+0000","mapping_epoch":32,"log_start":"41'6500","ondisk_log_start":"41'6500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18022400,"num_objects":4400,"num_object_clones":0,"num_object_copies":8800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13470,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223906+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.223906+0000","last_peered":"2024-09-17T15:33:11.223906+0000","last_clean":"2024-09-17T15:33:11.223906+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.223906+0000","last_undegraded":"2024-09-17T15:33:11.223906+0000","last_fullsized":"2024-09-17T15:33:11.223906+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16406","reported_seq":24698,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:12.576890+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:33:12.576890+0000","last_peered":"2024-09-17T15:33:12.576890+0000","last_clean":"2024-09-17T15:33:12.576890+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:33:12.576890+0000","last_undegraded":"2024-09-17T15:33:12.576890+0000","last_fullsized":"2024-09-17T15:33:12.576890+0000","mapping_epoch":42,"log_start":"41'6400","ondisk_log_start":"41'6400","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18067939,"num_objects":4412,"num_object_clones":0,"num_object_copies":8824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4412,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14406,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225444+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225444+0000","last_peered":"2024-09-17T15:33:11.225444+0000","last_clean":"2024-09-17T15:33:11.225444+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225444+0000","last_undegraded":"2024-09-17T15:33:11.225444+0000","last_fullsized":"2024-09-17T15:33:11.225444+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16204","reported_seq":16476,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:13.050495+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:33:13.050495+0000","last_peered":"2024-09-17T15:33:13.050495+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:33:13.050495+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'6200","ondisk_log_start":"41'6200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17768448,"num_objects":4338,"num_object_clones":0,"num_object_copies":8676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12800,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":395}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":112,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.228064+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.228064+0000","last_peered":"2024-09-17T15:33:11.228064+0000","last_clean":"2024-09-17T15:33:11.228064+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.228064+0000","last_undegraded":"2024-09-17T15:33:11.228064+0000","last_fullsized":"2024-09-17T15:33:11.228064+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16275","reported_seq":16490,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:12.941727+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:33:12.941727+0000","last_peered":"2024-09-17T15:33:12.941727+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:33:12.941727+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'6200","ondisk_log_start":"41'6200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17383907,"num_objects":4245,"num_object_clones":0,"num_object_copies":8490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4245,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14170,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":212}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16416","reported_seq":24830,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:12.777514+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:33:12.777514+0000","last_peered":"2024-09-17T15:33:12.777514+0000","last_clean":"2024-09-17T15:33:12.777514+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:33:12.777514+0000","last_undegraded":"2024-09-17T15:33:12.777514+0000","last_fullsized":"2024-09-17T15:33:12.777514+0000","mapping_epoch":46,"log_start":"41'6400","ondisk_log_start":"41'6400","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17694720,"num_objects":4320,"num_object_clones":0,"num_object_copies":8640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14178,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16663","reported_seq":16969,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:13.099085+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:33:13.099085+0000","last_peered":"2024-09-17T15:33:13.099085+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:33:13.099085+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'6600","ondisk_log_start":"41'6600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17625088,"num_objects":4303,"num_object_clones":0,"num_object_copies":8606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13211,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16471","reported_seq":16929,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:13.086677+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:33:13.086677+0000","last_peered":"2024-09-17T15:33:13.086677+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:33:13.086677+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'6400","ondisk_log_start":"41'6400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17616896,"num_objects":4301,"num_object_clones":0,"num_object_copies":8602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4007,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4301,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12758,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":294}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225014+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225014+0000","last_peered":"2024-09-17T15:33:11.225014+0000","last_clean":"2024-09-17T15:33:11.225014+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225014+0000","last_undegraded":"2024-09-17T15:33:11.225014+0000","last_fullsized":"2024-09-17T15:33:11.225014+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16562","reported_seq":16869,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:13.066278+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:33:13.066278+0000","last_peered":"2024-09-17T15:33:13.066278+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:33:13.066278+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'6500","ondisk_log_start":"41'6500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17711104,"num_objects":4324,"num_object_clones":0,"num_object_copies":8648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13349,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":344}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16356","reported_seq":24740,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:12.538441+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:33:12.538441+0000","last_peered":"2024-09-17T15:33:12.538441+0000","last_clean":"2024-09-17T15:33:12.538441+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:33:12.538441+0000","last_undegraded":"2024-09-17T15:33:12.538441+0000","last_fullsized":"2024-09-17T15:33:12.538441+0000","mapping_epoch":44,"log_start":"41'6300","ondisk_log_start":"41'6300","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17515462,"num_objects":4278,"num_object_clones":0,"num_object_copies":8556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4278,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12798,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":284242678,"num_objects":69405,"num_object_clones":0,"num_object_copies":138810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36418,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69405,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":223582,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13042,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":771534848,"data_stored":771400249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160747,"ondisk_log_size":160747,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051682,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610644,"kb_used_data":272132,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93761196,"statfs":{"total":96636764160,"available":96011464704,"internally_reserved":0,"allocated":278663168,"data_stored":278361570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":346615468},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,3,0,4,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724627,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":561076,"kb_used_data":208656,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93810764,"statfs":{"total":96636764160,"available":96062222336,"internally_reserved":0,"allocated":213663744,"data_stored":213340004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":360836779},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,5,2,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247910,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":694708,"kb_used_data":275748,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93677132,"statfs":{"total":96636764160,"available":95925383168,"internally_reserved":0,"allocated":282365952,"data_stored":282064349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":428994216},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":280485888,"data_stored":280449758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":212955136,"data_stored":212896869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":278093824,"data_stored":278053622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:33:14.610 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:33:15.169 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:33:15.170 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:33:15.193 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":305,"stamp":"2024-09-17T15:33:13.741275+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":284242697,"num_objects":69407,"num_object_clones":0,"num_object_copies":138814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36418,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69407,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":223584,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13043,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160750,"ondisk_log_size":160750,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1866428,"kb_used_data":756536,"kb_used_omap":13,"kb_used_meta":1109810,"kb_avail":281249092,"statfs":{"total":289910292480,"available":287999070208,"internally_reserved":0,"allocated":774692864,"data_stored":773765923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1136446463},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,6,12,6,9,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19218432,"num_objects":-4692,"num_object_clones":0,"num_object_copies":-9384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-377,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4692,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":377,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002253"},"pg_stats":[{"pgid":"2.f","version":"47'16348","reported_seq":16606,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:13.100459+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:33:13.100459+0000","last_peered":"2024-09-17T15:33:13.100459+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:33:13.100459+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'6300","ondisk_log_start":"41'6300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17612800,"num_objects":4300,"num_object_clones":0,"num_object_copies":8600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16348,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":316}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16500","reported_seq":16611,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:12.960536+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:33:12.960536+0000","last_peered":"2024-09-17T15:33:12.960536+0000","last_clean":"2024-09-17T15:33:12.960536+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:33:12.960536+0000","last_undegraded":"2024-09-17T15:33:12.960536+0000","last_fullsized":"2024-09-17T15:33:12.960536+0000","mapping_epoch":33,"log_start":"41'6400","ondisk_log_start":"41'6400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18154438,"num_objects":4434,"num_object_clones":0,"num_object_copies":8868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4434,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14857,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16565","reported_seq":16787,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:12.921509+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:33:12.921509+0000","last_peered":"2024-09-17T15:33:12.921509+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:33:12.921509+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'6500","ondisk_log_start":"41'6500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17891352,"num_objects":4369,"num_object_clones":0,"num_object_copies":8738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4369,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14722,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":244}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16567","reported_seq":16646,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:12.902634+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:33:12.902634+0000","last_peered":"2024-09-17T15:33:12.902634+0000","last_clean":"2024-09-17T15:33:12.902634+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:33:12.902634+0000","last_undegraded":"2024-09-17T15:33:12.902634+0000","last_fullsized":"2024-09-17T15:33:12.902634+0000","mapping_epoch":33,"log_start":"41'6500","ondisk_log_start":"41'6500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18034688,"num_objects":4403,"num_object_clones":0,"num_object_copies":8806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4403,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16567,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16320","reported_seq":16459,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:13.092068+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:33:13.092068+0000","last_peered":"2024-09-17T15:33:13.092068+0000","last_clean":"2024-09-17T15:33:13.092068+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:33:13.092068+0000","last_undegraded":"2024-09-17T15:33:13.092068+0000","last_fullsized":"2024-09-17T15:33:13.092068+0000","mapping_epoch":32,"log_start":"41'6300","ondisk_log_start":"41'6300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17748451,"num_objects":4334,"num_object_clones":0,"num_object_copies":8668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4334,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13302,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16829","reported_seq":17128,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:13.085067+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:33:13.085067+0000","last_peered":"2024-09-17T15:33:13.085067+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:33:13.085067+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'6800","ondisk_log_start":"41'6800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18112995,"num_objects":4423,"num_object_clones":0,"num_object_copies":8846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4423,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13328,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":231},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16549","reported_seq":16867,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:13.074818+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:33:13.074818+0000","last_peered":"2024-09-17T15:33:13.074818+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:33:13.074818+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'6500","ondisk_log_start":"41'6500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17281990,"num_objects":4221,"num_object_clones":0,"num_object_copies":8442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4221,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13318,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":132},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16516","reported_seq":16653,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:13.048733+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:33:13.048733+0000","last_peered":"2024-09-17T15:33:13.048733+0000","last_clean":"2024-09-17T15:33:13.048733+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:33:13.048733+0000","last_undegraded":"2024-09-17T15:33:13.048733+0000","last_fullsized":"2024-09-17T15:33:13.048733+0000","mapping_epoch":32,"log_start":"41'6500","ondisk_log_start":"41'6500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18022400,"num_objects":4400,"num_object_clones":0,"num_object_copies":8800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13470,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223906+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.223906+0000","last_peered":"2024-09-17T15:33:11.223906+0000","last_clean":"2024-09-17T15:33:11.223906+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.223906+0000","last_undegraded":"2024-09-17T15:33:11.223906+0000","last_fullsized":"2024-09-17T15:33:11.223906+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16406","reported_seq":24698,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:12.576890+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:33:12.576890+0000","last_peered":"2024-09-17T15:33:12.576890+0000","last_clean":"2024-09-17T15:33:12.576890+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:33:12.576890+0000","last_undegraded":"2024-09-17T15:33:12.576890+0000","last_fullsized":"2024-09-17T15:33:12.576890+0000","mapping_epoch":42,"log_start":"41'6400","ondisk_log_start":"41'6400","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18067939,"num_objects":4412,"num_object_clones":0,"num_object_copies":8824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4412,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14406,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225444+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225444+0000","last_peered":"2024-09-17T15:33:11.225444+0000","last_clean":"2024-09-17T15:33:11.225444+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225444+0000","last_undegraded":"2024-09-17T15:33:11.225444+0000","last_fullsized":"2024-09-17T15:33:11.225444+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16204","reported_seq":16476,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:13.050495+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:33:13.050495+0000","last_peered":"2024-09-17T15:33:13.050495+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:33:13.050495+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'6200","ondisk_log_start":"41'6200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17768448,"num_objects":4338,"num_object_clones":0,"num_object_copies":8676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12800,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":395}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":112,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.228064+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.228064+0000","last_peered":"2024-09-17T15:33:11.228064+0000","last_clean":"2024-09-17T15:33:11.228064+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.228064+0000","last_undegraded":"2024-09-17T15:33:11.228064+0000","last_fullsized":"2024-09-17T15:33:11.228064+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16275","reported_seq":16490,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:12.941727+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:33:12.941727+0000","last_peered":"2024-09-17T15:33:12.941727+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:33:12.941727+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'6200","ondisk_log_start":"41'6200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17383907,"num_objects":4245,"num_object_clones":0,"num_object_copies":8490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4245,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14170,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":212}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16416","reported_seq":24830,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:12.777514+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:33:12.777514+0000","last_peered":"2024-09-17T15:33:12.777514+0000","last_clean":"2024-09-17T15:33:12.777514+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:33:12.777514+0000","last_undegraded":"2024-09-17T15:33:12.777514+0000","last_fullsized":"2024-09-17T15:33:12.777514+0000","mapping_epoch":46,"log_start":"41'6400","ondisk_log_start":"41'6400","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17694720,"num_objects":4320,"num_object_clones":0,"num_object_copies":8640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14178,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16663","reported_seq":16969,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:13.099085+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:33:13.099085+0000","last_peered":"2024-09-17T15:33:13.099085+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:33:13.099085+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'6600","ondisk_log_start":"41'6600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17625088,"num_objects":4303,"num_object_clones":0,"num_object_copies":8606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13211,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16471","reported_seq":16929,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:13.086677+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:33:13.086677+0000","last_peered":"2024-09-17T15:33:13.086677+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:33:13.086677+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'6400","ondisk_log_start":"41'6400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17616896,"num_objects":4301,"num_object_clones":0,"num_object_copies":8602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4007,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4301,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12758,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":294}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225014+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225014+0000","last_peered":"2024-09-17T15:33:11.225014+0000","last_clean":"2024-09-17T15:33:11.225014+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225014+0000","last_undegraded":"2024-09-17T15:33:11.225014+0000","last_fullsized":"2024-09-17T15:33:11.225014+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16562","reported_seq":16869,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:13.066278+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:33:13.066278+0000","last_peered":"2024-09-17T15:33:13.066278+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:33:13.066278+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'6500","ondisk_log_start":"41'6500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17711104,"num_objects":4324,"num_object_clones":0,"num_object_copies":8648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13349,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":344}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16356","reported_seq":24740,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:12.538441+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:33:12.538441+0000","last_peered":"2024-09-17T15:33:12.538441+0000","last_clean":"2024-09-17T15:33:12.538441+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:33:12.538441+0000","last_undegraded":"2024-09-17T15:33:12.538441+0000","last_fullsized":"2024-09-17T15:33:12.538441+0000","mapping_epoch":44,"log_start":"41'6300","ondisk_log_start":"41'6300","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17515462,"num_objects":4278,"num_object_clones":0,"num_object_copies":8556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4278,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12798,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":284242678,"num_objects":69405,"num_object_clones":0,"num_object_copies":138810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36418,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69405,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":223582,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13042,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":771534848,"data_stored":771400249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160747,"ondisk_log_size":160747,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051682,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610644,"kb_used_data":272132,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93761196,"statfs":{"total":96636764160,"available":96011464704,"internally_reserved":0,"allocated":278663168,"data_stored":278361570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":346615468},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,3,0,4,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724627,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":561076,"kb_used_data":208656,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93810764,"statfs":{"total":96636764160,"available":96062222336,"internally_reserved":0,"allocated":213663744,"data_stored":213340004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":360836779},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,5,2,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247910,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":694708,"kb_used_data":275748,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93677132,"statfs":{"total":96636764160,"available":95925383168,"internally_reserved":0,"allocated":282365952,"data_stored":282064349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":428994216},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":280485888,"data_stored":280449758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":212955136,"data_stored":212896869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":278093824,"data_stored":278053622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:33:15.194 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:33:15.194 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'16348", 'reported_seq': 16606, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:13.100459+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:33:13.100459+0000', 'last_peered': '2024-09-17T15:33:13.100459+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:33:13.100459+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "41'6300", 'ondisk_log_start': "41'6300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17612800, 'num_objects': 4300, 'num_object_clones': 0, 'num_object_copies': 8600, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4300, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16348, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 316}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:15.194 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:33:15.194 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'16565", 'reported_seq': 16787, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:12.921509+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:33:12.921509+0000', 'last_peered': '2024-09-17T15:33:12.921509+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:33:12.921509+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "41'6500", 'ondisk_log_start': "41'6500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17891352, 'num_objects': 4369, 'num_object_clones': 0, 'num_object_copies': 8738, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4369, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14722, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 244}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:33:15.195 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:33:15.195 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'16829", 'reported_seq': 17128, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:13.085067+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:33:13.085067+0000', 'last_peered': '2024-09-17T15:33:13.085067+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:33:13.085067+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "41'6800", 'ondisk_log_start': "41'6800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18112995, 'num_objects': 4423, 'num_object_clones': 0, 'num_object_copies': 8846, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4423, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13328, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 231}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:15.195 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:33:15.195 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'16549", 'reported_seq': 16867, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:13.074818+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:33:13.074818+0000', 'last_peered': '2024-09-17T15:33:13.074818+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:33:13.074818+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "41'6500", 'ondisk_log_start': "41'6500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17281990, 'num_objects': 4221, 'num_object_clones': 0, 'num_object_copies': 8442, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4089, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4221, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 13318, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 85, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 132}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:15.195 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:33:15.195 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'16204", 'reported_seq': 16476, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:13.050495+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:33:13.050495+0000', 'last_peered': '2024-09-17T15:33:13.050495+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:33:13.050495+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "41'6200", 'ondisk_log_start': "41'6200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17768448, 'num_objects': 4338, 'num_object_clones': 0, 'num_object_copies': 8676, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4338, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12800, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 395}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:15.195 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:33:15.195 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'16275", 'reported_seq': 16490, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:12.941727+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:33:12.941727+0000', 'last_peered': '2024-09-17T15:33:12.941727+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:33:12.941727+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "41'6200", 'ondisk_log_start': "41'6200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17383907, 'num_objects': 4245, 'num_object_clones': 0, 'num_object_copies': 8490, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4034, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4245, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14170, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 212}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:33:15.196 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:33:15.196 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'16663", 'reported_seq': 16969, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:13.099085+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:33:13.099085+0000', 'last_peered': '2024-09-17T15:33:13.099085+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:33:13.099085+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "41'6600", 'ondisk_log_start': "41'6600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17625088, 'num_objects': 4303, 'num_object_clones': 0, 'num_object_copies': 8606, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4303, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13211, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 246}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:15.196 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:33:15.196 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'16471", 'reported_seq': 16929, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:13.086677+0000', 'last_change': '2024-09-17T15:33:09.763500+0000', 'last_active': '2024-09-17T15:33:13.086677+0000', 'last_peered': '2024-09-17T15:33:13.086677+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:33:13.086677+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "41'6400", 'ondisk_log_start': "41'6400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17616896, 'num_objects': 4301, 'num_object_clones': 0, 'num_object_copies': 8602, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4007, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4301, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12758, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 153, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 294}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:15.196 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:33:15.196 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'16562", 'reported_seq': 16869, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:13.066278+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:33:13.066278+0000', 'last_peered': '2024-09-17T15:33:13.066278+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:33:13.066278+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "41'6500", 'ondisk_log_start': "41'6500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17711104, 'num_objects': 4324, 'num_object_clones': 0, 'num_object_copies': 8648, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4324, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13349, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 344}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:15.196 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:33:15.920 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:33:15.920 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36418/138814 objects degraded (26.235%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":532,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":69407,"data_bytes":284242697,"bytes_used":1911222272,"bytes_avail":287999070208,"bytes_total":289910292480,"degraded_objects":36418,"degraded_total":138814,"degraded_ratio":0.26235105969138561,"recovering_objects_per_sec":47,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":377,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":586},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:33:15.940 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:33:15.941 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:33:16.495 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:33:16.498 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:33:16.516 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":306,"stamp":"2024-09-17T15:33:15.750102+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":284242697,"num_objects":69407,"num_object_clones":0,"num_object_copies":138814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36418,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69407,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":223584,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13043,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160750,"ondisk_log_size":160750,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1866428,"kb_used_data":756536,"kb_used_omap":13,"kb_used_meta":1109810,"kb_avail":281249092,"statfs":{"total":289910292480,"available":287999070208,"internally_reserved":0,"allocated":774692864,"data_stored":773765923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1136446463},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,6,12,6,9,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18022400,"num_objects":-4400,"num_object_clones":0,"num_object_copies":-8800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-377,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4400,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":377,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.010425"},"pg_stats":[{"pgid":"2.f","version":"47'16348","reported_seq":16606,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:13.100459+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:33:13.100459+0000","last_peered":"2024-09-17T15:33:13.100459+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:33:13.100459+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'6300","ondisk_log_start":"41'6300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17612800,"num_objects":4300,"num_object_clones":0,"num_object_copies":8600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16348,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":316}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16500","reported_seq":16611,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:12.960536+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:33:12.960536+0000","last_peered":"2024-09-17T15:33:12.960536+0000","last_clean":"2024-09-17T15:33:12.960536+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:33:12.960536+0000","last_undegraded":"2024-09-17T15:33:12.960536+0000","last_fullsized":"2024-09-17T15:33:12.960536+0000","mapping_epoch":33,"log_start":"41'6400","ondisk_log_start":"41'6400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18154438,"num_objects":4434,"num_object_clones":0,"num_object_copies":8868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4434,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14857,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16565","reported_seq":16787,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:12.921509+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:33:12.921509+0000","last_peered":"2024-09-17T15:33:12.921509+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:33:12.921509+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'6500","ondisk_log_start":"41'6500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17891352,"num_objects":4369,"num_object_clones":0,"num_object_copies":8738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4369,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14722,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":244}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16567","reported_seq":16646,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:12.902634+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:33:12.902634+0000","last_peered":"2024-09-17T15:33:12.902634+0000","last_clean":"2024-09-17T15:33:12.902634+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:33:12.902634+0000","last_undegraded":"2024-09-17T15:33:12.902634+0000","last_fullsized":"2024-09-17T15:33:12.902634+0000","mapping_epoch":33,"log_start":"41'6500","ondisk_log_start":"41'6500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18034688,"num_objects":4403,"num_object_clones":0,"num_object_copies":8806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4403,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16567,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16320","reported_seq":16459,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:13.092068+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:33:13.092068+0000","last_peered":"2024-09-17T15:33:13.092068+0000","last_clean":"2024-09-17T15:33:13.092068+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:33:13.092068+0000","last_undegraded":"2024-09-17T15:33:13.092068+0000","last_fullsized":"2024-09-17T15:33:13.092068+0000","mapping_epoch":32,"log_start":"41'6300","ondisk_log_start":"41'6300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17748451,"num_objects":4334,"num_object_clones":0,"num_object_copies":8668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4334,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13302,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16829","reported_seq":17128,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:13.085067+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:33:13.085067+0000","last_peered":"2024-09-17T15:33:13.085067+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:33:13.085067+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'6800","ondisk_log_start":"41'6800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18112995,"num_objects":4423,"num_object_clones":0,"num_object_copies":8846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4423,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13328,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":231},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16549","reported_seq":16867,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:13.074818+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:33:13.074818+0000","last_peered":"2024-09-17T15:33:13.074818+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:33:13.074818+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'6500","ondisk_log_start":"41'6500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17281990,"num_objects":4221,"num_object_clones":0,"num_object_copies":8442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4221,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13318,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":132},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16516","reported_seq":16653,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:13.048733+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:33:13.048733+0000","last_peered":"2024-09-17T15:33:13.048733+0000","last_clean":"2024-09-17T15:33:13.048733+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:33:13.048733+0000","last_undegraded":"2024-09-17T15:33:13.048733+0000","last_fullsized":"2024-09-17T15:33:13.048733+0000","mapping_epoch":32,"log_start":"41'6500","ondisk_log_start":"41'6500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18022400,"num_objects":4400,"num_object_clones":0,"num_object_copies":8800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13470,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223906+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.223906+0000","last_peered":"2024-09-17T15:33:11.223906+0000","last_clean":"2024-09-17T15:33:11.223906+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.223906+0000","last_undegraded":"2024-09-17T15:33:11.223906+0000","last_fullsized":"2024-09-17T15:33:11.223906+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16406","reported_seq":24698,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:12.576890+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:33:12.576890+0000","last_peered":"2024-09-17T15:33:12.576890+0000","last_clean":"2024-09-17T15:33:12.576890+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:33:12.576890+0000","last_undegraded":"2024-09-17T15:33:12.576890+0000","last_fullsized":"2024-09-17T15:33:12.576890+0000","mapping_epoch":42,"log_start":"41'6400","ondisk_log_start":"41'6400","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18067939,"num_objects":4412,"num_object_clones":0,"num_object_copies":8824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4412,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14406,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225444+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225444+0000","last_peered":"2024-09-17T15:33:11.225444+0000","last_clean":"2024-09-17T15:33:11.225444+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225444+0000","last_undegraded":"2024-09-17T15:33:11.225444+0000","last_fullsized":"2024-09-17T15:33:11.225444+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16204","reported_seq":16476,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:13.050495+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:33:13.050495+0000","last_peered":"2024-09-17T15:33:13.050495+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:33:13.050495+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'6200","ondisk_log_start":"41'6200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17768448,"num_objects":4338,"num_object_clones":0,"num_object_copies":8676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12800,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":395}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":112,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.228064+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.228064+0000","last_peered":"2024-09-17T15:33:11.228064+0000","last_clean":"2024-09-17T15:33:11.228064+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.228064+0000","last_undegraded":"2024-09-17T15:33:11.228064+0000","last_fullsized":"2024-09-17T15:33:11.228064+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16275","reported_seq":16490,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:12.941727+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:33:12.941727+0000","last_peered":"2024-09-17T15:33:12.941727+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:33:12.941727+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'6200","ondisk_log_start":"41'6200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17383907,"num_objects":4245,"num_object_clones":0,"num_object_copies":8490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4245,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14170,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":212}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16416","reported_seq":24830,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:12.777514+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:33:12.777514+0000","last_peered":"2024-09-17T15:33:12.777514+0000","last_clean":"2024-09-17T15:33:12.777514+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:33:12.777514+0000","last_undegraded":"2024-09-17T15:33:12.777514+0000","last_fullsized":"2024-09-17T15:33:12.777514+0000","mapping_epoch":46,"log_start":"41'6400","ondisk_log_start":"41'6400","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17694720,"num_objects":4320,"num_object_clones":0,"num_object_copies":8640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14178,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16663","reported_seq":16969,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:13.099085+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:33:13.099085+0000","last_peered":"2024-09-17T15:33:13.099085+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:33:13.099085+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'6600","ondisk_log_start":"41'6600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17625088,"num_objects":4303,"num_object_clones":0,"num_object_copies":8606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13211,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16471","reported_seq":16929,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:13.086677+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:33:13.086677+0000","last_peered":"2024-09-17T15:33:13.086677+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:33:13.086677+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'6400","ondisk_log_start":"41'6400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17616896,"num_objects":4301,"num_object_clones":0,"num_object_copies":8602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4007,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4301,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12758,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":294}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225014+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225014+0000","last_peered":"2024-09-17T15:33:11.225014+0000","last_clean":"2024-09-17T15:33:11.225014+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225014+0000","last_undegraded":"2024-09-17T15:33:11.225014+0000","last_fullsized":"2024-09-17T15:33:11.225014+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16562","reported_seq":16869,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:13.066278+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:33:13.066278+0000","last_peered":"2024-09-17T15:33:13.066278+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:33:13.066278+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'6500","ondisk_log_start":"41'6500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17711104,"num_objects":4324,"num_object_clones":0,"num_object_copies":8648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13349,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":344}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16356","reported_seq":24740,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:12.538441+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:33:12.538441+0000","last_peered":"2024-09-17T15:33:12.538441+0000","last_clean":"2024-09-17T15:33:12.538441+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:33:12.538441+0000","last_undegraded":"2024-09-17T15:33:12.538441+0000","last_fullsized":"2024-09-17T15:33:12.538441+0000","mapping_epoch":44,"log_start":"41'6300","ondisk_log_start":"41'6300","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17515462,"num_objects":4278,"num_object_clones":0,"num_object_copies":8556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4278,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12798,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":284242678,"num_objects":69405,"num_object_clones":0,"num_object_copies":138810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36418,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69405,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":223582,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13042,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":771534848,"data_stored":771400249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160747,"ondisk_log_size":160747,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051682,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610644,"kb_used_data":272132,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93761196,"statfs":{"total":96636764160,"available":96011464704,"internally_reserved":0,"allocated":278663168,"data_stored":278361570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":346615468},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,3,0,4,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724627,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":561076,"kb_used_data":208656,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93810764,"statfs":{"total":96636764160,"available":96062222336,"internally_reserved":0,"allocated":213663744,"data_stored":213340004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":360836779},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,5,2,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247910,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":694708,"kb_used_data":275748,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93677132,"statfs":{"total":96636764160,"available":95925383168,"internally_reserved":0,"allocated":282365952,"data_stored":282064349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":428994216},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":280485888,"data_stored":280449758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":212955136,"data_stored":212896869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":278093824,"data_stored":278053622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:33:19.518 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:33:20.053 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:33:20.053 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:33:20.075 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2024-09-17T15:33:19.751058+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":274187017,"num_objects":66952,"num_object_clones":0,"num_object_copies":133904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36326,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66952,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":226039,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13135,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160605,"ondisk_log_size":160605,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1846940,"kb_used_data":737064,"kb_used_omap":13,"kb_used_meta":1109810,"kb_avail":281268580,"statfs":{"total":289910292480,"available":288019025920,"internally_reserved":0,"allocated":754753536,"data_stored":753826595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1136446463},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,0,3,10,7,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19623936,"num_objects":-4791,"num_object_clones":0,"num_object_copies":-9582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-254,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4791,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.523525"},"pg_stats":[{"pgid":"2.f","version":"47'16502","reported_seq":16760,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:18.103254+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:33:18.103254+0000","last_peered":"2024-09-17T15:33:18.103254+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:33:18.103254+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'6500","ondisk_log_start":"41'6500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16982016,"num_objects":4146,"num_object_clones":0,"num_object_copies":8292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16502,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":162}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16652","reported_seq":16764,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:17.935457+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:33:17.935457+0000","last_peered":"2024-09-17T15:33:17.935457+0000","last_clean":"2024-09-17T15:33:17.935457+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:33:17.935457+0000","last_undegraded":"2024-09-17T15:33:17.935457+0000","last_fullsized":"2024-09-17T15:33:17.935457+0000","mapping_epoch":33,"log_start":"41'6600","ondisk_log_start":"41'6600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17531846,"num_objects":4282,"num_object_clones":0,"num_object_copies":8564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4282,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15009,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16705","reported_seq":16927,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:17.756283+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:33:17.756283+0000","last_peered":"2024-09-17T15:33:17.756283+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:33:17.756283+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'6700","ondisk_log_start":"41'6700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17317912,"num_objects":4229,"num_object_clones":0,"num_object_copies":8458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4229,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14862,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":104}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16728","reported_seq":16807,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:17.953239+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:33:17.953239+0000","last_peered":"2024-09-17T15:33:17.953239+0000","last_clean":"2024-09-17T15:33:17.953239+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:33:17.953239+0000","last_undegraded":"2024-09-17T15:33:17.953239+0000","last_fullsized":"2024-09-17T15:33:17.953239+0000","mapping_epoch":33,"log_start":"41'6700","ondisk_log_start":"41'6700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17375232,"num_objects":4242,"num_object_clones":0,"num_object_copies":8484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4242,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16728,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16490","reported_seq":16629,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:18.158160+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:33:18.158160+0000","last_peered":"2024-09-17T15:33:18.158160+0000","last_clean":"2024-09-17T15:33:18.158160+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:33:18.158160+0000","last_undegraded":"2024-09-17T15:33:18.158160+0000","last_fullsized":"2024-09-17T15:33:18.158160+0000","mapping_epoch":32,"log_start":"41'6400","ondisk_log_start":"41'6400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17052131,"num_objects":4164,"num_object_clones":0,"num_object_copies":8328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4164,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13472,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17003","reported_seq":17303,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:18.053096+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:33:18.053096+0000","last_peered":"2024-09-17T15:33:18.053096+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:33:18.053096+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'7000","ondisk_log_start":"41'7000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17400291,"num_objects":4249,"num_object_clones":0,"num_object_copies":8498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4249,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13502,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":57},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16701","reported_seq":17019,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:18.144851+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:33:18.144851+0000","last_peered":"2024-09-17T15:33:18.144851+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:33:18.144851+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'6700","ondisk_log_start":"41'6700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16659398,"num_objects":4069,"num_object_clones":0,"num_object_copies":8138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4069,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13470,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-20},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16663","reported_seq":16800,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:18.077071+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:33:18.077071+0000","last_peered":"2024-09-17T15:33:18.077071+0000","last_clean":"2024-09-17T15:33:18.077071+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:33:18.077071+0000","last_undegraded":"2024-09-17T15:33:18.077071+0000","last_fullsized":"2024-09-17T15:33:18.077071+0000","mapping_epoch":32,"log_start":"41'6600","ondisk_log_start":"41'6600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17420288,"num_objects":4253,"num_object_clones":0,"num_object_copies":8506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13617,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223906+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.223906+0000","last_peered":"2024-09-17T15:33:11.223906+0000","last_clean":"2024-09-17T15:33:11.223906+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.223906+0000","last_undegraded":"2024-09-17T15:33:11.223906+0000","last_fullsized":"2024-09-17T15:33:11.223906+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16576","reported_seq":24868,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:17.520312+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:33:17.520312+0000","last_peered":"2024-09-17T15:33:17.520312+0000","last_clean":"2024-09-17T15:33:17.520312+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:33:17.520312+0000","last_undegraded":"2024-09-17T15:33:17.520312+0000","last_fullsized":"2024-09-17T15:33:17.520312+0000","mapping_epoch":42,"log_start":"41'6500","ondisk_log_start":"41'6500","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17371619,"num_objects":4242,"num_object_clones":0,"num_object_copies":8484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4242,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14576,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225444+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225444+0000","last_peered":"2024-09-17T15:33:11.225444+0000","last_clean":"2024-09-17T15:33:11.225444+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225444+0000","last_undegraded":"2024-09-17T15:33:11.225444+0000","last_fullsized":"2024-09-17T15:33:11.225444+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16352","reported_seq":16626,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:18.142216+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:33:18.142216+0000","last_peered":"2024-09-17T15:33:18.142216+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:33:18.142216+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'6300","ondisk_log_start":"41'6300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17162240,"num_objects":4190,"num_object_clones":0,"num_object_copies":8380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12948,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":247}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":112,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.228064+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.228064+0000","last_peered":"2024-09-17T15:33:11.228064+0000","last_clean":"2024-09-17T15:33:11.228064+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.228064+0000","last_undegraded":"2024-09-17T15:33:11.228064+0000","last_fullsized":"2024-09-17T15:33:11.228064+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16424","reported_seq":16642,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:17.940259+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:33:17.940259+0000","last_peered":"2024-09-17T15:33:17.940259+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:33:17.940259+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'6400","ondisk_log_start":"41'6400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16773603,"num_objects":4096,"num_object_clones":0,"num_object_copies":8192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4096,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14319,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":63}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16573","reported_seq":24987,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:17.943226+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:33:17.943226+0000","last_peered":"2024-09-17T15:33:17.943226+0000","last_clean":"2024-09-17T15:33:17.943226+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:33:17.943226+0000","last_undegraded":"2024-09-17T15:33:17.943226+0000","last_fullsized":"2024-09-17T15:33:17.943226+0000","mapping_epoch":46,"log_start":"41'6500","ondisk_log_start":"41'6500","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17051648,"num_objects":4163,"num_object_clones":0,"num_object_copies":8326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14335,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16804","reported_seq":17110,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:18.188730+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:33:18.188730+0000","last_peered":"2024-09-17T15:33:18.188730+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:33:18.188730+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'6800","ondisk_log_start":"41'6800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17047552,"num_objects":4162,"num_object_clones":0,"num_object_copies":8324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13352,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":105}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16625","reported_seq":17267,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:18.190007+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:33:18.190007+0000","last_peered":"2024-09-17T15:33:18.190007+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:33:18.190007+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'6600","ondisk_log_start":"41'6600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16986112,"num_objects":4147,"num_object_clones":0,"num_object_copies":8294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3915,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12912,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":245,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":232}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225014+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225014+0000","last_peered":"2024-09-17T15:33:11.225014+0000","last_clean":"2024-09-17T15:33:11.225014+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225014+0000","last_undegraded":"2024-09-17T15:33:11.225014+0000","last_fullsized":"2024-09-17T15:33:11.225014+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16706","reported_seq":17012,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:18.106301+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:33:18.106301+0000","last_peered":"2024-09-17T15:33:18.106301+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:33:18.106301+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'6700","ondisk_log_start":"41'6700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17121280,"num_objects":4180,"num_object_clones":0,"num_object_copies":8360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13493,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":200}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16498","reported_seq":24882,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:17.593390+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:33:17.593390+0000","last_peered":"2024-09-17T15:33:17.593390+0000","last_clean":"2024-09-17T15:33:17.593390+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:33:17.593390+0000","last_undegraded":"2024-09-17T15:33:17.593390+0000","last_fullsized":"2024-09-17T15:33:17.593390+0000","mapping_epoch":44,"log_start":"41'6400","ondisk_log_start":"41'6400","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16933830,"num_objects":4136,"num_object_clones":0,"num_object_copies":8272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12940,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":274186998,"num_objects":66950,"num_object_clones":0,"num_object_copies":133900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36326,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66950,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":226037,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13134,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":751431680,"data_stored":751297081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160602,"ondisk_log_size":160602,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051683,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609820,"kb_used_data":267148,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93762020,"statfs":{"total":96636764160,"available":96012308480,"internally_reserved":0,"allocated":273559552,"data_stored":273257954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":350875308},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724628,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562116,"kb_used_data":201396,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93809724,"statfs":{"total":96636764160,"available":96061157376,"internally_reserved":0,"allocated":206229504,"data_stored":205905764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":369356459},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247911,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":675004,"kb_used_data":268520,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93696836,"statfs":{"total":96636764160,"available":95945560064,"internally_reserved":0,"allocated":274964480,"data_stored":274662877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":416214696},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,0,3,6,3,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":272941056,"data_stored":272904926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":205471744,"data_stored":205413477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":273018880,"data_stored":272978678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:33:20.076 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:33:20.642 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:33:20.642 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:33:20.670 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2024-09-17T15:33:19.751058+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":274187017,"num_objects":66952,"num_object_clones":0,"num_object_copies":133904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36326,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66952,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":226039,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13135,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160605,"ondisk_log_size":160605,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1846940,"kb_used_data":737064,"kb_used_omap":13,"kb_used_meta":1109810,"kb_avail":281268580,"statfs":{"total":289910292480,"available":288019025920,"internally_reserved":0,"allocated":754753536,"data_stored":753826595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1136446463},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,0,3,10,7,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19623936,"num_objects":-4791,"num_object_clones":0,"num_object_copies":-9582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-254,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4791,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.523525"},"pg_stats":[{"pgid":"2.f","version":"47'16502","reported_seq":16760,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:18.103254+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:33:18.103254+0000","last_peered":"2024-09-17T15:33:18.103254+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:33:18.103254+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'6500","ondisk_log_start":"41'6500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16982016,"num_objects":4146,"num_object_clones":0,"num_object_copies":8292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16502,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":162}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16652","reported_seq":16764,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:17.935457+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:33:17.935457+0000","last_peered":"2024-09-17T15:33:17.935457+0000","last_clean":"2024-09-17T15:33:17.935457+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:33:17.935457+0000","last_undegraded":"2024-09-17T15:33:17.935457+0000","last_fullsized":"2024-09-17T15:33:17.935457+0000","mapping_epoch":33,"log_start":"41'6600","ondisk_log_start":"41'6600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17531846,"num_objects":4282,"num_object_clones":0,"num_object_copies":8564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4282,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15009,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16705","reported_seq":16927,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:17.756283+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:33:17.756283+0000","last_peered":"2024-09-17T15:33:17.756283+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:33:17.756283+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'6700","ondisk_log_start":"41'6700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17317912,"num_objects":4229,"num_object_clones":0,"num_object_copies":8458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4229,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14862,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":104}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16728","reported_seq":16807,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:17.953239+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:33:17.953239+0000","last_peered":"2024-09-17T15:33:17.953239+0000","last_clean":"2024-09-17T15:33:17.953239+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:33:17.953239+0000","last_undegraded":"2024-09-17T15:33:17.953239+0000","last_fullsized":"2024-09-17T15:33:17.953239+0000","mapping_epoch":33,"log_start":"41'6700","ondisk_log_start":"41'6700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17375232,"num_objects":4242,"num_object_clones":0,"num_object_copies":8484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4242,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16728,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16490","reported_seq":16629,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:18.158160+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:33:18.158160+0000","last_peered":"2024-09-17T15:33:18.158160+0000","last_clean":"2024-09-17T15:33:18.158160+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:33:18.158160+0000","last_undegraded":"2024-09-17T15:33:18.158160+0000","last_fullsized":"2024-09-17T15:33:18.158160+0000","mapping_epoch":32,"log_start":"41'6400","ondisk_log_start":"41'6400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17052131,"num_objects":4164,"num_object_clones":0,"num_object_copies":8328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4164,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13472,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17003","reported_seq":17303,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:18.053096+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:33:18.053096+0000","last_peered":"2024-09-17T15:33:18.053096+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:33:18.053096+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'7000","ondisk_log_start":"41'7000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17400291,"num_objects":4249,"num_object_clones":0,"num_object_copies":8498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4249,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13502,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":57},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16701","reported_seq":17019,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:18.144851+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:33:18.144851+0000","last_peered":"2024-09-17T15:33:18.144851+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:33:18.144851+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'6700","ondisk_log_start":"41'6700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16659398,"num_objects":4069,"num_object_clones":0,"num_object_copies":8138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4069,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13470,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-20},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16663","reported_seq":16800,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:18.077071+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:33:18.077071+0000","last_peered":"2024-09-17T15:33:18.077071+0000","last_clean":"2024-09-17T15:33:18.077071+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:33:18.077071+0000","last_undegraded":"2024-09-17T15:33:18.077071+0000","last_fullsized":"2024-09-17T15:33:18.077071+0000","mapping_epoch":32,"log_start":"41'6600","ondisk_log_start":"41'6600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17420288,"num_objects":4253,"num_object_clones":0,"num_object_copies":8506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13617,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223906+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.223906+0000","last_peered":"2024-09-17T15:33:11.223906+0000","last_clean":"2024-09-17T15:33:11.223906+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.223906+0000","last_undegraded":"2024-09-17T15:33:11.223906+0000","last_fullsized":"2024-09-17T15:33:11.223906+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16576","reported_seq":24868,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:17.520312+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:33:17.520312+0000","last_peered":"2024-09-17T15:33:17.520312+0000","last_clean":"2024-09-17T15:33:17.520312+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:33:17.520312+0000","last_undegraded":"2024-09-17T15:33:17.520312+0000","last_fullsized":"2024-09-17T15:33:17.520312+0000","mapping_epoch":42,"log_start":"41'6500","ondisk_log_start":"41'6500","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17371619,"num_objects":4242,"num_object_clones":0,"num_object_copies":8484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4242,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14576,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225444+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225444+0000","last_peered":"2024-09-17T15:33:11.225444+0000","last_clean":"2024-09-17T15:33:11.225444+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225444+0000","last_undegraded":"2024-09-17T15:33:11.225444+0000","last_fullsized":"2024-09-17T15:33:11.225444+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16352","reported_seq":16626,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:18.142216+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:33:18.142216+0000","last_peered":"2024-09-17T15:33:18.142216+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:33:18.142216+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'6300","ondisk_log_start":"41'6300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17162240,"num_objects":4190,"num_object_clones":0,"num_object_copies":8380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12948,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":247}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":112,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.228064+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.228064+0000","last_peered":"2024-09-17T15:33:11.228064+0000","last_clean":"2024-09-17T15:33:11.228064+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.228064+0000","last_undegraded":"2024-09-17T15:33:11.228064+0000","last_fullsized":"2024-09-17T15:33:11.228064+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16424","reported_seq":16642,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:17.940259+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:33:17.940259+0000","last_peered":"2024-09-17T15:33:17.940259+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:33:17.940259+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'6400","ondisk_log_start":"41'6400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16773603,"num_objects":4096,"num_object_clones":0,"num_object_copies":8192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4096,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14319,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":63}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16573","reported_seq":24987,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:17.943226+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:33:17.943226+0000","last_peered":"2024-09-17T15:33:17.943226+0000","last_clean":"2024-09-17T15:33:17.943226+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:33:17.943226+0000","last_undegraded":"2024-09-17T15:33:17.943226+0000","last_fullsized":"2024-09-17T15:33:17.943226+0000","mapping_epoch":46,"log_start":"41'6500","ondisk_log_start":"41'6500","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17051648,"num_objects":4163,"num_object_clones":0,"num_object_copies":8326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14335,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16804","reported_seq":17110,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:18.188730+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:33:18.188730+0000","last_peered":"2024-09-17T15:33:18.188730+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:33:18.188730+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'6800","ondisk_log_start":"41'6800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17047552,"num_objects":4162,"num_object_clones":0,"num_object_copies":8324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13352,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":105}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16625","reported_seq":17267,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:18.190007+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:33:18.190007+0000","last_peered":"2024-09-17T15:33:18.190007+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:33:18.190007+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'6600","ondisk_log_start":"41'6600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16986112,"num_objects":4147,"num_object_clones":0,"num_object_copies":8294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3915,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12912,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":245,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":232}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225014+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225014+0000","last_peered":"2024-09-17T15:33:11.225014+0000","last_clean":"2024-09-17T15:33:11.225014+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225014+0000","last_undegraded":"2024-09-17T15:33:11.225014+0000","last_fullsized":"2024-09-17T15:33:11.225014+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16706","reported_seq":17012,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:18.106301+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:33:18.106301+0000","last_peered":"2024-09-17T15:33:18.106301+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:33:18.106301+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'6700","ondisk_log_start":"41'6700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17121280,"num_objects":4180,"num_object_clones":0,"num_object_copies":8360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13493,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":200}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16498","reported_seq":24882,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:17.593390+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:33:17.593390+0000","last_peered":"2024-09-17T15:33:17.593390+0000","last_clean":"2024-09-17T15:33:17.593390+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:33:17.593390+0000","last_undegraded":"2024-09-17T15:33:17.593390+0000","last_fullsized":"2024-09-17T15:33:17.593390+0000","mapping_epoch":44,"log_start":"41'6400","ondisk_log_start":"41'6400","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16933830,"num_objects":4136,"num_object_clones":0,"num_object_copies":8272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12940,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":274186998,"num_objects":66950,"num_object_clones":0,"num_object_copies":133900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36326,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66950,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":226037,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13134,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":751431680,"data_stored":751297081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160602,"ondisk_log_size":160602,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051683,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609820,"kb_used_data":267148,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93762020,"statfs":{"total":96636764160,"available":96012308480,"internally_reserved":0,"allocated":273559552,"data_stored":273257954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":350875308},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724628,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562116,"kb_used_data":201396,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93809724,"statfs":{"total":96636764160,"available":96061157376,"internally_reserved":0,"allocated":206229504,"data_stored":205905764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":369356459},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247911,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":675004,"kb_used_data":268520,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93696836,"statfs":{"total":96636764160,"available":95945560064,"internally_reserved":0,"allocated":274964480,"data_stored":274662877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":416214696},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,0,3,6,3,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":272941056,"data_stored":272904926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":205471744,"data_stored":205413477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":273018880,"data_stored":272978678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:33:20.671 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:33:20.671 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'16502", 'reported_seq': 16760, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:18.103254+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:33:18.103254+0000', 'last_peered': '2024-09-17T15:33:18.103254+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:33:18.103254+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "41'6500", 'ondisk_log_start': "41'6500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16982016, 'num_objects': 4146, 'num_object_clones': 0, 'num_object_copies': 8292, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4146, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16502, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 162}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:20.671 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:33:20.671 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'16705", 'reported_seq': 16927, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:17.756283+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:33:17.756283+0000', 'last_peered': '2024-09-17T15:33:17.756283+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:33:17.756283+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "41'6700", 'ondisk_log_start': "41'6700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17317912, 'num_objects': 4229, 'num_object_clones': 0, 'num_object_copies': 8458, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4229, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14862, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 104}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:33:20.671 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:33:20.671 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'17003", 'reported_seq': 17303, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:18.053096+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:33:18.053096+0000', 'last_peered': '2024-09-17T15:33:18.053096+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:33:18.053096+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "41'7000", 'ondisk_log_start': "41'7000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17400291, 'num_objects': 4249, 'num_object_clones': 0, 'num_object_copies': 8498, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4249, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13502, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 57}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:20.671 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:33:20.672 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'16701", 'reported_seq': 17019, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:18.144851+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:33:18.144851+0000', 'last_peered': '2024-09-17T15:33:18.144851+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:33:18.144851+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "41'6700", 'ondisk_log_start': "41'6700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16659398, 'num_objects': 4069, 'num_object_clones': 0, 'num_object_copies': 8138, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4089, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4069, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 13470, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 85, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -20}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:20.672 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:33:20.672 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'16352", 'reported_seq': 16626, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:18.142216+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:33:18.142216+0000', 'last_peered': '2024-09-17T15:33:18.142216+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:33:18.142216+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "41'6300", 'ondisk_log_start': "41'6300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17162240, 'num_objects': 4190, 'num_object_clones': 0, 'num_object_copies': 8380, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4190, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12948, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 247}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:20.672 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:33:20.672 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'16424", 'reported_seq': 16642, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:17.940259+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:33:17.940259+0000', 'last_peered': '2024-09-17T15:33:17.940259+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:33:17.940259+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "41'6400", 'ondisk_log_start': "41'6400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16773603, 'num_objects': 4096, 'num_object_clones': 0, 'num_object_copies': 8192, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4034, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4096, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14319, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 63}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:33:20.672 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:33:20.672 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'16804", 'reported_seq': 17110, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:18.188730+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:33:18.188730+0000', 'last_peered': '2024-09-17T15:33:18.188730+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:33:18.188730+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "41'6800", 'ondisk_log_start': "41'6800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17047552, 'num_objects': 4162, 'num_object_clones': 0, 'num_object_copies': 8324, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4162, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13352, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 105}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:20.672 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:33:20.672 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'16625", 'reported_seq': 17267, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:18.190007+0000', 'last_change': '2024-09-17T15:33:09.763500+0000', 'last_active': '2024-09-17T15:33:18.190007+0000', 'last_peered': '2024-09-17T15:33:18.190007+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:33:18.190007+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "41'6600", 'ondisk_log_start': "41'6600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16986112, 'num_objects': 4147, 'num_object_clones': 0, 'num_object_copies': 8294, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3915, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4147, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12912, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 245, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 232}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:20.672 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:33:20.673 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'16706", 'reported_seq': 17012, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:18.106301+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:33:18.106301+0000', 'last_peered': '2024-09-17T15:33:18.106301+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:33:18.106301+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "41'6700", 'ondisk_log_start': "41'6700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17121280, 'num_objects': 4180, 'num_object_clones': 0, 'num_object_copies': 8360, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4180, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13493, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 200}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:20.673 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:33:21.369 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:33:21.369 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36326/133904 objects degraded (27.128%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":538,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":66952,"data_bytes":274187017,"bytes_used":1891266560,"bytes_avail":288019025920,"bytes_total":289910292480,"degraded_objects":36326,"degraded_total":133904,"degraded_ratio":0.27128390488708326,"recovering_objects_per_sec":26,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":254,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":503},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:33:21.394 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:33:21.394 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:33:21.934 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:33:21.937 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:33:21.955 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":309,"stamp":"2024-09-17T15:33:21.751866+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":274187017,"num_objects":66952,"num_object_clones":0,"num_object_copies":133904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36326,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66952,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":226039,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13135,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160605,"ondisk_log_size":160605,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1846940,"kb_used_data":737064,"kb_used_omap":13,"kb_used_meta":1109810,"kb_avail":281268580,"statfs":{"total":289910292480,"available":288019025920,"internally_reserved":0,"allocated":754753536,"data_stored":753826595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1136446463},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,0,3,10,7,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19623936,"num_objects":-4791,"num_object_clones":0,"num_object_copies":-9582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-254,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4791,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.530045"},"pg_stats":[{"pgid":"2.f","version":"47'16502","reported_seq":16760,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:18.103254+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:33:18.103254+0000","last_peered":"2024-09-17T15:33:18.103254+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:33:18.103254+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'6500","ondisk_log_start":"41'6500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16982016,"num_objects":4146,"num_object_clones":0,"num_object_copies":8292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16502,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":162}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16652","reported_seq":16764,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:17.935457+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:33:17.935457+0000","last_peered":"2024-09-17T15:33:17.935457+0000","last_clean":"2024-09-17T15:33:17.935457+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:33:17.935457+0000","last_undegraded":"2024-09-17T15:33:17.935457+0000","last_fullsized":"2024-09-17T15:33:17.935457+0000","mapping_epoch":33,"log_start":"41'6600","ondisk_log_start":"41'6600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17531846,"num_objects":4282,"num_object_clones":0,"num_object_copies":8564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4282,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15009,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16705","reported_seq":16927,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:17.756283+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:33:17.756283+0000","last_peered":"2024-09-17T15:33:17.756283+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:33:17.756283+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'6700","ondisk_log_start":"41'6700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17317912,"num_objects":4229,"num_object_clones":0,"num_object_copies":8458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4229,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14862,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":104}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16728","reported_seq":16807,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:17.953239+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:33:17.953239+0000","last_peered":"2024-09-17T15:33:17.953239+0000","last_clean":"2024-09-17T15:33:17.953239+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:33:17.953239+0000","last_undegraded":"2024-09-17T15:33:17.953239+0000","last_fullsized":"2024-09-17T15:33:17.953239+0000","mapping_epoch":33,"log_start":"41'6700","ondisk_log_start":"41'6700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17375232,"num_objects":4242,"num_object_clones":0,"num_object_copies":8484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4242,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16728,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16490","reported_seq":16629,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:18.158160+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:33:18.158160+0000","last_peered":"2024-09-17T15:33:18.158160+0000","last_clean":"2024-09-17T15:33:18.158160+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:33:18.158160+0000","last_undegraded":"2024-09-17T15:33:18.158160+0000","last_fullsized":"2024-09-17T15:33:18.158160+0000","mapping_epoch":32,"log_start":"41'6400","ondisk_log_start":"41'6400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17052131,"num_objects":4164,"num_object_clones":0,"num_object_copies":8328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4164,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13472,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17003","reported_seq":17303,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:18.053096+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:33:18.053096+0000","last_peered":"2024-09-17T15:33:18.053096+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:33:18.053096+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'7000","ondisk_log_start":"41'7000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17400291,"num_objects":4249,"num_object_clones":0,"num_object_copies":8498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4249,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13502,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":57},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16701","reported_seq":17019,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:18.144851+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:33:18.144851+0000","last_peered":"2024-09-17T15:33:18.144851+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:33:18.144851+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'6700","ondisk_log_start":"41'6700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16659398,"num_objects":4069,"num_object_clones":0,"num_object_copies":8138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4069,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13470,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-20},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16663","reported_seq":16800,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:18.077071+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:33:18.077071+0000","last_peered":"2024-09-17T15:33:18.077071+0000","last_clean":"2024-09-17T15:33:18.077071+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:33:18.077071+0000","last_undegraded":"2024-09-17T15:33:18.077071+0000","last_fullsized":"2024-09-17T15:33:18.077071+0000","mapping_epoch":32,"log_start":"41'6600","ondisk_log_start":"41'6600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17420288,"num_objects":4253,"num_object_clones":0,"num_object_copies":8506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13617,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223906+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.223906+0000","last_peered":"2024-09-17T15:33:11.223906+0000","last_clean":"2024-09-17T15:33:11.223906+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.223906+0000","last_undegraded":"2024-09-17T15:33:11.223906+0000","last_fullsized":"2024-09-17T15:33:11.223906+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16576","reported_seq":24868,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:17.520312+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:33:17.520312+0000","last_peered":"2024-09-17T15:33:17.520312+0000","last_clean":"2024-09-17T15:33:17.520312+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:33:17.520312+0000","last_undegraded":"2024-09-17T15:33:17.520312+0000","last_fullsized":"2024-09-17T15:33:17.520312+0000","mapping_epoch":42,"log_start":"41'6500","ondisk_log_start":"41'6500","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17371619,"num_objects":4242,"num_object_clones":0,"num_object_copies":8484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4242,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14576,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225444+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225444+0000","last_peered":"2024-09-17T15:33:11.225444+0000","last_clean":"2024-09-17T15:33:11.225444+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225444+0000","last_undegraded":"2024-09-17T15:33:11.225444+0000","last_fullsized":"2024-09-17T15:33:11.225444+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16352","reported_seq":16626,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:18.142216+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:33:18.142216+0000","last_peered":"2024-09-17T15:33:18.142216+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:33:18.142216+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'6300","ondisk_log_start":"41'6300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17162240,"num_objects":4190,"num_object_clones":0,"num_object_copies":8380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12948,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":247}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":112,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.228064+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.228064+0000","last_peered":"2024-09-17T15:33:11.228064+0000","last_clean":"2024-09-17T15:33:11.228064+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.228064+0000","last_undegraded":"2024-09-17T15:33:11.228064+0000","last_fullsized":"2024-09-17T15:33:11.228064+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16424","reported_seq":16642,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:17.940259+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:33:17.940259+0000","last_peered":"2024-09-17T15:33:17.940259+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:33:17.940259+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'6400","ondisk_log_start":"41'6400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16773603,"num_objects":4096,"num_object_clones":0,"num_object_copies":8192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4096,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14319,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":63}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16573","reported_seq":24987,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:17.943226+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:33:17.943226+0000","last_peered":"2024-09-17T15:33:17.943226+0000","last_clean":"2024-09-17T15:33:17.943226+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:33:17.943226+0000","last_undegraded":"2024-09-17T15:33:17.943226+0000","last_fullsized":"2024-09-17T15:33:17.943226+0000","mapping_epoch":46,"log_start":"41'6500","ondisk_log_start":"41'6500","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17051648,"num_objects":4163,"num_object_clones":0,"num_object_copies":8326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14335,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16804","reported_seq":17110,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:18.188730+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:33:18.188730+0000","last_peered":"2024-09-17T15:33:18.188730+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:33:18.188730+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'6800","ondisk_log_start":"41'6800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17047552,"num_objects":4162,"num_object_clones":0,"num_object_copies":8324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13352,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":105}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16625","reported_seq":17267,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:18.190007+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:33:18.190007+0000","last_peered":"2024-09-17T15:33:18.190007+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:33:18.190007+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'6600","ondisk_log_start":"41'6600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16986112,"num_objects":4147,"num_object_clones":0,"num_object_copies":8294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3915,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12912,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":245,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":232}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225014+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225014+0000","last_peered":"2024-09-17T15:33:11.225014+0000","last_clean":"2024-09-17T15:33:11.225014+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225014+0000","last_undegraded":"2024-09-17T15:33:11.225014+0000","last_fullsized":"2024-09-17T15:33:11.225014+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16706","reported_seq":17012,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:18.106301+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:33:18.106301+0000","last_peered":"2024-09-17T15:33:18.106301+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:33:18.106301+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'6700","ondisk_log_start":"41'6700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17121280,"num_objects":4180,"num_object_clones":0,"num_object_copies":8360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13493,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":200}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16498","reported_seq":24882,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:17.593390+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:33:17.593390+0000","last_peered":"2024-09-17T15:33:17.593390+0000","last_clean":"2024-09-17T15:33:17.593390+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:33:17.593390+0000","last_undegraded":"2024-09-17T15:33:17.593390+0000","last_fullsized":"2024-09-17T15:33:17.593390+0000","mapping_epoch":44,"log_start":"41'6400","ondisk_log_start":"41'6400","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16933830,"num_objects":4136,"num_object_clones":0,"num_object_copies":8272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12940,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":274186998,"num_objects":66950,"num_object_clones":0,"num_object_copies":133900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36326,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66950,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":226037,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13134,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":751431680,"data_stored":751297081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160602,"ondisk_log_size":160602,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051683,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609820,"kb_used_data":267148,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93762020,"statfs":{"total":96636764160,"available":96012308480,"internally_reserved":0,"allocated":273559552,"data_stored":273257954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":350875308},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724628,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562116,"kb_used_data":201396,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93809724,"statfs":{"total":96636764160,"available":96061157376,"internally_reserved":0,"allocated":206229504,"data_stored":205905764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":369356459},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247911,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":675004,"kb_used_data":268520,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93696836,"statfs":{"total":96636764160,"available":95945560064,"internally_reserved":0,"allocated":274964480,"data_stored":274662877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":416214696},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,0,3,6,3,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":272941056,"data_stored":272904926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":205471744,"data_stored":205413477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":273018880,"data_stored":272978678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:33:24.957 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:33:25.501 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:33:25.502 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:33:25.523 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2024-09-17T15:33:23.752661+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":264069897,"num_objects":64482,"num_object_clones":0,"num_object_copies":128964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36227,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64482,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":228509,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13234,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160775,"ondisk_log_size":160775,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1844940,"kb_used_data":718424,"kb_used_omap":13,"kb_used_meta":1126450,"kb_avail":281270580,"statfs":{"total":289910292480,"available":288021073920,"internally_reserved":0,"allocated":735666176,"data_stored":734739235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1153485823},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,3,13,12,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29741056,"num_objects":-7261,"num_object_clones":0,"num_object_copies":-14522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-353,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7261,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":353,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.012079"},"pg_stats":[{"pgid":"2.f","version":"47'16665","reported_seq":16923,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:23.249227+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:33:23.249227+0000","last_peered":"2024-09-17T15:33:23.249227+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:33:23.249227+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'6600","ondisk_log_start":"41'6600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16314368,"num_objects":3983,"num_object_clones":0,"num_object_copies":7966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16665,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16808","reported_seq":16920,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:22.940224+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:33:22.940224+0000","last_peered":"2024-09-17T15:33:22.940224+0000","last_clean":"2024-09-17T15:33:22.940224+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:33:22.940224+0000","last_undegraded":"2024-09-17T15:33:22.940224+0000","last_fullsized":"2024-09-17T15:33:22.940224+0000","mapping_epoch":33,"log_start":"41'6800","ondisk_log_start":"41'6800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16892870,"num_objects":4126,"num_object_clones":0,"num_object_copies":8252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4126,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15165,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16860","reported_seq":17082,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:22.940193+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:33:22.940193+0000","last_peered":"2024-09-17T15:33:22.940193+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:33:22.940193+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'6800","ondisk_log_start":"41'6800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16683032,"num_objects":4074,"num_object_clones":0,"num_object_copies":8148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4074,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15017,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-51}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16896","reported_seq":16975,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:22.882139+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:33:22.882139+0000","last_peered":"2024-09-17T15:33:22.882139+0000","last_clean":"2024-09-17T15:33:22.882139+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:33:22.882139+0000","last_undegraded":"2024-09-17T15:33:22.882139+0000","last_fullsized":"2024-09-17T15:33:22.882139+0000","mapping_epoch":33,"log_start":"41'6800","ondisk_log_start":"41'6800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16687104,"num_objects":4074,"num_object_clones":0,"num_object_copies":8148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16896,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16663","reported_seq":16802,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:23.154224+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:33:23.154224+0000","last_peered":"2024-09-17T15:33:23.154224+0000","last_clean":"2024-09-17T15:33:23.154224+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:33:23.154224+0000","last_undegraded":"2024-09-17T15:33:23.154224+0000","last_fullsized":"2024-09-17T15:33:23.154224+0000","mapping_epoch":32,"log_start":"41'6600","ondisk_log_start":"41'6600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16343523,"num_objects":3991,"num_object_clones":0,"num_object_copies":7982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3991,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13645,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17129","reported_seq":17429,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:23.130644+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:33:23.130644+0000","last_peered":"2024-09-17T15:33:23.130644+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:33:23.130644+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'7100","ondisk_log_start":"41'7100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16884195,"num_objects":4123,"num_object_clones":0,"num_object_copies":8246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4123,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13628,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-69},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16850","reported_seq":17168,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:23.193280+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:33:23.193280+0000","last_peered":"2024-09-17T15:33:23.193280+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:33:23.193280+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'6800","ondisk_log_start":"41'6800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16049094,"num_objects":3920,"num_object_clones":0,"num_object_copies":7840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3920,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13619,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-169},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16837","reported_seq":16974,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:23.182674+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:33:23.182674+0000","last_peered":"2024-09-17T15:33:23.182674+0000","last_clean":"2024-09-17T15:33:23.182674+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:33:23.182674+0000","last_undegraded":"2024-09-17T15:33:23.182674+0000","last_fullsized":"2024-09-17T15:33:23.182674+0000","mapping_epoch":32,"log_start":"41'6800","ondisk_log_start":"41'6800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16707584,"num_objects":4079,"num_object_clones":0,"num_object_copies":8158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13791,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223906+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.223906+0000","last_peered":"2024-09-17T15:33:11.223906+0000","last_clean":"2024-09-17T15:33:11.223906+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.223906+0000","last_undegraded":"2024-09-17T15:33:11.223906+0000","last_fullsized":"2024-09-17T15:33:11.223906+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16744","reported_seq":25036,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:22.638470+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:33:22.638470+0000","last_peered":"2024-09-17T15:33:22.638470+0000","last_clean":"2024-09-17T15:33:22.638470+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:33:22.638470+0000","last_undegraded":"2024-09-17T15:33:22.638470+0000","last_fullsized":"2024-09-17T15:33:22.638470+0000","mapping_epoch":42,"log_start":"41'6700","ondisk_log_start":"41'6700","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16683491,"num_objects":4074,"num_object_clones":0,"num_object_copies":8148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4074,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14744,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225444+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225444+0000","last_peered":"2024-09-17T15:33:11.225444+0000","last_clean":"2024-09-17T15:33:11.225444+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225444+0000","last_undegraded":"2024-09-17T15:33:11.225444+0000","last_fullsized":"2024-09-17T15:33:11.225444+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16521","reported_seq":16793,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:23.190182+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:33:23.190182+0000","last_peered":"2024-09-17T15:33:23.190182+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:33:23.190182+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'6500","ondisk_log_start":"41'6500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16470016,"num_objects":4021,"num_object_clones":0,"num_object_copies":8042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13117,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":78}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":112,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.228064+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.228064+0000","last_peered":"2024-09-17T15:33:11.228064+0000","last_clean":"2024-09-17T15:33:11.228064+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.228064+0000","last_undegraded":"2024-09-17T15:33:11.228064+0000","last_fullsized":"2024-09-17T15:33:11.228064+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16578","reported_seq":16796,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:22.914593+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:33:22.914593+0000","last_peered":"2024-09-17T15:33:22.914593+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:33:22.914593+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'6500","ondisk_log_start":"41'6500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16142819,"num_objects":3942,"num_object_clones":0,"num_object_copies":7884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3942,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14473,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-91}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16724","reported_seq":25139,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:22.985969+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:33:22.985969+0000","last_peered":"2024-09-17T15:33:22.985969+0000","last_clean":"2024-09-17T15:33:22.985969+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:33:22.985969+0000","last_undegraded":"2024-09-17T15:33:22.985969+0000","last_fullsized":"2024-09-17T15:33:22.985969+0000","mapping_epoch":46,"log_start":"41'6700","ondisk_log_start":"41'6700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16433152,"num_objects":4012,"num_object_clones":0,"num_object_copies":8024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14486,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16945","reported_seq":17251,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:23.224510+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:33:23.224510+0000","last_peered":"2024-09-17T15:33:23.224510+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:33:23.224510+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'6900","ondisk_log_start":"41'6900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16470016,"num_objects":4021,"num_object_clones":0,"num_object_copies":8042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13493,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-36}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16762","reported_seq":17601,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:23.226097+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:33:23.226097+0000","last_peered":"2024-09-17T15:33:23.226097+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:33:23.226097+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'6700","ondisk_log_start":"41'6700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16424960,"num_objects":4010,"num_object_clones":0,"num_object_copies":8020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3816,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13049,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":194}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225014+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225014+0000","last_peered":"2024-09-17T15:33:11.225014+0000","last_clean":"2024-09-17T15:33:11.225014+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225014+0000","last_undegraded":"2024-09-17T15:33:11.225014+0000","last_fullsized":"2024-09-17T15:33:11.225014+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16851","reported_seq":17158,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:23.224498+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:33:23.224498+0000","last_peered":"2024-09-17T15:33:23.224498+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:33:23.224498+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'6800","ondisk_log_start":"41'6800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16527360,"num_objects":4035,"num_object_clones":0,"num_object_copies":8070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13638,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":55}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16639","reported_seq":25023,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:22.664388+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:33:22.664388+0000","last_peered":"2024-09-17T15:33:22.664388+0000","last_clean":"2024-09-17T15:33:22.664388+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:33:22.664388+0000","last_undegraded":"2024-09-17T15:33:22.664388+0000","last_fullsized":"2024-09-17T15:33:22.664388+0000","mapping_epoch":44,"log_start":"41'6600","ondisk_log_start":"41'6600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16356294,"num_objects":3995,"num_object_clones":0,"num_object_copies":7990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13081,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":264069878,"num_objects":64480,"num_object_clones":0,"num_object_copies":128960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36227,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64480,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":228507,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13233,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":731234304,"data_stored":731099705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160772,"ondisk_log_size":160772,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051684,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609852,"kb_used_data":263020,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93761988,"statfs":{"total":96636764160,"available":96012275712,"internally_reserved":0,"allocated":269332480,"data_stored":269030882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":355135148},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,0,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724629,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":559232,"kb_used_data":194352,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93812608,"statfs":{"total":96636764160,"available":96064110592,"internally_reserved":0,"allocated":199016448,"data_stored":198692708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":373616299},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,4,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247912,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":675856,"kb_used_data":261052,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93695984,"statfs":{"total":96636764160,"available":95944687616,"internally_reserved":0,"allocated":267317248,"data_stored":267015645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":424734376},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":265383936,"data_stored":265347806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":198119424,"data_stored":198061157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":267730944,"data_stored":267690742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:33:25.524 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:33:26.094 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:33:26.097 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:33:26.118 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":311,"stamp":"2024-09-17T15:33:25.753547+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":264069897,"num_objects":64482,"num_object_clones":0,"num_object_copies":128964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36227,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64482,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":228509,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13234,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160775,"ondisk_log_size":160775,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1844940,"kb_used_data":718424,"kb_used_omap":13,"kb_used_meta":1126450,"kb_avail":281270580,"statfs":{"total":289910292480,"available":288021073920,"internally_reserved":0,"allocated":735666176,"data_stored":734739235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1153485823},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,3,13,12,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20172800,"num_objects":-4925,"num_object_clones":0,"num_object_copies":-9850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-191,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4925,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":191,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.012271"},"pg_stats":[{"pgid":"2.f","version":"47'16665","reported_seq":16923,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:23.249227+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:33:23.249227+0000","last_peered":"2024-09-17T15:33:23.249227+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:33:23.249227+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'6600","ondisk_log_start":"41'6600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16314368,"num_objects":3983,"num_object_clones":0,"num_object_copies":7966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16665,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16808","reported_seq":16920,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:22.940224+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:33:22.940224+0000","last_peered":"2024-09-17T15:33:22.940224+0000","last_clean":"2024-09-17T15:33:22.940224+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:33:22.940224+0000","last_undegraded":"2024-09-17T15:33:22.940224+0000","last_fullsized":"2024-09-17T15:33:22.940224+0000","mapping_epoch":33,"log_start":"41'6800","ondisk_log_start":"41'6800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16892870,"num_objects":4126,"num_object_clones":0,"num_object_copies":8252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4126,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15165,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16860","reported_seq":17082,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:22.940193+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:33:22.940193+0000","last_peered":"2024-09-17T15:33:22.940193+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:33:22.940193+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'6800","ondisk_log_start":"41'6800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16683032,"num_objects":4074,"num_object_clones":0,"num_object_copies":8148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4074,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15017,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-51}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16896","reported_seq":16975,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:22.882139+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:33:22.882139+0000","last_peered":"2024-09-17T15:33:22.882139+0000","last_clean":"2024-09-17T15:33:22.882139+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:33:22.882139+0000","last_undegraded":"2024-09-17T15:33:22.882139+0000","last_fullsized":"2024-09-17T15:33:22.882139+0000","mapping_epoch":33,"log_start":"41'6800","ondisk_log_start":"41'6800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16687104,"num_objects":4074,"num_object_clones":0,"num_object_copies":8148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16896,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16663","reported_seq":16802,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:23.154224+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:33:23.154224+0000","last_peered":"2024-09-17T15:33:23.154224+0000","last_clean":"2024-09-17T15:33:23.154224+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:33:23.154224+0000","last_undegraded":"2024-09-17T15:33:23.154224+0000","last_fullsized":"2024-09-17T15:33:23.154224+0000","mapping_epoch":32,"log_start":"41'6600","ondisk_log_start":"41'6600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16343523,"num_objects":3991,"num_object_clones":0,"num_object_copies":7982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3991,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13645,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17129","reported_seq":17429,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:23.130644+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:33:23.130644+0000","last_peered":"2024-09-17T15:33:23.130644+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:33:23.130644+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'7100","ondisk_log_start":"41'7100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16884195,"num_objects":4123,"num_object_clones":0,"num_object_copies":8246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4123,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13628,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-69},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16850","reported_seq":17168,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:23.193280+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:33:23.193280+0000","last_peered":"2024-09-17T15:33:23.193280+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:33:23.193280+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'6800","ondisk_log_start":"41'6800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16049094,"num_objects":3920,"num_object_clones":0,"num_object_copies":7840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3920,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13619,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-169},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16837","reported_seq":16974,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:23.182674+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:33:23.182674+0000","last_peered":"2024-09-17T15:33:23.182674+0000","last_clean":"2024-09-17T15:33:23.182674+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:33:23.182674+0000","last_undegraded":"2024-09-17T15:33:23.182674+0000","last_fullsized":"2024-09-17T15:33:23.182674+0000","mapping_epoch":32,"log_start":"41'6800","ondisk_log_start":"41'6800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16707584,"num_objects":4079,"num_object_clones":0,"num_object_copies":8158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13791,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223906+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.223906+0000","last_peered":"2024-09-17T15:33:11.223906+0000","last_clean":"2024-09-17T15:33:11.223906+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.223906+0000","last_undegraded":"2024-09-17T15:33:11.223906+0000","last_fullsized":"2024-09-17T15:33:11.223906+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16744","reported_seq":25036,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:22.638470+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:33:22.638470+0000","last_peered":"2024-09-17T15:33:22.638470+0000","last_clean":"2024-09-17T15:33:22.638470+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:33:22.638470+0000","last_undegraded":"2024-09-17T15:33:22.638470+0000","last_fullsized":"2024-09-17T15:33:22.638470+0000","mapping_epoch":42,"log_start":"41'6700","ondisk_log_start":"41'6700","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16683491,"num_objects":4074,"num_object_clones":0,"num_object_copies":8148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4074,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14744,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225444+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225444+0000","last_peered":"2024-09-17T15:33:11.225444+0000","last_clean":"2024-09-17T15:33:11.225444+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225444+0000","last_undegraded":"2024-09-17T15:33:11.225444+0000","last_fullsized":"2024-09-17T15:33:11.225444+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16521","reported_seq":16793,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:23.190182+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:33:23.190182+0000","last_peered":"2024-09-17T15:33:23.190182+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:33:23.190182+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'6500","ondisk_log_start":"41'6500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16470016,"num_objects":4021,"num_object_clones":0,"num_object_copies":8042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13117,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":78}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":112,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.228064+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.228064+0000","last_peered":"2024-09-17T15:33:11.228064+0000","last_clean":"2024-09-17T15:33:11.228064+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.228064+0000","last_undegraded":"2024-09-17T15:33:11.228064+0000","last_fullsized":"2024-09-17T15:33:11.228064+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16578","reported_seq":16796,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:22.914593+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:33:22.914593+0000","last_peered":"2024-09-17T15:33:22.914593+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:33:22.914593+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'6500","ondisk_log_start":"41'6500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16142819,"num_objects":3942,"num_object_clones":0,"num_object_copies":7884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3942,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14473,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-91}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16724","reported_seq":25139,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:22.985969+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:33:22.985969+0000","last_peered":"2024-09-17T15:33:22.985969+0000","last_clean":"2024-09-17T15:33:22.985969+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:33:22.985969+0000","last_undegraded":"2024-09-17T15:33:22.985969+0000","last_fullsized":"2024-09-17T15:33:22.985969+0000","mapping_epoch":46,"log_start":"41'6700","ondisk_log_start":"41'6700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16433152,"num_objects":4012,"num_object_clones":0,"num_object_copies":8024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14486,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16945","reported_seq":17251,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:23.224510+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:33:23.224510+0000","last_peered":"2024-09-17T15:33:23.224510+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:33:23.224510+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'6900","ondisk_log_start":"41'6900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16470016,"num_objects":4021,"num_object_clones":0,"num_object_copies":8042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13493,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-36}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16762","reported_seq":17601,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:23.226097+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:33:23.226097+0000","last_peered":"2024-09-17T15:33:23.226097+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:33:23.226097+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'6700","ondisk_log_start":"41'6700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16424960,"num_objects":4010,"num_object_clones":0,"num_object_copies":8020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3816,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13049,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":194}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225014+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225014+0000","last_peered":"2024-09-17T15:33:11.225014+0000","last_clean":"2024-09-17T15:33:11.225014+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225014+0000","last_undegraded":"2024-09-17T15:33:11.225014+0000","last_fullsized":"2024-09-17T15:33:11.225014+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16851","reported_seq":17158,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:23.224498+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:33:23.224498+0000","last_peered":"2024-09-17T15:33:23.224498+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:33:23.224498+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'6800","ondisk_log_start":"41'6800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16527360,"num_objects":4035,"num_object_clones":0,"num_object_copies":8070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13638,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":55}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16639","reported_seq":25023,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:22.664388+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:33:22.664388+0000","last_peered":"2024-09-17T15:33:22.664388+0000","last_clean":"2024-09-17T15:33:22.664388+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:33:22.664388+0000","last_undegraded":"2024-09-17T15:33:22.664388+0000","last_fullsized":"2024-09-17T15:33:22.664388+0000","mapping_epoch":44,"log_start":"41'6600","ondisk_log_start":"41'6600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16356294,"num_objects":3995,"num_object_clones":0,"num_object_copies":7990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13081,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":264069878,"num_objects":64480,"num_object_clones":0,"num_object_copies":128960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36227,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64480,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":228507,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13233,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":731234304,"data_stored":731099705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160772,"ondisk_log_size":160772,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051684,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609852,"kb_used_data":263020,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93761988,"statfs":{"total":96636764160,"available":96012275712,"internally_reserved":0,"allocated":269332480,"data_stored":269030882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":355135148},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,0,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724629,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":559232,"kb_used_data":194352,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93812608,"statfs":{"total":96636764160,"available":96064110592,"internally_reserved":0,"allocated":199016448,"data_stored":198692708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":373616299},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,4,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247912,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":675856,"kb_used_data":261052,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93695984,"statfs":{"total":96636764160,"available":95944687616,"internally_reserved":0,"allocated":267317248,"data_stored":267015645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":424734376},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":265383936,"data_stored":265347806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":198119424,"data_stored":198061157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":267730944,"data_stored":267690742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:33:26.120 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:33:26.120 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'16665", 'reported_seq': 16923, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:23.249227+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:33:23.249227+0000', 'last_peered': '2024-09-17T15:33:23.249227+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:33:23.249227+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "41'6600", 'ondisk_log_start': "41'6600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16314368, 'num_objects': 3983, 'num_object_clones': 0, 'num_object_copies': 7966, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3983, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16665, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:26.120 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:33:26.120 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'16860", 'reported_seq': 17082, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:22.940193+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:33:22.940193+0000', 'last_peered': '2024-09-17T15:33:22.940193+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:33:22.940193+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "41'6800", 'ondisk_log_start': "41'6800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16683032, 'num_objects': 4074, 'num_object_clones': 0, 'num_object_copies': 8148, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4074, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15017, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -51}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:33:26.120 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:33:26.120 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'17129", 'reported_seq': 17429, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:23.130644+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:33:23.130644+0000', 'last_peered': '2024-09-17T15:33:23.130644+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:33:23.130644+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "41'7100", 'ondisk_log_start': "41'7100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16884195, 'num_objects': 4123, 'num_object_clones': 0, 'num_object_copies': 8246, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4123, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13628, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -69}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:26.120 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:33:26.120 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'16850", 'reported_seq': 17168, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:23.193280+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:33:23.193280+0000', 'last_peered': '2024-09-17T15:33:23.193280+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:33:23.193280+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "41'6800", 'ondisk_log_start': "41'6800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16049094, 'num_objects': 3920, 'num_object_clones': 0, 'num_object_copies': 7840, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4089, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3920, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 13619, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 85, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -169}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:26.121 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:33:26.121 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'16521", 'reported_seq': 16793, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:23.190182+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:33:23.190182+0000', 'last_peered': '2024-09-17T15:33:23.190182+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:33:23.190182+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "41'6500", 'ondisk_log_start': "41'6500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16470016, 'num_objects': 4021, 'num_object_clones': 0, 'num_object_copies': 8042, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4021, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13117, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 78}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:26.121 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:33:26.121 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'16578", 'reported_seq': 16796, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:22.914593+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:33:22.914593+0000', 'last_peered': '2024-09-17T15:33:22.914593+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:33:22.914593+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "41'6500", 'ondisk_log_start': "41'6500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16142819, 'num_objects': 3942, 'num_object_clones': 0, 'num_object_copies': 7884, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4034, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3942, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14473, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -91}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:33:26.121 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:33:26.121 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'16945", 'reported_seq': 17251, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:23.224510+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:33:23.224510+0000', 'last_peered': '2024-09-17T15:33:23.224510+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:33:23.224510+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "41'6900", 'ondisk_log_start': "41'6900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16470016, 'num_objects': 4021, 'num_object_clones': 0, 'num_object_copies': 8042, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4021, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13493, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -36}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:26.121 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:33:26.121 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'16762", 'reported_seq': 17601, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:23.226097+0000', 'last_change': '2024-09-17T15:33:09.763500+0000', 'last_active': '2024-09-17T15:33:23.226097+0000', 'last_peered': '2024-09-17T15:33:23.226097+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:33:23.226097+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "41'6700", 'ondisk_log_start': "41'6700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16424960, 'num_objects': 4010, 'num_object_clones': 0, 'num_object_copies': 8020, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3816, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4010, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13049, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 194}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:26.121 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:33:26.121 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'16851", 'reported_seq': 17158, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:23.224498+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:33:23.224498+0000', 'last_peered': '2024-09-17T15:33:23.224498+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:33:23.224498+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "41'6800", 'ondisk_log_start': "41'6800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16527360, 'num_objects': 4035, 'num_object_clones': 0, 'num_object_copies': 8070, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4035, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13638, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 55}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:26.122 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:33:26.846 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:33:26.846 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36227/128964 objects degraded (28.091%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":543,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":64482,"data_bytes":264069897,"bytes_used":1889218560,"bytes_avail":288021073920,"bytes_total":289910292480,"degraded_objects":36227,"degraded_total":128964,"degraded_ratio":0.2809078502527837,"recovering_objects_per_sec":15,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":191,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":409},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:33:26.865 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:33:26.865 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:33:27.452 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:33:27.458 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:33:27.486 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":311,"stamp":"2024-09-17T15:33:25.753547+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":264069897,"num_objects":64482,"num_object_clones":0,"num_object_copies":128964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36227,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64482,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":228509,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13234,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160775,"ondisk_log_size":160775,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1844940,"kb_used_data":718424,"kb_used_omap":13,"kb_used_meta":1126450,"kb_avail":281270580,"statfs":{"total":289910292480,"available":288021073920,"internally_reserved":0,"allocated":735666176,"data_stored":734739235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1153485823},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,3,13,12,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20172800,"num_objects":-4925,"num_object_clones":0,"num_object_copies":-9850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-191,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4925,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":191,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.012271"},"pg_stats":[{"pgid":"2.f","version":"47'16665","reported_seq":16923,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:23.249227+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:33:23.249227+0000","last_peered":"2024-09-17T15:33:23.249227+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:33:23.249227+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'6600","ondisk_log_start":"41'6600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16314368,"num_objects":3983,"num_object_clones":0,"num_object_copies":7966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16665,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16808","reported_seq":16920,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:22.940224+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:33:22.940224+0000","last_peered":"2024-09-17T15:33:22.940224+0000","last_clean":"2024-09-17T15:33:22.940224+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:33:22.940224+0000","last_undegraded":"2024-09-17T15:33:22.940224+0000","last_fullsized":"2024-09-17T15:33:22.940224+0000","mapping_epoch":33,"log_start":"41'6800","ondisk_log_start":"41'6800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16892870,"num_objects":4126,"num_object_clones":0,"num_object_copies":8252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4126,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15165,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16860","reported_seq":17082,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:22.940193+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:33:22.940193+0000","last_peered":"2024-09-17T15:33:22.940193+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:33:22.940193+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'6800","ondisk_log_start":"41'6800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16683032,"num_objects":4074,"num_object_clones":0,"num_object_copies":8148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4074,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15017,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-51}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16896","reported_seq":16975,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:22.882139+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:33:22.882139+0000","last_peered":"2024-09-17T15:33:22.882139+0000","last_clean":"2024-09-17T15:33:22.882139+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:33:22.882139+0000","last_undegraded":"2024-09-17T15:33:22.882139+0000","last_fullsized":"2024-09-17T15:33:22.882139+0000","mapping_epoch":33,"log_start":"41'6800","ondisk_log_start":"41'6800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16687104,"num_objects":4074,"num_object_clones":0,"num_object_copies":8148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16896,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16663","reported_seq":16802,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:23.154224+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:33:23.154224+0000","last_peered":"2024-09-17T15:33:23.154224+0000","last_clean":"2024-09-17T15:33:23.154224+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:33:23.154224+0000","last_undegraded":"2024-09-17T15:33:23.154224+0000","last_fullsized":"2024-09-17T15:33:23.154224+0000","mapping_epoch":32,"log_start":"41'6600","ondisk_log_start":"41'6600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16343523,"num_objects":3991,"num_object_clones":0,"num_object_copies":7982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3991,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13645,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17129","reported_seq":17429,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:23.130644+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:33:23.130644+0000","last_peered":"2024-09-17T15:33:23.130644+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:33:23.130644+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'7100","ondisk_log_start":"41'7100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16884195,"num_objects":4123,"num_object_clones":0,"num_object_copies":8246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4123,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13628,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-69},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16850","reported_seq":17168,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:23.193280+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:33:23.193280+0000","last_peered":"2024-09-17T15:33:23.193280+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:33:23.193280+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'6800","ondisk_log_start":"41'6800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16049094,"num_objects":3920,"num_object_clones":0,"num_object_copies":7840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3920,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13619,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-169},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16837","reported_seq":16974,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:23.182674+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:33:23.182674+0000","last_peered":"2024-09-17T15:33:23.182674+0000","last_clean":"2024-09-17T15:33:23.182674+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:33:23.182674+0000","last_undegraded":"2024-09-17T15:33:23.182674+0000","last_fullsized":"2024-09-17T15:33:23.182674+0000","mapping_epoch":32,"log_start":"41'6800","ondisk_log_start":"41'6800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16707584,"num_objects":4079,"num_object_clones":0,"num_object_copies":8158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13791,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223906+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.223906+0000","last_peered":"2024-09-17T15:33:11.223906+0000","last_clean":"2024-09-17T15:33:11.223906+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.223906+0000","last_undegraded":"2024-09-17T15:33:11.223906+0000","last_fullsized":"2024-09-17T15:33:11.223906+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16744","reported_seq":25036,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:22.638470+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:33:22.638470+0000","last_peered":"2024-09-17T15:33:22.638470+0000","last_clean":"2024-09-17T15:33:22.638470+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:33:22.638470+0000","last_undegraded":"2024-09-17T15:33:22.638470+0000","last_fullsized":"2024-09-17T15:33:22.638470+0000","mapping_epoch":42,"log_start":"41'6700","ondisk_log_start":"41'6700","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16683491,"num_objects":4074,"num_object_clones":0,"num_object_copies":8148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4074,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14744,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225444+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225444+0000","last_peered":"2024-09-17T15:33:11.225444+0000","last_clean":"2024-09-17T15:33:11.225444+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225444+0000","last_undegraded":"2024-09-17T15:33:11.225444+0000","last_fullsized":"2024-09-17T15:33:11.225444+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16521","reported_seq":16793,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:23.190182+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:33:23.190182+0000","last_peered":"2024-09-17T15:33:23.190182+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:33:23.190182+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'6500","ondisk_log_start":"41'6500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16470016,"num_objects":4021,"num_object_clones":0,"num_object_copies":8042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13117,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":78}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":112,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.228064+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.228064+0000","last_peered":"2024-09-17T15:33:11.228064+0000","last_clean":"2024-09-17T15:33:11.228064+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.228064+0000","last_undegraded":"2024-09-17T15:33:11.228064+0000","last_fullsized":"2024-09-17T15:33:11.228064+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16578","reported_seq":16796,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:22.914593+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:33:22.914593+0000","last_peered":"2024-09-17T15:33:22.914593+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:33:22.914593+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'6500","ondisk_log_start":"41'6500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16142819,"num_objects":3942,"num_object_clones":0,"num_object_copies":7884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3942,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14473,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-91}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16724","reported_seq":25139,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:22.985969+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:33:22.985969+0000","last_peered":"2024-09-17T15:33:22.985969+0000","last_clean":"2024-09-17T15:33:22.985969+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:33:22.985969+0000","last_undegraded":"2024-09-17T15:33:22.985969+0000","last_fullsized":"2024-09-17T15:33:22.985969+0000","mapping_epoch":46,"log_start":"41'6700","ondisk_log_start":"41'6700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16433152,"num_objects":4012,"num_object_clones":0,"num_object_copies":8024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14486,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16945","reported_seq":17251,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:23.224510+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:33:23.224510+0000","last_peered":"2024-09-17T15:33:23.224510+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:33:23.224510+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'6900","ondisk_log_start":"41'6900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16470016,"num_objects":4021,"num_object_clones":0,"num_object_copies":8042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13493,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-36}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16762","reported_seq":17601,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:23.226097+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:33:23.226097+0000","last_peered":"2024-09-17T15:33:23.226097+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:33:23.226097+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'6700","ondisk_log_start":"41'6700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16424960,"num_objects":4010,"num_object_clones":0,"num_object_copies":8020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3816,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13049,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":194}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225014+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225014+0000","last_peered":"2024-09-17T15:33:11.225014+0000","last_clean":"2024-09-17T15:33:11.225014+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225014+0000","last_undegraded":"2024-09-17T15:33:11.225014+0000","last_fullsized":"2024-09-17T15:33:11.225014+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16851","reported_seq":17158,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:23.224498+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:33:23.224498+0000","last_peered":"2024-09-17T15:33:23.224498+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:33:23.224498+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'6800","ondisk_log_start":"41'6800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16527360,"num_objects":4035,"num_object_clones":0,"num_object_copies":8070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13638,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":55}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16639","reported_seq":25023,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:22.664388+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:33:22.664388+0000","last_peered":"2024-09-17T15:33:22.664388+0000","last_clean":"2024-09-17T15:33:22.664388+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:33:22.664388+0000","last_undegraded":"2024-09-17T15:33:22.664388+0000","last_fullsized":"2024-09-17T15:33:22.664388+0000","mapping_epoch":44,"log_start":"41'6600","ondisk_log_start":"41'6600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16356294,"num_objects":3995,"num_object_clones":0,"num_object_copies":7990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13081,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":264069878,"num_objects":64480,"num_object_clones":0,"num_object_copies":128960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36227,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64480,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":228507,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13233,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":731234304,"data_stored":731099705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160772,"ondisk_log_size":160772,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051684,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609852,"kb_used_data":263020,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93761988,"statfs":{"total":96636764160,"available":96012275712,"internally_reserved":0,"allocated":269332480,"data_stored":269030882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":355135148},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,0,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724629,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":559232,"kb_used_data":194352,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93812608,"statfs":{"total":96636764160,"available":96064110592,"internally_reserved":0,"allocated":199016448,"data_stored":198692708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":373616299},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,4,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247912,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":675856,"kb_used_data":261052,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93695984,"statfs":{"total":96636764160,"available":95944687616,"internally_reserved":0,"allocated":267317248,"data_stored":267015645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":424734376},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":265383936,"data_stored":265347806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":198119424,"data_stored":198061157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":267730944,"data_stored":267690742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:33:30.489 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:33:31.033 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:33:31.033 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:33:31.061 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":313,"stamp":"2024-09-17T15:33:29.755028+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":254104329,"num_objects":62049,"num_object_clones":0,"num_object_copies":124098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36144,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62049,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":230942,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13317,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160708,"ondisk_log_size":160708,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1811316,"kb_used_data":697280,"kb_used_omap":13,"kb_used_meta":1113970,"kb_avail":281304204,"statfs":{"total":289910292480,"available":288055504896,"internally_reserved":0,"allocated":714014720,"data_stored":713087779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1140706303},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,4,8,11,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28860416,"num_objects":-7046,"num_object_clones":0,"num_object_copies":-14092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-274,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7046,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7046,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004416"},"pg_stats":[{"pgid":"2.f","version":"47'16807","reported_seq":17066,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:28.129765+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:33:28.129765+0000","last_peered":"2024-09-17T15:33:28.129765+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:33:28.129765+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'6800","ondisk_log_start":"41'6800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15732736,"num_objects":3841,"num_object_clones":0,"num_object_copies":7682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16807,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-143}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16967","reported_seq":17079,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:27.984204+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:33:27.984204+0000","last_peered":"2024-09-17T15:33:27.984204+0000","last_clean":"2024-09-17T15:33:27.984204+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:33:27.984204+0000","last_undegraded":"2024-09-17T15:33:27.984204+0000","last_fullsized":"2024-09-17T15:33:27.984204+0000","mapping_epoch":33,"log_start":"41'6900","ondisk_log_start":"41'6900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16241606,"num_objects":3967,"num_object_clones":0,"num_object_copies":7934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3967,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15324,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17014","reported_seq":17229,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:27.901093+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:33:27.901093+0000","last_peered":"2024-09-17T15:33:27.901093+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:33:27.901093+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'7000","ondisk_log_start":"41'7000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16052248,"num_objects":3920,"num_object_clones":0,"num_object_copies":7840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3920,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15171,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17061","reported_seq":17140,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:27.929466+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:33:27.929466+0000","last_peered":"2024-09-17T15:33:27.929466+0000","last_clean":"2024-09-17T15:33:27.929466+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:33:27.929466+0000","last_undegraded":"2024-09-17T15:33:27.929466+0000","last_fullsized":"2024-09-17T15:33:27.929466+0000","mapping_epoch":33,"log_start":"41'7000","ondisk_log_start":"41'7000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16011264,"num_objects":3909,"num_object_clones":0,"num_object_copies":7818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17061,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16815","reported_seq":16954,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:28.224021+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:33:28.224021+0000","last_peered":"2024-09-17T15:33:28.224021+0000","last_clean":"2024-09-17T15:33:28.224021+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:33:28.224021+0000","last_undegraded":"2024-09-17T15:33:28.224021+0000","last_fullsized":"2024-09-17T15:33:28.224021+0000","mapping_epoch":32,"log_start":"41'6800","ondisk_log_start":"41'6800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15720931,"num_objects":3839,"num_object_clones":0,"num_object_copies":7678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3839,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13797,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17288","reported_seq":17585,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:28.244422+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:33:28.244422+0000","last_peered":"2024-09-17T15:33:28.244422+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:33:28.244422+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'7200","ondisk_log_start":"41'7200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16232931,"num_objects":3964,"num_object_clones":0,"num_object_copies":7928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3964,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13787,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-228},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16979","reported_seq":17297,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:28.210419+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:33:28.210419+0000","last_peered":"2024-09-17T15:33:28.210419+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:33:28.210419+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'6900","ondisk_log_start":"41'6900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15520710,"num_objects":3791,"num_object_clones":0,"num_object_copies":7582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3791,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13748,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-298},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16978","reported_seq":17115,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:28.178835+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:33:28.178835+0000","last_peered":"2024-09-17T15:33:28.178835+0000","last_clean":"2024-09-17T15:33:28.178835+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:33:28.178835+0000","last_undegraded":"2024-09-17T15:33:28.178835+0000","last_fullsized":"2024-09-17T15:33:28.178835+0000","mapping_epoch":32,"log_start":"41'6900","ondisk_log_start":"41'6900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16130048,"num_objects":3938,"num_object_clones":0,"num_object_copies":7876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13932,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223906+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.223906+0000","last_peered":"2024-09-17T15:33:11.223906+0000","last_clean":"2024-09-17T15:33:11.223906+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.223906+0000","last_undegraded":"2024-09-17T15:33:11.223906+0000","last_fullsized":"2024-09-17T15:33:11.223906+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16903","reported_seq":25195,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:27.727390+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:33:27.727390+0000","last_peered":"2024-09-17T15:33:27.727390+0000","last_clean":"2024-09-17T15:33:27.727390+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:33:27.727390+0000","last_undegraded":"2024-09-17T15:33:27.727390+0000","last_fullsized":"2024-09-17T15:33:27.727390+0000","mapping_epoch":42,"log_start":"41'6900","ondisk_log_start":"41'6900","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16032227,"num_objects":3915,"num_object_clones":0,"num_object_copies":7830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3915,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14903,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225444+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225444+0000","last_peered":"2024-09-17T15:33:11.225444+0000","last_clean":"2024-09-17T15:33:11.225444+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225444+0000","last_undegraded":"2024-09-17T15:33:11.225444+0000","last_fullsized":"2024-09-17T15:33:11.225444+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16663","reported_seq":16936,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:28.192022+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:33:28.192022+0000","last_peered":"2024-09-17T15:33:28.192022+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:33:28.192022+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'6600","ondisk_log_start":"41'6600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15888384,"num_objects":3879,"num_object_clones":0,"num_object_copies":7758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3879,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13259,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-64}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16713","reported_seq":16931,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:27.910280+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:33:27.910280+0000","last_peered":"2024-09-17T15:33:27.910280+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:33:27.910280+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'6700","ondisk_log_start":"41'6700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15589859,"num_objects":3807,"num_object_clones":0,"num_object_copies":7614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3807,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14608,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-226}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16887","reported_seq":25302,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:27.938415+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:33:27.938415+0000","last_peered":"2024-09-17T15:33:27.938415+0000","last_clean":"2024-09-17T15:33:27.938415+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:33:27.938415+0000","last_undegraded":"2024-09-17T15:33:27.938415+0000","last_fullsized":"2024-09-17T15:33:27.938415+0000","mapping_epoch":46,"log_start":"41'6800","ondisk_log_start":"41'6800","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15765504,"num_objects":3849,"num_object_clones":0,"num_object_copies":7698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3849,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14649,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17105","reported_seq":17411,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:28.252917+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:33:28.252917+0000","last_peered":"2024-09-17T15:33:28.252917+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:33:28.252917+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'7100","ondisk_log_start":"41'7100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15814656,"num_objects":3861,"num_object_clones":0,"num_object_copies":7722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13653,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-196}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16929","reported_seq":17935,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:28.243974+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:33:28.243974+0000","last_peered":"2024-09-17T15:33:28.243974+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:33:28.243974+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'6900","ondisk_log_start":"41'6900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15740928,"num_objects":3843,"num_object_clones":0,"num_object_copies":7686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3733,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13216,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":427,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":110}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225014+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225014+0000","last_peered":"2024-09-17T15:33:11.225014+0000","last_clean":"2024-09-17T15:33:11.225014+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225014+0000","last_undegraded":"2024-09-17T15:33:11.225014+0000","last_fullsized":"2024-09-17T15:33:11.225014+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17018","reported_seq":17324,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:28.255607+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:33:28.255607+0000","last_peered":"2024-09-17T15:33:28.255607+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:33:28.255607+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'7000","ondisk_log_start":"41'7000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15843328,"num_objects":3868,"num_object_clones":0,"num_object_copies":7736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13805,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-112}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16778","reported_seq":25162,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:27.734573+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:33:27.734573+0000","last_peered":"2024-09-17T15:33:27.734573+0000","last_clean":"2024-09-17T15:33:27.734573+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:33:27.734573+0000","last_undegraded":"2024-09-17T15:33:27.734573+0000","last_fullsized":"2024-09-17T15:33:27.734573+0000","mapping_epoch":44,"log_start":"41'6700","ondisk_log_start":"41'6700","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15786950,"num_objects":3856,"num_object_clones":0,"num_object_copies":7712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3856,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13220,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":254104310,"num_objects":62047,"num_object_clones":0,"num_object_copies":124094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36144,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62047,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":230940,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13316,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":711327744,"data_stored":711193145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160705,"ondisk_log_size":160705,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051685,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":596332,"kb_used_data":257816,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93775508,"statfs":{"total":96636764160,"available":96026120192,"internally_reserved":0,"allocated":264003584,"data_stored":263701986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":346615468},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,2,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724630,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":543608,"kb_used_data":187048,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93828232,"statfs":{"total":96636764160,"available":96080109568,"internally_reserved":0,"allocated":191537152,"data_stored":191213412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":365096619},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,1,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247913,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":671376,"kb_used_data":252416,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93700464,"statfs":{"total":96636764160,"available":95949275136,"internally_reserved":0,"allocated":258473984,"data_stored":258172381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":428994216},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,5,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":257933312,"data_stored":257897182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":190672896,"data_stored":190614629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":262721536,"data_stored":262681334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:33:31.062 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:33:31.632 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:33:31.632 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:33:31.658 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":313,"stamp":"2024-09-17T15:33:29.755028+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":254104329,"num_objects":62049,"num_object_clones":0,"num_object_copies":124098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36144,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62049,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":230942,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13317,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160708,"ondisk_log_size":160708,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1811316,"kb_used_data":697280,"kb_used_omap":13,"kb_used_meta":1113970,"kb_avail":281304204,"statfs":{"total":289910292480,"available":288055504896,"internally_reserved":0,"allocated":714014720,"data_stored":713087779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1140706303},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,4,8,11,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28860416,"num_objects":-7046,"num_object_clones":0,"num_object_copies":-14092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-274,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7046,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7046,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004416"},"pg_stats":[{"pgid":"2.f","version":"47'16807","reported_seq":17066,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:28.129765+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:33:28.129765+0000","last_peered":"2024-09-17T15:33:28.129765+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:33:28.129765+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'6800","ondisk_log_start":"41'6800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15732736,"num_objects":3841,"num_object_clones":0,"num_object_copies":7682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16807,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-143}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16967","reported_seq":17079,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:27.984204+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:33:27.984204+0000","last_peered":"2024-09-17T15:33:27.984204+0000","last_clean":"2024-09-17T15:33:27.984204+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:33:27.984204+0000","last_undegraded":"2024-09-17T15:33:27.984204+0000","last_fullsized":"2024-09-17T15:33:27.984204+0000","mapping_epoch":33,"log_start":"41'6900","ondisk_log_start":"41'6900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16241606,"num_objects":3967,"num_object_clones":0,"num_object_copies":7934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3967,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15324,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17014","reported_seq":17229,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:27.901093+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:33:27.901093+0000","last_peered":"2024-09-17T15:33:27.901093+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:33:27.901093+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'7000","ondisk_log_start":"41'7000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16052248,"num_objects":3920,"num_object_clones":0,"num_object_copies":7840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3920,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15171,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17061","reported_seq":17140,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:27.929466+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:33:27.929466+0000","last_peered":"2024-09-17T15:33:27.929466+0000","last_clean":"2024-09-17T15:33:27.929466+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:33:27.929466+0000","last_undegraded":"2024-09-17T15:33:27.929466+0000","last_fullsized":"2024-09-17T15:33:27.929466+0000","mapping_epoch":33,"log_start":"41'7000","ondisk_log_start":"41'7000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16011264,"num_objects":3909,"num_object_clones":0,"num_object_copies":7818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17061,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16815","reported_seq":16954,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:28.224021+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:33:28.224021+0000","last_peered":"2024-09-17T15:33:28.224021+0000","last_clean":"2024-09-17T15:33:28.224021+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:33:28.224021+0000","last_undegraded":"2024-09-17T15:33:28.224021+0000","last_fullsized":"2024-09-17T15:33:28.224021+0000","mapping_epoch":32,"log_start":"41'6800","ondisk_log_start":"41'6800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15720931,"num_objects":3839,"num_object_clones":0,"num_object_copies":7678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3839,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13797,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17288","reported_seq":17585,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:28.244422+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:33:28.244422+0000","last_peered":"2024-09-17T15:33:28.244422+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:33:28.244422+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'7200","ondisk_log_start":"41'7200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16232931,"num_objects":3964,"num_object_clones":0,"num_object_copies":7928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3964,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13787,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-228},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16979","reported_seq":17297,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:28.210419+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:33:28.210419+0000","last_peered":"2024-09-17T15:33:28.210419+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:33:28.210419+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'6900","ondisk_log_start":"41'6900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15520710,"num_objects":3791,"num_object_clones":0,"num_object_copies":7582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3791,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13748,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-298},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16978","reported_seq":17115,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:28.178835+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:33:28.178835+0000","last_peered":"2024-09-17T15:33:28.178835+0000","last_clean":"2024-09-17T15:33:28.178835+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:33:28.178835+0000","last_undegraded":"2024-09-17T15:33:28.178835+0000","last_fullsized":"2024-09-17T15:33:28.178835+0000","mapping_epoch":32,"log_start":"41'6900","ondisk_log_start":"41'6900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16130048,"num_objects":3938,"num_object_clones":0,"num_object_copies":7876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13932,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223906+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.223906+0000","last_peered":"2024-09-17T15:33:11.223906+0000","last_clean":"2024-09-17T15:33:11.223906+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.223906+0000","last_undegraded":"2024-09-17T15:33:11.223906+0000","last_fullsized":"2024-09-17T15:33:11.223906+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16903","reported_seq":25195,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:27.727390+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:33:27.727390+0000","last_peered":"2024-09-17T15:33:27.727390+0000","last_clean":"2024-09-17T15:33:27.727390+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:33:27.727390+0000","last_undegraded":"2024-09-17T15:33:27.727390+0000","last_fullsized":"2024-09-17T15:33:27.727390+0000","mapping_epoch":42,"log_start":"41'6900","ondisk_log_start":"41'6900","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16032227,"num_objects":3915,"num_object_clones":0,"num_object_copies":7830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3915,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14903,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225444+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225444+0000","last_peered":"2024-09-17T15:33:11.225444+0000","last_clean":"2024-09-17T15:33:11.225444+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225444+0000","last_undegraded":"2024-09-17T15:33:11.225444+0000","last_fullsized":"2024-09-17T15:33:11.225444+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16663","reported_seq":16936,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:28.192022+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:33:28.192022+0000","last_peered":"2024-09-17T15:33:28.192022+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:33:28.192022+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'6600","ondisk_log_start":"41'6600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15888384,"num_objects":3879,"num_object_clones":0,"num_object_copies":7758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3879,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13259,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-64}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16713","reported_seq":16931,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:27.910280+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:33:27.910280+0000","last_peered":"2024-09-17T15:33:27.910280+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:33:27.910280+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'6700","ondisk_log_start":"41'6700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15589859,"num_objects":3807,"num_object_clones":0,"num_object_copies":7614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3807,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14608,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-226}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16887","reported_seq":25302,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:27.938415+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:33:27.938415+0000","last_peered":"2024-09-17T15:33:27.938415+0000","last_clean":"2024-09-17T15:33:27.938415+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:33:27.938415+0000","last_undegraded":"2024-09-17T15:33:27.938415+0000","last_fullsized":"2024-09-17T15:33:27.938415+0000","mapping_epoch":46,"log_start":"41'6800","ondisk_log_start":"41'6800","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15765504,"num_objects":3849,"num_object_clones":0,"num_object_copies":7698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3849,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14649,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17105","reported_seq":17411,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:28.252917+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:33:28.252917+0000","last_peered":"2024-09-17T15:33:28.252917+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:33:28.252917+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'7100","ondisk_log_start":"41'7100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15814656,"num_objects":3861,"num_object_clones":0,"num_object_copies":7722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13653,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-196}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16929","reported_seq":17935,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:28.243974+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:33:28.243974+0000","last_peered":"2024-09-17T15:33:28.243974+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:33:28.243974+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'6900","ondisk_log_start":"41'6900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15740928,"num_objects":3843,"num_object_clones":0,"num_object_copies":7686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3733,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13216,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":427,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":110}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225014+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225014+0000","last_peered":"2024-09-17T15:33:11.225014+0000","last_clean":"2024-09-17T15:33:11.225014+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225014+0000","last_undegraded":"2024-09-17T15:33:11.225014+0000","last_fullsized":"2024-09-17T15:33:11.225014+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17018","reported_seq":17324,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:28.255607+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:33:28.255607+0000","last_peered":"2024-09-17T15:33:28.255607+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:33:28.255607+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'7000","ondisk_log_start":"41'7000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15843328,"num_objects":3868,"num_object_clones":0,"num_object_copies":7736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13805,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-112}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16778","reported_seq":25162,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:27.734573+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:33:27.734573+0000","last_peered":"2024-09-17T15:33:27.734573+0000","last_clean":"2024-09-17T15:33:27.734573+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:33:27.734573+0000","last_undegraded":"2024-09-17T15:33:27.734573+0000","last_fullsized":"2024-09-17T15:33:27.734573+0000","mapping_epoch":44,"log_start":"41'6700","ondisk_log_start":"41'6700","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15786950,"num_objects":3856,"num_object_clones":0,"num_object_copies":7712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3856,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13220,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":254104310,"num_objects":62047,"num_object_clones":0,"num_object_copies":124094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36144,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62047,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":230940,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13316,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":711327744,"data_stored":711193145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160705,"ondisk_log_size":160705,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051685,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":596332,"kb_used_data":257816,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93775508,"statfs":{"total":96636764160,"available":96026120192,"internally_reserved":0,"allocated":264003584,"data_stored":263701986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":346615468},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,2,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724630,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":543608,"kb_used_data":187048,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93828232,"statfs":{"total":96636764160,"available":96080109568,"internally_reserved":0,"allocated":191537152,"data_stored":191213412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":365096619},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,1,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247913,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":671376,"kb_used_data":252416,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93700464,"statfs":{"total":96636764160,"available":95949275136,"internally_reserved":0,"allocated":258473984,"data_stored":258172381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":428994216},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,5,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":257933312,"data_stored":257897182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":190672896,"data_stored":190614629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":262721536,"data_stored":262681334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:33:31.659 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:33:31.660 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'16807", 'reported_seq': 17066, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:28.129765+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:33:28.129765+0000', 'last_peered': '2024-09-17T15:33:28.129765+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:33:28.129765+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "41'6800", 'ondisk_log_start': "41'6800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15732736, 'num_objects': 3841, 'num_object_clones': 0, 'num_object_copies': 7682, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3841, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16807, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -143}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:31.660 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:33:31.660 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'17014", 'reported_seq': 17229, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:27.901093+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:33:27.901093+0000', 'last_peered': '2024-09-17T15:33:27.901093+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:33:27.901093+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "41'7000", 'ondisk_log_start': "41'7000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16052248, 'num_objects': 3920, 'num_object_clones': 0, 'num_object_copies': 7840, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3920, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15171, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -205}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:33:31.660 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:33:31.660 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'17288", 'reported_seq': 17585, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:28.244422+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:33:28.244422+0000', 'last_peered': '2024-09-17T15:33:28.244422+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:33:28.244422+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "41'7200", 'ondisk_log_start': "41'7200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16232931, 'num_objects': 3964, 'num_object_clones': 0, 'num_object_copies': 7928, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3964, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13787, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -228}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:31.660 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:33:31.660 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'16979", 'reported_seq': 17297, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:28.210419+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:33:28.210419+0000', 'last_peered': '2024-09-17T15:33:28.210419+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:33:28.210419+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "41'6900", 'ondisk_log_start': "41'6900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15520710, 'num_objects': 3791, 'num_object_clones': 0, 'num_object_copies': 7582, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4089, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3791, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 13748, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 85, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -298}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:31.660 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:33:31.660 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'16663", 'reported_seq': 16936, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:28.192022+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:33:28.192022+0000', 'last_peered': '2024-09-17T15:33:28.192022+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:33:28.192022+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "41'6600", 'ondisk_log_start': "41'6600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15888384, 'num_objects': 3879, 'num_object_clones': 0, 'num_object_copies': 7758, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3879, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13259, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -64}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:31.660 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:33:31.661 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'16713", 'reported_seq': 16931, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:27.910280+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:33:27.910280+0000', 'last_peered': '2024-09-17T15:33:27.910280+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:33:27.910280+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "41'6700", 'ondisk_log_start': "41'6700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15589859, 'num_objects': 3807, 'num_object_clones': 0, 'num_object_copies': 7614, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4034, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3807, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14608, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -226}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:33:31.661 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:33:31.661 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'17105", 'reported_seq': 17411, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:28.252917+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:33:28.252917+0000', 'last_peered': '2024-09-17T15:33:28.252917+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:33:28.252917+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "41'7100", 'ondisk_log_start': "41'7100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15814656, 'num_objects': 3861, 'num_object_clones': 0, 'num_object_copies': 7722, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3861, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13653, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -196}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:31.661 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:33:31.661 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'16929", 'reported_seq': 17935, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:28.243974+0000', 'last_change': '2024-09-17T15:33:09.763500+0000', 'last_active': '2024-09-17T15:33:28.243974+0000', 'last_peered': '2024-09-17T15:33:28.243974+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:33:28.243974+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "41'6900", 'ondisk_log_start': "41'6900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15740928, 'num_objects': 3843, 'num_object_clones': 0, 'num_object_copies': 7686, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3733, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3843, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13216, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 427, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 110}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:31.661 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:33:31.661 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'17018", 'reported_seq': 17324, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:28.255607+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:33:28.255607+0000', 'last_peered': '2024-09-17T15:33:28.255607+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:33:28.255607+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "41'7000", 'ondisk_log_start': "41'7000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15843328, 'num_objects': 3868, 'num_object_clones': 0, 'num_object_copies': 7736, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3868, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13805, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -112}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:31.661 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:33:32.347 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:33:32.347 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36144/124098 objects degraded (29.125%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":549,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":62049,"data_bytes":254104329,"bytes_used":1854787584,"bytes_avail":288055504896,"bytes_total":289910292480,"degraded_objects":36144,"degraded_total":124098,"degraded_ratio":0.29125368660252382,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":274,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":586},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:33:32.372 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:33:32.373 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:33:32.943 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:33:32.944 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:33:32.968 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2024-09-17T15:33:31.755881+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":254104329,"num_objects":62049,"num_object_clones":0,"num_object_copies":124098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36144,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62049,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":230942,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13317,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160708,"ondisk_log_size":160708,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1811316,"kb_used_data":697280,"kb_used_omap":13,"kb_used_meta":1113970,"kb_avail":281304204,"statfs":{"total":289910292480,"available":288055504896,"internally_reserved":0,"allocated":714014720,"data_stored":713087779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1140706303},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,4,8,11,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20082688,"num_objects":-4903,"num_object_clones":0,"num_object_copies":-9806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-182,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4903,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004823"},"pg_stats":[{"pgid":"2.f","version":"47'16807","reported_seq":17066,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:28.129765+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:33:28.129765+0000","last_peered":"2024-09-17T15:33:28.129765+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:33:28.129765+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'6800","ondisk_log_start":"41'6800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15732736,"num_objects":3841,"num_object_clones":0,"num_object_copies":7682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16807,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-143}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16967","reported_seq":17079,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:27.984204+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:33:27.984204+0000","last_peered":"2024-09-17T15:33:27.984204+0000","last_clean":"2024-09-17T15:33:27.984204+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:33:27.984204+0000","last_undegraded":"2024-09-17T15:33:27.984204+0000","last_fullsized":"2024-09-17T15:33:27.984204+0000","mapping_epoch":33,"log_start":"41'6900","ondisk_log_start":"41'6900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16241606,"num_objects":3967,"num_object_clones":0,"num_object_copies":7934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3967,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15324,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17014","reported_seq":17229,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:27.901093+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:33:27.901093+0000","last_peered":"2024-09-17T15:33:27.901093+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:33:27.901093+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'7000","ondisk_log_start":"41'7000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16052248,"num_objects":3920,"num_object_clones":0,"num_object_copies":7840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3920,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15171,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17061","reported_seq":17140,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:27.929466+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:33:27.929466+0000","last_peered":"2024-09-17T15:33:27.929466+0000","last_clean":"2024-09-17T15:33:27.929466+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:33:27.929466+0000","last_undegraded":"2024-09-17T15:33:27.929466+0000","last_fullsized":"2024-09-17T15:33:27.929466+0000","mapping_epoch":33,"log_start":"41'7000","ondisk_log_start":"41'7000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16011264,"num_objects":3909,"num_object_clones":0,"num_object_copies":7818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17061,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16815","reported_seq":16954,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:28.224021+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:33:28.224021+0000","last_peered":"2024-09-17T15:33:28.224021+0000","last_clean":"2024-09-17T15:33:28.224021+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:33:28.224021+0000","last_undegraded":"2024-09-17T15:33:28.224021+0000","last_fullsized":"2024-09-17T15:33:28.224021+0000","mapping_epoch":32,"log_start":"41'6800","ondisk_log_start":"41'6800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15720931,"num_objects":3839,"num_object_clones":0,"num_object_copies":7678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3839,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13797,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17288","reported_seq":17585,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:28.244422+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:33:28.244422+0000","last_peered":"2024-09-17T15:33:28.244422+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:33:28.244422+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'7200","ondisk_log_start":"41'7200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16232931,"num_objects":3964,"num_object_clones":0,"num_object_copies":7928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3964,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13787,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-228},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16979","reported_seq":17297,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:28.210419+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:33:28.210419+0000","last_peered":"2024-09-17T15:33:28.210419+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:33:28.210419+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'6900","ondisk_log_start":"41'6900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15520710,"num_objects":3791,"num_object_clones":0,"num_object_copies":7582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3791,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13748,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-298},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16978","reported_seq":17115,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:28.178835+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:33:28.178835+0000","last_peered":"2024-09-17T15:33:28.178835+0000","last_clean":"2024-09-17T15:33:28.178835+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:33:28.178835+0000","last_undegraded":"2024-09-17T15:33:28.178835+0000","last_fullsized":"2024-09-17T15:33:28.178835+0000","mapping_epoch":32,"log_start":"41'6900","ondisk_log_start":"41'6900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16130048,"num_objects":3938,"num_object_clones":0,"num_object_copies":7876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13932,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223906+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.223906+0000","last_peered":"2024-09-17T15:33:11.223906+0000","last_clean":"2024-09-17T15:33:11.223906+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.223906+0000","last_undegraded":"2024-09-17T15:33:11.223906+0000","last_fullsized":"2024-09-17T15:33:11.223906+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16903","reported_seq":25195,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:27.727390+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:33:27.727390+0000","last_peered":"2024-09-17T15:33:27.727390+0000","last_clean":"2024-09-17T15:33:27.727390+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:33:27.727390+0000","last_undegraded":"2024-09-17T15:33:27.727390+0000","last_fullsized":"2024-09-17T15:33:27.727390+0000","mapping_epoch":42,"log_start":"41'6900","ondisk_log_start":"41'6900","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16032227,"num_objects":3915,"num_object_clones":0,"num_object_copies":7830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3915,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14903,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225444+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225444+0000","last_peered":"2024-09-17T15:33:11.225444+0000","last_clean":"2024-09-17T15:33:11.225444+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225444+0000","last_undegraded":"2024-09-17T15:33:11.225444+0000","last_fullsized":"2024-09-17T15:33:11.225444+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16663","reported_seq":16936,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:28.192022+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:33:28.192022+0000","last_peered":"2024-09-17T15:33:28.192022+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:33:28.192022+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'6600","ondisk_log_start":"41'6600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15888384,"num_objects":3879,"num_object_clones":0,"num_object_copies":7758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3879,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13259,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-64}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16713","reported_seq":16931,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:27.910280+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:33:27.910280+0000","last_peered":"2024-09-17T15:33:27.910280+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:33:27.910280+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'6700","ondisk_log_start":"41'6700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15589859,"num_objects":3807,"num_object_clones":0,"num_object_copies":7614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3807,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14608,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-226}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16887","reported_seq":25302,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:27.938415+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:33:27.938415+0000","last_peered":"2024-09-17T15:33:27.938415+0000","last_clean":"2024-09-17T15:33:27.938415+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:33:27.938415+0000","last_undegraded":"2024-09-17T15:33:27.938415+0000","last_fullsized":"2024-09-17T15:33:27.938415+0000","mapping_epoch":46,"log_start":"41'6800","ondisk_log_start":"41'6800","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15765504,"num_objects":3849,"num_object_clones":0,"num_object_copies":7698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3849,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14649,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17105","reported_seq":17411,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:28.252917+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:33:28.252917+0000","last_peered":"2024-09-17T15:33:28.252917+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:33:28.252917+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'7100","ondisk_log_start":"41'7100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15814656,"num_objects":3861,"num_object_clones":0,"num_object_copies":7722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13653,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-196}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16929","reported_seq":17935,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:28.243974+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:33:28.243974+0000","last_peered":"2024-09-17T15:33:28.243974+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:33:28.243974+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'6900","ondisk_log_start":"41'6900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15740928,"num_objects":3843,"num_object_clones":0,"num_object_copies":7686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3733,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13216,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":427,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":110}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225014+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225014+0000","last_peered":"2024-09-17T15:33:11.225014+0000","last_clean":"2024-09-17T15:33:11.225014+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225014+0000","last_undegraded":"2024-09-17T15:33:11.225014+0000","last_fullsized":"2024-09-17T15:33:11.225014+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17018","reported_seq":17324,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:28.255607+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:33:28.255607+0000","last_peered":"2024-09-17T15:33:28.255607+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:33:28.255607+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'7000","ondisk_log_start":"41'7000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15843328,"num_objects":3868,"num_object_clones":0,"num_object_copies":7736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13805,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-112}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16778","reported_seq":25162,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:27.734573+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:33:27.734573+0000","last_peered":"2024-09-17T15:33:27.734573+0000","last_clean":"2024-09-17T15:33:27.734573+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:33:27.734573+0000","last_undegraded":"2024-09-17T15:33:27.734573+0000","last_fullsized":"2024-09-17T15:33:27.734573+0000","mapping_epoch":44,"log_start":"41'6700","ondisk_log_start":"41'6700","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15786950,"num_objects":3856,"num_object_clones":0,"num_object_copies":7712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3856,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13220,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":254104310,"num_objects":62047,"num_object_clones":0,"num_object_copies":124094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36144,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62047,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":230940,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13316,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":711327744,"data_stored":711193145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160705,"ondisk_log_size":160705,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051685,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":596332,"kb_used_data":257816,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93775508,"statfs":{"total":96636764160,"available":96026120192,"internally_reserved":0,"allocated":264003584,"data_stored":263701986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":346615468},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,2,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724630,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":543608,"kb_used_data":187048,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93828232,"statfs":{"total":96636764160,"available":96080109568,"internally_reserved":0,"allocated":191537152,"data_stored":191213412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":365096619},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,1,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247913,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":671376,"kb_used_data":252416,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93700464,"statfs":{"total":96636764160,"available":95949275136,"internally_reserved":0,"allocated":258473984,"data_stored":258172381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":428994216},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,5,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":257933312,"data_stored":257897182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":190672896,"data_stored":190614629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":262721536,"data_stored":262681334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:33:35.970 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:33:36.506 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:33:36.509 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:33:36.528 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2024-09-17T15:33:35.757279+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":244294409,"num_objects":59654,"num_object_clones":0,"num_object_copies":119308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36030,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59654,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":233337,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13431,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160703,"ondisk_log_size":160703,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1792172,"kb_used_data":678132,"kb_used_omap":13,"kb_used_meta":1113970,"kb_avail":281323348,"statfs":{"total":289910292480,"available":288075108352,"internally_reserved":0,"allocated":694407168,"data_stored":693480227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1140706303},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,4,9,9,8,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19775488,"num_objects":-4828,"num_object_clones":0,"num_object_copies":-9656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-197,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4828,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004617"},"pg_stats":[{"pgid":"2.f","version":"47'16960","reported_seq":17219,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:33.203297+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:33:33.203297+0000","last_peered":"2024-09-17T15:33:33.203297+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:33:33.203297+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'6900","ondisk_log_start":"41'6900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15106048,"num_objects":3688,"num_object_clones":0,"num_object_copies":7376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-296}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17110","reported_seq":17222,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:32.957683+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:33:32.957683+0000","last_peered":"2024-09-17T15:33:32.957683+0000","last_clean":"2024-09-17T15:33:32.957683+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:33:32.957683+0000","last_undegraded":"2024-09-17T15:33:32.957683+0000","last_fullsized":"2024-09-17T15:33:32.957683+0000","mapping_epoch":33,"log_start":"41'7100","ondisk_log_start":"41'7100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15655878,"num_objects":3824,"num_object_clones":0,"num_object_copies":7648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3824,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15467,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17148","reported_seq":17370,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:32.964093+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:33:32.964093+0000","last_peered":"2024-09-17T15:33:32.964093+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:33:32.964093+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'7100","ondisk_log_start":"41'7100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15503384,"num_objects":3786,"num_object_clones":0,"num_object_copies":7572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3786,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15305,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-339}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17214","reported_seq":17293,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:32.984827+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:33:32.984827+0000","last_peered":"2024-09-17T15:33:32.984827+0000","last_clean":"2024-09-17T15:33:32.984827+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:33:32.984827+0000","last_undegraded":"2024-09-17T15:33:32.984827+0000","last_fullsized":"2024-09-17T15:33:32.984827+0000","mapping_epoch":33,"log_start":"41'7200","ondisk_log_start":"41'7200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15384576,"num_objects":3756,"num_object_clones":0,"num_object_copies":7512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17214,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16966","reported_seq":17105,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:33.283591+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:33:33.283591+0000","last_peered":"2024-09-17T15:33:33.283591+0000","last_clean":"2024-09-17T15:33:33.283591+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:33:33.283591+0000","last_undegraded":"2024-09-17T15:33:33.283591+0000","last_fullsized":"2024-09-17T15:33:33.283591+0000","mapping_epoch":32,"log_start":"41'6900","ondisk_log_start":"41'6900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15102435,"num_objects":3688,"num_object_clones":0,"num_object_copies":7376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3688,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13948,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17447","reported_seq":17747,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:33.258653+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:33:33.258653+0000","last_peered":"2024-09-17T15:33:33.258653+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:33:33.258653+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'7400","ondisk_log_start":"41'7400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15581667,"num_objects":3805,"num_object_clones":0,"num_object_copies":7610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3805,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13946,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-387},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17121","reported_seq":17439,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:33.214109+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:33:33.214109+0000","last_peered":"2024-09-17T15:33:33.214109+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:33:33.214109+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'7100","ondisk_log_start":"41'7100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14939078,"num_objects":3649,"num_object_clones":0,"num_object_copies":7298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3649,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13890,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-440},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17125","reported_seq":17262,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:33.209105+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:33:33.209105+0000","last_peered":"2024-09-17T15:33:33.209105+0000","last_clean":"2024-09-17T15:33:33.209105+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:33:33.209105+0000","last_undegraded":"2024-09-17T15:33:33.209105+0000","last_fullsized":"2024-09-17T15:33:33.209105+0000","mapping_epoch":32,"log_start":"41'7100","ondisk_log_start":"41'7100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15527936,"num_objects":3791,"num_object_clones":0,"num_object_copies":7582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14079,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223906+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.223906+0000","last_peered":"2024-09-17T15:33:11.223906+0000","last_clean":"2024-09-17T15:33:11.223906+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.223906+0000","last_undegraded":"2024-09-17T15:33:11.223906+0000","last_fullsized":"2024-09-17T15:33:11.223906+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17052","reported_seq":25344,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:32.711474+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:33:32.711474+0000","last_peered":"2024-09-17T15:33:32.711474+0000","last_clean":"2024-09-17T15:33:32.711474+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:33:32.711474+0000","last_undegraded":"2024-09-17T15:33:32.711474+0000","last_fullsized":"2024-09-17T15:33:32.711474+0000","mapping_epoch":42,"log_start":"41'7000","ondisk_log_start":"41'7000","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15421923,"num_objects":3766,"num_object_clones":0,"num_object_copies":7532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3766,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15052,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225444+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225444+0000","last_peered":"2024-09-17T15:33:11.225444+0000","last_clean":"2024-09-17T15:33:11.225444+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225444+0000","last_undegraded":"2024-09-17T15:33:11.225444+0000","last_fullsized":"2024-09-17T15:33:11.225444+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16816","reported_seq":17090,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:33.270552+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:33:33.270552+0000","last_peered":"2024-09-17T15:33:33.270552+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:33:33.270552+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'6800","ondisk_log_start":"41'6800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15261696,"num_objects":3726,"num_object_clones":0,"num_object_copies":7452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13412,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-217}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16891","reported_seq":17109,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:32.956259+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:33:32.956259+0000","last_peered":"2024-09-17T15:33:32.956259+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:33:32.956259+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'6800","ondisk_log_start":"41'6800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14860771,"num_objects":3629,"num_object_clones":0,"num_object_copies":7258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3629,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14786,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-404}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17029","reported_seq":25444,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:32.893594+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:33:32.893594+0000","last_peered":"2024-09-17T15:33:32.893594+0000","last_clean":"2024-09-17T15:33:32.893594+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:33:32.893594+0000","last_undegraded":"2024-09-17T15:33:32.893594+0000","last_fullsized":"2024-09-17T15:33:32.893594+0000","mapping_epoch":46,"log_start":"41'7000","ondisk_log_start":"41'7000","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15183872,"num_objects":3707,"num_object_clones":0,"num_object_copies":7414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14791,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17259","reported_seq":17563,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:33.300140+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:33:33.300140+0000","last_peered":"2024-09-17T15:33:33.300140+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:33:33.300140+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'7200","ondisk_log_start":"41'7200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15183872,"num_objects":3707,"num_object_clones":0,"num_object_copies":7414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13807,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-350}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17064","reported_seq":18298,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:33.290224+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:33:33.290224+0000","last_peered":"2024-09-17T15:33:33.290224+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:33:33.290224+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'7000","ondisk_log_start":"41'7000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15187968,"num_objects":3708,"num_object_clones":0,"num_object_copies":7416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3619,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13351,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":541,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":89}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225014+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225014+0000","last_peered":"2024-09-17T15:33:11.225014+0000","last_clean":"2024-09-17T15:33:11.225014+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225014+0000","last_undegraded":"2024-09-17T15:33:11.225014+0000","last_fullsized":"2024-09-17T15:33:11.225014+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17156","reported_seq":17463,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:33.269642+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:33:33.269642+0000","last_peered":"2024-09-17T15:33:33.269642+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:33:33.269642+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'7100","ondisk_log_start":"41'7100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15278080,"num_objects":3730,"num_object_clones":0,"num_object_copies":7460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13943,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-250}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16942","reported_seq":25326,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:32.784491+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:33:32.784491+0000","last_peered":"2024-09-17T15:33:32.784491+0000","last_clean":"2024-09-17T15:33:32.784491+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:33:32.784491+0000","last_undegraded":"2024-09-17T15:33:32.784491+0000","last_fullsized":"2024-09-17T15:33:32.784491+0000","mapping_epoch":44,"log_start":"41'6900","ondisk_log_start":"41'6900","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15115206,"num_objects":3692,"num_object_clones":0,"num_object_copies":7384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13384,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":244294390,"num_objects":59652,"num_object_clones":0,"num_object_copies":119304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36030,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59652,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":233335,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13430,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":691544064,"data_stored":691409465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160700,"ondisk_log_size":160700,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051686,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":595628,"kb_used_data":252956,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93776212,"statfs":{"total":96636764160,"available":96026841088,"internally_reserved":0,"allocated":259026944,"data_stored":258725346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":350875308},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724631,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":540624,"kb_used_data":179904,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93831216,"statfs":{"total":96636764160,"available":96083165184,"internally_reserved":0,"allocated":184221696,"data_stored":183897956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":369356459},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,5,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247914,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":655920,"kb_used_data":245272,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93715920,"statfs":{"total":96636764160,"available":95965102080,"internally_reserved":0,"allocated":251158528,"data_stored":250856925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":420474536},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,3,5,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":250580992,"data_stored":250544862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":183136256,"data_stored":183077989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257826816,"data_stored":257786614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:33:36.529 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:33:37.086 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:33:37.087 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:33:37.112 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2024-09-17T15:33:35.757279+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":244294409,"num_objects":59654,"num_object_clones":0,"num_object_copies":119308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36030,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59654,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":233337,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13431,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160703,"ondisk_log_size":160703,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1792172,"kb_used_data":678132,"kb_used_omap":13,"kb_used_meta":1113970,"kb_avail":281323348,"statfs":{"total":289910292480,"available":288075108352,"internally_reserved":0,"allocated":694407168,"data_stored":693480227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1140706303},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,4,9,9,8,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19775488,"num_objects":-4828,"num_object_clones":0,"num_object_copies":-9656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-197,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4828,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004617"},"pg_stats":[{"pgid":"2.f","version":"47'16960","reported_seq":17219,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:33.203297+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:33:33.203297+0000","last_peered":"2024-09-17T15:33:33.203297+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:33:33.203297+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'6900","ondisk_log_start":"41'6900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15106048,"num_objects":3688,"num_object_clones":0,"num_object_copies":7376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-296}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17110","reported_seq":17222,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:32.957683+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:33:32.957683+0000","last_peered":"2024-09-17T15:33:32.957683+0000","last_clean":"2024-09-17T15:33:32.957683+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:33:32.957683+0000","last_undegraded":"2024-09-17T15:33:32.957683+0000","last_fullsized":"2024-09-17T15:33:32.957683+0000","mapping_epoch":33,"log_start":"41'7100","ondisk_log_start":"41'7100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15655878,"num_objects":3824,"num_object_clones":0,"num_object_copies":7648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3824,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15467,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17148","reported_seq":17370,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:32.964093+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:33:32.964093+0000","last_peered":"2024-09-17T15:33:32.964093+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:33:32.964093+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'7100","ondisk_log_start":"41'7100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15503384,"num_objects":3786,"num_object_clones":0,"num_object_copies":7572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3786,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15305,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-339}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17214","reported_seq":17293,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:32.984827+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:33:32.984827+0000","last_peered":"2024-09-17T15:33:32.984827+0000","last_clean":"2024-09-17T15:33:32.984827+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:33:32.984827+0000","last_undegraded":"2024-09-17T15:33:32.984827+0000","last_fullsized":"2024-09-17T15:33:32.984827+0000","mapping_epoch":33,"log_start":"41'7200","ondisk_log_start":"41'7200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15384576,"num_objects":3756,"num_object_clones":0,"num_object_copies":7512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17214,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16966","reported_seq":17105,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:33.283591+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:33:33.283591+0000","last_peered":"2024-09-17T15:33:33.283591+0000","last_clean":"2024-09-17T15:33:33.283591+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:33:33.283591+0000","last_undegraded":"2024-09-17T15:33:33.283591+0000","last_fullsized":"2024-09-17T15:33:33.283591+0000","mapping_epoch":32,"log_start":"41'6900","ondisk_log_start":"41'6900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15102435,"num_objects":3688,"num_object_clones":0,"num_object_copies":7376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3688,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13948,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17447","reported_seq":17747,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:33.258653+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:33:33.258653+0000","last_peered":"2024-09-17T15:33:33.258653+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:33:33.258653+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'7400","ondisk_log_start":"41'7400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15581667,"num_objects":3805,"num_object_clones":0,"num_object_copies":7610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3805,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13946,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-387},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17121","reported_seq":17439,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:33.214109+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:33:33.214109+0000","last_peered":"2024-09-17T15:33:33.214109+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:33:33.214109+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'7100","ondisk_log_start":"41'7100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14939078,"num_objects":3649,"num_object_clones":0,"num_object_copies":7298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3649,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13890,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-440},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17125","reported_seq":17262,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:33.209105+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:33:33.209105+0000","last_peered":"2024-09-17T15:33:33.209105+0000","last_clean":"2024-09-17T15:33:33.209105+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:33:33.209105+0000","last_undegraded":"2024-09-17T15:33:33.209105+0000","last_fullsized":"2024-09-17T15:33:33.209105+0000","mapping_epoch":32,"log_start":"41'7100","ondisk_log_start":"41'7100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15527936,"num_objects":3791,"num_object_clones":0,"num_object_copies":7582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14079,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223906+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.223906+0000","last_peered":"2024-09-17T15:33:11.223906+0000","last_clean":"2024-09-17T15:33:11.223906+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.223906+0000","last_undegraded":"2024-09-17T15:33:11.223906+0000","last_fullsized":"2024-09-17T15:33:11.223906+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17052","reported_seq":25344,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:32.711474+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:33:32.711474+0000","last_peered":"2024-09-17T15:33:32.711474+0000","last_clean":"2024-09-17T15:33:32.711474+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:33:32.711474+0000","last_undegraded":"2024-09-17T15:33:32.711474+0000","last_fullsized":"2024-09-17T15:33:32.711474+0000","mapping_epoch":42,"log_start":"41'7000","ondisk_log_start":"41'7000","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15421923,"num_objects":3766,"num_object_clones":0,"num_object_copies":7532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3766,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15052,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225444+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225444+0000","last_peered":"2024-09-17T15:33:11.225444+0000","last_clean":"2024-09-17T15:33:11.225444+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225444+0000","last_undegraded":"2024-09-17T15:33:11.225444+0000","last_fullsized":"2024-09-17T15:33:11.225444+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16816","reported_seq":17090,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:33.270552+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:33:33.270552+0000","last_peered":"2024-09-17T15:33:33.270552+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:33:33.270552+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'6800","ondisk_log_start":"41'6800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15261696,"num_objects":3726,"num_object_clones":0,"num_object_copies":7452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13412,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-217}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16891","reported_seq":17109,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:32.956259+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:33:32.956259+0000","last_peered":"2024-09-17T15:33:32.956259+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:33:32.956259+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'6800","ondisk_log_start":"41'6800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14860771,"num_objects":3629,"num_object_clones":0,"num_object_copies":7258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3629,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14786,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-404}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17029","reported_seq":25444,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:32.893594+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:33:32.893594+0000","last_peered":"2024-09-17T15:33:32.893594+0000","last_clean":"2024-09-17T15:33:32.893594+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:33:32.893594+0000","last_undegraded":"2024-09-17T15:33:32.893594+0000","last_fullsized":"2024-09-17T15:33:32.893594+0000","mapping_epoch":46,"log_start":"41'7000","ondisk_log_start":"41'7000","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15183872,"num_objects":3707,"num_object_clones":0,"num_object_copies":7414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14791,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17259","reported_seq":17563,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:33.300140+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:33:33.300140+0000","last_peered":"2024-09-17T15:33:33.300140+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:33:33.300140+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'7200","ondisk_log_start":"41'7200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15183872,"num_objects":3707,"num_object_clones":0,"num_object_copies":7414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13807,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-350}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17064","reported_seq":18298,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:33.290224+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:33:33.290224+0000","last_peered":"2024-09-17T15:33:33.290224+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:33:33.290224+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'7000","ondisk_log_start":"41'7000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15187968,"num_objects":3708,"num_object_clones":0,"num_object_copies":7416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3619,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13351,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":541,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":89}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225014+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225014+0000","last_peered":"2024-09-17T15:33:11.225014+0000","last_clean":"2024-09-17T15:33:11.225014+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225014+0000","last_undegraded":"2024-09-17T15:33:11.225014+0000","last_fullsized":"2024-09-17T15:33:11.225014+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17156","reported_seq":17463,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:33.269642+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:33:33.269642+0000","last_peered":"2024-09-17T15:33:33.269642+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:33:33.269642+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'7100","ondisk_log_start":"41'7100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15278080,"num_objects":3730,"num_object_clones":0,"num_object_copies":7460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13943,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-250}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16942","reported_seq":25326,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:32.784491+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:33:32.784491+0000","last_peered":"2024-09-17T15:33:32.784491+0000","last_clean":"2024-09-17T15:33:32.784491+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:33:32.784491+0000","last_undegraded":"2024-09-17T15:33:32.784491+0000","last_fullsized":"2024-09-17T15:33:32.784491+0000","mapping_epoch":44,"log_start":"41'6900","ondisk_log_start":"41'6900","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15115206,"num_objects":3692,"num_object_clones":0,"num_object_copies":7384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13384,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":244294390,"num_objects":59652,"num_object_clones":0,"num_object_copies":119304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36030,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59652,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":233335,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13430,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":691544064,"data_stored":691409465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160700,"ondisk_log_size":160700,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051686,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":595628,"kb_used_data":252956,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93776212,"statfs":{"total":96636764160,"available":96026841088,"internally_reserved":0,"allocated":259026944,"data_stored":258725346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":350875308},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724631,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":540624,"kb_used_data":179904,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93831216,"statfs":{"total":96636764160,"available":96083165184,"internally_reserved":0,"allocated":184221696,"data_stored":183897956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":369356459},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,5,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247914,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":655920,"kb_used_data":245272,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93715920,"statfs":{"total":96636764160,"available":95965102080,"internally_reserved":0,"allocated":251158528,"data_stored":250856925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":420474536},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,3,5,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":250580992,"data_stored":250544862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":183136256,"data_stored":183077989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257826816,"data_stored":257786614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:33:37.114 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:33:37.114 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'16960", 'reported_seq': 17219, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:33.203297+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:33:33.203297+0000', 'last_peered': '2024-09-17T15:33:33.203297+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:33:33.203297+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "41'6900", 'ondisk_log_start': "41'6900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15106048, 'num_objects': 3688, 'num_object_clones': 0, 'num_object_copies': 7376, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3688, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16960, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -296}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:37.114 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:33:37.114 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'17148", 'reported_seq': 17370, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:32.964093+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:33:32.964093+0000', 'last_peered': '2024-09-17T15:33:32.964093+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:33:32.964093+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "41'7100", 'ondisk_log_start': "41'7100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15503384, 'num_objects': 3786, 'num_object_clones': 0, 'num_object_copies': 7572, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3786, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15305, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -339}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:33:37.114 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:33:37.114 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'17447", 'reported_seq': 17747, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:33.258653+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:33:33.258653+0000', 'last_peered': '2024-09-17T15:33:33.258653+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:33:33.258653+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "41'7400", 'ondisk_log_start': "41'7400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15581667, 'num_objects': 3805, 'num_object_clones': 0, 'num_object_copies': 7610, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3805, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13946, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -387}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:37.114 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:33:37.114 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'17121", 'reported_seq': 17439, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:33.214109+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:33:33.214109+0000', 'last_peered': '2024-09-17T15:33:33.214109+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:33:33.214109+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "41'7100", 'ondisk_log_start': "41'7100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14939078, 'num_objects': 3649, 'num_object_clones': 0, 'num_object_copies': 7298, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4089, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3649, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 13890, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 85, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -440}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:37.115 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:33:37.115 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'16816", 'reported_seq': 17090, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:33.270552+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:33:33.270552+0000', 'last_peered': '2024-09-17T15:33:33.270552+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:33:33.270552+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "41'6800", 'ondisk_log_start': "41'6800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15261696, 'num_objects': 3726, 'num_object_clones': 0, 'num_object_copies': 7452, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3726, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13412, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -217}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:37.115 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:33:37.115 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'16891", 'reported_seq': 17109, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:32.956259+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:33:32.956259+0000', 'last_peered': '2024-09-17T15:33:32.956259+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:33:32.956259+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "41'6800", 'ondisk_log_start': "41'6800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14860771, 'num_objects': 3629, 'num_object_clones': 0, 'num_object_copies': 7258, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4034, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3629, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14786, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -404}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:33:37.115 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:33:37.115 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'17259", 'reported_seq': 17563, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:33.300140+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:33:33.300140+0000', 'last_peered': '2024-09-17T15:33:33.300140+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:33:33.300140+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "41'7200", 'ondisk_log_start': "41'7200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15183872, 'num_objects': 3707, 'num_object_clones': 0, 'num_object_copies': 7414, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3707, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13807, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -350}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:37.115 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:33:37.115 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'17064", 'reported_seq': 18298, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:33.290224+0000', 'last_change': '2024-09-17T15:33:09.763500+0000', 'last_active': '2024-09-17T15:33:33.290224+0000', 'last_peered': '2024-09-17T15:33:33.290224+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:33:33.290224+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "41'7000", 'ondisk_log_start': "41'7000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15187968, 'num_objects': 3708, 'num_object_clones': 0, 'num_object_copies': 7416, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3619, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3708, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13351, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 541, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 89}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:37.115 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:33:37.115 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'17156", 'reported_seq': 17463, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:33.269642+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:33:33.269642+0000', 'last_peered': '2024-09-17T15:33:33.269642+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:33:33.269642+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "41'7100", 'ondisk_log_start': "41'7100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15278080, 'num_objects': 3730, 'num_object_clones': 0, 'num_object_copies': 7460, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3730, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13943, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -250}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:37.116 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:33:37.807 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:33:37.807 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36030/119308 objects degraded (30.199%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":554,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":59654,"data_bytes":244294409,"bytes_used":1835184128,"bytes_avail":288075108352,"bytes_total":289910292480,"degraded_objects":36030,"degraded_total":119308,"degraded_ratio":0.30199148422570155,"recovering_objects_per_sec":16,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":197,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":402},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:33:37.835 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:33:37.836 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:33:38.401 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:33:38.401 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:33:38.427 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2024-09-17T15:33:37.758046+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":244294409,"num_objects":59654,"num_object_clones":0,"num_object_copies":119308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36030,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59654,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":233337,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13431,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160703,"ondisk_log_size":160703,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1792172,"kb_used_data":678132,"kb_used_omap":13,"kb_used_meta":1113970,"kb_avail":281323348,"statfs":{"total":289910292480,"available":288075108352,"internally_reserved":0,"allocated":694407168,"data_stored":693480227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1140706303},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,4,9,9,8,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19775488,"num_objects":-4828,"num_object_clones":0,"num_object_copies":-9656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-197,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4828,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004499"},"pg_stats":[{"pgid":"2.f","version":"47'16960","reported_seq":17219,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:33.203297+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:33:33.203297+0000","last_peered":"2024-09-17T15:33:33.203297+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:33:33.203297+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'6900","ondisk_log_start":"41'6900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15106048,"num_objects":3688,"num_object_clones":0,"num_object_copies":7376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-296}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17110","reported_seq":17222,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:32.957683+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:33:32.957683+0000","last_peered":"2024-09-17T15:33:32.957683+0000","last_clean":"2024-09-17T15:33:32.957683+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:33:32.957683+0000","last_undegraded":"2024-09-17T15:33:32.957683+0000","last_fullsized":"2024-09-17T15:33:32.957683+0000","mapping_epoch":33,"log_start":"41'7100","ondisk_log_start":"41'7100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15655878,"num_objects":3824,"num_object_clones":0,"num_object_copies":7648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3824,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15467,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17148","reported_seq":17370,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:32.964093+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:33:32.964093+0000","last_peered":"2024-09-17T15:33:32.964093+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:33:32.964093+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'7100","ondisk_log_start":"41'7100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15503384,"num_objects":3786,"num_object_clones":0,"num_object_copies":7572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3786,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15305,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-339}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17214","reported_seq":17293,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:32.984827+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:33:32.984827+0000","last_peered":"2024-09-17T15:33:32.984827+0000","last_clean":"2024-09-17T15:33:32.984827+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:33:32.984827+0000","last_undegraded":"2024-09-17T15:33:32.984827+0000","last_fullsized":"2024-09-17T15:33:32.984827+0000","mapping_epoch":33,"log_start":"41'7200","ondisk_log_start":"41'7200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15384576,"num_objects":3756,"num_object_clones":0,"num_object_copies":7512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17214,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16966","reported_seq":17105,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:33.283591+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:33:33.283591+0000","last_peered":"2024-09-17T15:33:33.283591+0000","last_clean":"2024-09-17T15:33:33.283591+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:33:33.283591+0000","last_undegraded":"2024-09-17T15:33:33.283591+0000","last_fullsized":"2024-09-17T15:33:33.283591+0000","mapping_epoch":32,"log_start":"41'6900","ondisk_log_start":"41'6900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15102435,"num_objects":3688,"num_object_clones":0,"num_object_copies":7376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3688,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13948,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17447","reported_seq":17747,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:33.258653+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:33:33.258653+0000","last_peered":"2024-09-17T15:33:33.258653+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:33:33.258653+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'7400","ondisk_log_start":"41'7400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15581667,"num_objects":3805,"num_object_clones":0,"num_object_copies":7610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3805,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13946,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-387},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17121","reported_seq":17439,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:33.214109+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:33:33.214109+0000","last_peered":"2024-09-17T15:33:33.214109+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:33:33.214109+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'7100","ondisk_log_start":"41'7100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14939078,"num_objects":3649,"num_object_clones":0,"num_object_copies":7298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3649,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13890,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-440},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17125","reported_seq":17262,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:33.209105+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:33:33.209105+0000","last_peered":"2024-09-17T15:33:33.209105+0000","last_clean":"2024-09-17T15:33:33.209105+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:33:33.209105+0000","last_undegraded":"2024-09-17T15:33:33.209105+0000","last_fullsized":"2024-09-17T15:33:33.209105+0000","mapping_epoch":32,"log_start":"41'7100","ondisk_log_start":"41'7100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15527936,"num_objects":3791,"num_object_clones":0,"num_object_copies":7582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14079,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223906+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.223906+0000","last_peered":"2024-09-17T15:33:11.223906+0000","last_clean":"2024-09-17T15:33:11.223906+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.223906+0000","last_undegraded":"2024-09-17T15:33:11.223906+0000","last_fullsized":"2024-09-17T15:33:11.223906+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17052","reported_seq":25344,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:32.711474+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:33:32.711474+0000","last_peered":"2024-09-17T15:33:32.711474+0000","last_clean":"2024-09-17T15:33:32.711474+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:33:32.711474+0000","last_undegraded":"2024-09-17T15:33:32.711474+0000","last_fullsized":"2024-09-17T15:33:32.711474+0000","mapping_epoch":42,"log_start":"41'7000","ondisk_log_start":"41'7000","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15421923,"num_objects":3766,"num_object_clones":0,"num_object_copies":7532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3766,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15052,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225444+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225444+0000","last_peered":"2024-09-17T15:33:11.225444+0000","last_clean":"2024-09-17T15:33:11.225444+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225444+0000","last_undegraded":"2024-09-17T15:33:11.225444+0000","last_fullsized":"2024-09-17T15:33:11.225444+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16816","reported_seq":17090,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:33.270552+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:33:33.270552+0000","last_peered":"2024-09-17T15:33:33.270552+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:33:33.270552+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'6800","ondisk_log_start":"41'6800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15261696,"num_objects":3726,"num_object_clones":0,"num_object_copies":7452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13412,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-217}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16891","reported_seq":17109,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:32.956259+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:33:32.956259+0000","last_peered":"2024-09-17T15:33:32.956259+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:33:32.956259+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'6800","ondisk_log_start":"41'6800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14860771,"num_objects":3629,"num_object_clones":0,"num_object_copies":7258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3629,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14786,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-404}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17029","reported_seq":25444,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:32.893594+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:33:32.893594+0000","last_peered":"2024-09-17T15:33:32.893594+0000","last_clean":"2024-09-17T15:33:32.893594+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:33:32.893594+0000","last_undegraded":"2024-09-17T15:33:32.893594+0000","last_fullsized":"2024-09-17T15:33:32.893594+0000","mapping_epoch":46,"log_start":"41'7000","ondisk_log_start":"41'7000","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15183872,"num_objects":3707,"num_object_clones":0,"num_object_copies":7414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14791,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17259","reported_seq":17563,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:33.300140+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:33:33.300140+0000","last_peered":"2024-09-17T15:33:33.300140+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:33:33.300140+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'7200","ondisk_log_start":"41'7200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15183872,"num_objects":3707,"num_object_clones":0,"num_object_copies":7414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13807,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-350}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17064","reported_seq":18298,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:33.290224+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:33:33.290224+0000","last_peered":"2024-09-17T15:33:33.290224+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:33:33.290224+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'7000","ondisk_log_start":"41'7000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15187968,"num_objects":3708,"num_object_clones":0,"num_object_copies":7416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3619,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13351,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":541,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":89}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225014+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225014+0000","last_peered":"2024-09-17T15:33:11.225014+0000","last_clean":"2024-09-17T15:33:11.225014+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225014+0000","last_undegraded":"2024-09-17T15:33:11.225014+0000","last_fullsized":"2024-09-17T15:33:11.225014+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17156","reported_seq":17463,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:33.269642+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:33:33.269642+0000","last_peered":"2024-09-17T15:33:33.269642+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:33:33.269642+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'7100","ondisk_log_start":"41'7100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15278080,"num_objects":3730,"num_object_clones":0,"num_object_copies":7460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13943,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-250}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16942","reported_seq":25326,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:32.784491+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:33:32.784491+0000","last_peered":"2024-09-17T15:33:32.784491+0000","last_clean":"2024-09-17T15:33:32.784491+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:33:32.784491+0000","last_undegraded":"2024-09-17T15:33:32.784491+0000","last_fullsized":"2024-09-17T15:33:32.784491+0000","mapping_epoch":44,"log_start":"41'6900","ondisk_log_start":"41'6900","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15115206,"num_objects":3692,"num_object_clones":0,"num_object_copies":7384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13384,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":244294390,"num_objects":59652,"num_object_clones":0,"num_object_copies":119304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36030,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59652,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":233335,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13430,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":691544064,"data_stored":691409465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160700,"ondisk_log_size":160700,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051686,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":595628,"kb_used_data":252956,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93776212,"statfs":{"total":96636764160,"available":96026841088,"internally_reserved":0,"allocated":259026944,"data_stored":258725346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":350875308},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724631,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":540624,"kb_used_data":179904,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93831216,"statfs":{"total":96636764160,"available":96083165184,"internally_reserved":0,"allocated":184221696,"data_stored":183897956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":369356459},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,5,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247914,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":655920,"kb_used_data":245272,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93715920,"statfs":{"total":96636764160,"available":95965102080,"internally_reserved":0,"allocated":251158528,"data_stored":250856925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":420474536},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,3,5,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":250580992,"data_stored":250544862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":183136256,"data_stored":183077989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257826816,"data_stored":257786614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:33:41.429 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:33:41.988 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:33:41.988 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:33:42.013 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":319,"stamp":"2024-09-17T15:33:41.759492+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":234328841,"num_objects":57221,"num_object_clones":0,"num_object_copies":114442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35924,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57221,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":235770,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13537,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1802964,"kb_used_data":658396,"kb_used_omap":13,"kb_used_meta":1144498,"kb_avail":281312556,"statfs":{"total":289910292480,"available":288064057344,"internally_reserved":0,"allocated":674197504,"data_stored":673270563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1171966975},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,6,6,4,10,8,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19775488,"num_objects":-4828,"num_object_clones":0,"num_object_copies":-9656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-220,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4828,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":220,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004463"},"pg_stats":[{"pgid":"2.f","version":"47'17103","reported_seq":17361,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:38.312254+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:33:38.312254+0000","last_peered":"2024-09-17T15:33:38.312254+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:33:38.312254+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'7100","ondisk_log_start":"41'7100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14520320,"num_objects":3545,"num_object_clones":0,"num_object_copies":7090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17103,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-439}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17259","reported_seq":17371,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:38.016666+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:33:38.016666+0000","last_peered":"2024-09-17T15:33:38.016666+0000","last_clean":"2024-09-17T15:33:38.016666+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:33:38.016666+0000","last_undegraded":"2024-09-17T15:33:38.016666+0000","last_fullsized":"2024-09-17T15:33:38.016666+0000","mapping_epoch":33,"log_start":"41'7200","ondisk_log_start":"41'7200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15045574,"num_objects":3675,"num_object_clones":0,"num_object_copies":7350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3675,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15616,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17283","reported_seq":17505,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:37.981159+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:33:37.981159+0000","last_peered":"2024-09-17T15:33:37.981159+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:33:37.981159+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'7200","ondisk_log_start":"41'7200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14950424,"num_objects":3651,"num_object_clones":0,"num_object_copies":7302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3651,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15440,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-474}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17378","reported_seq":17456,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:38.000715+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:33:38.000715+0000","last_peered":"2024-09-17T15:33:38.000715+0000","last_clean":"2024-09-17T15:33:38.000715+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:33:38.000715+0000","last_undegraded":"2024-09-17T15:33:38.000715+0000","last_fullsized":"2024-09-17T15:33:38.000715+0000","mapping_epoch":33,"log_start":"41'7300","ondisk_log_start":"41'7300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14712832,"num_objects":3592,"num_object_clones":0,"num_object_copies":7184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17378,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17109","reported_seq":17248,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:38.292628+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:33:38.292628+0000","last_peered":"2024-09-17T15:33:38.292628+0000","last_clean":"2024-09-17T15:33:38.292628+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:33:38.292628+0000","last_undegraded":"2024-09-17T15:33:38.292628+0000","last_fullsized":"2024-09-17T15:33:38.292628+0000","mapping_epoch":32,"log_start":"41'7100","ondisk_log_start":"41'7100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14516707,"num_objects":3545,"num_object_clones":0,"num_object_copies":7090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3545,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14091,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17622","reported_seq":17922,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:38.290094+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:33:38.290094+0000","last_peered":"2024-09-17T15:33:38.290094+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:33:38.290094+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'7600","ondisk_log_start":"41'7600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14864867,"num_objects":3630,"num_object_clones":0,"num_object_copies":7260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3630,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14121,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-562},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17281","reported_seq":17599,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:38.324073+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:33:38.324073+0000","last_peered":"2024-09-17T15:33:38.324073+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:33:38.324073+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'7200","ondisk_log_start":"41'7200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14283718,"num_objects":3489,"num_object_clones":0,"num_object_copies":6978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3489,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14050,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-600},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17289","reported_seq":17426,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:38.297628+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:33:38.297628+0000","last_peered":"2024-09-17T15:33:38.297628+0000","last_clean":"2024-09-17T15:33:38.297628+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:33:38.297628+0000","last_undegraded":"2024-09-17T15:33:38.297628+0000","last_fullsized":"2024-09-17T15:33:38.297628+0000","mapping_epoch":32,"log_start":"41'7200","ondisk_log_start":"41'7200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14856192,"num_objects":3627,"num_object_clones":0,"num_object_copies":7254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14243,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223906+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.223906+0000","last_peered":"2024-09-17T15:33:11.223906+0000","last_clean":"2024-09-17T15:33:11.223906+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.223906+0000","last_undegraded":"2024-09-17T15:33:11.223906+0000","last_fullsized":"2024-09-17T15:33:11.223906+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17190","reported_seq":25482,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:37.850713+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:33:37.850713+0000","last_peered":"2024-09-17T15:33:37.850713+0000","last_clean":"2024-09-17T15:33:37.850713+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:33:37.850713+0000","last_undegraded":"2024-09-17T15:33:37.850713+0000","last_fullsized":"2024-09-17T15:33:37.850713+0000","mapping_epoch":42,"log_start":"41'7100","ondisk_log_start":"41'7100","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14856675,"num_objects":3628,"num_object_clones":0,"num_object_copies":7256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3628,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15190,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225444+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225444+0000","last_peered":"2024-09-17T15:33:11.225444+0000","last_clean":"2024-09-17T15:33:11.225444+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225444+0000","last_undegraded":"2024-09-17T15:33:11.225444+0000","last_fullsized":"2024-09-17T15:33:11.225444+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16988","reported_seq":17261,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:38.342158+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:33:38.342158+0000","last_peered":"2024-09-17T15:33:38.342158+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:33:38.342158+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'6900","ondisk_log_start":"41'6900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14557184,"num_objects":3554,"num_object_clones":0,"num_object_copies":7108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3554,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13584,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-389}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17023","reported_seq":17241,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:37.949444+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:33:37.949444+0000","last_peered":"2024-09-17T15:33:37.949444+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:33:37.949444+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'7000","ondisk_log_start":"41'7000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14320099,"num_objects":3497,"num_object_clones":0,"num_object_copies":6994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3497,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14918,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-536}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17180","reported_seq":25595,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:38.010148+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:33:38.010148+0000","last_peered":"2024-09-17T15:33:38.010148+0000","last_clean":"2024-09-17T15:33:38.010148+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:33:38.010148+0000","last_undegraded":"2024-09-17T15:33:38.010148+0000","last_fullsized":"2024-09-17T15:33:38.010148+0000","mapping_epoch":46,"log_start":"41'7100","ondisk_log_start":"41'7100","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14565376,"num_objects":3556,"num_object_clones":0,"num_object_copies":7112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14942,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17406","reported_seq":17713,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:38.268665+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:33:38.268665+0000","last_peered":"2024-09-17T15:33:38.268665+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:33:38.268665+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'7400","ondisk_log_start":"41'7400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14581760,"num_objects":3560,"num_object_clones":0,"num_object_copies":7120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13954,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-497}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17202","reported_seq":18648,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:38.350668+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:33:38.350668+0000","last_peered":"2024-09-17T15:33:38.350668+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:33:38.350668+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'7200","ondisk_log_start":"41'7200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14622720,"num_objects":3570,"num_object_clones":0,"num_object_copies":7140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3513,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13489,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":647,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":57}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225014+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225014+0000","last_peered":"2024-09-17T15:33:11.225014+0000","last_clean":"2024-09-17T15:33:11.225014+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225014+0000","last_undegraded":"2024-09-17T15:33:11.225014+0000","last_fullsized":"2024-09-17T15:33:11.225014+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17328","reported_seq":17636,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:38.373434+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:33:38.373434+0000","last_peered":"2024-09-17T15:33:38.373434+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:33:38.373434+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'7300","ondisk_log_start":"41'7300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14573568,"num_objects":3558,"num_object_clones":0,"num_object_copies":7116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14115,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-422}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17092","reported_seq":25476,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:37.703788+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:33:37.703788+0000","last_peered":"2024-09-17T15:33:37.703788+0000","last_clean":"2024-09-17T15:33:37.703788+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:33:37.703788+0000","last_undegraded":"2024-09-17T15:33:37.703788+0000","last_fullsized":"2024-09-17T15:33:37.703788+0000","mapping_epoch":44,"log_start":"41'7000","ondisk_log_start":"41'7000","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14500806,"num_objects":3542,"num_object_clones":0,"num_object_copies":7084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13534,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":234328822,"num_objects":57219,"num_object_clones":0,"num_object_copies":114438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35924,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57219,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":235768,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13536,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":671477760,"data_stored":671343161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160833,"ondisk_log_size":160833,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051687,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":595088,"kb_used_data":248256,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93776752,"statfs":{"total":96636764160,"available":96027394048,"internally_reserved":0,"allocated":254214144,"data_stored":253912546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":355135148},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,0,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724632,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":555380,"kb_used_data":172452,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93816460,"statfs":{"total":96636764160,"available":96068055040,"internally_reserved":0,"allocated":176590848,"data_stored":176267108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":392097451},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,1,2,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247915,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":652496,"kb_used_data":237688,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93719344,"statfs":{"total":96636764160,"available":95968608256,"internally_reserved":0,"allocated":243392512,"data_stored":243090909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":424734376},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,1,3,5,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":242860032,"data_stored":242823902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":175640576,"data_stored":175582309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":252977152,"data_stored":252936950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:33:42.014 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:33:42.593 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:33:42.595 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:33:42.617 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":319,"stamp":"2024-09-17T15:33:41.759492+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":234328841,"num_objects":57221,"num_object_clones":0,"num_object_copies":114442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35924,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57221,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":235770,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13537,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1802964,"kb_used_data":658396,"kb_used_omap":13,"kb_used_meta":1144498,"kb_avail":281312556,"statfs":{"total":289910292480,"available":288064057344,"internally_reserved":0,"allocated":674197504,"data_stored":673270563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1171966975},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,6,6,4,10,8,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19775488,"num_objects":-4828,"num_object_clones":0,"num_object_copies":-9656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-220,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4828,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":220,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004463"},"pg_stats":[{"pgid":"2.f","version":"47'17103","reported_seq":17361,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:38.312254+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:33:38.312254+0000","last_peered":"2024-09-17T15:33:38.312254+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:33:38.312254+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'7100","ondisk_log_start":"41'7100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14520320,"num_objects":3545,"num_object_clones":0,"num_object_copies":7090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17103,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-439}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17259","reported_seq":17371,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:38.016666+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:33:38.016666+0000","last_peered":"2024-09-17T15:33:38.016666+0000","last_clean":"2024-09-17T15:33:38.016666+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:33:38.016666+0000","last_undegraded":"2024-09-17T15:33:38.016666+0000","last_fullsized":"2024-09-17T15:33:38.016666+0000","mapping_epoch":33,"log_start":"41'7200","ondisk_log_start":"41'7200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15045574,"num_objects":3675,"num_object_clones":0,"num_object_copies":7350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3675,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15616,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17283","reported_seq":17505,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:37.981159+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:33:37.981159+0000","last_peered":"2024-09-17T15:33:37.981159+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:33:37.981159+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'7200","ondisk_log_start":"41'7200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14950424,"num_objects":3651,"num_object_clones":0,"num_object_copies":7302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3651,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15440,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-474}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17378","reported_seq":17456,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:38.000715+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:33:38.000715+0000","last_peered":"2024-09-17T15:33:38.000715+0000","last_clean":"2024-09-17T15:33:38.000715+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:33:38.000715+0000","last_undegraded":"2024-09-17T15:33:38.000715+0000","last_fullsized":"2024-09-17T15:33:38.000715+0000","mapping_epoch":33,"log_start":"41'7300","ondisk_log_start":"41'7300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14712832,"num_objects":3592,"num_object_clones":0,"num_object_copies":7184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17378,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17109","reported_seq":17248,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:38.292628+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:33:38.292628+0000","last_peered":"2024-09-17T15:33:38.292628+0000","last_clean":"2024-09-17T15:33:38.292628+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:33:38.292628+0000","last_undegraded":"2024-09-17T15:33:38.292628+0000","last_fullsized":"2024-09-17T15:33:38.292628+0000","mapping_epoch":32,"log_start":"41'7100","ondisk_log_start":"41'7100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14516707,"num_objects":3545,"num_object_clones":0,"num_object_copies":7090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3545,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14091,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17622","reported_seq":17922,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:38.290094+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:33:38.290094+0000","last_peered":"2024-09-17T15:33:38.290094+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:33:38.290094+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'7600","ondisk_log_start":"41'7600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14864867,"num_objects":3630,"num_object_clones":0,"num_object_copies":7260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3630,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14121,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-562},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17281","reported_seq":17599,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:38.324073+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:33:38.324073+0000","last_peered":"2024-09-17T15:33:38.324073+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:33:38.324073+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'7200","ondisk_log_start":"41'7200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14283718,"num_objects":3489,"num_object_clones":0,"num_object_copies":6978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3489,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14050,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-600},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17289","reported_seq":17426,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:38.297628+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:33:38.297628+0000","last_peered":"2024-09-17T15:33:38.297628+0000","last_clean":"2024-09-17T15:33:38.297628+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:33:38.297628+0000","last_undegraded":"2024-09-17T15:33:38.297628+0000","last_fullsized":"2024-09-17T15:33:38.297628+0000","mapping_epoch":32,"log_start":"41'7200","ondisk_log_start":"41'7200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14856192,"num_objects":3627,"num_object_clones":0,"num_object_copies":7254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14243,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223906+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.223906+0000","last_peered":"2024-09-17T15:33:11.223906+0000","last_clean":"2024-09-17T15:33:11.223906+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.223906+0000","last_undegraded":"2024-09-17T15:33:11.223906+0000","last_fullsized":"2024-09-17T15:33:11.223906+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17190","reported_seq":25482,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:37.850713+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:33:37.850713+0000","last_peered":"2024-09-17T15:33:37.850713+0000","last_clean":"2024-09-17T15:33:37.850713+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:33:37.850713+0000","last_undegraded":"2024-09-17T15:33:37.850713+0000","last_fullsized":"2024-09-17T15:33:37.850713+0000","mapping_epoch":42,"log_start":"41'7100","ondisk_log_start":"41'7100","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14856675,"num_objects":3628,"num_object_clones":0,"num_object_copies":7256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3628,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15190,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225444+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225444+0000","last_peered":"2024-09-17T15:33:11.225444+0000","last_clean":"2024-09-17T15:33:11.225444+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225444+0000","last_undegraded":"2024-09-17T15:33:11.225444+0000","last_fullsized":"2024-09-17T15:33:11.225444+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16988","reported_seq":17261,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:38.342158+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:33:38.342158+0000","last_peered":"2024-09-17T15:33:38.342158+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:33:38.342158+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'6900","ondisk_log_start":"41'6900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14557184,"num_objects":3554,"num_object_clones":0,"num_object_copies":7108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3554,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13584,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-389}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17023","reported_seq":17241,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:37.949444+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:33:37.949444+0000","last_peered":"2024-09-17T15:33:37.949444+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:33:37.949444+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'7000","ondisk_log_start":"41'7000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14320099,"num_objects":3497,"num_object_clones":0,"num_object_copies":6994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3497,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14918,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-536}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17180","reported_seq":25595,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:38.010148+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:33:38.010148+0000","last_peered":"2024-09-17T15:33:38.010148+0000","last_clean":"2024-09-17T15:33:38.010148+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:33:38.010148+0000","last_undegraded":"2024-09-17T15:33:38.010148+0000","last_fullsized":"2024-09-17T15:33:38.010148+0000","mapping_epoch":46,"log_start":"41'7100","ondisk_log_start":"41'7100","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14565376,"num_objects":3556,"num_object_clones":0,"num_object_copies":7112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14942,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17406","reported_seq":17713,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:38.268665+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:33:38.268665+0000","last_peered":"2024-09-17T15:33:38.268665+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:33:38.268665+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'7400","ondisk_log_start":"41'7400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14581760,"num_objects":3560,"num_object_clones":0,"num_object_copies":7120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13954,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-497}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17202","reported_seq":18648,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:38.350668+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:33:38.350668+0000","last_peered":"2024-09-17T15:33:38.350668+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:33:38.350668+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'7200","ondisk_log_start":"41'7200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14622720,"num_objects":3570,"num_object_clones":0,"num_object_copies":7140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3513,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13489,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":647,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":57}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225014+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225014+0000","last_peered":"2024-09-17T15:33:11.225014+0000","last_clean":"2024-09-17T15:33:11.225014+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225014+0000","last_undegraded":"2024-09-17T15:33:11.225014+0000","last_fullsized":"2024-09-17T15:33:11.225014+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17328","reported_seq":17636,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:38.373434+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:33:38.373434+0000","last_peered":"2024-09-17T15:33:38.373434+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:33:38.373434+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'7300","ondisk_log_start":"41'7300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14573568,"num_objects":3558,"num_object_clones":0,"num_object_copies":7116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14115,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-422}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17092","reported_seq":25476,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:37.703788+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:33:37.703788+0000","last_peered":"2024-09-17T15:33:37.703788+0000","last_clean":"2024-09-17T15:33:37.703788+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:33:37.703788+0000","last_undegraded":"2024-09-17T15:33:37.703788+0000","last_fullsized":"2024-09-17T15:33:37.703788+0000","mapping_epoch":44,"log_start":"41'7000","ondisk_log_start":"41'7000","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14500806,"num_objects":3542,"num_object_clones":0,"num_object_copies":7084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13534,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":234328822,"num_objects":57219,"num_object_clones":0,"num_object_copies":114438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35924,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57219,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":235768,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13536,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":671477760,"data_stored":671343161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160833,"ondisk_log_size":160833,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051687,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":595088,"kb_used_data":248256,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93776752,"statfs":{"total":96636764160,"available":96027394048,"internally_reserved":0,"allocated":254214144,"data_stored":253912546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":355135148},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,0,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724632,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":555380,"kb_used_data":172452,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93816460,"statfs":{"total":96636764160,"available":96068055040,"internally_reserved":0,"allocated":176590848,"data_stored":176267108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":392097451},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,1,2,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247915,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":652496,"kb_used_data":237688,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93719344,"statfs":{"total":96636764160,"available":95968608256,"internally_reserved":0,"allocated":243392512,"data_stored":243090909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":424734376},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,1,3,5,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":242860032,"data_stored":242823902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":175640576,"data_stored":175582309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":252977152,"data_stored":252936950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:33:42.619 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:33:42.619 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'17103", 'reported_seq': 17361, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:38.312254+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:33:38.312254+0000', 'last_peered': '2024-09-17T15:33:38.312254+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:33:38.312254+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "41'7100", 'ondisk_log_start': "41'7100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14520320, 'num_objects': 3545, 'num_object_clones': 0, 'num_object_copies': 7090, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3545, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17103, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -439}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:42.619 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:33:42.619 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'17283", 'reported_seq': 17505, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:37.981159+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:33:37.981159+0000', 'last_peered': '2024-09-17T15:33:37.981159+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:33:37.981159+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "41'7200", 'ondisk_log_start': "41'7200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14950424, 'num_objects': 3651, 'num_object_clones': 0, 'num_object_copies': 7302, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3651, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15440, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -474}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:33:42.619 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:33:42.619 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'17622", 'reported_seq': 17922, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:38.290094+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:33:38.290094+0000', 'last_peered': '2024-09-17T15:33:38.290094+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:33:38.290094+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "41'7600", 'ondisk_log_start': "41'7600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14864867, 'num_objects': 3630, 'num_object_clones': 0, 'num_object_copies': 7260, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3630, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14121, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -562}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:42.619 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:33:42.619 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'17281", 'reported_seq': 17599, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:38.324073+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:33:38.324073+0000', 'last_peered': '2024-09-17T15:33:38.324073+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:33:38.324073+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "41'7200", 'ondisk_log_start': "41'7200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14283718, 'num_objects': 3489, 'num_object_clones': 0, 'num_object_copies': 6978, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4089, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3489, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 14050, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 85, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -600}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:42.619 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:33:42.619 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'16988", 'reported_seq': 17261, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:38.342158+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:33:38.342158+0000', 'last_peered': '2024-09-17T15:33:38.342158+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:33:38.342158+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "41'6900", 'ondisk_log_start': "41'6900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14557184, 'num_objects': 3554, 'num_object_clones': 0, 'num_object_copies': 7108, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3554, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13584, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -389}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:42.620 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:33:42.620 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'17023", 'reported_seq': 17241, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:37.949444+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:33:37.949444+0000', 'last_peered': '2024-09-17T15:33:37.949444+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:33:37.949444+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "41'7000", 'ondisk_log_start': "41'7000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14320099, 'num_objects': 3497, 'num_object_clones': 0, 'num_object_copies': 6994, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4034, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3497, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14918, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -536}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:33:42.620 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:33:42.620 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'17406", 'reported_seq': 17713, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:38.268665+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:33:38.268665+0000', 'last_peered': '2024-09-17T15:33:38.268665+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:33:38.268665+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "41'7400", 'ondisk_log_start': "41'7400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14581760, 'num_objects': 3560, 'num_object_clones': 0, 'num_object_copies': 7120, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3560, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13954, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -497}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:42.620 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:33:42.620 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'17202", 'reported_seq': 18648, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:38.350668+0000', 'last_change': '2024-09-17T15:33:09.763500+0000', 'last_active': '2024-09-17T15:33:38.350668+0000', 'last_peered': '2024-09-17T15:33:38.350668+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:33:38.350668+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "41'7200", 'ondisk_log_start': "41'7200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14622720, 'num_objects': 3570, 'num_object_clones': 0, 'num_object_copies': 7140, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3513, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3570, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13489, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 647, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 57}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:42.620 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:33:42.620 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'17328", 'reported_seq': 17636, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:38.373434+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:33:38.373434+0000', 'last_peered': '2024-09-17T15:33:38.373434+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:33:38.373434+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "41'7300", 'ondisk_log_start': "41'7300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14573568, 'num_objects': 3558, 'num_object_clones': 0, 'num_object_copies': 7116, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3558, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14115, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -422}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:42.621 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:33:43.331 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:33:43.331 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35924/114442 objects degraded (31.391%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":560,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":57221,"data_bytes":234328841,"bytes_used":1846235136,"bytes_avail":288064057344,"bytes_total":289910292480,"degraded_objects":35924,"degraded_total":114442,"degraded_ratio":0.3139057339088796,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":220,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":402},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:33:43.360 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:33:43.360 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:33:43.931 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:33:43.933 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:33:43.955 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":320,"stamp":"2024-09-17T15:33:43.760266+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":224215817,"num_objects":54752,"num_object_clones":0,"num_object_copies":109504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35821,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54752,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":238239,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13640,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160805,"ondisk_log_size":160805,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1787648,"kb_used_data":638924,"kb_used_omap":13,"kb_used_meta":1148658,"kb_avail":281327872,"statfs":{"total":289910292480,"available":288079740928,"internally_reserved":0,"allocated":654258176,"data_stored":653331235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1176226815},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,6,6,11,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29888512,"num_objects":-7297,"num_object_clones":0,"num_object_copies":-14594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7297,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004384"},"pg_stats":[{"pgid":"2.f","version":"47'17282","reported_seq":17541,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:43.403570+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:33:43.403570+0000","last_peered":"2024-09-17T15:33:43.403570+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:33:43.403570+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'7200","ondisk_log_start":"41'7200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13787136,"num_objects":3366,"num_object_clones":0,"num_object_copies":6732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3366,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17282,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-618}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17421","reported_seq":17533,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:43.017911+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:33:43.017911+0000","last_peered":"2024-09-17T15:33:43.017911+0000","last_clean":"2024-09-17T15:33:43.017911+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:33:43.017911+0000","last_undegraded":"2024-09-17T15:33:43.017911+0000","last_fullsized":"2024-09-17T15:33:43.017911+0000","mapping_epoch":33,"log_start":"41'7400","ondisk_log_start":"41'7400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14382022,"num_objects":3513,"num_object_clones":0,"num_object_copies":7026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3513,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15778,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17433","reported_seq":17655,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:43.029696+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:33:43.029696+0000","last_peered":"2024-09-17T15:33:43.029696+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:33:43.029696+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'7400","ondisk_log_start":"41'7400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14336024,"num_objects":3501,"num_object_clones":0,"num_object_copies":7002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3501,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15590,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-624}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17518","reported_seq":17597,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:42.898148+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:33:42.898148+0000","last_peered":"2024-09-17T15:33:42.898148+0000","last_clean":"2024-09-17T15:33:42.898148+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:33:42.898148+0000","last_undegraded":"2024-09-17T15:33:42.898148+0000","last_fullsized":"2024-09-17T15:33:42.898148+0000","mapping_epoch":33,"log_start":"41'7500","ondisk_log_start":"41'7500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14139392,"num_objects":3452,"num_object_clones":0,"num_object_copies":6904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17518,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17247","reported_seq":17386,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:43.422634+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:33:43.422634+0000","last_peered":"2024-09-17T15:33:43.422634+0000","last_clean":"2024-09-17T15:33:43.422634+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:33:43.422634+0000","last_undegraded":"2024-09-17T15:33:43.422634+0000","last_fullsized":"2024-09-17T15:33:43.422634+0000","mapping_epoch":32,"log_start":"41'7200","ondisk_log_start":"41'7200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13951459,"num_objects":3407,"num_object_clones":0,"num_object_copies":6814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3407,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14229,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17767","reported_seq":18067,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:43.302534+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:33:43.302534+0000","last_peered":"2024-09-17T15:33:43.302534+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:33:43.302534+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'7700","ondisk_log_start":"41'7700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14270947,"num_objects":3485,"num_object_clones":0,"num_object_copies":6970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3485,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14266,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-707},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17441","reported_seq":17759,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:43.316077+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:33:43.316077+0000","last_peered":"2024-09-17T15:33:43.316077+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:33:43.316077+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'7400","ondisk_log_start":"41'7400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13628358,"num_objects":3329,"num_object_clones":0,"num_object_copies":6658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3329,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14210,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-760},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17449","reported_seq":17586,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:43.386994+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:33:43.386994+0000","last_peered":"2024-09-17T15:33:43.386994+0000","last_clean":"2024-09-17T15:33:43.386994+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:33:43.386994+0000","last_undegraded":"2024-09-17T15:33:43.386994+0000","last_fullsized":"2024-09-17T15:33:43.386994+0000","mapping_epoch":32,"log_start":"41'7400","ondisk_log_start":"41'7400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14200832,"num_objects":3467,"num_object_clones":0,"num_object_copies":6934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14403,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223906+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.223906+0000","last_peered":"2024-09-17T15:33:11.223906+0000","last_clean":"2024-09-17T15:33:11.223906+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.223906+0000","last_undegraded":"2024-09-17T15:33:11.223906+0000","last_fullsized":"2024-09-17T15:33:11.223906+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17336","reported_seq":25628,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:42.883427+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:33:42.883427+0000","last_peered":"2024-09-17T15:33:42.883427+0000","last_clean":"2024-09-17T15:33:42.883427+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:33:42.883427+0000","last_undegraded":"2024-09-17T15:33:42.883427+0000","last_fullsized":"2024-09-17T15:33:42.883427+0000","mapping_epoch":42,"log_start":"41'7300","ondisk_log_start":"41'7300","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14258659,"num_objects":3482,"num_object_clones":0,"num_object_copies":6964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3482,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15336,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17139","reported_seq":17413,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:43.243081+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:33:43.243081+0000","last_peered":"2024-09-17T15:33:43.243081+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:33:43.243081+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'7100","ondisk_log_start":"41'7100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13938688,"num_objects":3403,"num_object_clones":0,"num_object_copies":6806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3403,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13735,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-540}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17187","reported_seq":17403,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:43.040616+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:33:43.040616+0000","last_peered":"2024-09-17T15:33:43.040616+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:33:43.040616+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'7100","ondisk_log_start":"41'7100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13648355,"num_objects":3333,"num_object_clones":0,"num_object_copies":6666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3333,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15082,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-700}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17335","reported_seq":25750,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:43.032917+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:33:43.032917+0000","last_peered":"2024-09-17T15:33:43.032917+0000","last_clean":"2024-09-17T15:33:43.032917+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:33:43.032917+0000","last_undegraded":"2024-09-17T15:33:43.032917+0000","last_fullsized":"2024-09-17T15:33:43.032917+0000","mapping_epoch":46,"log_start":"41'7300","ondisk_log_start":"41'7300","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13930496,"num_objects":3401,"num_object_clones":0,"num_object_copies":6802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15097,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17571","reported_seq":17877,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:43.407244+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:33:43.407244+0000","last_peered":"2024-09-17T15:33:43.407244+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:33:43.407244+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'7500","ondisk_log_start":"41'7500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13905920,"num_objects":3395,"num_object_clones":0,"num_object_copies":6790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14119,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-662}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17361","reported_seq":19013,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:43.429810+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:33:43.429810+0000","last_peered":"2024-09-17T15:33:43.429810+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:33:43.429810+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'7300","ondisk_log_start":"41'7300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13971456,"num_objects":3411,"num_object_clones":0,"num_object_copies":6822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3410,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13648,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":750,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17469","reported_seq":17777,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:43.347576+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:33:43.347576+0000","last_peered":"2024-09-17T15:33:43.347576+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:33:43.347576+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'7400","ondisk_log_start":"41'7400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13996032,"num_objects":3417,"num_object_clones":0,"num_object_copies":6834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3417,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14256,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-563}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17246","reported_seq":25630,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:42.902235+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:33:42.902235+0000","last_peered":"2024-09-17T15:33:42.902235+0000","last_clean":"2024-09-17T15:33:42.902235+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:33:42.902235+0000","last_undegraded":"2024-09-17T15:33:42.902235+0000","last_fullsized":"2024-09-17T15:33:42.902235+0000","mapping_epoch":44,"log_start":"41'7200","ondisk_log_start":"41'7200","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13870022,"num_objects":3388,"num_object_clones":0,"num_object_copies":6776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13688,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":224215798,"num_objects":54750,"num_object_clones":0,"num_object_copies":109500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35821,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54750,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":238237,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13639,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":651354112,"data_stored":651219513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160802,"ondisk_log_size":160802,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051688,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594500,"kb_used_data":243508,"kb_used_omap":4,"kb_used_meta":350971,"kb_avail":93777340,"statfs":{"total":96636764160,"available":96027996160,"internally_reserved":0,"allocated":249352192,"data_stored":249050594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":359394988},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,3,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724633,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539720,"kb_used_data":165112,"kb_used_omap":4,"kb_used_meta":374587,"kb_avail":93832120,"statfs":{"total":96636764160,"available":96084090880,"internally_reserved":0,"allocated":169074688,"data_stored":168750948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":383577771},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,3,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247916,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":653428,"kb_used_data":230304,"kb_used_omap":4,"kb_used_meta":423099,"kb_avail":93718412,"statfs":{"total":96636764160,"available":95967653888,"internally_reserved":0,"allocated":235831296,"data_stored":235529693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":433254056},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,0,7,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":235261952,"data_stored":235225822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":168034304,"data_stored":167976037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":248057856,"data_stored":248017654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:33:46.957 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:33:47.545 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:33:47.545 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:33:47.567 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2024-09-17T15:33:45.761027+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":224215817,"num_objects":54752,"num_object_clones":0,"num_object_copies":109504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35821,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54752,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":238239,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13640,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160805,"ondisk_log_size":160805,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1787648,"kb_used_data":638924,"kb_used_omap":13,"kb_used_meta":1148658,"kb_avail":281327872,"statfs":{"total":289910292480,"available":288079740928,"internally_reserved":0,"allocated":654258176,"data_stored":653331235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1176226815},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,6,6,11,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20078592,"num_objects":-4902,"num_object_clones":0,"num_object_copies":-9804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-209,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4902,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004492"},"pg_stats":[{"pgid":"2.f","version":"47'17282","reported_seq":17541,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:43.403570+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:33:43.403570+0000","last_peered":"2024-09-17T15:33:43.403570+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:33:43.403570+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'7200","ondisk_log_start":"41'7200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13787136,"num_objects":3366,"num_object_clones":0,"num_object_copies":6732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3366,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17282,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-618}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17421","reported_seq":17533,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:43.017911+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:33:43.017911+0000","last_peered":"2024-09-17T15:33:43.017911+0000","last_clean":"2024-09-17T15:33:43.017911+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:33:43.017911+0000","last_undegraded":"2024-09-17T15:33:43.017911+0000","last_fullsized":"2024-09-17T15:33:43.017911+0000","mapping_epoch":33,"log_start":"41'7400","ondisk_log_start":"41'7400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14382022,"num_objects":3513,"num_object_clones":0,"num_object_copies":7026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3513,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15778,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17433","reported_seq":17655,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:43.029696+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:33:43.029696+0000","last_peered":"2024-09-17T15:33:43.029696+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:33:43.029696+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'7400","ondisk_log_start":"41'7400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14336024,"num_objects":3501,"num_object_clones":0,"num_object_copies":7002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3501,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15590,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-624}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17518","reported_seq":17597,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:42.898148+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:33:42.898148+0000","last_peered":"2024-09-17T15:33:42.898148+0000","last_clean":"2024-09-17T15:33:42.898148+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:33:42.898148+0000","last_undegraded":"2024-09-17T15:33:42.898148+0000","last_fullsized":"2024-09-17T15:33:42.898148+0000","mapping_epoch":33,"log_start":"41'7500","ondisk_log_start":"41'7500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14139392,"num_objects":3452,"num_object_clones":0,"num_object_copies":6904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17518,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17247","reported_seq":17386,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:43.422634+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:33:43.422634+0000","last_peered":"2024-09-17T15:33:43.422634+0000","last_clean":"2024-09-17T15:33:43.422634+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:33:43.422634+0000","last_undegraded":"2024-09-17T15:33:43.422634+0000","last_fullsized":"2024-09-17T15:33:43.422634+0000","mapping_epoch":32,"log_start":"41'7200","ondisk_log_start":"41'7200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13951459,"num_objects":3407,"num_object_clones":0,"num_object_copies":6814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3407,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14229,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17767","reported_seq":18067,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:43.302534+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:33:43.302534+0000","last_peered":"2024-09-17T15:33:43.302534+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:33:43.302534+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'7700","ondisk_log_start":"41'7700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14270947,"num_objects":3485,"num_object_clones":0,"num_object_copies":6970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3485,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14266,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-707},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17441","reported_seq":17759,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:43.316077+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:33:43.316077+0000","last_peered":"2024-09-17T15:33:43.316077+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:33:43.316077+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'7400","ondisk_log_start":"41'7400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13628358,"num_objects":3329,"num_object_clones":0,"num_object_copies":6658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3329,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14210,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-760},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17449","reported_seq":17586,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:43.386994+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:33:43.386994+0000","last_peered":"2024-09-17T15:33:43.386994+0000","last_clean":"2024-09-17T15:33:43.386994+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:33:43.386994+0000","last_undegraded":"2024-09-17T15:33:43.386994+0000","last_fullsized":"2024-09-17T15:33:43.386994+0000","mapping_epoch":32,"log_start":"41'7400","ondisk_log_start":"41'7400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14200832,"num_objects":3467,"num_object_clones":0,"num_object_copies":6934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14403,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223906+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.223906+0000","last_peered":"2024-09-17T15:33:11.223906+0000","last_clean":"2024-09-17T15:33:11.223906+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.223906+0000","last_undegraded":"2024-09-17T15:33:11.223906+0000","last_fullsized":"2024-09-17T15:33:11.223906+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17336","reported_seq":25628,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:42.883427+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:33:42.883427+0000","last_peered":"2024-09-17T15:33:42.883427+0000","last_clean":"2024-09-17T15:33:42.883427+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:33:42.883427+0000","last_undegraded":"2024-09-17T15:33:42.883427+0000","last_fullsized":"2024-09-17T15:33:42.883427+0000","mapping_epoch":42,"log_start":"41'7300","ondisk_log_start":"41'7300","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14258659,"num_objects":3482,"num_object_clones":0,"num_object_copies":6964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3482,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15336,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17139","reported_seq":17413,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:43.243081+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:33:43.243081+0000","last_peered":"2024-09-17T15:33:43.243081+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:33:43.243081+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'7100","ondisk_log_start":"41'7100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13938688,"num_objects":3403,"num_object_clones":0,"num_object_copies":6806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3403,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13735,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-540}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17187","reported_seq":17403,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:43.040616+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:33:43.040616+0000","last_peered":"2024-09-17T15:33:43.040616+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:33:43.040616+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'7100","ondisk_log_start":"41'7100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13648355,"num_objects":3333,"num_object_clones":0,"num_object_copies":6666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3333,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15082,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-700}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17335","reported_seq":25750,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:43.032917+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:33:43.032917+0000","last_peered":"2024-09-17T15:33:43.032917+0000","last_clean":"2024-09-17T15:33:43.032917+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:33:43.032917+0000","last_undegraded":"2024-09-17T15:33:43.032917+0000","last_fullsized":"2024-09-17T15:33:43.032917+0000","mapping_epoch":46,"log_start":"41'7300","ondisk_log_start":"41'7300","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13930496,"num_objects":3401,"num_object_clones":0,"num_object_copies":6802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15097,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17571","reported_seq":17877,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:43.407244+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:33:43.407244+0000","last_peered":"2024-09-17T15:33:43.407244+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:33:43.407244+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'7500","ondisk_log_start":"41'7500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13905920,"num_objects":3395,"num_object_clones":0,"num_object_copies":6790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14119,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-662}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17361","reported_seq":19013,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:43.429810+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:33:43.429810+0000","last_peered":"2024-09-17T15:33:43.429810+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:33:43.429810+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'7300","ondisk_log_start":"41'7300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13971456,"num_objects":3411,"num_object_clones":0,"num_object_copies":6822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3410,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13648,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":750,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17469","reported_seq":17777,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:43.347576+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:33:43.347576+0000","last_peered":"2024-09-17T15:33:43.347576+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:33:43.347576+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'7400","ondisk_log_start":"41'7400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13996032,"num_objects":3417,"num_object_clones":0,"num_object_copies":6834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3417,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14256,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-563}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17246","reported_seq":25630,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:42.902235+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:33:42.902235+0000","last_peered":"2024-09-17T15:33:42.902235+0000","last_clean":"2024-09-17T15:33:42.902235+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:33:42.902235+0000","last_undegraded":"2024-09-17T15:33:42.902235+0000","last_fullsized":"2024-09-17T15:33:42.902235+0000","mapping_epoch":44,"log_start":"41'7200","ondisk_log_start":"41'7200","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13870022,"num_objects":3388,"num_object_clones":0,"num_object_copies":6776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13688,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":224215798,"num_objects":54750,"num_object_clones":0,"num_object_copies":109500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35821,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54750,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":238237,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13639,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":651354112,"data_stored":651219513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160802,"ondisk_log_size":160802,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051688,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594500,"kb_used_data":243508,"kb_used_omap":4,"kb_used_meta":350971,"kb_avail":93777340,"statfs":{"total":96636764160,"available":96027996160,"internally_reserved":0,"allocated":249352192,"data_stored":249050594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":359394988},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,3,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724633,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539720,"kb_used_data":165112,"kb_used_omap":4,"kb_used_meta":374587,"kb_avail":93832120,"statfs":{"total":96636764160,"available":96084090880,"internally_reserved":0,"allocated":169074688,"data_stored":168750948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":383577771},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,3,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247916,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":653428,"kb_used_data":230304,"kb_used_omap":4,"kb_used_meta":423099,"kb_avail":93718412,"statfs":{"total":96636764160,"available":95967653888,"internally_reserved":0,"allocated":235831296,"data_stored":235529693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":433254056},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,0,7,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":235261952,"data_stored":235225822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":168034304,"data_stored":167976037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":248057856,"data_stored":248017654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:33:47.569 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:33:48.157 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:33:48.158 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:33:48.180 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-09-17T15:33:47.761638+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":224215817,"num_objects":54752,"num_object_clones":0,"num_object_copies":109504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35821,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54752,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":238239,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13640,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160805,"ondisk_log_size":160805,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1787648,"kb_used_data":638924,"kb_used_omap":13,"kb_used_meta":1148658,"kb_avail":281327872,"statfs":{"total":289910292480,"available":288079740928,"internally_reserved":0,"allocated":654258176,"data_stored":653331235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1176226815},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,6,6,11,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20078592,"num_objects":-4902,"num_object_clones":0,"num_object_copies":-9804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-209,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4902,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004359"},"pg_stats":[{"pgid":"2.f","version":"47'17282","reported_seq":17541,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:43.403570+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:33:43.403570+0000","last_peered":"2024-09-17T15:33:43.403570+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:33:43.403570+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'7200","ondisk_log_start":"41'7200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13787136,"num_objects":3366,"num_object_clones":0,"num_object_copies":6732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3366,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17282,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-618}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17421","reported_seq":17533,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:43.017911+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:33:43.017911+0000","last_peered":"2024-09-17T15:33:43.017911+0000","last_clean":"2024-09-17T15:33:43.017911+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:33:43.017911+0000","last_undegraded":"2024-09-17T15:33:43.017911+0000","last_fullsized":"2024-09-17T15:33:43.017911+0000","mapping_epoch":33,"log_start":"41'7400","ondisk_log_start":"41'7400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14382022,"num_objects":3513,"num_object_clones":0,"num_object_copies":7026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3513,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15778,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17433","reported_seq":17655,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:43.029696+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:33:43.029696+0000","last_peered":"2024-09-17T15:33:43.029696+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:33:43.029696+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'7400","ondisk_log_start":"41'7400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14336024,"num_objects":3501,"num_object_clones":0,"num_object_copies":7002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3501,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15590,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-624}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17518","reported_seq":17597,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:42.898148+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:33:42.898148+0000","last_peered":"2024-09-17T15:33:42.898148+0000","last_clean":"2024-09-17T15:33:42.898148+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:33:42.898148+0000","last_undegraded":"2024-09-17T15:33:42.898148+0000","last_fullsized":"2024-09-17T15:33:42.898148+0000","mapping_epoch":33,"log_start":"41'7500","ondisk_log_start":"41'7500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14139392,"num_objects":3452,"num_object_clones":0,"num_object_copies":6904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17518,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17247","reported_seq":17386,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:43.422634+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:33:43.422634+0000","last_peered":"2024-09-17T15:33:43.422634+0000","last_clean":"2024-09-17T15:33:43.422634+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:33:43.422634+0000","last_undegraded":"2024-09-17T15:33:43.422634+0000","last_fullsized":"2024-09-17T15:33:43.422634+0000","mapping_epoch":32,"log_start":"41'7200","ondisk_log_start":"41'7200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13951459,"num_objects":3407,"num_object_clones":0,"num_object_copies":6814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3407,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14229,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17767","reported_seq":18067,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:43.302534+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:33:43.302534+0000","last_peered":"2024-09-17T15:33:43.302534+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:33:43.302534+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'7700","ondisk_log_start":"41'7700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14270947,"num_objects":3485,"num_object_clones":0,"num_object_copies":6970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3485,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14266,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-707},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17441","reported_seq":17759,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:43.316077+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:33:43.316077+0000","last_peered":"2024-09-17T15:33:43.316077+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:33:43.316077+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'7400","ondisk_log_start":"41'7400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13628358,"num_objects":3329,"num_object_clones":0,"num_object_copies":6658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3329,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14210,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-760},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17449","reported_seq":17586,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:43.386994+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:33:43.386994+0000","last_peered":"2024-09-17T15:33:43.386994+0000","last_clean":"2024-09-17T15:33:43.386994+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:33:43.386994+0000","last_undegraded":"2024-09-17T15:33:43.386994+0000","last_fullsized":"2024-09-17T15:33:43.386994+0000","mapping_epoch":32,"log_start":"41'7400","ondisk_log_start":"41'7400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14200832,"num_objects":3467,"num_object_clones":0,"num_object_copies":6934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14403,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223906+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.223906+0000","last_peered":"2024-09-17T15:33:11.223906+0000","last_clean":"2024-09-17T15:33:11.223906+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.223906+0000","last_undegraded":"2024-09-17T15:33:11.223906+0000","last_fullsized":"2024-09-17T15:33:11.223906+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17336","reported_seq":25628,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:42.883427+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:33:42.883427+0000","last_peered":"2024-09-17T15:33:42.883427+0000","last_clean":"2024-09-17T15:33:42.883427+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:33:42.883427+0000","last_undegraded":"2024-09-17T15:33:42.883427+0000","last_fullsized":"2024-09-17T15:33:42.883427+0000","mapping_epoch":42,"log_start":"41'7300","ondisk_log_start":"41'7300","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14258659,"num_objects":3482,"num_object_clones":0,"num_object_copies":6964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3482,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15336,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17139","reported_seq":17413,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:43.243081+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:33:43.243081+0000","last_peered":"2024-09-17T15:33:43.243081+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:33:43.243081+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'7100","ondisk_log_start":"41'7100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13938688,"num_objects":3403,"num_object_clones":0,"num_object_copies":6806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3403,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13735,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-540}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17187","reported_seq":17403,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:43.040616+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:33:43.040616+0000","last_peered":"2024-09-17T15:33:43.040616+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:33:43.040616+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'7100","ondisk_log_start":"41'7100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13648355,"num_objects":3333,"num_object_clones":0,"num_object_copies":6666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3333,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15082,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-700}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17335","reported_seq":25750,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:43.032917+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:33:43.032917+0000","last_peered":"2024-09-17T15:33:43.032917+0000","last_clean":"2024-09-17T15:33:43.032917+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:33:43.032917+0000","last_undegraded":"2024-09-17T15:33:43.032917+0000","last_fullsized":"2024-09-17T15:33:43.032917+0000","mapping_epoch":46,"log_start":"41'7300","ondisk_log_start":"41'7300","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13930496,"num_objects":3401,"num_object_clones":0,"num_object_copies":6802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15097,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17571","reported_seq":17877,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:43.407244+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:33:43.407244+0000","last_peered":"2024-09-17T15:33:43.407244+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:33:43.407244+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'7500","ondisk_log_start":"41'7500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13905920,"num_objects":3395,"num_object_clones":0,"num_object_copies":6790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14119,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-662}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17361","reported_seq":19013,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:43.429810+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:33:43.429810+0000","last_peered":"2024-09-17T15:33:43.429810+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:33:43.429810+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'7300","ondisk_log_start":"41'7300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13971456,"num_objects":3411,"num_object_clones":0,"num_object_copies":6822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3410,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13648,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":750,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17469","reported_seq":17777,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:43.347576+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:33:43.347576+0000","last_peered":"2024-09-17T15:33:43.347576+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:33:43.347576+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'7400","ondisk_log_start":"41'7400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13996032,"num_objects":3417,"num_object_clones":0,"num_object_copies":6834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3417,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14256,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-563}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17246","reported_seq":25630,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:42.902235+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:33:42.902235+0000","last_peered":"2024-09-17T15:33:42.902235+0000","last_clean":"2024-09-17T15:33:42.902235+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:33:42.902235+0000","last_undegraded":"2024-09-17T15:33:42.902235+0000","last_fullsized":"2024-09-17T15:33:42.902235+0000","mapping_epoch":44,"log_start":"41'7200","ondisk_log_start":"41'7200","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13870022,"num_objects":3388,"num_object_clones":0,"num_object_copies":6776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13688,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":224215798,"num_objects":54750,"num_object_clones":0,"num_object_copies":109500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35821,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54750,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":238237,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13639,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":651354112,"data_stored":651219513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160802,"ondisk_log_size":160802,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051688,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594500,"kb_used_data":243508,"kb_used_omap":4,"kb_used_meta":350971,"kb_avail":93777340,"statfs":{"total":96636764160,"available":96027996160,"internally_reserved":0,"allocated":249352192,"data_stored":249050594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":359394988},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,3,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724633,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539720,"kb_used_data":165112,"kb_used_omap":4,"kb_used_meta":374587,"kb_avail":93832120,"statfs":{"total":96636764160,"available":96084090880,"internally_reserved":0,"allocated":169074688,"data_stored":168750948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":383577771},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,3,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247916,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":653428,"kb_used_data":230304,"kb_used_omap":4,"kb_used_meta":423099,"kb_avail":93718412,"statfs":{"total":96636764160,"available":95967653888,"internally_reserved":0,"allocated":235831296,"data_stored":235529693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":433254056},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,0,7,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":235261952,"data_stored":235225822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":168034304,"data_stored":167976037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":248057856,"data_stored":248017654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:33:48.181 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:33:48.181 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'17282", 'reported_seq': 17541, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:43.403570+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:33:43.403570+0000', 'last_peered': '2024-09-17T15:33:43.403570+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:33:43.403570+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "41'7200", 'ondisk_log_start': "41'7200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13787136, 'num_objects': 3366, 'num_object_clones': 0, 'num_object_copies': 6732, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3366, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17282, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -618}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:48.182 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:33:48.182 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'17433", 'reported_seq': 17655, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:43.029696+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:33:43.029696+0000', 'last_peered': '2024-09-17T15:33:43.029696+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:33:43.029696+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "41'7400", 'ondisk_log_start': "41'7400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14336024, 'num_objects': 3501, 'num_object_clones': 0, 'num_object_copies': 7002, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3501, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15590, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -624}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:33:48.182 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:33:48.182 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'17767", 'reported_seq': 18067, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:43.302534+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:33:43.302534+0000', 'last_peered': '2024-09-17T15:33:43.302534+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:33:43.302534+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "41'7700", 'ondisk_log_start': "41'7700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14270947, 'num_objects': 3485, 'num_object_clones': 0, 'num_object_copies': 6970, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3485, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14266, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -707}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:48.182 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:33:48.182 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'17441", 'reported_seq': 17759, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:43.316077+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:33:43.316077+0000', 'last_peered': '2024-09-17T15:33:43.316077+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:33:43.316077+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "41'7400", 'ondisk_log_start': "41'7400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13628358, 'num_objects': 3329, 'num_object_clones': 0, 'num_object_copies': 6658, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4089, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3329, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 14210, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 85, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -760}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:48.182 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:33:48.182 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'17139", 'reported_seq': 17413, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:43.243081+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:33:43.243081+0000', 'last_peered': '2024-09-17T15:33:43.243081+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:33:43.243081+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "41'7100", 'ondisk_log_start': "41'7100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13938688, 'num_objects': 3403, 'num_object_clones': 0, 'num_object_copies': 6806, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3403, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13735, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -540}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:48.182 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:33:48.183 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'17187", 'reported_seq': 17403, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:43.040616+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:33:43.040616+0000', 'last_peered': '2024-09-17T15:33:43.040616+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:33:43.040616+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "41'7100", 'ondisk_log_start': "41'7100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13648355, 'num_objects': 3333, 'num_object_clones': 0, 'num_object_copies': 6666, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4034, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3333, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15082, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -700}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:33:48.183 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:33:48.183 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'17571", 'reported_seq': 17877, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:43.407244+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:33:43.407244+0000', 'last_peered': '2024-09-17T15:33:43.407244+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:33:43.407244+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "41'7500", 'ondisk_log_start': "41'7500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13905920, 'num_objects': 3395, 'num_object_clones': 0, 'num_object_copies': 6790, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3395, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14119, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -662}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:48.183 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:33:48.183 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'17361", 'reported_seq': 19013, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:43.429810+0000', 'last_change': '2024-09-17T15:33:09.763500+0000', 'last_active': '2024-09-17T15:33:43.429810+0000', 'last_peered': '2024-09-17T15:33:43.429810+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:33:43.429810+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "41'7300", 'ondisk_log_start': "41'7300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13971456, 'num_objects': 3411, 'num_object_clones': 0, 'num_object_copies': 6822, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3410, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3411, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13648, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 750, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:48.183 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:33:48.183 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'17469", 'reported_seq': 17777, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:43.347576+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:33:43.347576+0000', 'last_peered': '2024-09-17T15:33:43.347576+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:33:43.347576+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "41'7400", 'ondisk_log_start': "41'7400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13996032, 'num_objects': 3417, 'num_object_clones': 0, 'num_object_copies': 6834, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3417, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14256, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -563}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:48.183 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:33:48.873 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:33:48.873 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35821/109504 objects degraded (32.712%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":565,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":54752,"data_bytes":224215817,"bytes_used":1830551552,"bytes_avail":288079740928,"bytes_total":289910292480,"degraded_objects":35821,"degraded_total":109504,"degraded_ratio":0.32712047048509646,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":209,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":408},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:33:48.904 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:33:48.904 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:33:49.664 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:33:49.667 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:33:49.688 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-09-17T15:33:47.761638+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":224215817,"num_objects":54752,"num_object_clones":0,"num_object_copies":109504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35821,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54752,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":238239,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13640,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160805,"ondisk_log_size":160805,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1787648,"kb_used_data":638924,"kb_used_omap":13,"kb_used_meta":1148658,"kb_avail":281327872,"statfs":{"total":289910292480,"available":288079740928,"internally_reserved":0,"allocated":654258176,"data_stored":653331235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1176226815},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,6,6,11,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20078592,"num_objects":-4902,"num_object_clones":0,"num_object_copies":-9804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-209,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4902,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004359"},"pg_stats":[{"pgid":"2.f","version":"47'17282","reported_seq":17541,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:43.403570+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:33:43.403570+0000","last_peered":"2024-09-17T15:33:43.403570+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:33:43.403570+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'7200","ondisk_log_start":"41'7200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13787136,"num_objects":3366,"num_object_clones":0,"num_object_copies":6732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3366,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17282,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-618}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17421","reported_seq":17533,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:43.017911+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:33:43.017911+0000","last_peered":"2024-09-17T15:33:43.017911+0000","last_clean":"2024-09-17T15:33:43.017911+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:33:43.017911+0000","last_undegraded":"2024-09-17T15:33:43.017911+0000","last_fullsized":"2024-09-17T15:33:43.017911+0000","mapping_epoch":33,"log_start":"41'7400","ondisk_log_start":"41'7400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14382022,"num_objects":3513,"num_object_clones":0,"num_object_copies":7026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3513,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15778,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17433","reported_seq":17655,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:43.029696+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:33:43.029696+0000","last_peered":"2024-09-17T15:33:43.029696+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:33:43.029696+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'7400","ondisk_log_start":"41'7400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14336024,"num_objects":3501,"num_object_clones":0,"num_object_copies":7002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3501,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15590,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-624}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17518","reported_seq":17597,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:42.898148+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:33:42.898148+0000","last_peered":"2024-09-17T15:33:42.898148+0000","last_clean":"2024-09-17T15:33:42.898148+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:33:42.898148+0000","last_undegraded":"2024-09-17T15:33:42.898148+0000","last_fullsized":"2024-09-17T15:33:42.898148+0000","mapping_epoch":33,"log_start":"41'7500","ondisk_log_start":"41'7500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14139392,"num_objects":3452,"num_object_clones":0,"num_object_copies":6904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17518,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17247","reported_seq":17386,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:43.422634+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:33:43.422634+0000","last_peered":"2024-09-17T15:33:43.422634+0000","last_clean":"2024-09-17T15:33:43.422634+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:33:43.422634+0000","last_undegraded":"2024-09-17T15:33:43.422634+0000","last_fullsized":"2024-09-17T15:33:43.422634+0000","mapping_epoch":32,"log_start":"41'7200","ondisk_log_start":"41'7200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13951459,"num_objects":3407,"num_object_clones":0,"num_object_copies":6814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3407,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14229,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17767","reported_seq":18067,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:43.302534+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:33:43.302534+0000","last_peered":"2024-09-17T15:33:43.302534+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:33:43.302534+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'7700","ondisk_log_start":"41'7700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14270947,"num_objects":3485,"num_object_clones":0,"num_object_copies":6970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3485,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14266,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-707},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17441","reported_seq":17759,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:43.316077+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:33:43.316077+0000","last_peered":"2024-09-17T15:33:43.316077+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:33:43.316077+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'7400","ondisk_log_start":"41'7400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13628358,"num_objects":3329,"num_object_clones":0,"num_object_copies":6658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3329,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14210,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-760},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17449","reported_seq":17586,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:43.386994+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:33:43.386994+0000","last_peered":"2024-09-17T15:33:43.386994+0000","last_clean":"2024-09-17T15:33:43.386994+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:33:43.386994+0000","last_undegraded":"2024-09-17T15:33:43.386994+0000","last_fullsized":"2024-09-17T15:33:43.386994+0000","mapping_epoch":32,"log_start":"41'7400","ondisk_log_start":"41'7400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14200832,"num_objects":3467,"num_object_clones":0,"num_object_copies":6934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14403,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223906+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.223906+0000","last_peered":"2024-09-17T15:33:11.223906+0000","last_clean":"2024-09-17T15:33:11.223906+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.223906+0000","last_undegraded":"2024-09-17T15:33:11.223906+0000","last_fullsized":"2024-09-17T15:33:11.223906+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17336","reported_seq":25628,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:42.883427+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:33:42.883427+0000","last_peered":"2024-09-17T15:33:42.883427+0000","last_clean":"2024-09-17T15:33:42.883427+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:33:42.883427+0000","last_undegraded":"2024-09-17T15:33:42.883427+0000","last_fullsized":"2024-09-17T15:33:42.883427+0000","mapping_epoch":42,"log_start":"41'7300","ondisk_log_start":"41'7300","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14258659,"num_objects":3482,"num_object_clones":0,"num_object_copies":6964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3482,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15336,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17139","reported_seq":17413,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:43.243081+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:33:43.243081+0000","last_peered":"2024-09-17T15:33:43.243081+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:33:43.243081+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'7100","ondisk_log_start":"41'7100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13938688,"num_objects":3403,"num_object_clones":0,"num_object_copies":6806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3403,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13735,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-540}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17187","reported_seq":17403,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:43.040616+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:33:43.040616+0000","last_peered":"2024-09-17T15:33:43.040616+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:33:43.040616+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'7100","ondisk_log_start":"41'7100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13648355,"num_objects":3333,"num_object_clones":0,"num_object_copies":6666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3333,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15082,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-700}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17335","reported_seq":25750,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:43.032917+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:33:43.032917+0000","last_peered":"2024-09-17T15:33:43.032917+0000","last_clean":"2024-09-17T15:33:43.032917+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:33:43.032917+0000","last_undegraded":"2024-09-17T15:33:43.032917+0000","last_fullsized":"2024-09-17T15:33:43.032917+0000","mapping_epoch":46,"log_start":"41'7300","ondisk_log_start":"41'7300","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13930496,"num_objects":3401,"num_object_clones":0,"num_object_copies":6802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15097,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17571","reported_seq":17877,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:43.407244+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:33:43.407244+0000","last_peered":"2024-09-17T15:33:43.407244+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:33:43.407244+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'7500","ondisk_log_start":"41'7500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13905920,"num_objects":3395,"num_object_clones":0,"num_object_copies":6790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14119,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-662}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17361","reported_seq":19013,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:43.429810+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:33:43.429810+0000","last_peered":"2024-09-17T15:33:43.429810+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:33:43.429810+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'7300","ondisk_log_start":"41'7300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13971456,"num_objects":3411,"num_object_clones":0,"num_object_copies":6822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3410,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13648,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":750,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17469","reported_seq":17777,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:43.347576+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:33:43.347576+0000","last_peered":"2024-09-17T15:33:43.347576+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:33:43.347576+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'7400","ondisk_log_start":"41'7400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13996032,"num_objects":3417,"num_object_clones":0,"num_object_copies":6834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3417,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14256,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-563}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17246","reported_seq":25630,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:42.902235+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:33:42.902235+0000","last_peered":"2024-09-17T15:33:42.902235+0000","last_clean":"2024-09-17T15:33:42.902235+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:33:42.902235+0000","last_undegraded":"2024-09-17T15:33:42.902235+0000","last_fullsized":"2024-09-17T15:33:42.902235+0000","mapping_epoch":44,"log_start":"41'7200","ondisk_log_start":"41'7200","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13870022,"num_objects":3388,"num_object_clones":0,"num_object_copies":6776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13688,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":224215798,"num_objects":54750,"num_object_clones":0,"num_object_copies":109500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35821,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54750,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":238237,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13639,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":651354112,"data_stored":651219513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160802,"ondisk_log_size":160802,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051688,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594500,"kb_used_data":243508,"kb_used_omap":4,"kb_used_meta":350971,"kb_avail":93777340,"statfs":{"total":96636764160,"available":96027996160,"internally_reserved":0,"allocated":249352192,"data_stored":249050594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":359394988},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,3,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724633,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539720,"kb_used_data":165112,"kb_used_omap":4,"kb_used_meta":374587,"kb_avail":93832120,"statfs":{"total":96636764160,"available":96084090880,"internally_reserved":0,"allocated":169074688,"data_stored":168750948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":383577771},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,3,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247916,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":653428,"kb_used_data":230304,"kb_used_omap":4,"kb_used_meta":423099,"kb_avail":93718412,"statfs":{"total":96636764160,"available":95967653888,"internally_reserved":0,"allocated":235831296,"data_stored":235529693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":433254056},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,0,7,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":235261952,"data_stored":235225822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":168034304,"data_stored":167976037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":248057856,"data_stored":248017654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:33:52.690 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:33:53.356 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:33:53.357 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:33:53.387 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2024-09-17T15:33:51.763597+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":214041353,"num_objects":52268,"num_object_clones":0,"num_object_copies":104536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35711,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52268,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":240723,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13750,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160889,"ondisk_log_size":160889,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1751132,"kb_used_data":619040,"kb_used_omap":13,"kb_used_meta":1132018,"kb_avail":281364388,"statfs":{"total":289910292480,"available":288117133312,"internally_reserved":0,"allocated":633896960,"data_stored":632970019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1159187455},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,3,6,9,4,2,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20287488,"num_objects":-4953,"num_object_clones":0,"num_object_copies":-9906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-213,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4953,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004919"},"pg_stats":[{"pgid":"2.f","version":"47'17430","reported_seq":17689,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:48.324055+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:33:48.324055+0000","last_peered":"2024-09-17T15:33:48.324055+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:33:48.324055+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'7400","ondisk_log_start":"41'7400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13180928,"num_objects":3218,"num_object_clones":0,"num_object_copies":6436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17430,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-766}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17562","reported_seq":17674,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:47.981618+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:33:47.981618+0000","last_peered":"2024-09-17T15:33:47.981618+0000","last_clean":"2024-09-17T15:33:47.981618+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:33:47.981618+0000","last_undegraded":"2024-09-17T15:33:47.981618+0000","last_fullsized":"2024-09-17T15:33:47.981618+0000","mapping_epoch":33,"log_start":"41'7500","ondisk_log_start":"41'7500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13804486,"num_objects":3372,"num_object_clones":0,"num_object_copies":6744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3372,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15919,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17591","reported_seq":17813,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:47.996763+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:33:47.996763+0000","last_peered":"2024-09-17T15:33:47.996763+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:33:47.996763+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'7500","ondisk_log_start":"41'7500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13688856,"num_objects":3343,"num_object_clones":0,"num_object_copies":6686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3343,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15748,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-782}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17696","reported_seq":17775,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:48.022411+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:33:48.022411+0000","last_peered":"2024-09-17T15:33:48.022411+0000","last_clean":"2024-09-17T15:33:48.022411+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:33:48.022411+0000","last_undegraded":"2024-09-17T15:33:48.022411+0000","last_fullsized":"2024-09-17T15:33:48.022411+0000","mapping_epoch":33,"log_start":"41'7600","ondisk_log_start":"41'7600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13410304,"num_objects":3274,"num_object_clones":0,"num_object_copies":6548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3274,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17696,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17389","reported_seq":17528,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:48.419550+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:33:48.419550+0000","last_peered":"2024-09-17T15:33:48.419550+0000","last_clean":"2024-09-17T15:33:48.419550+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:33:48.419550+0000","last_undegraded":"2024-09-17T15:33:48.419550+0000","last_fullsized":"2024-09-17T15:33:48.419550+0000","mapping_epoch":32,"log_start":"41'7300","ondisk_log_start":"41'7300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13369827,"num_objects":3265,"num_object_clones":0,"num_object_copies":6530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3265,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14371,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17949","reported_seq":18249,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:48.423855+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:33:48.423855+0000","last_peered":"2024-09-17T15:33:48.423855+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:33:48.423855+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'7900","ondisk_log_start":"41'7900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13525475,"num_objects":3303,"num_object_clones":0,"num_object_copies":6606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3303,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14448,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-889},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17600","reported_seq":17918,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:48.474189+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:33:48.474189+0000","last_peered":"2024-09-17T15:33:48.474189+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:33:48.474189+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'7500","ondisk_log_start":"41'7500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12977094,"num_objects":3170,"num_object_clones":0,"num_object_copies":6340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3170,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14369,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-919},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17611","reported_seq":17749,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:48.444555+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:33:48.444555+0000","last_peered":"2024-09-17T15:33:48.444555+0000","last_clean":"2024-09-17T15:33:48.444555+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:33:48.444555+0000","last_undegraded":"2024-09-17T15:33:48.444555+0000","last_fullsized":"2024-09-17T15:33:48.444555+0000","mapping_epoch":32,"log_start":"41'7600","ondisk_log_start":"41'7600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13537280,"num_objects":3305,"num_object_clones":0,"num_object_copies":6610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14565,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223906+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.223906+0000","last_peered":"2024-09-17T15:33:11.223906+0000","last_clean":"2024-09-17T15:33:11.223906+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.223906+0000","last_undegraded":"2024-09-17T15:33:11.223906+0000","last_fullsized":"2024-09-17T15:33:11.223906+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17495","reported_seq":25787,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:47.951775+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:33:47.951775+0000","last_peered":"2024-09-17T15:33:47.951775+0000","last_clean":"2024-09-17T15:33:47.951775+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:33:47.951775+0000","last_undegraded":"2024-09-17T15:33:47.951775+0000","last_fullsized":"2024-09-17T15:33:47.951775+0000","mapping_epoch":42,"log_start":"41'7400","ondisk_log_start":"41'7400","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13607395,"num_objects":3323,"num_object_clones":0,"num_object_copies":6646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3323,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15495,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17307","reported_seq":17581,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:48.368436+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:33:48.368436+0000","last_peered":"2024-09-17T15:33:48.368436+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:33:48.368436+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'7300","ondisk_log_start":"41'7300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13250560,"num_objects":3235,"num_object_clones":0,"num_object_copies":6470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13903,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-708}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17336","reported_seq":17552,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:48.048161+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:33:48.048161+0000","last_peered":"2024-09-17T15:33:48.048161+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:33:48.048161+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'7300","ondisk_log_start":"41'7300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13038051,"num_objects":3184,"num_object_clones":0,"num_object_copies":6368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3184,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15231,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-849}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17510","reported_seq":25925,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:48.020776+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:33:48.020776+0000","last_peered":"2024-09-17T15:33:48.020776+0000","last_clean":"2024-09-17T15:33:48.020776+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:33:48.020776+0000","last_undegraded":"2024-09-17T15:33:48.020776+0000","last_fullsized":"2024-09-17T15:33:48.020776+0000","mapping_epoch":46,"log_start":"41'7500","ondisk_log_start":"41'7500","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13213696,"num_objects":3226,"num_object_clones":0,"num_object_copies":6452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3226,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15272,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17686","reported_seq":17993,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:48.471729+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:33:48.471729+0000","last_peered":"2024-09-17T15:33:48.471729+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:33:48.471729+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'7600","ondisk_log_start":"41'7600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13434880,"num_objects":3280,"num_object_clones":0,"num_object_copies":6560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14234,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-777}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17507","reported_seq":19377,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:48.450106+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:33:48.450106+0000","last_peered":"2024-09-17T15:33:48.450106+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:33:48.450106+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'7500","ondisk_log_start":"41'7500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13373440,"num_objects":3265,"num_object_clones":0,"num_object_copies":6530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3300,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3265,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13794,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":860,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-35}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17617","reported_seq":17925,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:48.473395+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:33:48.473395+0000","last_peered":"2024-09-17T15:33:48.473395+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:33:48.473395+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'7600","ondisk_log_start":"41'7600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13389824,"num_objects":3269,"num_object_clones":0,"num_object_copies":6538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3269,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14404,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-711}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17400","reported_seq":25784,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:47.886681+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:33:47.886681+0000","last_peered":"2024-09-17T15:33:47.886681+0000","last_clean":"2024-09-17T15:33:47.886681+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:33:47.886681+0000","last_undegraded":"2024-09-17T15:33:47.886681+0000","last_fullsized":"2024-09-17T15:33:47.886681+0000","mapping_epoch":44,"log_start":"41'7300","ondisk_log_start":"41'7300","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13239238,"num_objects":3234,"num_object_clones":0,"num_object_copies":6468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13842,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":214041334,"num_objects":52266,"num_object_clones":0,"num_object_copies":104532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35711,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52266,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":240721,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13749,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":630976512,"data_stored":630841913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160886,"ondisk_log_size":160886,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051689,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576984,"kb_used_data":238472,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93794856,"statfs":{"total":96636764160,"available":96045932544,"internally_reserved":0,"allocated":244195328,"data_stored":243893730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":346615468},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724634,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":540556,"kb_used_data":157616,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93831284,"statfs":{"total":96636764160,"available":96083234816,"internally_reserved":0,"allocated":161398784,"data_stored":161075044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":392097451},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,3,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247917,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":633592,"kb_used_data":222952,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93738248,"statfs":{"total":96636764160,"available":95987965952,"internally_reserved":0,"allocated":228302848,"data_stored":228001245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":420474536},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,4,2,2,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":227700736,"data_stored":227664606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":160403456,"data_stored":160345189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":242872320,"data_stored":242832118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:33:53.388 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:33:53.948 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:33:53.950 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:33:53.973 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":325,"stamp":"2024-09-17T15:33:53.764102+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":203858697,"num_objects":49782,"num_object_clones":0,"num_object_copies":99564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49782,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":243209,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13859,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160875,"ondisk_log_size":160875,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1745156,"kb_used_data":599172,"kb_used_omap":13,"kb_used_meta":1145906,"kb_avail":281370364,"statfs":{"total":289910292480,"available":288123252736,"internally_reserved":0,"allocated":613552128,"data_stored":612625187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1173408767},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,0,11,7,14,10,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-30470144,"num_objects":-7439,"num_object_clones":0,"num_object_copies":-14878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-322,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7439,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7439,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004610"},"pg_stats":[{"pgid":"2.f","version":"47'17583","reported_seq":17839,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:53.591096+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:33:53.591096+0000","last_peered":"2024-09-17T15:33:53.591096+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:33:53.591096+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'7500","ondisk_log_start":"41'7500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12554240,"num_objects":3065,"num_object_clones":0,"num_object_copies":6130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17583,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-919}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17724","reported_seq":17832,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:53.003389+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:33:53.003389+0000","last_peered":"2024-09-17T15:33:53.003389+0000","last_clean":"2024-09-17T15:33:53.003389+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:33:53.003389+0000","last_undegraded":"2024-09-17T15:33:53.003389+0000","last_fullsized":"2024-09-17T15:33:53.003389+0000","mapping_epoch":33,"log_start":"41'7700","ondisk_log_start":"41'7700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13140934,"num_objects":3210,"num_object_clones":0,"num_object_copies":6420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3210,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16081,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17746","reported_seq":17968,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:53.021463+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:33:53.021463+0000","last_peered":"2024-09-17T15:33:53.021463+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:33:53.021463+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'7700","ondisk_log_start":"41'7700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13053976,"num_objects":3188,"num_object_clones":0,"num_object_copies":6376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3188,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15903,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-937}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17833","reported_seq":17912,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:52.948041+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:33:52.948041+0000","last_peered":"2024-09-17T15:33:52.948041+0000","last_clean":"2024-09-17T15:33:52.948041+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:33:52.948041+0000","last_undegraded":"2024-09-17T15:33:52.948041+0000","last_fullsized":"2024-09-17T15:33:52.948041+0000","mapping_epoch":33,"log_start":"41'7800","ondisk_log_start":"41'7800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12849152,"num_objects":3137,"num_object_clones":0,"num_object_copies":6274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17833,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17543","reported_seq":17682,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:53.483157+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:33:53.483157+0000","last_peered":"2024-09-17T15:33:53.483157+0000","last_clean":"2024-09-17T15:33:53.483157+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:33:53.483157+0000","last_undegraded":"2024-09-17T15:33:53.483157+0000","last_fullsized":"2024-09-17T15:33:53.483157+0000","mapping_epoch":32,"log_start":"41'7500","ondisk_log_start":"41'7500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12739043,"num_objects":3111,"num_object_clones":0,"num_object_copies":6222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3111,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14525,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18142","reported_seq":18442,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:53.424881+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:33:53.424881+0000","last_peered":"2024-09-17T15:33:53.424881+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:33:53.424881+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'8100","ondisk_log_start":"41'8100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12734947,"num_objects":3110,"num_object_clones":0,"num_object_copies":6220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3110,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14641,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1082},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17761","reported_seq":18079,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:53.497631+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:33:53.497631+0000","last_peered":"2024-09-17T15:33:53.497631+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:33:53.497631+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'7700","ondisk_log_start":"41'7700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12317638,"num_objects":3009,"num_object_clones":0,"num_object_copies":6018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3009,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14530,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1080},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17763","reported_seq":17901,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:53.482217+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:33:53.482217+0000","last_peered":"2024-09-17T15:33:53.482217+0000","last_clean":"2024-09-17T15:33:53.482217+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:33:53.482217+0000","last_undegraded":"2024-09-17T15:33:53.482217+0000","last_fullsized":"2024-09-17T15:33:53.482217+0000","mapping_epoch":32,"log_start":"41'7700","ondisk_log_start":"41'7700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12914688,"num_objects":3153,"num_object_clones":0,"num_object_copies":6306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3153,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14717,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223906+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.223906+0000","last_peered":"2024-09-17T15:33:11.223906+0000","last_clean":"2024-09-17T15:33:11.223906+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.223906+0000","last_undegraded":"2024-09-17T15:33:11.223906+0000","last_fullsized":"2024-09-17T15:33:11.223906+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17648","reported_seq":25940,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:53.001148+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:33:53.001148+0000","last_peered":"2024-09-17T15:33:53.001148+0000","last_clean":"2024-09-17T15:33:53.001148+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:33:53.001148+0000","last_undegraded":"2024-09-17T15:33:53.001148+0000","last_fullsized":"2024-09-17T15:33:53.001148+0000","mapping_epoch":42,"log_start":"41'7600","ondisk_log_start":"41'7600","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12980707,"num_objects":3170,"num_object_clones":0,"num_object_copies":6340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3170,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15648,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17473","reported_seq":17747,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:53.447601+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:33:53.447601+0000","last_peered":"2024-09-17T15:33:53.447601+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:33:53.447601+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'7400","ondisk_log_start":"41'7400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12570624,"num_objects":3069,"num_object_clones":0,"num_object_copies":6138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14069,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-874}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17464","reported_seq":17683,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:52.956372+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:33:52.956372+0000","last_peered":"2024-09-17T15:33:52.956372+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:33:52.956372+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'7400","ondisk_log_start":"41'7400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12513763,"num_objects":3056,"num_object_clones":0,"num_object_copies":6112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3056,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15359,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-977}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17661","reported_seq":26076,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:53.073232+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:33:53.073232+0000","last_peered":"2024-09-17T15:33:53.073232+0000","last_clean":"2024-09-17T15:33:53.073232+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:33:53.073232+0000","last_undegraded":"2024-09-17T15:33:53.073232+0000","last_fullsized":"2024-09-17T15:33:53.073232+0000","mapping_epoch":46,"log_start":"41'7600","ondisk_log_start":"41'7600","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12595200,"num_objects":3075,"num_object_clones":0,"num_object_copies":6150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3075,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15423,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17836","reported_seq":18143,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:53.533471+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:33:53.533471+0000","last_peered":"2024-09-17T15:33:53.533471+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:33:53.533471+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'7800","ondisk_log_start":"41'7800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12820480,"num_objects":3130,"num_object_clones":0,"num_object_copies":6260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14384,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-927}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17667","reported_seq":19758,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:53.525543+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:33:53.525543+0000","last_peered":"2024-09-17T15:33:53.525543+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:33:53.525543+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'7600","ondisk_log_start":"41'7600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12718080,"num_objects":3105,"num_object_clones":0,"num_object_copies":6210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3191,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13954,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":969,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-86}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17756","reported_seq":18064,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:53.449399+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:33:53.449399+0000","last_peered":"2024-09-17T15:33:53.449399+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:33:53.449399+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'7700","ondisk_log_start":"41'7700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12820480,"num_objects":3130,"num_object_clones":0,"num_object_copies":6260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14543,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-850}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17572","reported_seq":25956,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:52.956164+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:33:52.956164+0000","last_peered":"2024-09-17T15:33:52.956164+0000","last_clean":"2024-09-17T15:33:52.956164+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:33:52.956164+0000","last_undegraded":"2024-09-17T15:33:52.956164+0000","last_fullsized":"2024-09-17T15:33:52.956164+0000","mapping_epoch":44,"log_start":"41'7500","ondisk_log_start":"41'7500","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12534726,"num_objects":3062,"num_object_clones":0,"num_object_copies":6124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14014,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":203858678,"num_objects":49780,"num_object_clones":0,"num_object_copies":99560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49780,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":243207,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13858,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":610639872,"data_stored":610505273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160872,"ondisk_log_size":160872,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051690,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576376,"kb_used_data":233700,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93795464,"statfs":{"total":96636764160,"available":96046555136,"internally_reserved":0,"allocated":239308800,"data_stored":239007202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":350875308},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,7,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724635,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":520544,"kb_used_data":150096,"kb_used_omap":4,"kb_used_meta":370427,"kb_avail":93851296,"statfs":{"total":96636764160,"available":96103727104,"internally_reserved":0,"allocated":153698304,"data_stored":153374564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":379317931},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247918,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":648236,"kb_used_data":215376,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93723604,"statfs":{"total":96636764160,"available":95972970496,"internally_reserved":0,"allocated":220545024,"data_stored":220243421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":443215528},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,0,1,2,7,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":219852800,"data_stored":219816670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":152748032,"data_stored":152689765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":238039040,"data_stored":237998838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:33:53.974 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:33:53.974 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'17583", 'reported_seq': 17839, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:53.591096+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:33:53.591096+0000', 'last_peered': '2024-09-17T15:33:53.591096+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:33:53.591096+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "41'7500", 'ondisk_log_start': "41'7500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12554240, 'num_objects': 3065, 'num_object_clones': 0, 'num_object_copies': 6130, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3065, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17583, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -919}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:53.974 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:33:53.974 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'17746", 'reported_seq': 17968, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:53.021463+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:33:53.021463+0000', 'last_peered': '2024-09-17T15:33:53.021463+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:33:53.021463+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "41'7700", 'ondisk_log_start': "41'7700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13053976, 'num_objects': 3188, 'num_object_clones': 0, 'num_object_copies': 6376, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3188, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15903, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -937}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:33:53.974 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:33:53.974 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'18142", 'reported_seq': 18442, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:53.424881+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:33:53.424881+0000', 'last_peered': '2024-09-17T15:33:53.424881+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:33:53.424881+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "41'8100", 'ondisk_log_start': "41'8100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12734947, 'num_objects': 3110, 'num_object_clones': 0, 'num_object_copies': 6220, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3110, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14641, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1082}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:53.975 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:33:53.975 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'17761", 'reported_seq': 18079, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:53.497631+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:33:53.497631+0000', 'last_peered': '2024-09-17T15:33:53.497631+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:33:53.497631+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "41'7700", 'ondisk_log_start': "41'7700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12317638, 'num_objects': 3009, 'num_object_clones': 0, 'num_object_copies': 6018, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4089, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3009, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 14530, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 85, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1080}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:53.975 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:33:53.975 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'17473", 'reported_seq': 17747, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:53.447601+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:33:53.447601+0000', 'last_peered': '2024-09-17T15:33:53.447601+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:33:53.447601+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "41'7400", 'ondisk_log_start': "41'7400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12570624, 'num_objects': 3069, 'num_object_clones': 0, 'num_object_copies': 6138, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3069, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14069, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -874}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:53.975 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:33:53.975 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'17464", 'reported_seq': 17683, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:52.956372+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:33:52.956372+0000', 'last_peered': '2024-09-17T15:33:52.956372+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:33:52.956372+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "41'7400", 'ondisk_log_start': "41'7400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12513763, 'num_objects': 3056, 'num_object_clones': 0, 'num_object_copies': 6112, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4034, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3056, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15359, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -977}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:33:53.975 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:33:53.975 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'17836", 'reported_seq': 18143, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:53.533471+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:33:53.533471+0000', 'last_peered': '2024-09-17T15:33:53.533471+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:33:53.533471+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "41'7800", 'ondisk_log_start': "41'7800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12820480, 'num_objects': 3130, 'num_object_clones': 0, 'num_object_copies': 6260, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3130, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14384, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -927}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:53.975 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:33:53.975 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'17667", 'reported_seq': 19758, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:53.525543+0000', 'last_change': '2024-09-17T15:33:09.763500+0000', 'last_active': '2024-09-17T15:33:53.525543+0000', 'last_peered': '2024-09-17T15:33:53.525543+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:33:53.525543+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "41'7600", 'ondisk_log_start': "41'7600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12718080, 'num_objects': 3105, 'num_object_clones': 0, 'num_object_copies': 6210, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3191, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3105, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13954, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 969, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -86}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:53.975 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:33:53.976 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'17756", 'reported_seq': 18064, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:53.449399+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:33:53.449399+0000', 'last_peered': '2024-09-17T15:33:53.449399+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:33:53.449399+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "41'7700", 'ondisk_log_start': "41'7700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12820480, 'num_objects': 3130, 'num_object_clones': 0, 'num_object_copies': 6260, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3130, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14543, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -850}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:53.976 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:33:54.664 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:33:54.665 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35602/99564 objects degraded (35.758%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":571,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":49782,"data_bytes":203858697,"bytes_used":1787039744,"bytes_avail":288123252736,"bytes_total":289910292480,"degraded_objects":35602,"degraded_total":99564,"degraded_ratio":0.35757904463460688,"recovering_objects_per_sec":26,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":322,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":619},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:33:54.693 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:33:54.693 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:33:55.285 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:33:55.288 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:33:55.307 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":325,"stamp":"2024-09-17T15:33:53.764102+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":203858697,"num_objects":49782,"num_object_clones":0,"num_object_copies":99564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49782,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":243209,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13859,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160875,"ondisk_log_size":160875,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1745156,"kb_used_data":599172,"kb_used_omap":13,"kb_used_meta":1145906,"kb_avail":281370364,"statfs":{"total":289910292480,"available":288123252736,"internally_reserved":0,"allocated":613552128,"data_stored":612625187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1173408767},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,0,11,7,14,10,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-30470144,"num_objects":-7439,"num_object_clones":0,"num_object_copies":-14878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-322,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7439,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7439,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004610"},"pg_stats":[{"pgid":"2.f","version":"47'17583","reported_seq":17839,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:53.591096+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:33:53.591096+0000","last_peered":"2024-09-17T15:33:53.591096+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:33:53.591096+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'7500","ondisk_log_start":"41'7500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12554240,"num_objects":3065,"num_object_clones":0,"num_object_copies":6130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17583,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-919}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17724","reported_seq":17832,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:53.003389+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:33:53.003389+0000","last_peered":"2024-09-17T15:33:53.003389+0000","last_clean":"2024-09-17T15:33:53.003389+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:33:53.003389+0000","last_undegraded":"2024-09-17T15:33:53.003389+0000","last_fullsized":"2024-09-17T15:33:53.003389+0000","mapping_epoch":33,"log_start":"41'7700","ondisk_log_start":"41'7700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13140934,"num_objects":3210,"num_object_clones":0,"num_object_copies":6420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3210,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16081,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17746","reported_seq":17968,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:53.021463+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:33:53.021463+0000","last_peered":"2024-09-17T15:33:53.021463+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:33:53.021463+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'7700","ondisk_log_start":"41'7700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13053976,"num_objects":3188,"num_object_clones":0,"num_object_copies":6376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3188,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15903,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-937}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17833","reported_seq":17912,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:52.948041+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:33:52.948041+0000","last_peered":"2024-09-17T15:33:52.948041+0000","last_clean":"2024-09-17T15:33:52.948041+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:33:52.948041+0000","last_undegraded":"2024-09-17T15:33:52.948041+0000","last_fullsized":"2024-09-17T15:33:52.948041+0000","mapping_epoch":33,"log_start":"41'7800","ondisk_log_start":"41'7800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12849152,"num_objects":3137,"num_object_clones":0,"num_object_copies":6274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17833,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17543","reported_seq":17682,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:53.483157+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:33:53.483157+0000","last_peered":"2024-09-17T15:33:53.483157+0000","last_clean":"2024-09-17T15:33:53.483157+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:33:53.483157+0000","last_undegraded":"2024-09-17T15:33:53.483157+0000","last_fullsized":"2024-09-17T15:33:53.483157+0000","mapping_epoch":32,"log_start":"41'7500","ondisk_log_start":"41'7500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12739043,"num_objects":3111,"num_object_clones":0,"num_object_copies":6222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3111,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14525,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18142","reported_seq":18442,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:53.424881+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:33:53.424881+0000","last_peered":"2024-09-17T15:33:53.424881+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:33:53.424881+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'8100","ondisk_log_start":"41'8100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12734947,"num_objects":3110,"num_object_clones":0,"num_object_copies":6220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3110,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14641,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1082},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17761","reported_seq":18079,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:53.497631+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:33:53.497631+0000","last_peered":"2024-09-17T15:33:53.497631+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:33:53.497631+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'7700","ondisk_log_start":"41'7700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12317638,"num_objects":3009,"num_object_clones":0,"num_object_copies":6018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3009,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14530,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1080},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17763","reported_seq":17901,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:53.482217+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:33:53.482217+0000","last_peered":"2024-09-17T15:33:53.482217+0000","last_clean":"2024-09-17T15:33:53.482217+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:33:53.482217+0000","last_undegraded":"2024-09-17T15:33:53.482217+0000","last_fullsized":"2024-09-17T15:33:53.482217+0000","mapping_epoch":32,"log_start":"41'7700","ondisk_log_start":"41'7700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12914688,"num_objects":3153,"num_object_clones":0,"num_object_copies":6306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3153,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14717,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223906+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.223906+0000","last_peered":"2024-09-17T15:33:11.223906+0000","last_clean":"2024-09-17T15:33:11.223906+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.223906+0000","last_undegraded":"2024-09-17T15:33:11.223906+0000","last_fullsized":"2024-09-17T15:33:11.223906+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17648","reported_seq":25940,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:53.001148+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:33:53.001148+0000","last_peered":"2024-09-17T15:33:53.001148+0000","last_clean":"2024-09-17T15:33:53.001148+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:33:53.001148+0000","last_undegraded":"2024-09-17T15:33:53.001148+0000","last_fullsized":"2024-09-17T15:33:53.001148+0000","mapping_epoch":42,"log_start":"41'7600","ondisk_log_start":"41'7600","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12980707,"num_objects":3170,"num_object_clones":0,"num_object_copies":6340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3170,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15648,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17473","reported_seq":17747,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:53.447601+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:33:53.447601+0000","last_peered":"2024-09-17T15:33:53.447601+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:33:53.447601+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'7400","ondisk_log_start":"41'7400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12570624,"num_objects":3069,"num_object_clones":0,"num_object_copies":6138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14069,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-874}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17464","reported_seq":17683,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:52.956372+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:33:52.956372+0000","last_peered":"2024-09-17T15:33:52.956372+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:33:52.956372+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'7400","ondisk_log_start":"41'7400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12513763,"num_objects":3056,"num_object_clones":0,"num_object_copies":6112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3056,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15359,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-977}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17661","reported_seq":26076,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:53.073232+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:33:53.073232+0000","last_peered":"2024-09-17T15:33:53.073232+0000","last_clean":"2024-09-17T15:33:53.073232+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:33:53.073232+0000","last_undegraded":"2024-09-17T15:33:53.073232+0000","last_fullsized":"2024-09-17T15:33:53.073232+0000","mapping_epoch":46,"log_start":"41'7600","ondisk_log_start":"41'7600","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12595200,"num_objects":3075,"num_object_clones":0,"num_object_copies":6150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3075,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15423,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17836","reported_seq":18143,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:53.533471+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:33:53.533471+0000","last_peered":"2024-09-17T15:33:53.533471+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:33:53.533471+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'7800","ondisk_log_start":"41'7800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12820480,"num_objects":3130,"num_object_clones":0,"num_object_copies":6260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14384,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-927}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17667","reported_seq":19758,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:53.525543+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:33:53.525543+0000","last_peered":"2024-09-17T15:33:53.525543+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:33:53.525543+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'7600","ondisk_log_start":"41'7600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12718080,"num_objects":3105,"num_object_clones":0,"num_object_copies":6210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3191,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13954,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":969,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-86}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17756","reported_seq":18064,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:53.449399+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:33:53.449399+0000","last_peered":"2024-09-17T15:33:53.449399+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:33:53.449399+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'7700","ondisk_log_start":"41'7700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12820480,"num_objects":3130,"num_object_clones":0,"num_object_copies":6260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14543,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-850}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17572","reported_seq":25956,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:52.956164+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:33:52.956164+0000","last_peered":"2024-09-17T15:33:52.956164+0000","last_clean":"2024-09-17T15:33:52.956164+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:33:52.956164+0000","last_undegraded":"2024-09-17T15:33:52.956164+0000","last_fullsized":"2024-09-17T15:33:52.956164+0000","mapping_epoch":44,"log_start":"41'7500","ondisk_log_start":"41'7500","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12534726,"num_objects":3062,"num_object_clones":0,"num_object_copies":6124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14014,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":203858678,"num_objects":49780,"num_object_clones":0,"num_object_copies":99560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49780,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":243207,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13858,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":610639872,"data_stored":610505273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160872,"ondisk_log_size":160872,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051690,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576376,"kb_used_data":233700,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93795464,"statfs":{"total":96636764160,"available":96046555136,"internally_reserved":0,"allocated":239308800,"data_stored":239007202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":350875308},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,7,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724635,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":520544,"kb_used_data":150096,"kb_used_omap":4,"kb_used_meta":370427,"kb_avail":93851296,"statfs":{"total":96636764160,"available":96103727104,"internally_reserved":0,"allocated":153698304,"data_stored":153374564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":379317931},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247918,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":648236,"kb_used_data":215376,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93723604,"statfs":{"total":96636764160,"available":95972970496,"internally_reserved":0,"allocated":220545024,"data_stored":220243421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":443215528},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,0,1,2,7,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":219852800,"data_stored":219816670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":152748032,"data_stored":152689765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":238039040,"data_stored":237998838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:33:58.309 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:33:58.875 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:33:58.877 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:33:58.903 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-09-17T15:33:57.765530+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":203858697,"num_objects":49782,"num_object_clones":0,"num_object_copies":99564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49782,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":243209,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13859,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160875,"ondisk_log_size":160875,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1745156,"kb_used_data":599172,"kb_used_omap":13,"kb_used_meta":1145906,"kb_avail":281370364,"statfs":{"total":289910292480,"available":288123252736,"internally_reserved":0,"allocated":613552128,"data_stored":612625187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1173408767},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,0,11,7,14,10,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20357120,"num_objects":-4970,"num_object_clones":0,"num_object_copies":-9940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-219,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4970,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":219,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004503"},"pg_stats":[{"pgid":"2.f","version":"47'17583","reported_seq":17839,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:53.591096+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:33:53.591096+0000","last_peered":"2024-09-17T15:33:53.591096+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:33:53.591096+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'7500","ondisk_log_start":"41'7500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12554240,"num_objects":3065,"num_object_clones":0,"num_object_copies":6130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17583,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-919}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17724","reported_seq":17832,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:53.003389+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:33:53.003389+0000","last_peered":"2024-09-17T15:33:53.003389+0000","last_clean":"2024-09-17T15:33:53.003389+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:33:53.003389+0000","last_undegraded":"2024-09-17T15:33:53.003389+0000","last_fullsized":"2024-09-17T15:33:53.003389+0000","mapping_epoch":33,"log_start":"41'7700","ondisk_log_start":"41'7700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13140934,"num_objects":3210,"num_object_clones":0,"num_object_copies":6420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3210,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16081,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17746","reported_seq":17968,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:53.021463+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:33:53.021463+0000","last_peered":"2024-09-17T15:33:53.021463+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:33:53.021463+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'7700","ondisk_log_start":"41'7700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13053976,"num_objects":3188,"num_object_clones":0,"num_object_copies":6376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3188,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15903,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-937}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17833","reported_seq":17912,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:52.948041+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:33:52.948041+0000","last_peered":"2024-09-17T15:33:52.948041+0000","last_clean":"2024-09-17T15:33:52.948041+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:33:52.948041+0000","last_undegraded":"2024-09-17T15:33:52.948041+0000","last_fullsized":"2024-09-17T15:33:52.948041+0000","mapping_epoch":33,"log_start":"41'7800","ondisk_log_start":"41'7800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12849152,"num_objects":3137,"num_object_clones":0,"num_object_copies":6274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17833,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17543","reported_seq":17682,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:53.483157+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:33:53.483157+0000","last_peered":"2024-09-17T15:33:53.483157+0000","last_clean":"2024-09-17T15:33:53.483157+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:33:53.483157+0000","last_undegraded":"2024-09-17T15:33:53.483157+0000","last_fullsized":"2024-09-17T15:33:53.483157+0000","mapping_epoch":32,"log_start":"41'7500","ondisk_log_start":"41'7500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12739043,"num_objects":3111,"num_object_clones":0,"num_object_copies":6222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3111,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14525,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18142","reported_seq":18442,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:53.424881+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:33:53.424881+0000","last_peered":"2024-09-17T15:33:53.424881+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:33:53.424881+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'8100","ondisk_log_start":"41'8100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12734947,"num_objects":3110,"num_object_clones":0,"num_object_copies":6220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3110,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14641,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1082},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17761","reported_seq":18079,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:53.497631+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:33:53.497631+0000","last_peered":"2024-09-17T15:33:53.497631+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:33:53.497631+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'7700","ondisk_log_start":"41'7700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12317638,"num_objects":3009,"num_object_clones":0,"num_object_copies":6018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3009,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14530,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1080},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17763","reported_seq":17901,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:53.482217+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:33:53.482217+0000","last_peered":"2024-09-17T15:33:53.482217+0000","last_clean":"2024-09-17T15:33:53.482217+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:33:53.482217+0000","last_undegraded":"2024-09-17T15:33:53.482217+0000","last_fullsized":"2024-09-17T15:33:53.482217+0000","mapping_epoch":32,"log_start":"41'7700","ondisk_log_start":"41'7700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12914688,"num_objects":3153,"num_object_clones":0,"num_object_copies":6306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3153,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14717,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223906+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.223906+0000","last_peered":"2024-09-17T15:33:11.223906+0000","last_clean":"2024-09-17T15:33:11.223906+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.223906+0000","last_undegraded":"2024-09-17T15:33:11.223906+0000","last_fullsized":"2024-09-17T15:33:11.223906+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17648","reported_seq":25940,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:53.001148+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:33:53.001148+0000","last_peered":"2024-09-17T15:33:53.001148+0000","last_clean":"2024-09-17T15:33:53.001148+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:33:53.001148+0000","last_undegraded":"2024-09-17T15:33:53.001148+0000","last_fullsized":"2024-09-17T15:33:53.001148+0000","mapping_epoch":42,"log_start":"41'7600","ondisk_log_start":"41'7600","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12980707,"num_objects":3170,"num_object_clones":0,"num_object_copies":6340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3170,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15648,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17473","reported_seq":17747,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:53.447601+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:33:53.447601+0000","last_peered":"2024-09-17T15:33:53.447601+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:33:53.447601+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'7400","ondisk_log_start":"41'7400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12570624,"num_objects":3069,"num_object_clones":0,"num_object_copies":6138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14069,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-874}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17464","reported_seq":17683,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:52.956372+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:33:52.956372+0000","last_peered":"2024-09-17T15:33:52.956372+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:33:52.956372+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'7400","ondisk_log_start":"41'7400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12513763,"num_objects":3056,"num_object_clones":0,"num_object_copies":6112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3056,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15359,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-977}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17661","reported_seq":26076,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:53.073232+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:33:53.073232+0000","last_peered":"2024-09-17T15:33:53.073232+0000","last_clean":"2024-09-17T15:33:53.073232+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:33:53.073232+0000","last_undegraded":"2024-09-17T15:33:53.073232+0000","last_fullsized":"2024-09-17T15:33:53.073232+0000","mapping_epoch":46,"log_start":"41'7600","ondisk_log_start":"41'7600","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12595200,"num_objects":3075,"num_object_clones":0,"num_object_copies":6150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3075,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15423,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17836","reported_seq":18143,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:53.533471+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:33:53.533471+0000","last_peered":"2024-09-17T15:33:53.533471+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:33:53.533471+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'7800","ondisk_log_start":"41'7800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12820480,"num_objects":3130,"num_object_clones":0,"num_object_copies":6260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14384,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-927}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17667","reported_seq":19758,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:53.525543+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:33:53.525543+0000","last_peered":"2024-09-17T15:33:53.525543+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:33:53.525543+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'7600","ondisk_log_start":"41'7600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12718080,"num_objects":3105,"num_object_clones":0,"num_object_copies":6210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3191,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13954,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":969,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-86}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17756","reported_seq":18064,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:53.449399+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:33:53.449399+0000","last_peered":"2024-09-17T15:33:53.449399+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:33:53.449399+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'7700","ondisk_log_start":"41'7700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12820480,"num_objects":3130,"num_object_clones":0,"num_object_copies":6260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14543,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-850}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17572","reported_seq":25956,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:52.956164+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:33:52.956164+0000","last_peered":"2024-09-17T15:33:52.956164+0000","last_clean":"2024-09-17T15:33:52.956164+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:33:52.956164+0000","last_undegraded":"2024-09-17T15:33:52.956164+0000","last_fullsized":"2024-09-17T15:33:52.956164+0000","mapping_epoch":44,"log_start":"41'7500","ondisk_log_start":"41'7500","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12534726,"num_objects":3062,"num_object_clones":0,"num_object_copies":6124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14014,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":203858678,"num_objects":49780,"num_object_clones":0,"num_object_copies":99560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49780,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":243207,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13858,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":610639872,"data_stored":610505273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160872,"ondisk_log_size":160872,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051690,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576376,"kb_used_data":233700,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93795464,"statfs":{"total":96636764160,"available":96046555136,"internally_reserved":0,"allocated":239308800,"data_stored":239007202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":350875308},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,7,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724635,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":520544,"kb_used_data":150096,"kb_used_omap":4,"kb_used_meta":370427,"kb_avail":93851296,"statfs":{"total":96636764160,"available":96103727104,"internally_reserved":0,"allocated":153698304,"data_stored":153374564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":379317931},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247918,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":648236,"kb_used_data":215376,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93723604,"statfs":{"total":96636764160,"available":95972970496,"internally_reserved":0,"allocated":220545024,"data_stored":220243421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":443215528},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,0,1,2,7,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":219852800,"data_stored":219816670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":152748032,"data_stored":152689765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":238039040,"data_stored":237998838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:33:58.905 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:33:59.491 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:33:59.493 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:33:59.515 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-09-17T15:33:57.765530+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":203858697,"num_objects":49782,"num_object_clones":0,"num_object_copies":99564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49782,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":243209,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13859,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160875,"ondisk_log_size":160875,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1745156,"kb_used_data":599172,"kb_used_omap":13,"kb_used_meta":1145906,"kb_avail":281370364,"statfs":{"total":289910292480,"available":288123252736,"internally_reserved":0,"allocated":613552128,"data_stored":612625187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1173408767},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,0,11,7,14,10,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20357120,"num_objects":-4970,"num_object_clones":0,"num_object_copies":-9940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-219,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4970,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":219,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004503"},"pg_stats":[{"pgid":"2.f","version":"47'17583","reported_seq":17839,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:53.591096+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:33:53.591096+0000","last_peered":"2024-09-17T15:33:53.591096+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:33:53.591096+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'7500","ondisk_log_start":"41'7500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12554240,"num_objects":3065,"num_object_clones":0,"num_object_copies":6130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17583,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-919}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17724","reported_seq":17832,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:53.003389+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:33:53.003389+0000","last_peered":"2024-09-17T15:33:53.003389+0000","last_clean":"2024-09-17T15:33:53.003389+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:33:53.003389+0000","last_undegraded":"2024-09-17T15:33:53.003389+0000","last_fullsized":"2024-09-17T15:33:53.003389+0000","mapping_epoch":33,"log_start":"41'7700","ondisk_log_start":"41'7700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13140934,"num_objects":3210,"num_object_clones":0,"num_object_copies":6420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3210,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16081,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17746","reported_seq":17968,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:53.021463+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:33:53.021463+0000","last_peered":"2024-09-17T15:33:53.021463+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:33:53.021463+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'7700","ondisk_log_start":"41'7700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13053976,"num_objects":3188,"num_object_clones":0,"num_object_copies":6376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3188,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15903,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-937}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17833","reported_seq":17912,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:52.948041+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:33:52.948041+0000","last_peered":"2024-09-17T15:33:52.948041+0000","last_clean":"2024-09-17T15:33:52.948041+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:33:52.948041+0000","last_undegraded":"2024-09-17T15:33:52.948041+0000","last_fullsized":"2024-09-17T15:33:52.948041+0000","mapping_epoch":33,"log_start":"41'7800","ondisk_log_start":"41'7800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12849152,"num_objects":3137,"num_object_clones":0,"num_object_copies":6274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17833,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17543","reported_seq":17682,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:53.483157+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:33:53.483157+0000","last_peered":"2024-09-17T15:33:53.483157+0000","last_clean":"2024-09-17T15:33:53.483157+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:33:53.483157+0000","last_undegraded":"2024-09-17T15:33:53.483157+0000","last_fullsized":"2024-09-17T15:33:53.483157+0000","mapping_epoch":32,"log_start":"41'7500","ondisk_log_start":"41'7500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12739043,"num_objects":3111,"num_object_clones":0,"num_object_copies":6222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3111,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14525,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18142","reported_seq":18442,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:53.424881+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:33:53.424881+0000","last_peered":"2024-09-17T15:33:53.424881+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:33:53.424881+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'8100","ondisk_log_start":"41'8100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12734947,"num_objects":3110,"num_object_clones":0,"num_object_copies":6220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3110,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14641,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1082},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17761","reported_seq":18079,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:53.497631+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:33:53.497631+0000","last_peered":"2024-09-17T15:33:53.497631+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:33:53.497631+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'7700","ondisk_log_start":"41'7700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12317638,"num_objects":3009,"num_object_clones":0,"num_object_copies":6018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3009,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14530,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1080},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17763","reported_seq":17901,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:53.482217+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:33:53.482217+0000","last_peered":"2024-09-17T15:33:53.482217+0000","last_clean":"2024-09-17T15:33:53.482217+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:33:53.482217+0000","last_undegraded":"2024-09-17T15:33:53.482217+0000","last_fullsized":"2024-09-17T15:33:53.482217+0000","mapping_epoch":32,"log_start":"41'7700","ondisk_log_start":"41'7700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12914688,"num_objects":3153,"num_object_clones":0,"num_object_copies":6306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3153,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14717,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223906+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.223906+0000","last_peered":"2024-09-17T15:33:11.223906+0000","last_clean":"2024-09-17T15:33:11.223906+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.223906+0000","last_undegraded":"2024-09-17T15:33:11.223906+0000","last_fullsized":"2024-09-17T15:33:11.223906+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17648","reported_seq":25940,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:53.001148+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:33:53.001148+0000","last_peered":"2024-09-17T15:33:53.001148+0000","last_clean":"2024-09-17T15:33:53.001148+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:33:53.001148+0000","last_undegraded":"2024-09-17T15:33:53.001148+0000","last_fullsized":"2024-09-17T15:33:53.001148+0000","mapping_epoch":42,"log_start":"41'7600","ondisk_log_start":"41'7600","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12980707,"num_objects":3170,"num_object_clones":0,"num_object_copies":6340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3170,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15648,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17473","reported_seq":17747,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:53.447601+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:33:53.447601+0000","last_peered":"2024-09-17T15:33:53.447601+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:33:53.447601+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'7400","ondisk_log_start":"41'7400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12570624,"num_objects":3069,"num_object_clones":0,"num_object_copies":6138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14069,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-874}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17464","reported_seq":17683,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:52.956372+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:33:52.956372+0000","last_peered":"2024-09-17T15:33:52.956372+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:33:52.956372+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'7400","ondisk_log_start":"41'7400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12513763,"num_objects":3056,"num_object_clones":0,"num_object_copies":6112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3056,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15359,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-977}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17661","reported_seq":26076,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:53.073232+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:33:53.073232+0000","last_peered":"2024-09-17T15:33:53.073232+0000","last_clean":"2024-09-17T15:33:53.073232+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:33:53.073232+0000","last_undegraded":"2024-09-17T15:33:53.073232+0000","last_fullsized":"2024-09-17T15:33:53.073232+0000","mapping_epoch":46,"log_start":"41'7600","ondisk_log_start":"41'7600","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12595200,"num_objects":3075,"num_object_clones":0,"num_object_copies":6150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3075,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15423,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17836","reported_seq":18143,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:53.533471+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:33:53.533471+0000","last_peered":"2024-09-17T15:33:53.533471+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:33:53.533471+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'7800","ondisk_log_start":"41'7800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12820480,"num_objects":3130,"num_object_clones":0,"num_object_copies":6260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14384,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-927}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17667","reported_seq":19758,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:53.525543+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:33:53.525543+0000","last_peered":"2024-09-17T15:33:53.525543+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:33:53.525543+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'7600","ondisk_log_start":"41'7600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12718080,"num_objects":3105,"num_object_clones":0,"num_object_copies":6210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3191,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13954,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":969,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-86}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17756","reported_seq":18064,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:53.449399+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:33:53.449399+0000","last_peered":"2024-09-17T15:33:53.449399+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:33:53.449399+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'7700","ondisk_log_start":"41'7700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12820480,"num_objects":3130,"num_object_clones":0,"num_object_copies":6260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14543,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-850}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17572","reported_seq":25956,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:52.956164+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:33:52.956164+0000","last_peered":"2024-09-17T15:33:52.956164+0000","last_clean":"2024-09-17T15:33:52.956164+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:33:52.956164+0000","last_undegraded":"2024-09-17T15:33:52.956164+0000","last_fullsized":"2024-09-17T15:33:52.956164+0000","mapping_epoch":44,"log_start":"41'7500","ondisk_log_start":"41'7500","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12534726,"num_objects":3062,"num_object_clones":0,"num_object_copies":6124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14014,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":203858678,"num_objects":49780,"num_object_clones":0,"num_object_copies":99560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49780,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":243207,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13858,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":610639872,"data_stored":610505273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160872,"ondisk_log_size":160872,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051690,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576376,"kb_used_data":233700,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93795464,"statfs":{"total":96636764160,"available":96046555136,"internally_reserved":0,"allocated":239308800,"data_stored":239007202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":350875308},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,7,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724635,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":520544,"kb_used_data":150096,"kb_used_omap":4,"kb_used_meta":370427,"kb_avail":93851296,"statfs":{"total":96636764160,"available":96103727104,"internally_reserved":0,"allocated":153698304,"data_stored":153374564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":379317931},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247918,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":648236,"kb_used_data":215376,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93723604,"statfs":{"total":96636764160,"available":95972970496,"internally_reserved":0,"allocated":220545024,"data_stored":220243421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":443215528},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,0,1,2,7,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":219852800,"data_stored":219816670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":152748032,"data_stored":152689765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":238039040,"data_stored":237998838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:33:59.516 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:33:59.516 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'17583", 'reported_seq': 17839, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:53.591096+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:33:53.591096+0000', 'last_peered': '2024-09-17T15:33:53.591096+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:33:53.591096+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "41'7500", 'ondisk_log_start': "41'7500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12554240, 'num_objects': 3065, 'num_object_clones': 0, 'num_object_copies': 6130, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3065, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17583, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -919}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:59.516 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:33:59.517 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'17746", 'reported_seq': 17968, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:53.021463+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:33:53.021463+0000', 'last_peered': '2024-09-17T15:33:53.021463+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:33:53.021463+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "41'7700", 'ondisk_log_start': "41'7700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13053976, 'num_objects': 3188, 'num_object_clones': 0, 'num_object_copies': 6376, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3188, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15903, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -937}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:33:59.517 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:33:59.517 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'18142", 'reported_seq': 18442, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:53.424881+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:33:53.424881+0000', 'last_peered': '2024-09-17T15:33:53.424881+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:33:53.424881+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "41'8100", 'ondisk_log_start': "41'8100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12734947, 'num_objects': 3110, 'num_object_clones': 0, 'num_object_copies': 6220, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3110, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14641, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1082}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:59.517 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:33:59.517 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'17761", 'reported_seq': 18079, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:53.497631+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:33:53.497631+0000', 'last_peered': '2024-09-17T15:33:53.497631+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:33:53.497631+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "41'7700", 'ondisk_log_start': "41'7700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12317638, 'num_objects': 3009, 'num_object_clones': 0, 'num_object_copies': 6018, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4089, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3009, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 14530, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 85, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1080}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:59.517 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:33:59.517 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'17473", 'reported_seq': 17747, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:53.447601+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:33:53.447601+0000', 'last_peered': '2024-09-17T15:33:53.447601+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:33:53.447601+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "41'7400", 'ondisk_log_start': "41'7400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12570624, 'num_objects': 3069, 'num_object_clones': 0, 'num_object_copies': 6138, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3069, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14069, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -874}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:59.517 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:33:59.517 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'17464", 'reported_seq': 17683, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:52.956372+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:33:52.956372+0000', 'last_peered': '2024-09-17T15:33:52.956372+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:33:52.956372+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "41'7400", 'ondisk_log_start': "41'7400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12513763, 'num_objects': 3056, 'num_object_clones': 0, 'num_object_copies': 6112, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4034, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3056, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15359, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -977}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:33:59.518 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:33:59.518 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'17836", 'reported_seq': 18143, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:53.533471+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:33:53.533471+0000', 'last_peered': '2024-09-17T15:33:53.533471+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:33:53.533471+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "41'7800", 'ondisk_log_start': "41'7800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12820480, 'num_objects': 3130, 'num_object_clones': 0, 'num_object_copies': 6260, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3130, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14384, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -927}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:59.518 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:33:59.518 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'17667", 'reported_seq': 19758, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:53.525543+0000', 'last_change': '2024-09-17T15:33:09.763500+0000', 'last_active': '2024-09-17T15:33:53.525543+0000', 'last_peered': '2024-09-17T15:33:53.525543+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:33:53.525543+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "41'7600", 'ondisk_log_start': "41'7600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12718080, 'num_objects': 3105, 'num_object_clones': 0, 'num_object_copies': 6210, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3191, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3105, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13954, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 969, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -86}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:59.518 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:33:59.518 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'17756", 'reported_seq': 18064, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:33:53.449399+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:33:53.449399+0000', 'last_peered': '2024-09-17T15:33:53.449399+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:33:53.449399+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "41'7700", 'ondisk_log_start': "41'7700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12820480, 'num_objects': 3130, 'num_object_clones': 0, 'num_object_copies': 6260, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3130, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14543, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -850}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:33:59.519 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:34:00.193 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:34:00.193 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35496/94626 objects degraded (37.512%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":577,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":47313,"data_bytes":193745673,"bytes_used":1771503616,"bytes_avail":288138788864,"bytes_total":289910292480,"degraded_objects":35496,"degraded_total":94626,"degraded_ratio":0.37511888910024721,"recovering_objects_per_sec":27,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":325,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":619},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:34:00.220 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:34:00.220 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:34:00.761 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:34:00.762 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:34:00.793 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":328,"stamp":"2024-09-17T15:33:59.766345+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":193745673,"num_objects":47313,"num_object_clones":0,"num_object_copies":94626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35496,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47313,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":245678,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13965,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160844,"ondisk_log_size":160844,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1729984,"kb_used_data":579852,"kb_used_omap":13,"kb_used_meta":1150066,"kb_avail":281385536,"statfs":{"total":289910292480,"available":288138788864,"internally_reserved":0,"allocated":593768448,"data_stored":592841507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1177668607},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,1,3,8,15,10,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-30470144,"num_objects":-7439,"num_object_clones":0,"num_object_copies":-14878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7439,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7439,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":325,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004706"},"pg_stats":[{"pgid":"2.f","version":"47'17745","reported_seq":18002,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:58.526262+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:33:58.526262+0000","last_peered":"2024-09-17T15:33:58.526262+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:33:58.526262+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'7700","ondisk_log_start":"41'7700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11890688,"num_objects":2903,"num_object_clones":0,"num_object_copies":5806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17745,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1081}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17870","reported_seq":17982,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:58.098391+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:33:58.098391+0000","last_peered":"2024-09-17T15:33:58.098391+0000","last_clean":"2024-09-17T15:33:58.098391+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:33:58.098391+0000","last_undegraded":"2024-09-17T15:33:58.098391+0000","last_fullsized":"2024-09-17T15:33:58.098391+0000","mapping_epoch":33,"log_start":"41'7800","ondisk_log_start":"41'7800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12542918,"num_objects":3064,"num_object_clones":0,"num_object_copies":6128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3064,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16227,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17885","reported_seq":18107,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:58.048487+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:33:58.048487+0000","last_peered":"2024-09-17T15:33:58.048487+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:33:58.048487+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'7800","ondisk_log_start":"41'7800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12484632,"num_objects":3049,"num_object_clones":0,"num_object_copies":6098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3049,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16042,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1076}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18007","reported_seq":18086,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:58.055374+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:33:58.055374+0000","last_peered":"2024-09-17T15:33:58.055374+0000","last_clean":"2024-09-17T15:33:58.055374+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:33:58.055374+0000","last_undegraded":"2024-09-17T15:33:58.055374+0000","last_fullsized":"2024-09-17T15:33:58.055374+0000","mapping_epoch":33,"log_start":"41'8000","ondisk_log_start":"41'8000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12136448,"num_objects":2963,"num_object_clones":0,"num_object_copies":5926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18007,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17692","reported_seq":17829,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:58.583396+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:33:58.583396+0000","last_peered":"2024-09-17T15:33:58.583396+0000","last_clean":"2024-09-17T15:33:58.583396+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:33:58.583396+0000","last_undegraded":"2024-09-17T15:33:58.583396+0000","last_fullsized":"2024-09-17T15:33:58.583396+0000","mapping_epoch":32,"log_start":"41'7600","ondisk_log_start":"41'7600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12128739,"num_objects":2962,"num_object_clones":0,"num_object_copies":5924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2962,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14674,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18308","reported_seq":18608,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:58.626735+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:33:58.626735+0000","last_peered":"2024-09-17T15:33:58.626735+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:33:58.626735+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'8300","ondisk_log_start":"41'8300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12055011,"num_objects":2944,"num_object_clones":0,"num_object_copies":5888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2944,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14807,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1248},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17893","reported_seq":18212,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:58.608025+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:33:58.608025+0000","last_peered":"2024-09-17T15:33:58.608025+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:33:58.608025+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'7800","ondisk_log_start":"41'7800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11776966,"num_objects":2877,"num_object_clones":0,"num_object_copies":5754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2877,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14662,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1212},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17930","reported_seq":18068,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:58.588609+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:33:58.588609+0000","last_peered":"2024-09-17T15:33:58.588609+0000","last_clean":"2024-09-17T15:33:58.588609+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:33:58.588609+0000","last_undegraded":"2024-09-17T15:33:58.588609+0000","last_fullsized":"2024-09-17T15:33:58.588609+0000","mapping_epoch":32,"log_start":"41'7900","ondisk_log_start":"41'7900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12230656,"num_objects":2986,"num_object_clones":0,"num_object_copies":5972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2986,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14884,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223906+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.223906+0000","last_peered":"2024-09-17T15:33:11.223906+0000","last_clean":"2024-09-17T15:33:11.223906+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.223906+0000","last_undegraded":"2024-09-17T15:33:11.223906+0000","last_fullsized":"2024-09-17T15:33:11.223906+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17810","reported_seq":26101,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:58.045242+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:33:58.045242+0000","last_peered":"2024-09-17T15:33:58.045242+0000","last_clean":"2024-09-17T15:33:58.045242+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:33:58.045242+0000","last_undegraded":"2024-09-17T15:33:58.045242+0000","last_fullsized":"2024-09-17T15:33:58.045242+0000","mapping_epoch":42,"log_start":"41'7800","ondisk_log_start":"41'7800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12317155,"num_objects":3008,"num_object_clones":0,"num_object_copies":6016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3008,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15810,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17625","reported_seq":17899,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:58.562860+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:33:58.562860+0000","last_peered":"2024-09-17T15:33:58.562860+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:33:58.562860+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'7600","ondisk_log_start":"41'7600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11948032,"num_objects":2917,"num_object_clones":0,"num_object_copies":5834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14221,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1026}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17623","reported_seq":17842,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:58.005924+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:33:58.005924+0000","last_peered":"2024-09-17T15:33:58.005924+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:33:58.005924+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'7600","ondisk_log_start":"41'7600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11862499,"num_objects":2897,"num_object_clones":0,"num_object_copies":5794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2897,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15518,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1136}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17799","reported_seq":26214,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:58.094182+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:33:58.094182+0000","last_peered":"2024-09-17T15:33:58.094182+0000","last_clean":"2024-09-17T15:33:58.094182+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:33:58.094182+0000","last_undegraded":"2024-09-17T15:33:58.094182+0000","last_fullsized":"2024-09-17T15:33:58.094182+0000","mapping_epoch":46,"log_start":"41'7700","ondisk_log_start":"41'7700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12029952,"num_objects":2937,"num_object_clones":0,"num_object_copies":5874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2937,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15561,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17995","reported_seq":18302,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:58.525371+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:33:58.525371+0000","last_peered":"2024-09-17T15:33:58.525371+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:33:58.525371+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'7900","ondisk_log_start":"41'7900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12169216,"num_objects":2971,"num_object_clones":0,"num_object_copies":5942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14543,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1086}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17827","reported_seq":20130,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:58.611793+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:33:58.611793+0000","last_peered":"2024-09-17T15:33:58.611793+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:33:58.611793+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'7800","ondisk_log_start":"41'7800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12062720,"num_objects":2945,"num_object_clones":0,"num_object_copies":5890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3085,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14114,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1075,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-140}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17900","reported_seq":18208,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:33:58.557250+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:33:58.557250+0000","last_peered":"2024-09-17T15:33:58.557250+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:33:58.557250+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'7800","ondisk_log_start":"41'7800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12230656,"num_objects":2986,"num_object_clones":0,"num_object_copies":5972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2986,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14687,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-994}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17732","reported_seq":26116,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:58.022748+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:33:58.022748+0000","last_peered":"2024-09-17T15:33:58.022748+0000","last_clean":"2024-09-17T15:33:58.022748+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:33:58.022748+0000","last_undegraded":"2024-09-17T15:33:58.022748+0000","last_fullsized":"2024-09-17T15:33:58.022748+0000","mapping_epoch":44,"log_start":"41'7700","ondisk_log_start":"41'7700","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11879366,"num_objects":2902,"num_object_clones":0,"num_object_copies":5804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14174,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":193745654,"num_objects":47311,"num_object_clones":0,"num_object_copies":94622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35496,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47311,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":245676,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13964,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":590434304,"data_stored":590299705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160841,"ondisk_log_size":160841,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051691,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":575724,"kb_used_data":228892,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93796116,"statfs":{"total":96636764160,"available":96047222784,"internally_reserved":0,"allocated":234385408,"data_stored":234083810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":355135148},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,1,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724636,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":521820,"kb_used_data":143052,"kb_used_omap":4,"kb_used_meta":378747,"kb_avail":93850020,"statfs":{"total":96636764160,"available":96102420480,"internally_reserved":0,"allocated":146485248,"data_stored":146161508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":387837611},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,8,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247919,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":632440,"kb_used_data":207908,"kb_used_omap":4,"kb_used_meta":424507,"kb_avail":93739400,"statfs":{"total":96636764160,"available":95989145600,"internally_reserved":0,"allocated":212897792,"data_stored":212596189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":434695848},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,1,5,5,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":212205568,"data_stored":212169438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":145235968,"data_stored":145177701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":232992768,"data_stored":232952566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:34:03.795 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:34:04.360 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:34:04.360 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:34:04.388 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":330,"stamp":"2024-09-17T15:34:03.768196+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":183730953,"num_objects":44868,"num_object_clones":0,"num_object_copies":89736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35388,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44868,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":248123,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14073,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160889,"ondisk_log_size":160889,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1706188,"kb_used_data":560216,"kb_used_omap":13,"kb_used_meta":1145906,"kb_avail":281409332,"statfs":{"total":289910292480,"available":288163155968,"internally_reserved":0,"allocated":573661184,"data_stored":572734243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1173408767},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,7,2,7,7,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-30310400,"num_objects":-7400,"num_object_clones":0,"num_object_copies":-14800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7400,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004598"},"pg_stats":[{"pgid":"2.f","version":"47'17900","reported_seq":18158,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:03.605262+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:34:03.605262+0000","last_peered":"2024-09-17T15:34:03.605262+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:34:03.605262+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'7800","ondisk_log_start":"41'7800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11255808,"num_objects":2748,"num_object_clones":0,"num_object_copies":5496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17900,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1236}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18041","reported_seq":18150,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:03.068072+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:34:03.068072+0000","last_peered":"2024-09-17T15:34:03.068072+0000","last_clean":"2024-09-17T15:34:03.068072+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:34:03.068072+0000","last_undegraded":"2024-09-17T15:34:03.068072+0000","last_fullsized":"2024-09-17T15:34:03.068072+0000","mapping_epoch":33,"log_start":"41'8000","ondisk_log_start":"41'8000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11842502,"num_objects":2893,"num_object_clones":0,"num_object_copies":5786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2893,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16398,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18032","reported_seq":18254,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:03.072442+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:34:03.072442+0000","last_peered":"2024-09-17T15:34:03.072442+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:34:03.072442+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'8000","ondisk_log_start":"41'8000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11882520,"num_objects":2902,"num_object_clones":0,"num_object_copies":5804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2902,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16189,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1223}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18152","reported_seq":18231,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:03.115355+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:34:03.115355+0000","last_peered":"2024-09-17T15:34:03.115355+0000","last_clean":"2024-09-17T15:34:03.115355+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:34:03.115355+0000","last_undegraded":"2024-09-17T15:34:03.115355+0000","last_fullsized":"2024-09-17T15:34:03.115355+0000","mapping_epoch":33,"log_start":"41'8100","ondisk_log_start":"41'8100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11542528,"num_objects":2818,"num_object_clones":0,"num_object_copies":5636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2818,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18152,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17838","reported_seq":17976,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:03.648962+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:34:03.648962+0000","last_peered":"2024-09-17T15:34:03.648962+0000","last_clean":"2024-09-17T15:34:03.648962+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:34:03.648962+0000","last_undegraded":"2024-09-17T15:34:03.648962+0000","last_fullsized":"2024-09-17T15:34:03.648962+0000","mapping_epoch":32,"log_start":"41'7800","ondisk_log_start":"41'7800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11530723,"num_objects":2816,"num_object_clones":0,"num_object_copies":5632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2816,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14820,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18455","reported_seq":18756,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:03.548055+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:34:03.548055+0000","last_peered":"2024-09-17T15:34:03.548055+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:34:03.548055+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'8400","ondisk_log_start":"41'8400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11452899,"num_objects":2797,"num_object_clones":0,"num_object_copies":5594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2797,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14954,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1395},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18051","reported_seq":18369,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:03.659675+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:34:03.659675+0000","last_peered":"2024-09-17T15:34:03.659675+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:34:03.659675+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'8000","ondisk_log_start":"41'8000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11129798,"num_objects":2719,"num_object_clones":0,"num_object_copies":5438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2719,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14820,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1370},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18108","reported_seq":18246,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:03.605143+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:34:03.605143+0000","last_peered":"2024-09-17T15:34:03.605143+0000","last_clean":"2024-09-17T15:34:03.605143+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:34:03.605143+0000","last_undegraded":"2024-09-17T15:34:03.605143+0000","last_fullsized":"2024-09-17T15:34:03.605143+0000","mapping_epoch":32,"log_start":"41'8100","ondisk_log_start":"41'8100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11501568,"num_objects":2808,"num_object_clones":0,"num_object_copies":5616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2808,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15062,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223906+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.223906+0000","last_peered":"2024-09-17T15:33:11.223906+0000","last_clean":"2024-09-17T15:33:11.223906+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.223906+0000","last_undegraded":"2024-09-17T15:33:11.223906+0000","last_fullsized":"2024-09-17T15:33:11.223906+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17961","reported_seq":26252,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:03.083391+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:34:03.083391+0000","last_peered":"2024-09-17T15:34:03.083391+0000","last_clean":"2024-09-17T15:34:03.083391+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:34:03.083391+0000","last_undegraded":"2024-09-17T15:34:03.083391+0000","last_fullsized":"2024-09-17T15:34:03.083391+0000","mapping_epoch":42,"log_start":"41'7900","ondisk_log_start":"41'7900","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11698659,"num_objects":2857,"num_object_clones":0,"num_object_copies":5714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2857,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15961,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17778","reported_seq":18052,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:03.516815+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:34:03.516815+0000","last_peered":"2024-09-17T15:34:03.516815+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:34:03.516815+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'7700","ondisk_log_start":"41'7700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11321344,"num_objects":2764,"num_object_clones":0,"num_object_copies":5528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14374,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1179}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17762","reported_seq":17981,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:03.034759+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:34:03.034759+0000","last_peered":"2024-09-17T15:34:03.034759+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:34:03.034759+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'7700","ondisk_log_start":"41'7700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11293155,"num_objects":2758,"num_object_clones":0,"num_object_copies":5516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2758,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15657,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1275}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17951","reported_seq":26366,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:03.054401+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:34:03.054401+0000","last_peered":"2024-09-17T15:34:03.054401+0000","last_clean":"2024-09-17T15:34:03.054401+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:34:03.054401+0000","last_undegraded":"2024-09-17T15:34:03.054401+0000","last_fullsized":"2024-09-17T15:34:03.054401+0000","mapping_epoch":46,"log_start":"41'7900","ondisk_log_start":"41'7900","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11407360,"num_objects":2785,"num_object_clones":0,"num_object_copies":5570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15713,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18150","reported_seq":18455,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:03.651948+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:34:03.651948+0000","last_peered":"2024-09-17T15:34:03.651948+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:34:03.651948+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'8100","ondisk_log_start":"41'8100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11534336,"num_objects":2816,"num_object_clones":0,"num_object_copies":5632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2816,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14698,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1241}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17965","reported_seq":20484,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:03.656431+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:34:03.656431+0000","last_peered":"2024-09-17T15:34:03.656431+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:34:03.656431+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'7900","ondisk_log_start":"41'7900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11497472,"num_objects":2807,"num_object_clones":0,"num_object_copies":5614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2977,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14252,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1183,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-170}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18062","reported_seq":18370,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:03.629015+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:34:03.629015+0000","last_peered":"2024-09-17T15:34:03.629015+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:34:03.629015+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'8000","ondisk_log_start":"41'8000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11567104,"num_objects":2824,"num_object_clones":0,"num_object_copies":5648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14849,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1156}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17880","reported_seq":26264,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:02.976983+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:34:02.976983+0000","last_peered":"2024-09-17T15:34:02.976983+0000","last_clean":"2024-09-17T15:34:02.976983+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:34:02.976983+0000","last_undegraded":"2024-09-17T15:34:02.976983+0000","last_fullsized":"2024-09-17T15:34:02.976983+0000","mapping_epoch":44,"log_start":"41'7800","ondisk_log_start":"41'7800","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11273158,"num_objects":2754,"num_object_clones":0,"num_object_copies":5508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14322,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":183730934,"num_objects":44866,"num_object_clones":0,"num_object_copies":89732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35388,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44866,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":248121,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14072,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":570449920,"data_stored":570315321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160886,"ondisk_log_size":160886,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051692,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":558236,"kb_used_data":223880,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93813604,"statfs":{"total":96636764160,"available":96065130496,"internally_reserved":0,"allocated":229253120,"data_stored":228951522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":342355628},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724637,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518756,"kb_used_data":135828,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93853084,"statfs":{"total":96636764160,"available":96105558016,"internally_reserved":0,"allocated":139087872,"data_stored":138764132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":392097451},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,6,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247920,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":629196,"kb_used_data":200508,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93742644,"statfs":{"total":96636764160,"available":95992467456,"internally_reserved":0,"allocated":205320192,"data_stored":205018589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":438955688},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,2,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":204603392,"data_stored":204567262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":137871360,"data_stored":137813093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":227975168,"data_stored":227934966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:34:04.390 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:34:04.938 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:34:04.938 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:34:04.973 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":330,"stamp":"2024-09-17T15:34:03.768196+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":183730953,"num_objects":44868,"num_object_clones":0,"num_object_copies":89736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35388,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44868,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":248123,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14073,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160889,"ondisk_log_size":160889,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1706188,"kb_used_data":560216,"kb_used_omap":13,"kb_used_meta":1145906,"kb_avail":281409332,"statfs":{"total":289910292480,"available":288163155968,"internally_reserved":0,"allocated":573661184,"data_stored":572734243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1173408767},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,7,2,7,7,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-30310400,"num_objects":-7400,"num_object_clones":0,"num_object_copies":-14800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7400,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004598"},"pg_stats":[{"pgid":"2.f","version":"47'17900","reported_seq":18158,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:03.605262+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:34:03.605262+0000","last_peered":"2024-09-17T15:34:03.605262+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:34:03.605262+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'7800","ondisk_log_start":"41'7800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11255808,"num_objects":2748,"num_object_clones":0,"num_object_copies":5496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17900,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1236}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18041","reported_seq":18150,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:03.068072+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:34:03.068072+0000","last_peered":"2024-09-17T15:34:03.068072+0000","last_clean":"2024-09-17T15:34:03.068072+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:34:03.068072+0000","last_undegraded":"2024-09-17T15:34:03.068072+0000","last_fullsized":"2024-09-17T15:34:03.068072+0000","mapping_epoch":33,"log_start":"41'8000","ondisk_log_start":"41'8000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11842502,"num_objects":2893,"num_object_clones":0,"num_object_copies":5786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2893,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16398,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18032","reported_seq":18254,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:03.072442+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:34:03.072442+0000","last_peered":"2024-09-17T15:34:03.072442+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:34:03.072442+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'8000","ondisk_log_start":"41'8000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11882520,"num_objects":2902,"num_object_clones":0,"num_object_copies":5804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2902,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16189,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1223}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18152","reported_seq":18231,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:03.115355+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:34:03.115355+0000","last_peered":"2024-09-17T15:34:03.115355+0000","last_clean":"2024-09-17T15:34:03.115355+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:34:03.115355+0000","last_undegraded":"2024-09-17T15:34:03.115355+0000","last_fullsized":"2024-09-17T15:34:03.115355+0000","mapping_epoch":33,"log_start":"41'8100","ondisk_log_start":"41'8100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11542528,"num_objects":2818,"num_object_clones":0,"num_object_copies":5636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2818,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18152,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17838","reported_seq":17976,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:03.648962+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:34:03.648962+0000","last_peered":"2024-09-17T15:34:03.648962+0000","last_clean":"2024-09-17T15:34:03.648962+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:34:03.648962+0000","last_undegraded":"2024-09-17T15:34:03.648962+0000","last_fullsized":"2024-09-17T15:34:03.648962+0000","mapping_epoch":32,"log_start":"41'7800","ondisk_log_start":"41'7800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11530723,"num_objects":2816,"num_object_clones":0,"num_object_copies":5632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2816,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14820,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18455","reported_seq":18756,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:03.548055+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:34:03.548055+0000","last_peered":"2024-09-17T15:34:03.548055+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:34:03.548055+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'8400","ondisk_log_start":"41'8400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11452899,"num_objects":2797,"num_object_clones":0,"num_object_copies":5594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2797,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14954,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1395},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18051","reported_seq":18369,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:03.659675+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:34:03.659675+0000","last_peered":"2024-09-17T15:34:03.659675+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:34:03.659675+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'8000","ondisk_log_start":"41'8000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11129798,"num_objects":2719,"num_object_clones":0,"num_object_copies":5438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2719,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14820,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1370},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18108","reported_seq":18246,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:03.605143+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:34:03.605143+0000","last_peered":"2024-09-17T15:34:03.605143+0000","last_clean":"2024-09-17T15:34:03.605143+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:34:03.605143+0000","last_undegraded":"2024-09-17T15:34:03.605143+0000","last_fullsized":"2024-09-17T15:34:03.605143+0000","mapping_epoch":32,"log_start":"41'8100","ondisk_log_start":"41'8100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11501568,"num_objects":2808,"num_object_clones":0,"num_object_copies":5616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2808,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15062,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223906+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.223906+0000","last_peered":"2024-09-17T15:33:11.223906+0000","last_clean":"2024-09-17T15:33:11.223906+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.223906+0000","last_undegraded":"2024-09-17T15:33:11.223906+0000","last_fullsized":"2024-09-17T15:33:11.223906+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17961","reported_seq":26252,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:03.083391+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:34:03.083391+0000","last_peered":"2024-09-17T15:34:03.083391+0000","last_clean":"2024-09-17T15:34:03.083391+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:34:03.083391+0000","last_undegraded":"2024-09-17T15:34:03.083391+0000","last_fullsized":"2024-09-17T15:34:03.083391+0000","mapping_epoch":42,"log_start":"41'7900","ondisk_log_start":"41'7900","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11698659,"num_objects":2857,"num_object_clones":0,"num_object_copies":5714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2857,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15961,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17778","reported_seq":18052,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:03.516815+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:34:03.516815+0000","last_peered":"2024-09-17T15:34:03.516815+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:34:03.516815+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'7700","ondisk_log_start":"41'7700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11321344,"num_objects":2764,"num_object_clones":0,"num_object_copies":5528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14374,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1179}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17762","reported_seq":17981,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:03.034759+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:34:03.034759+0000","last_peered":"2024-09-17T15:34:03.034759+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:34:03.034759+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'7700","ondisk_log_start":"41'7700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11293155,"num_objects":2758,"num_object_clones":0,"num_object_copies":5516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2758,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15657,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1275}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17951","reported_seq":26366,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:03.054401+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:34:03.054401+0000","last_peered":"2024-09-17T15:34:03.054401+0000","last_clean":"2024-09-17T15:34:03.054401+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:34:03.054401+0000","last_undegraded":"2024-09-17T15:34:03.054401+0000","last_fullsized":"2024-09-17T15:34:03.054401+0000","mapping_epoch":46,"log_start":"41'7900","ondisk_log_start":"41'7900","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11407360,"num_objects":2785,"num_object_clones":0,"num_object_copies":5570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15713,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18150","reported_seq":18455,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:03.651948+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:34:03.651948+0000","last_peered":"2024-09-17T15:34:03.651948+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:34:03.651948+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'8100","ondisk_log_start":"41'8100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11534336,"num_objects":2816,"num_object_clones":0,"num_object_copies":5632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2816,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14698,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1241}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17965","reported_seq":20484,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:03.656431+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:34:03.656431+0000","last_peered":"2024-09-17T15:34:03.656431+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:34:03.656431+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'7900","ondisk_log_start":"41'7900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11497472,"num_objects":2807,"num_object_clones":0,"num_object_copies":5614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2977,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14252,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1183,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-170}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18062","reported_seq":18370,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:03.629015+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:34:03.629015+0000","last_peered":"2024-09-17T15:34:03.629015+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:34:03.629015+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'8000","ondisk_log_start":"41'8000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11567104,"num_objects":2824,"num_object_clones":0,"num_object_copies":5648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14849,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1156}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17880","reported_seq":26264,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:02.976983+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:34:02.976983+0000","last_peered":"2024-09-17T15:34:02.976983+0000","last_clean":"2024-09-17T15:34:02.976983+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:34:02.976983+0000","last_undegraded":"2024-09-17T15:34:02.976983+0000","last_fullsized":"2024-09-17T15:34:02.976983+0000","mapping_epoch":44,"log_start":"41'7800","ondisk_log_start":"41'7800","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11273158,"num_objects":2754,"num_object_clones":0,"num_object_copies":5508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14322,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":183730934,"num_objects":44866,"num_object_clones":0,"num_object_copies":89732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35388,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44866,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":248121,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14072,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":570449920,"data_stored":570315321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160886,"ondisk_log_size":160886,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051692,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":558236,"kb_used_data":223880,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93813604,"statfs":{"total":96636764160,"available":96065130496,"internally_reserved":0,"allocated":229253120,"data_stored":228951522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":342355628},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724637,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518756,"kb_used_data":135828,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93853084,"statfs":{"total":96636764160,"available":96105558016,"internally_reserved":0,"allocated":139087872,"data_stored":138764132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":392097451},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,6,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247920,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":629196,"kb_used_data":200508,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93742644,"statfs":{"total":96636764160,"available":95992467456,"internally_reserved":0,"allocated":205320192,"data_stored":205018589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":438955688},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,2,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":204603392,"data_stored":204567262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":137871360,"data_stored":137813093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":227975168,"data_stored":227934966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:34:04.974 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:34:04.974 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'17900", 'reported_seq': 18158, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:03.605262+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:34:03.605262+0000', 'last_peered': '2024-09-17T15:34:03.605262+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:34:03.605262+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "41'7800", 'ondisk_log_start': "41'7800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11255808, 'num_objects': 2748, 'num_object_clones': 0, 'num_object_copies': 5496, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2748, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17900, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1236}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:04.975 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:34:04.975 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'18032", 'reported_seq': 18254, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:03.072442+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:34:03.072442+0000', 'last_peered': '2024-09-17T15:34:03.072442+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:34:03.072442+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "41'8000", 'ondisk_log_start': "41'8000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11882520, 'num_objects': 2902, 'num_object_clones': 0, 'num_object_copies': 5804, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2902, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16189, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1223}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:34:04.975 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:34:04.975 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'18455", 'reported_seq': 18756, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:03.548055+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:34:03.548055+0000', 'last_peered': '2024-09-17T15:34:03.548055+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:34:03.548055+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "41'8400", 'ondisk_log_start': "41'8400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11452899, 'num_objects': 2797, 'num_object_clones': 0, 'num_object_copies': 5594, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2797, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14954, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1395}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:04.975 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:34:04.975 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'18051", 'reported_seq': 18369, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:03.659675+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:34:03.659675+0000', 'last_peered': '2024-09-17T15:34:03.659675+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:34:03.659675+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "41'8000", 'ondisk_log_start': "41'8000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11129798, 'num_objects': 2719, 'num_object_clones': 0, 'num_object_copies': 5438, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4089, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2719, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 14820, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 85, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1370}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:04.975 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:34:04.975 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'17778", 'reported_seq': 18052, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:03.516815+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:34:03.516815+0000', 'last_peered': '2024-09-17T15:34:03.516815+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:34:03.516815+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "41'7700", 'ondisk_log_start': "41'7700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11321344, 'num_objects': 2764, 'num_object_clones': 0, 'num_object_copies': 5528, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2764, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14374, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1179}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:04.975 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:34:04.975 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'17762", 'reported_seq': 17981, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:03.034759+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:34:03.034759+0000', 'last_peered': '2024-09-17T15:34:03.034759+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:34:03.034759+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "41'7700", 'ondisk_log_start': "41'7700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11293155, 'num_objects': 2758, 'num_object_clones': 0, 'num_object_copies': 5516, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4034, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2758, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15657, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -1275}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:34:04.976 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:34:04.976 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'18150", 'reported_seq': 18455, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:03.651948+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:34:03.651948+0000', 'last_peered': '2024-09-17T15:34:03.651948+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:34:03.651948+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "41'8100", 'ondisk_log_start': "41'8100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11534336, 'num_objects': 2816, 'num_object_clones': 0, 'num_object_copies': 5632, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2816, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14698, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1241}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:04.976 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:34:04.976 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'17965", 'reported_seq': 20484, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:03.656431+0000', 'last_change': '2024-09-17T15:33:09.763500+0000', 'last_active': '2024-09-17T15:34:03.656431+0000', 'last_peered': '2024-09-17T15:34:03.656431+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:34:03.656431+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "41'7900", 'ondisk_log_start': "41'7900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11497472, 'num_objects': 2807, 'num_object_clones': 0, 'num_object_copies': 5614, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2977, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2807, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14252, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1183, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -170}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:04.976 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:34:04.976 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'18062", 'reported_seq': 18370, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:03.629015+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:34:03.629015+0000', 'last_peered': '2024-09-17T15:34:03.629015+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:34:03.629015+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "41'8000", 'ondisk_log_start': "41'8000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11567104, 'num_objects': 2824, 'num_object_clones': 0, 'num_object_copies': 5648, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2824, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14849, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1156}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:04.976 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:34:05.724 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:34:05.724 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35388/89736 objects degraded (39.436%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":582,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":44868,"data_bytes":183730953,"bytes_used":1747136512,"bytes_avail":288163155968,"bytes_total":289910292480,"degraded_objects":35388,"degraded_total":89736,"degraded_ratio":0.39435677988767048,"recovering_objects_per_sec":26,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":323,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":616},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:34:05.747 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:34:05.747 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:34:06.297 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:34:06.297 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:34:06.327 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2024-09-17T15:34:05.768865+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":183730953,"num_objects":44868,"num_object_clones":0,"num_object_copies":89736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35388,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44868,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":248123,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14073,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160889,"ondisk_log_size":160889,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1706188,"kb_used_data":560216,"kb_used_omap":13,"kb_used_meta":1145906,"kb_avail":281409332,"statfs":{"total":289910292480,"available":288163155968,"internally_reserved":0,"allocated":573661184,"data_stored":572734243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1173408767},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,7,2,7,7,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20127744,"num_objects":-4914,"num_object_clones":0,"num_object_copies":-9828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-214,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4914,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":214,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004763"},"pg_stats":[{"pgid":"2.f","version":"47'17900","reported_seq":18158,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:03.605262+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:34:03.605262+0000","last_peered":"2024-09-17T15:34:03.605262+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:34:03.605262+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'7800","ondisk_log_start":"41'7800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11255808,"num_objects":2748,"num_object_clones":0,"num_object_copies":5496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17900,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1236}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18041","reported_seq":18150,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:03.068072+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:34:03.068072+0000","last_peered":"2024-09-17T15:34:03.068072+0000","last_clean":"2024-09-17T15:34:03.068072+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:34:03.068072+0000","last_undegraded":"2024-09-17T15:34:03.068072+0000","last_fullsized":"2024-09-17T15:34:03.068072+0000","mapping_epoch":33,"log_start":"41'8000","ondisk_log_start":"41'8000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11842502,"num_objects":2893,"num_object_clones":0,"num_object_copies":5786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2893,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16398,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18032","reported_seq":18254,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:03.072442+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:34:03.072442+0000","last_peered":"2024-09-17T15:34:03.072442+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:34:03.072442+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'8000","ondisk_log_start":"41'8000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11882520,"num_objects":2902,"num_object_clones":0,"num_object_copies":5804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2902,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16189,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1223}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18152","reported_seq":18231,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:03.115355+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:34:03.115355+0000","last_peered":"2024-09-17T15:34:03.115355+0000","last_clean":"2024-09-17T15:34:03.115355+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:34:03.115355+0000","last_undegraded":"2024-09-17T15:34:03.115355+0000","last_fullsized":"2024-09-17T15:34:03.115355+0000","mapping_epoch":33,"log_start":"41'8100","ondisk_log_start":"41'8100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11542528,"num_objects":2818,"num_object_clones":0,"num_object_copies":5636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2818,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18152,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17838","reported_seq":17976,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:03.648962+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:34:03.648962+0000","last_peered":"2024-09-17T15:34:03.648962+0000","last_clean":"2024-09-17T15:34:03.648962+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:34:03.648962+0000","last_undegraded":"2024-09-17T15:34:03.648962+0000","last_fullsized":"2024-09-17T15:34:03.648962+0000","mapping_epoch":32,"log_start":"41'7800","ondisk_log_start":"41'7800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11530723,"num_objects":2816,"num_object_clones":0,"num_object_copies":5632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2816,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14820,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18455","reported_seq":18756,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:03.548055+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:34:03.548055+0000","last_peered":"2024-09-17T15:34:03.548055+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:34:03.548055+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"41'8400","ondisk_log_start":"41'8400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11452899,"num_objects":2797,"num_object_clones":0,"num_object_copies":5594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2797,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14954,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1395},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18051","reported_seq":18369,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:03.659675+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:34:03.659675+0000","last_peered":"2024-09-17T15:34:03.659675+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:34:03.659675+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'8000","ondisk_log_start":"41'8000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11129798,"num_objects":2719,"num_object_clones":0,"num_object_copies":5438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2719,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14820,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1370},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18108","reported_seq":18246,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:03.605143+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:34:03.605143+0000","last_peered":"2024-09-17T15:34:03.605143+0000","last_clean":"2024-09-17T15:34:03.605143+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:34:03.605143+0000","last_undegraded":"2024-09-17T15:34:03.605143+0000","last_fullsized":"2024-09-17T15:34:03.605143+0000","mapping_epoch":32,"log_start":"41'8100","ondisk_log_start":"41'8100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11501568,"num_objects":2808,"num_object_clones":0,"num_object_copies":5616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2808,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15062,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223906+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.223906+0000","last_peered":"2024-09-17T15:33:11.223906+0000","last_clean":"2024-09-17T15:33:11.223906+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.223906+0000","last_undegraded":"2024-09-17T15:33:11.223906+0000","last_fullsized":"2024-09-17T15:33:11.223906+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17961","reported_seq":26252,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:03.083391+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:34:03.083391+0000","last_peered":"2024-09-17T15:34:03.083391+0000","last_clean":"2024-09-17T15:34:03.083391+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:34:03.083391+0000","last_undegraded":"2024-09-17T15:34:03.083391+0000","last_fullsized":"2024-09-17T15:34:03.083391+0000","mapping_epoch":42,"log_start":"41'7900","ondisk_log_start":"41'7900","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11698659,"num_objects":2857,"num_object_clones":0,"num_object_copies":5714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2857,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15961,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17778","reported_seq":18052,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:03.516815+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:34:03.516815+0000","last_peered":"2024-09-17T15:34:03.516815+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:34:03.516815+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'7700","ondisk_log_start":"41'7700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11321344,"num_objects":2764,"num_object_clones":0,"num_object_copies":5528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14374,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1179}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17762","reported_seq":17981,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:03.034759+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:34:03.034759+0000","last_peered":"2024-09-17T15:34:03.034759+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:34:03.034759+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'7700","ondisk_log_start":"41'7700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11293155,"num_objects":2758,"num_object_clones":0,"num_object_copies":5516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2758,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15657,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1275}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17951","reported_seq":26366,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:03.054401+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:34:03.054401+0000","last_peered":"2024-09-17T15:34:03.054401+0000","last_clean":"2024-09-17T15:34:03.054401+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:34:03.054401+0000","last_undegraded":"2024-09-17T15:34:03.054401+0000","last_fullsized":"2024-09-17T15:34:03.054401+0000","mapping_epoch":46,"log_start":"41'7900","ondisk_log_start":"41'7900","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11407360,"num_objects":2785,"num_object_clones":0,"num_object_copies":5570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15713,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18150","reported_seq":18455,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:03.651948+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:34:03.651948+0000","last_peered":"2024-09-17T15:34:03.651948+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:34:03.651948+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'8100","ondisk_log_start":"41'8100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11534336,"num_objects":2816,"num_object_clones":0,"num_object_copies":5632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2816,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14698,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1241}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17965","reported_seq":20484,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:03.656431+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:34:03.656431+0000","last_peered":"2024-09-17T15:34:03.656431+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:34:03.656431+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'7900","ondisk_log_start":"41'7900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11497472,"num_objects":2807,"num_object_clones":0,"num_object_copies":5614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2977,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14252,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1183,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-170}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18062","reported_seq":18370,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:03.629015+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:34:03.629015+0000","last_peered":"2024-09-17T15:34:03.629015+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:34:03.629015+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'8000","ondisk_log_start":"41'8000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11567104,"num_objects":2824,"num_object_clones":0,"num_object_copies":5648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14849,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1156}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17880","reported_seq":26264,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:02.976983+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:34:02.976983+0000","last_peered":"2024-09-17T15:34:02.976983+0000","last_clean":"2024-09-17T15:34:02.976983+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:34:02.976983+0000","last_undegraded":"2024-09-17T15:34:02.976983+0000","last_fullsized":"2024-09-17T15:34:02.976983+0000","mapping_epoch":44,"log_start":"41'7800","ondisk_log_start":"41'7800","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11273158,"num_objects":2754,"num_object_clones":0,"num_object_copies":5508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14322,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":183730934,"num_objects":44866,"num_object_clones":0,"num_object_copies":89732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35388,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44866,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":248121,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14072,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":570449920,"data_stored":570315321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160886,"ondisk_log_size":160886,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051692,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":558236,"kb_used_data":223880,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93813604,"statfs":{"total":96636764160,"available":96065130496,"internally_reserved":0,"allocated":229253120,"data_stored":228951522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":342355628},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724637,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518756,"kb_used_data":135828,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93853084,"statfs":{"total":96636764160,"available":96105558016,"internally_reserved":0,"allocated":139087872,"data_stored":138764132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":392097451},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,6,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247920,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":629196,"kb_used_data":200508,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93742644,"statfs":{"total":96636764160,"available":95992467456,"internally_reserved":0,"allocated":205320192,"data_stored":205018589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":438955688},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,2,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":204603392,"data_stored":204567262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":137871360,"data_stored":137813093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":227975168,"data_stored":227934966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:34:09.329 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:34:09.898 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:34:09.899 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:34:09.924 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":333,"stamp":"2024-09-17T15:34:09.770484+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":173744905,"num_objects":42430,"num_object_clones":0,"num_object_copies":84860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35281,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42430,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":250561,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14180,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160827,"ondisk_log_size":160827,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1690772,"kb_used_data":540632,"kb_used_omap":13,"kb_used_meta":1150066,"kb_avail":281424748,"statfs":{"total":289910292480,"available":288178941952,"internally_reserved":0,"allocated":553607168,"data_stored":552680227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1177668607},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,11,3,13,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-30113792,"num_objects":-7352,"num_object_clones":0,"num_object_copies":-14704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-321,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7352,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":321,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004953"},"pg_stats":[{"pgid":"2.f","version":"47'18052","reported_seq":18311,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:08.681383+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:34:08.681383+0000","last_peered":"2024-09-17T15:34:08.681383+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:34:08.681383+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'8000","ondisk_log_start":"41'8000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10633216,"num_objects":2596,"num_object_clones":0,"num_object_copies":5192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1388}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18212","reported_seq":18324,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:08.131346+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:34:08.131346+0000","last_peered":"2024-09-17T15:34:08.131346+0000","last_clean":"2024-09-17T15:34:08.131346+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:34:08.131346+0000","last_undegraded":"2024-09-17T15:34:08.131346+0000","last_fullsized":"2024-09-17T15:34:08.131346+0000","mapping_epoch":33,"log_start":"41'8200","ondisk_log_start":"41'8200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11142086,"num_objects":2722,"num_object_clones":0,"num_object_copies":5444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2722,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16569,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18185","reported_seq":18407,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:08.128315+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:34:08.128315+0000","last_peered":"2024-09-17T15:34:08.128315+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:34:08.128315+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'8100","ondisk_log_start":"41'8100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11255832,"num_objects":2749,"num_object_clones":0,"num_object_copies":5498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2749,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16342,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1376}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18289","reported_seq":18368,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:08.131066+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:34:08.131066+0000","last_peered":"2024-09-17T15:34:08.131066+0000","last_clean":"2024-09-17T15:34:08.131066+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:34:08.131066+0000","last_undegraded":"2024-09-17T15:34:08.131066+0000","last_fullsized":"2024-09-17T15:34:08.131066+0000","mapping_epoch":33,"log_start":"41'8200","ondisk_log_start":"41'8200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10981376,"num_objects":2681,"num_object_clones":0,"num_object_copies":5362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2681,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18289,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17971","reported_seq":18110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:08.631712+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:34:08.631712+0000","last_peered":"2024-09-17T15:34:08.631712+0000","last_clean":"2024-09-17T15:34:08.631712+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:34:08.631712+0000","last_undegraded":"2024-09-17T15:34:08.631712+0000","last_fullsized":"2024-09-17T15:34:08.631712+0000","mapping_epoch":32,"log_start":"41'7900","ondisk_log_start":"41'7900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10985955,"num_objects":2683,"num_object_clones":0,"num_object_copies":5366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2683,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14953,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18599","reported_seq":18900,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:08.671675+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:34:08.671675+0000","last_peered":"2024-09-17T15:34:08.671675+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:34:08.671675+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'8500","ondisk_log_start":"43'8500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10863075,"num_objects":2653,"num_object_clones":0,"num_object_copies":5306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2653,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15098,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1539},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18187","reported_seq":18505,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:08.644069+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:34:08.644069+0000","last_peered":"2024-09-17T15:34:08.644069+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:34:08.644069+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'8100","ondisk_log_start":"41'8100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10572742,"num_objects":2583,"num_object_clones":0,"num_object_copies":5166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2583,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14956,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1506},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18268","reported_seq":18406,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:08.536779+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:34:08.536779+0000","last_peered":"2024-09-17T15:34:08.536779+0000","last_clean":"2024-09-17T15:34:08.536779+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:34:08.536779+0000","last_undegraded":"2024-09-17T15:34:08.536779+0000","last_fullsized":"2024-09-17T15:34:08.536779+0000","mapping_epoch":32,"log_start":"41'8200","ondisk_log_start":"41'8200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10846208,"num_objects":2648,"num_object_clones":0,"num_object_copies":5296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15222,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223906+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.223906+0000","last_peered":"2024-09-17T15:33:11.223906+0000","last_clean":"2024-09-17T15:33:11.223906+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.223906+0000","last_undegraded":"2024-09-17T15:33:11.223906+0000","last_fullsized":"2024-09-17T15:33:11.223906+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18117","reported_seq":26409,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:08.044804+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:34:08.044804+0000","last_peered":"2024-09-17T15:34:08.044804+0000","last_clean":"2024-09-17T15:34:08.044804+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:34:08.044804+0000","last_undegraded":"2024-09-17T15:34:08.044804+0000","last_fullsized":"2024-09-17T15:34:08.044804+0000","mapping_epoch":42,"log_start":"41'8100","ondisk_log_start":"41'8100","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11059683,"num_objects":2701,"num_object_clones":0,"num_object_copies":5402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2701,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16117,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17936","reported_seq":18210,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:08.664939+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:34:08.664939+0000","last_peered":"2024-09-17T15:34:08.664939+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:34:08.664939+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'7900","ondisk_log_start":"41'7900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10674176,"num_objects":2606,"num_object_clones":0,"num_object_copies":5212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14532,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1337}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17897","reported_seq":18116,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:08.029374+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:34:08.029374+0000","last_peered":"2024-09-17T15:34:08.029374+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:34:08.029374+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'7800","ondisk_log_start":"41'7800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10740195,"num_objects":2623,"num_object_clones":0,"num_object_copies":5246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2623,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15792,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1410}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18122","reported_seq":26537,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:08.103267+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:34:08.103267+0000","last_peered":"2024-09-17T15:34:08.103267+0000","last_clean":"2024-09-17T15:34:08.103267+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:34:08.103267+0000","last_undegraded":"2024-09-17T15:34:08.103267+0000","last_fullsized":"2024-09-17T15:34:08.103267+0000","mapping_epoch":46,"log_start":"41'8100","ondisk_log_start":"41'8100","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10706944,"num_objects":2614,"num_object_clones":0,"num_object_copies":5228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15884,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18309","reported_seq":18616,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:08.664414+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:34:08.664414+0000","last_peered":"2024-09-17T15:34:08.664414+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:34:08.664414+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'8300","ondisk_log_start":"41'8300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10883072,"num_objects":2657,"num_object_clones":0,"num_object_copies":5314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14857,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1400}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18119","reported_seq":20852,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:08.628138+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:34:08.628138+0000","last_peered":"2024-09-17T15:34:08.628138+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:34:08.628138+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'8100","ondisk_log_start":"41'8100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10866688,"num_objects":2653,"num_object_clones":0,"num_object_copies":5306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2653,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14406,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1290,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-217}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18224","reported_seq":18532,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:08.582546+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:34:08.582546+0000","last_peered":"2024-09-17T15:34:08.582546+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:34:08.582546+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'8200","ondisk_log_start":"41'8200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10903552,"num_objects":2662,"num_object_clones":0,"num_object_copies":5324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15011,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1318}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18037","reported_seq":26421,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:08.122965+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:34:08.122965+0000","last_peered":"2024-09-17T15:34:08.122965+0000","last_clean":"2024-09-17T15:34:08.122965+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:34:08.122965+0000","last_undegraded":"2024-09-17T15:34:08.122965+0000","last_fullsized":"2024-09-17T15:34:08.122965+0000","mapping_epoch":44,"log_start":"41'8000","ondisk_log_start":"41'8000","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10630086,"num_objects":2597,"num_object_clones":0,"num_object_copies":5194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2597,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14479,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":173744886,"num_objects":42428,"num_object_clones":0,"num_object_copies":84856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35281,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42428,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":250559,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14179,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":550408192,"data_stored":550273593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160824,"ondisk_log_size":160824,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051693,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":557468,"kb_used_data":218956,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93814372,"statfs":{"total":96636764160,"available":96065916928,"internally_reserved":0,"allocated":224210944,"data_stored":223909346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":346615468},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,0,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724638,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":503100,"kb_used_data":128484,"kb_used_omap":4,"kb_used_meta":374587,"kb_avail":93868740,"statfs":{"total":96636764160,"available":96121589760,"internally_reserved":0,"allocated":131567616,"data_stored":131243876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":383577771},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,6,1,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247921,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":630204,"kb_used_data":193192,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93741636,"statfs":{"total":96636764160,"available":95991435264,"internally_reserved":0,"allocated":197828608,"data_stored":197527005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":447475368},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":197173248,"data_stored":197137118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":130289664,"data_stored":130231397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":222945280,"data_stored":222905078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:34:09.926 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:34:10.467 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:34:10.467 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:34:10.495 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":333,"stamp":"2024-09-17T15:34:09.770484+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":173744905,"num_objects":42430,"num_object_clones":0,"num_object_copies":84860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35281,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42430,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":250561,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14180,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160827,"ondisk_log_size":160827,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1690772,"kb_used_data":540632,"kb_used_omap":13,"kb_used_meta":1150066,"kb_avail":281424748,"statfs":{"total":289910292480,"available":288178941952,"internally_reserved":0,"allocated":553607168,"data_stored":552680227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1177668607},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,11,3,13,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-30113792,"num_objects":-7352,"num_object_clones":0,"num_object_copies":-14704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-321,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7352,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":321,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004953"},"pg_stats":[{"pgid":"2.f","version":"47'18052","reported_seq":18311,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:08.681383+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:34:08.681383+0000","last_peered":"2024-09-17T15:34:08.681383+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:34:08.681383+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'8000","ondisk_log_start":"41'8000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10633216,"num_objects":2596,"num_object_clones":0,"num_object_copies":5192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1388}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18212","reported_seq":18324,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:08.131346+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:34:08.131346+0000","last_peered":"2024-09-17T15:34:08.131346+0000","last_clean":"2024-09-17T15:34:08.131346+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:34:08.131346+0000","last_undegraded":"2024-09-17T15:34:08.131346+0000","last_fullsized":"2024-09-17T15:34:08.131346+0000","mapping_epoch":33,"log_start":"41'8200","ondisk_log_start":"41'8200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11142086,"num_objects":2722,"num_object_clones":0,"num_object_copies":5444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2722,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16569,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18185","reported_seq":18407,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:08.128315+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:34:08.128315+0000","last_peered":"2024-09-17T15:34:08.128315+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:34:08.128315+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'8100","ondisk_log_start":"41'8100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11255832,"num_objects":2749,"num_object_clones":0,"num_object_copies":5498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2749,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16342,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1376}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18289","reported_seq":18368,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:08.131066+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:34:08.131066+0000","last_peered":"2024-09-17T15:34:08.131066+0000","last_clean":"2024-09-17T15:34:08.131066+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:34:08.131066+0000","last_undegraded":"2024-09-17T15:34:08.131066+0000","last_fullsized":"2024-09-17T15:34:08.131066+0000","mapping_epoch":33,"log_start":"41'8200","ondisk_log_start":"41'8200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10981376,"num_objects":2681,"num_object_clones":0,"num_object_copies":5362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2681,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18289,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17971","reported_seq":18110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:08.631712+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:34:08.631712+0000","last_peered":"2024-09-17T15:34:08.631712+0000","last_clean":"2024-09-17T15:34:08.631712+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:34:08.631712+0000","last_undegraded":"2024-09-17T15:34:08.631712+0000","last_fullsized":"2024-09-17T15:34:08.631712+0000","mapping_epoch":32,"log_start":"41'7900","ondisk_log_start":"41'7900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10985955,"num_objects":2683,"num_object_clones":0,"num_object_copies":5366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2683,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14953,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18599","reported_seq":18900,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:08.671675+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:34:08.671675+0000","last_peered":"2024-09-17T15:34:08.671675+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:34:08.671675+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'8500","ondisk_log_start":"43'8500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10863075,"num_objects":2653,"num_object_clones":0,"num_object_copies":5306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2653,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15098,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1539},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18187","reported_seq":18505,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:08.644069+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:34:08.644069+0000","last_peered":"2024-09-17T15:34:08.644069+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:34:08.644069+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'8100","ondisk_log_start":"41'8100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10572742,"num_objects":2583,"num_object_clones":0,"num_object_copies":5166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2583,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14956,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1506},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18268","reported_seq":18406,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:08.536779+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:34:08.536779+0000","last_peered":"2024-09-17T15:34:08.536779+0000","last_clean":"2024-09-17T15:34:08.536779+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:34:08.536779+0000","last_undegraded":"2024-09-17T15:34:08.536779+0000","last_fullsized":"2024-09-17T15:34:08.536779+0000","mapping_epoch":32,"log_start":"41'8200","ondisk_log_start":"41'8200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10846208,"num_objects":2648,"num_object_clones":0,"num_object_copies":5296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15222,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223906+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.223906+0000","last_peered":"2024-09-17T15:33:11.223906+0000","last_clean":"2024-09-17T15:33:11.223906+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.223906+0000","last_undegraded":"2024-09-17T15:33:11.223906+0000","last_fullsized":"2024-09-17T15:33:11.223906+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18117","reported_seq":26409,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:08.044804+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:34:08.044804+0000","last_peered":"2024-09-17T15:34:08.044804+0000","last_clean":"2024-09-17T15:34:08.044804+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:34:08.044804+0000","last_undegraded":"2024-09-17T15:34:08.044804+0000","last_fullsized":"2024-09-17T15:34:08.044804+0000","mapping_epoch":42,"log_start":"41'8100","ondisk_log_start":"41'8100","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11059683,"num_objects":2701,"num_object_clones":0,"num_object_copies":5402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2701,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16117,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17936","reported_seq":18210,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:08.664939+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:34:08.664939+0000","last_peered":"2024-09-17T15:34:08.664939+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:34:08.664939+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'7900","ondisk_log_start":"41'7900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10674176,"num_objects":2606,"num_object_clones":0,"num_object_copies":5212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14532,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1337}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17897","reported_seq":18116,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:08.029374+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:34:08.029374+0000","last_peered":"2024-09-17T15:34:08.029374+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:34:08.029374+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'7800","ondisk_log_start":"41'7800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10740195,"num_objects":2623,"num_object_clones":0,"num_object_copies":5246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2623,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15792,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1410}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18122","reported_seq":26537,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:08.103267+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:34:08.103267+0000","last_peered":"2024-09-17T15:34:08.103267+0000","last_clean":"2024-09-17T15:34:08.103267+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:34:08.103267+0000","last_undegraded":"2024-09-17T15:34:08.103267+0000","last_fullsized":"2024-09-17T15:34:08.103267+0000","mapping_epoch":46,"log_start":"41'8100","ondisk_log_start":"41'8100","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10706944,"num_objects":2614,"num_object_clones":0,"num_object_copies":5228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15884,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18309","reported_seq":18616,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:08.664414+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:34:08.664414+0000","last_peered":"2024-09-17T15:34:08.664414+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:34:08.664414+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'8300","ondisk_log_start":"41'8300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10883072,"num_objects":2657,"num_object_clones":0,"num_object_copies":5314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14857,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1400}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18119","reported_seq":20852,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:08.628138+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:34:08.628138+0000","last_peered":"2024-09-17T15:34:08.628138+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:34:08.628138+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'8100","ondisk_log_start":"41'8100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10866688,"num_objects":2653,"num_object_clones":0,"num_object_copies":5306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2653,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14406,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1290,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-217}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18224","reported_seq":18532,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:08.582546+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:34:08.582546+0000","last_peered":"2024-09-17T15:34:08.582546+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:34:08.582546+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'8200","ondisk_log_start":"41'8200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10903552,"num_objects":2662,"num_object_clones":0,"num_object_copies":5324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15011,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1318}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18037","reported_seq":26421,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:08.122965+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:34:08.122965+0000","last_peered":"2024-09-17T15:34:08.122965+0000","last_clean":"2024-09-17T15:34:08.122965+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:34:08.122965+0000","last_undegraded":"2024-09-17T15:34:08.122965+0000","last_fullsized":"2024-09-17T15:34:08.122965+0000","mapping_epoch":44,"log_start":"41'8000","ondisk_log_start":"41'8000","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10630086,"num_objects":2597,"num_object_clones":0,"num_object_copies":5194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2597,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14479,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":173744886,"num_objects":42428,"num_object_clones":0,"num_object_copies":84856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35281,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42428,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":250559,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14179,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":550408192,"data_stored":550273593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160824,"ondisk_log_size":160824,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051693,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":557468,"kb_used_data":218956,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93814372,"statfs":{"total":96636764160,"available":96065916928,"internally_reserved":0,"allocated":224210944,"data_stored":223909346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":346615468},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,0,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724638,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":503100,"kb_used_data":128484,"kb_used_omap":4,"kb_used_meta":374587,"kb_avail":93868740,"statfs":{"total":96636764160,"available":96121589760,"internally_reserved":0,"allocated":131567616,"data_stored":131243876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":383577771},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,6,1,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247921,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":630204,"kb_used_data":193192,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93741636,"statfs":{"total":96636764160,"available":95991435264,"internally_reserved":0,"allocated":197828608,"data_stored":197527005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":447475368},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":197173248,"data_stored":197137118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":130289664,"data_stored":130231397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":222945280,"data_stored":222905078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:34:10.497 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:34:10.497 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'18052", 'reported_seq': 18311, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:08.681383+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:34:08.681383+0000', 'last_peered': '2024-09-17T15:34:08.681383+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:34:08.681383+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "41'8000", 'ondisk_log_start': "41'8000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10633216, 'num_objects': 2596, 'num_object_clones': 0, 'num_object_copies': 5192, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2596, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18052, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1388}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:10.497 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:34:10.497 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'18185", 'reported_seq': 18407, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:08.128315+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:34:08.128315+0000', 'last_peered': '2024-09-17T15:34:08.128315+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:34:08.128315+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "41'8100", 'ondisk_log_start': "41'8100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11255832, 'num_objects': 2749, 'num_object_clones': 0, 'num_object_copies': 5498, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2749, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16342, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1376}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:34:10.497 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:34:10.497 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'18599", 'reported_seq': 18900, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:08.671675+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:34:08.671675+0000', 'last_peered': '2024-09-17T15:34:08.671675+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:34:08.671675+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "43'8500", 'ondisk_log_start': "43'8500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10863075, 'num_objects': 2653, 'num_object_clones': 0, 'num_object_copies': 5306, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2653, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15098, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1539}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:10.497 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:34:10.498 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'18187", 'reported_seq': 18505, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:08.644069+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:34:08.644069+0000', 'last_peered': '2024-09-17T15:34:08.644069+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:34:08.644069+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "41'8100", 'ondisk_log_start': "41'8100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10572742, 'num_objects': 2583, 'num_object_clones': 0, 'num_object_copies': 5166, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4089, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2583, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 14956, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 85, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1506}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:10.498 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:34:10.498 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'17936", 'reported_seq': 18210, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:08.664939+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:34:08.664939+0000', 'last_peered': '2024-09-17T15:34:08.664939+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:34:08.664939+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "41'7900", 'ondisk_log_start': "41'7900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10674176, 'num_objects': 2606, 'num_object_clones': 0, 'num_object_copies': 5212, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2606, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14532, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1337}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:10.498 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:34:10.498 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'17897", 'reported_seq': 18116, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:08.029374+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:34:08.029374+0000', 'last_peered': '2024-09-17T15:34:08.029374+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:34:08.029374+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "41'7800", 'ondisk_log_start': "41'7800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10740195, 'num_objects': 2623, 'num_object_clones': 0, 'num_object_copies': 5246, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4034, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2623, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15792, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -1410}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:34:10.498 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:34:10.498 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'18309", 'reported_seq': 18616, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:08.664414+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:34:08.664414+0000', 'last_peered': '2024-09-17T15:34:08.664414+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:34:08.664414+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "41'8300", 'ondisk_log_start': "41'8300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10883072, 'num_objects': 2657, 'num_object_clones': 0, 'num_object_copies': 5314, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2657, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14857, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1400}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:10.498 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:34:10.498 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'18119", 'reported_seq': 20852, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:08.628138+0000', 'last_change': '2024-09-17T15:33:09.763500+0000', 'last_active': '2024-09-17T15:34:08.628138+0000', 'last_peered': '2024-09-17T15:34:08.628138+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:34:08.628138+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "41'8100", 'ondisk_log_start': "41'8100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10866688, 'num_objects': 2653, 'num_object_clones': 0, 'num_object_copies': 5306, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2870, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2653, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14406, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1290, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -217}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:10.498 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:34:10.498 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'18224", 'reported_seq': 18532, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:08.582546+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:34:08.582546+0000', 'last_peered': '2024-09-17T15:34:08.582546+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:34:08.582546+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "41'8200", 'ondisk_log_start': "41'8200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10903552, 'num_objects': 2662, 'num_object_clones': 0, 'num_object_copies': 5324, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2662, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15011, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1318}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:10.499 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:34:11.210 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:34:11.210 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35281/84860 objects degraded (41.576%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":588,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":42430,"data_bytes":173744905,"bytes_used":1731350528,"bytes_avail":288178941952,"bytes_total":289910292480,"degraded_objects":35281,"degraded_total":84860,"degraded_ratio":0.41575536177233091,"recovering_objects_per_sec":26,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":321,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":612},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:34:11.233 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:34:11.233 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:34:11.815 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:34:11.815 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:34:11.838 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":334,"stamp":"2024-09-17T15:34:11.771369+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":173744905,"num_objects":42430,"num_object_clones":0,"num_object_copies":84860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35281,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42430,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":250561,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14180,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160827,"ondisk_log_size":160827,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1690772,"kb_used_data":540632,"kb_used_omap":13,"kb_used_meta":1150066,"kb_avail":281424748,"statfs":{"total":289910292480,"available":288178941952,"internally_reserved":0,"allocated":553607168,"data_stored":552680227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1177668607},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,11,3,13,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20000768,"num_objects":-4883,"num_object_clones":0,"num_object_copies":-9766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-215,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4883,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":215,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005023"},"pg_stats":[{"pgid":"2.f","version":"47'18052","reported_seq":18311,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:08.681383+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:34:08.681383+0000","last_peered":"2024-09-17T15:34:08.681383+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:34:08.681383+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'8000","ondisk_log_start":"41'8000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10633216,"num_objects":2596,"num_object_clones":0,"num_object_copies":5192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1388}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18212","reported_seq":18324,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:08.131346+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:34:08.131346+0000","last_peered":"2024-09-17T15:34:08.131346+0000","last_clean":"2024-09-17T15:34:08.131346+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:34:08.131346+0000","last_undegraded":"2024-09-17T15:34:08.131346+0000","last_fullsized":"2024-09-17T15:34:08.131346+0000","mapping_epoch":33,"log_start":"41'8200","ondisk_log_start":"41'8200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11142086,"num_objects":2722,"num_object_clones":0,"num_object_copies":5444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2722,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16569,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18185","reported_seq":18407,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:08.128315+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:34:08.128315+0000","last_peered":"2024-09-17T15:34:08.128315+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:34:08.128315+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"41'8100","ondisk_log_start":"41'8100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11255832,"num_objects":2749,"num_object_clones":0,"num_object_copies":5498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2749,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16342,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1376}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18289","reported_seq":18368,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:08.131066+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:34:08.131066+0000","last_peered":"2024-09-17T15:34:08.131066+0000","last_clean":"2024-09-17T15:34:08.131066+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:34:08.131066+0000","last_undegraded":"2024-09-17T15:34:08.131066+0000","last_fullsized":"2024-09-17T15:34:08.131066+0000","mapping_epoch":33,"log_start":"41'8200","ondisk_log_start":"41'8200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10981376,"num_objects":2681,"num_object_clones":0,"num_object_copies":5362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2681,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18289,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17971","reported_seq":18110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:08.631712+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:34:08.631712+0000","last_peered":"2024-09-17T15:34:08.631712+0000","last_clean":"2024-09-17T15:34:08.631712+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:34:08.631712+0000","last_undegraded":"2024-09-17T15:34:08.631712+0000","last_fullsized":"2024-09-17T15:34:08.631712+0000","mapping_epoch":32,"log_start":"41'7900","ondisk_log_start":"41'7900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10985955,"num_objects":2683,"num_object_clones":0,"num_object_copies":5366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2683,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14953,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18599","reported_seq":18900,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:08.671675+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:34:08.671675+0000","last_peered":"2024-09-17T15:34:08.671675+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:34:08.671675+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'8500","ondisk_log_start":"43'8500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10863075,"num_objects":2653,"num_object_clones":0,"num_object_copies":5306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2653,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15098,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1539},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18187","reported_seq":18505,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:08.644069+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:34:08.644069+0000","last_peered":"2024-09-17T15:34:08.644069+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:34:08.644069+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"41'8100","ondisk_log_start":"41'8100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10572742,"num_objects":2583,"num_object_clones":0,"num_object_copies":5166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2583,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14956,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1506},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18268","reported_seq":18406,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:08.536779+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:34:08.536779+0000","last_peered":"2024-09-17T15:34:08.536779+0000","last_clean":"2024-09-17T15:34:08.536779+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:34:08.536779+0000","last_undegraded":"2024-09-17T15:34:08.536779+0000","last_fullsized":"2024-09-17T15:34:08.536779+0000","mapping_epoch":32,"log_start":"41'8200","ondisk_log_start":"41'8200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10846208,"num_objects":2648,"num_object_clones":0,"num_object_copies":5296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15222,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223906+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.223906+0000","last_peered":"2024-09-17T15:33:11.223906+0000","last_clean":"2024-09-17T15:33:11.223906+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.223906+0000","last_undegraded":"2024-09-17T15:33:11.223906+0000","last_fullsized":"2024-09-17T15:33:11.223906+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18117","reported_seq":26409,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:08.044804+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:34:08.044804+0000","last_peered":"2024-09-17T15:34:08.044804+0000","last_clean":"2024-09-17T15:34:08.044804+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:34:08.044804+0000","last_undegraded":"2024-09-17T15:34:08.044804+0000","last_fullsized":"2024-09-17T15:34:08.044804+0000","mapping_epoch":42,"log_start":"41'8100","ondisk_log_start":"41'8100","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11059683,"num_objects":2701,"num_object_clones":0,"num_object_copies":5402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2701,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16117,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17936","reported_seq":18210,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:08.664939+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:34:08.664939+0000","last_peered":"2024-09-17T15:34:08.664939+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:34:08.664939+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'7900","ondisk_log_start":"41'7900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10674176,"num_objects":2606,"num_object_clones":0,"num_object_copies":5212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14532,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1337}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17897","reported_seq":18116,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:08.029374+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:34:08.029374+0000","last_peered":"2024-09-17T15:34:08.029374+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:34:08.029374+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'7800","ondisk_log_start":"41'7800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10740195,"num_objects":2623,"num_object_clones":0,"num_object_copies":5246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2623,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15792,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1410}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18122","reported_seq":26537,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:08.103267+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:34:08.103267+0000","last_peered":"2024-09-17T15:34:08.103267+0000","last_clean":"2024-09-17T15:34:08.103267+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:34:08.103267+0000","last_undegraded":"2024-09-17T15:34:08.103267+0000","last_fullsized":"2024-09-17T15:34:08.103267+0000","mapping_epoch":46,"log_start":"41'8100","ondisk_log_start":"41'8100","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10706944,"num_objects":2614,"num_object_clones":0,"num_object_copies":5228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15884,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18309","reported_seq":18616,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:08.664414+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:34:08.664414+0000","last_peered":"2024-09-17T15:34:08.664414+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:34:08.664414+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"41'8300","ondisk_log_start":"41'8300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10883072,"num_objects":2657,"num_object_clones":0,"num_object_copies":5314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14857,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1400}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18119","reported_seq":20852,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:08.628138+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:34:08.628138+0000","last_peered":"2024-09-17T15:34:08.628138+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:34:08.628138+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'8100","ondisk_log_start":"41'8100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10866688,"num_objects":2653,"num_object_clones":0,"num_object_copies":5306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2653,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14406,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1290,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-217}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18224","reported_seq":18532,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:08.582546+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:34:08.582546+0000","last_peered":"2024-09-17T15:34:08.582546+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:34:08.582546+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'8200","ondisk_log_start":"41'8200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10903552,"num_objects":2662,"num_object_clones":0,"num_object_copies":5324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15011,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1318}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18037","reported_seq":26421,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:08.122965+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:34:08.122965+0000","last_peered":"2024-09-17T15:34:08.122965+0000","last_clean":"2024-09-17T15:34:08.122965+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:34:08.122965+0000","last_undegraded":"2024-09-17T15:34:08.122965+0000","last_fullsized":"2024-09-17T15:34:08.122965+0000","mapping_epoch":44,"log_start":"41'8000","ondisk_log_start":"41'8000","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10630086,"num_objects":2597,"num_object_clones":0,"num_object_copies":5194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2597,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14479,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":173744886,"num_objects":42428,"num_object_clones":0,"num_object_copies":84856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35281,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42428,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":250559,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14179,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":550408192,"data_stored":550273593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160824,"ondisk_log_size":160824,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051693,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":557468,"kb_used_data":218956,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93814372,"statfs":{"total":96636764160,"available":96065916928,"internally_reserved":0,"allocated":224210944,"data_stored":223909346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":346615468},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,0,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724638,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":503100,"kb_used_data":128484,"kb_used_omap":4,"kb_used_meta":374587,"kb_avail":93868740,"statfs":{"total":96636764160,"available":96121589760,"internally_reserved":0,"allocated":131567616,"data_stored":131243876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":383577771},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,6,1,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247921,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":630204,"kb_used_data":193192,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93741636,"statfs":{"total":96636764160,"available":95991435264,"internally_reserved":0,"allocated":197828608,"data_stored":197527005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":447475368},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":197173248,"data_stored":197137118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":130289664,"data_stored":130231397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":222945280,"data_stored":222905078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:34:14.840 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:34:15.458 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:34:15.460 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:34:15.490 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2024-09-17T15:34:13.772153+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":163951369,"num_objects":40039,"num_object_clones":0,"num_object_copies":80078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35180,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40039,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":252952,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14281,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160918,"ondisk_log_size":160918,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1703692,"kb_used_data":521612,"kb_used_omap":13,"kb_used_meta":1182002,"kb_avail":281411828,"statfs":{"total":289910292480,"available":288165711872,"internally_reserved":0,"allocated":534130688,"data_stored":533203747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1210371071},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,9,13,7,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29794304,"num_objects":-7274,"num_object_clones":0,"num_object_copies":-14548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-316,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7274,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7274,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":316,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004540"},"pg_stats":[{"pgid":"2.f","version":"47'18187","reported_seq":18447,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:13.732123+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:34:13.732123+0000","last_peered":"2024-09-17T15:34:13.732123+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:34:13.732123+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'8100","ondisk_log_start":"41'8100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10080256,"num_objects":2461,"num_object_clones":0,"num_object_copies":4922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2461,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18187,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1523}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18364","reported_seq":18476,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:13.093295+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:34:13.093295+0000","last_peered":"2024-09-17T15:34:13.093295+0000","last_clean":"2024-09-17T15:34:13.093295+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:34:13.093295+0000","last_undegraded":"2024-09-17T15:34:13.093295+0000","last_fullsized":"2024-09-17T15:34:13.093295+0000","mapping_epoch":33,"log_start":"43'8300","ondisk_log_start":"43'8300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10519494,"num_objects":2570,"num_object_clones":0,"num_object_copies":5140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2570,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16721,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18344","reported_seq":18566,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:13.125305+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:34:13.125305+0000","last_peered":"2024-09-17T15:34:13.125305+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:34:13.125305+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'8300","ondisk_log_start":"43'8300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10604568,"num_objects":2590,"num_object_clones":0,"num_object_copies":5180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2590,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16501,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1535}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18440","reported_seq":18519,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:13.108815+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:34:13.108815+0000","last_peered":"2024-09-17T15:34:13.108815+0000","last_clean":"2024-09-17T15:34:13.108815+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:34:13.108815+0000","last_undegraded":"2024-09-17T15:34:13.108815+0000","last_fullsized":"2024-09-17T15:34:13.108815+0000","mapping_epoch":33,"log_start":"43'8400","ondisk_log_start":"43'8400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10362880,"num_objects":2530,"num_object_clones":0,"num_object_copies":5060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18440,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18107","reported_seq":18246,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:13.719967+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:34:13.719967+0000","last_peered":"2024-09-17T15:34:13.719967+0000","last_clean":"2024-09-17T15:34:13.719967+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:34:13.719967+0000","last_undegraded":"2024-09-17T15:34:13.719967+0000","last_fullsized":"2024-09-17T15:34:13.719967+0000","mapping_epoch":32,"log_start":"41'8100","ondisk_log_start":"41'8100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10428899,"num_objects":2547,"num_object_clones":0,"num_object_copies":5094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2547,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15089,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18731","reported_seq":19032,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:13.622749+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:34:13.622749+0000","last_peered":"2024-09-17T15:34:13.622749+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:34:13.622749+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'8700","ondisk_log_start":"43'8700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10322403,"num_objects":2521,"num_object_clones":0,"num_object_copies":5042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2521,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15230,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1671},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18337","reported_seq":18656,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:13.714528+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:34:13.714528+0000","last_peered":"2024-09-17T15:34:13.714528+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:34:13.714528+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"42'8300","ondisk_log_start":"42'8300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9958342,"num_objects":2433,"num_object_clones":0,"num_object_copies":4866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2433,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15106,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1656},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18415","reported_seq":18553,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:13.704736+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:34:13.704736+0000","last_peered":"2024-09-17T15:34:13.704736+0000","last_clean":"2024-09-17T15:34:13.704736+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:34:13.704736+0000","last_undegraded":"2024-09-17T15:34:13.704736+0000","last_fullsized":"2024-09-17T15:34:13.704736+0000","mapping_epoch":32,"log_start":"43'8400","ondisk_log_start":"43'8400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10244096,"num_objects":2501,"num_object_clones":0,"num_object_copies":5002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15369,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223906+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.223906+0000","last_peered":"2024-09-17T15:33:11.223906+0000","last_clean":"2024-09-17T15:33:11.223906+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.223906+0000","last_undegraded":"2024-09-17T15:33:11.223906+0000","last_fullsized":"2024-09-17T15:33:11.223906+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18269","reported_seq":26561,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:13.103335+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:34:13.103335+0000","last_peered":"2024-09-17T15:34:13.103335+0000","last_clean":"2024-09-17T15:34:13.103335+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:34:13.103335+0000","last_undegraded":"2024-09-17T15:34:13.103335+0000","last_fullsized":"2024-09-17T15:34:13.103335+0000","mapping_epoch":42,"log_start":"41'8200","ondisk_log_start":"41'8200","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10437091,"num_objects":2549,"num_object_clones":0,"num_object_copies":5098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2549,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16269,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18072","reported_seq":18345,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:13.621566+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:34:13.621566+0000","last_peered":"2024-09-17T15:34:13.621566+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:34:13.621566+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'8000","ondisk_log_start":"41'8000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10117120,"num_objects":2470,"num_object_clones":0,"num_object_copies":4940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2470,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14668,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1473}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18055","reported_seq":18274,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:13.158709+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:34:13.158709+0000","last_peered":"2024-09-17T15:34:13.158709+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:34:13.158709+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'8000","ondisk_log_start":"41'8000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10093027,"num_objects":2465,"num_object_clones":0,"num_object_copies":4930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2465,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15950,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1568}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18275","reported_seq":26690,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:13.120257+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:34:13.120257+0000","last_peered":"2024-09-17T15:34:13.120257+0000","last_clean":"2024-09-17T15:34:13.120257+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:34:13.120257+0000","last_undegraded":"2024-09-17T15:34:13.120257+0000","last_fullsized":"2024-09-17T15:34:13.120257+0000","mapping_epoch":46,"log_start":"41'8200","ondisk_log_start":"41'8200","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10080256,"num_objects":2461,"num_object_clones":0,"num_object_copies":4922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2461,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16037,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18484","reported_seq":18791,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:13.711759+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:34:13.711759+0000","last_peered":"2024-09-17T15:34:13.711759+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:34:13.711759+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'8400","ondisk_log_start":"43'8400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10166272,"num_objects":2482,"num_object_clones":0,"num_object_copies":4964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15032,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1575}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18285","reported_seq":21219,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:13.719632+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:34:13.719632+0000","last_peered":"2024-09-17T15:34:13.719632+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:34:13.719632+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'8200","ondisk_log_start":"41'8200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10186752,"num_objects":2487,"num_object_clones":0,"num_object_copies":4974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2769,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14572,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1391,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-282}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18388","reported_seq":18696,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:13.677068+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:34:13.677068+0000","last_peered":"2024-09-17T15:34:13.677068+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:34:13.677068+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'8300","ondisk_log_start":"41'8300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10231808,"num_objects":2498,"num_object_clones":0,"num_object_copies":4996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2498,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15175,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1482}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18162","reported_seq":26546,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:13.090577+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:34:13.090577+0000","last_peered":"2024-09-17T15:34:13.090577+0000","last_clean":"2024-09-17T15:34:13.090577+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:34:13.090577+0000","last_undegraded":"2024-09-17T15:34:13.090577+0000","last_fullsized":"2024-09-17T15:34:13.090577+0000","mapping_epoch":44,"log_start":"41'8100","ondisk_log_start":"41'8100","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10118086,"num_objects":2472,"num_object_clones":0,"num_object_copies":4944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14604,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":163951350,"num_objects":40037,"num_object_clones":0,"num_object_copies":80074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35180,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40037,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":252950,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14280,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":530874368,"data_stored":530739769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160915,"ondisk_log_size":160915,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051694,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":556832,"kb_used_data":214160,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93815008,"statfs":{"total":96636764160,"available":96066568192,"internally_reserved":0,"allocated":219299840,"data_stored":218998242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":350875308},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,4,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724639,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":536244,"kb_used_data":121368,"kb_used_omap":4,"kb_used_meta":414843,"kb_avail":93835596,"statfs":{"total":96636764160,"available":96087650304,"internally_reserved":0,"allocated":124280832,"data_stored":123957092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":424799915},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,6,0,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247922,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610616,"kb_used_data":186084,"kb_used_omap":4,"kb_used_meta":424507,"kb_avail":93761224,"statfs":{"total":96636764160,"available":96011493376,"internally_reserved":0,"allocated":190550016,"data_stored":190248413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":434695848},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,9,0,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":189865984,"data_stored":189829854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":122949632,"data_stored":122891365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":218058752,"data_stored":218018550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:34:15.492 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:34:16.075 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:34:16.078 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:34:16.098 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2024-09-17T15:34:15.773011+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":163951369,"num_objects":40039,"num_object_clones":0,"num_object_copies":80078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35180,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40039,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":252952,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14281,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160918,"ondisk_log_size":160918,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1703692,"kb_used_data":521612,"kb_used_omap":13,"kb_used_meta":1182002,"kb_avail":281411828,"statfs":{"total":289910292480,"available":288165711872,"internally_reserved":0,"allocated":534130688,"data_stored":533203747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1210371071},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,9,13,7,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19779584,"num_objects":-4829,"num_object_clones":0,"num_object_copies":-9658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4829,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004815"},"pg_stats":[{"pgid":"2.f","version":"47'18187","reported_seq":18447,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:13.732123+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:34:13.732123+0000","last_peered":"2024-09-17T15:34:13.732123+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:34:13.732123+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'8100","ondisk_log_start":"41'8100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10080256,"num_objects":2461,"num_object_clones":0,"num_object_copies":4922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2461,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18187,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1523}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18364","reported_seq":18476,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:13.093295+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:34:13.093295+0000","last_peered":"2024-09-17T15:34:13.093295+0000","last_clean":"2024-09-17T15:34:13.093295+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:34:13.093295+0000","last_undegraded":"2024-09-17T15:34:13.093295+0000","last_fullsized":"2024-09-17T15:34:13.093295+0000","mapping_epoch":33,"log_start":"43'8300","ondisk_log_start":"43'8300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10519494,"num_objects":2570,"num_object_clones":0,"num_object_copies":5140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2570,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16721,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18344","reported_seq":18566,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:13.125305+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:34:13.125305+0000","last_peered":"2024-09-17T15:34:13.125305+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:34:13.125305+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'8300","ondisk_log_start":"43'8300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10604568,"num_objects":2590,"num_object_clones":0,"num_object_copies":5180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2590,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16501,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1535}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18440","reported_seq":18519,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:13.108815+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:34:13.108815+0000","last_peered":"2024-09-17T15:34:13.108815+0000","last_clean":"2024-09-17T15:34:13.108815+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:34:13.108815+0000","last_undegraded":"2024-09-17T15:34:13.108815+0000","last_fullsized":"2024-09-17T15:34:13.108815+0000","mapping_epoch":33,"log_start":"43'8400","ondisk_log_start":"43'8400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10362880,"num_objects":2530,"num_object_clones":0,"num_object_copies":5060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18440,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18107","reported_seq":18246,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:13.719967+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:34:13.719967+0000","last_peered":"2024-09-17T15:34:13.719967+0000","last_clean":"2024-09-17T15:34:13.719967+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:34:13.719967+0000","last_undegraded":"2024-09-17T15:34:13.719967+0000","last_fullsized":"2024-09-17T15:34:13.719967+0000","mapping_epoch":32,"log_start":"41'8100","ondisk_log_start":"41'8100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10428899,"num_objects":2547,"num_object_clones":0,"num_object_copies":5094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2547,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15089,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18731","reported_seq":19032,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:13.622749+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:34:13.622749+0000","last_peered":"2024-09-17T15:34:13.622749+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:34:13.622749+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'8700","ondisk_log_start":"43'8700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10322403,"num_objects":2521,"num_object_clones":0,"num_object_copies":5042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2521,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15230,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1671},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18337","reported_seq":18656,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:13.714528+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:34:13.714528+0000","last_peered":"2024-09-17T15:34:13.714528+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:34:13.714528+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"42'8300","ondisk_log_start":"42'8300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9958342,"num_objects":2433,"num_object_clones":0,"num_object_copies":4866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2433,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15106,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1656},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18415","reported_seq":18553,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:13.704736+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:34:13.704736+0000","last_peered":"2024-09-17T15:34:13.704736+0000","last_clean":"2024-09-17T15:34:13.704736+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:34:13.704736+0000","last_undegraded":"2024-09-17T15:34:13.704736+0000","last_fullsized":"2024-09-17T15:34:13.704736+0000","mapping_epoch":32,"log_start":"43'8400","ondisk_log_start":"43'8400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10244096,"num_objects":2501,"num_object_clones":0,"num_object_copies":5002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15369,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223906+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.223906+0000","last_peered":"2024-09-17T15:33:11.223906+0000","last_clean":"2024-09-17T15:33:11.223906+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.223906+0000","last_undegraded":"2024-09-17T15:33:11.223906+0000","last_fullsized":"2024-09-17T15:33:11.223906+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18269","reported_seq":26561,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:13.103335+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:34:13.103335+0000","last_peered":"2024-09-17T15:34:13.103335+0000","last_clean":"2024-09-17T15:34:13.103335+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:34:13.103335+0000","last_undegraded":"2024-09-17T15:34:13.103335+0000","last_fullsized":"2024-09-17T15:34:13.103335+0000","mapping_epoch":42,"log_start":"41'8200","ondisk_log_start":"41'8200","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10437091,"num_objects":2549,"num_object_clones":0,"num_object_copies":5098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2549,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16269,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18072","reported_seq":18345,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:13.621566+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:34:13.621566+0000","last_peered":"2024-09-17T15:34:13.621566+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:34:13.621566+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'8000","ondisk_log_start":"41'8000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10117120,"num_objects":2470,"num_object_clones":0,"num_object_copies":4940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2470,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14668,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1473}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18055","reported_seq":18274,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:13.158709+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:34:13.158709+0000","last_peered":"2024-09-17T15:34:13.158709+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:34:13.158709+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'8000","ondisk_log_start":"41'8000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10093027,"num_objects":2465,"num_object_clones":0,"num_object_copies":4930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2465,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15950,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1568}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18275","reported_seq":26690,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:13.120257+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:34:13.120257+0000","last_peered":"2024-09-17T15:34:13.120257+0000","last_clean":"2024-09-17T15:34:13.120257+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:34:13.120257+0000","last_undegraded":"2024-09-17T15:34:13.120257+0000","last_fullsized":"2024-09-17T15:34:13.120257+0000","mapping_epoch":46,"log_start":"41'8200","ondisk_log_start":"41'8200","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10080256,"num_objects":2461,"num_object_clones":0,"num_object_copies":4922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2461,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16037,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18484","reported_seq":18791,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:13.711759+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:34:13.711759+0000","last_peered":"2024-09-17T15:34:13.711759+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:34:13.711759+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'8400","ondisk_log_start":"43'8400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10166272,"num_objects":2482,"num_object_clones":0,"num_object_copies":4964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15032,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1575}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18285","reported_seq":21219,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:13.719632+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:34:13.719632+0000","last_peered":"2024-09-17T15:34:13.719632+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:34:13.719632+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'8200","ondisk_log_start":"41'8200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10186752,"num_objects":2487,"num_object_clones":0,"num_object_copies":4974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2769,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14572,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1391,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-282}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18388","reported_seq":18696,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:13.677068+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:34:13.677068+0000","last_peered":"2024-09-17T15:34:13.677068+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:34:13.677068+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'8300","ondisk_log_start":"41'8300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10231808,"num_objects":2498,"num_object_clones":0,"num_object_copies":4996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2498,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15175,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1482}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18162","reported_seq":26546,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:13.090577+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:34:13.090577+0000","last_peered":"2024-09-17T15:34:13.090577+0000","last_clean":"2024-09-17T15:34:13.090577+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:34:13.090577+0000","last_undegraded":"2024-09-17T15:34:13.090577+0000","last_fullsized":"2024-09-17T15:34:13.090577+0000","mapping_epoch":44,"log_start":"41'8100","ondisk_log_start":"41'8100","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10118086,"num_objects":2472,"num_object_clones":0,"num_object_copies":4944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14604,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":163951350,"num_objects":40037,"num_object_clones":0,"num_object_copies":80074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35180,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40037,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":252950,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14280,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":530874368,"data_stored":530739769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160915,"ondisk_log_size":160915,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051694,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":556832,"kb_used_data":214160,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93815008,"statfs":{"total":96636764160,"available":96066568192,"internally_reserved":0,"allocated":219299840,"data_stored":218998242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":350875308},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,4,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724639,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":536244,"kb_used_data":121368,"kb_used_omap":4,"kb_used_meta":414843,"kb_avail":93835596,"statfs":{"total":96636764160,"available":96087650304,"internally_reserved":0,"allocated":124280832,"data_stored":123957092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":424799915},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,6,0,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247922,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610616,"kb_used_data":186084,"kb_used_omap":4,"kb_used_meta":424507,"kb_avail":93761224,"statfs":{"total":96636764160,"available":96011493376,"internally_reserved":0,"allocated":190550016,"data_stored":190248413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":434695848},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,9,0,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":189865984,"data_stored":189829854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":122949632,"data_stored":122891365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":218058752,"data_stored":218018550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:34:16.100 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:34:16.100 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'18187", 'reported_seq': 18447, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:13.732123+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:34:13.732123+0000', 'last_peered': '2024-09-17T15:34:13.732123+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:34:13.732123+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "41'8100", 'ondisk_log_start': "41'8100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10080256, 'num_objects': 2461, 'num_object_clones': 0, 'num_object_copies': 4922, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2461, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18187, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1523}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:16.100 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:34:16.100 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'18344", 'reported_seq': 18566, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:13.125305+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:34:13.125305+0000', 'last_peered': '2024-09-17T15:34:13.125305+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:34:13.125305+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "43'8300", 'ondisk_log_start': "43'8300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10604568, 'num_objects': 2590, 'num_object_clones': 0, 'num_object_copies': 5180, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2590, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16501, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1535}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:34:16.101 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:34:16.101 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'18731", 'reported_seq': 19032, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:13.622749+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:34:13.622749+0000', 'last_peered': '2024-09-17T15:34:13.622749+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:34:13.622749+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "43'8700", 'ondisk_log_start': "43'8700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10322403, 'num_objects': 2521, 'num_object_clones': 0, 'num_object_copies': 5042, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2521, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15230, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1671}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:16.101 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:34:16.101 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'18337", 'reported_seq': 18656, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:13.714528+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:34:13.714528+0000', 'last_peered': '2024-09-17T15:34:13.714528+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:34:13.714528+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "42'8300", 'ondisk_log_start': "42'8300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9958342, 'num_objects': 2433, 'num_object_clones': 0, 'num_object_copies': 4866, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4089, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2433, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 15106, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 85, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1656}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:16.101 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:34:16.101 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'18072", 'reported_seq': 18345, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:13.621566+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:34:13.621566+0000', 'last_peered': '2024-09-17T15:34:13.621566+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:34:13.621566+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "41'8000", 'ondisk_log_start': "41'8000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10117120, 'num_objects': 2470, 'num_object_clones': 0, 'num_object_copies': 4940, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2470, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14668, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1473}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:16.101 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:34:16.101 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'18055", 'reported_seq': 18274, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:13.158709+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:34:13.158709+0000', 'last_peered': '2024-09-17T15:34:13.158709+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:34:13.158709+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "41'8000", 'ondisk_log_start': "41'8000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10093027, 'num_objects': 2465, 'num_object_clones': 0, 'num_object_copies': 4930, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4034, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2465, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15950, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -1568}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:34:16.101 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:34:16.102 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'18484", 'reported_seq': 18791, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:13.711759+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:34:13.711759+0000', 'last_peered': '2024-09-17T15:34:13.711759+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:34:13.711759+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "43'8400", 'ondisk_log_start': "43'8400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10166272, 'num_objects': 2482, 'num_object_clones': 0, 'num_object_copies': 4964, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2482, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15032, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1575}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:16.102 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:34:16.102 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'18285", 'reported_seq': 21219, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:13.719632+0000', 'last_change': '2024-09-17T15:33:09.763500+0000', 'last_active': '2024-09-17T15:34:13.719632+0000', 'last_peered': '2024-09-17T15:34:13.719632+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:34:13.719632+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "41'8200", 'ondisk_log_start': "41'8200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10186752, 'num_objects': 2487, 'num_object_clones': 0, 'num_object_copies': 4974, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2769, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2487, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14572, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1391, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -282}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:16.102 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:34:16.102 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'18388", 'reported_seq': 18696, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:13.677068+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:34:13.677068+0000', 'last_peered': '2024-09-17T15:34:13.677068+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:34:13.677068+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "41'8300", 'ondisk_log_start': "41'8300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10231808, 'num_objects': 2498, 'num_object_clones': 0, 'num_object_copies': 4996, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2498, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15175, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1482}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:16.102 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:34:16.825 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:34:16.825 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35180/80078 objects degraded (43.932%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":593,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":40039,"data_bytes":163951369,"bytes_used":1744580608,"bytes_avail":288165711872,"bytes_total":289910292480,"degraded_objects":35180,"degraded_total":80078,"degraded_ratio":0.43932166138015433,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":208,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":402},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:34:16.850 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:34:16.851 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:34:17.411 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:34:17.412 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:34:17.434 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2024-09-17T15:34:15.773011+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":163951369,"num_objects":40039,"num_object_clones":0,"num_object_copies":80078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35180,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40039,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":252952,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14281,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160918,"ondisk_log_size":160918,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1703692,"kb_used_data":521612,"kb_used_omap":13,"kb_used_meta":1182002,"kb_avail":281411828,"statfs":{"total":289910292480,"available":288165711872,"internally_reserved":0,"allocated":534130688,"data_stored":533203747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1210371071},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,9,13,7,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19779584,"num_objects":-4829,"num_object_clones":0,"num_object_copies":-9658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4829,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004815"},"pg_stats":[{"pgid":"2.f","version":"47'18187","reported_seq":18447,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:13.732123+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:34:13.732123+0000","last_peered":"2024-09-17T15:34:13.732123+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:34:13.732123+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"41'8100","ondisk_log_start":"41'8100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10080256,"num_objects":2461,"num_object_clones":0,"num_object_copies":4922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2461,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18187,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1523}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18364","reported_seq":18476,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:13.093295+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:34:13.093295+0000","last_peered":"2024-09-17T15:34:13.093295+0000","last_clean":"2024-09-17T15:34:13.093295+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:34:13.093295+0000","last_undegraded":"2024-09-17T15:34:13.093295+0000","last_fullsized":"2024-09-17T15:34:13.093295+0000","mapping_epoch":33,"log_start":"43'8300","ondisk_log_start":"43'8300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10519494,"num_objects":2570,"num_object_clones":0,"num_object_copies":5140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2570,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16721,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18344","reported_seq":18566,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:13.125305+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:34:13.125305+0000","last_peered":"2024-09-17T15:34:13.125305+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:34:13.125305+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'8300","ondisk_log_start":"43'8300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10604568,"num_objects":2590,"num_object_clones":0,"num_object_copies":5180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2590,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16501,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1535}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18440","reported_seq":18519,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:13.108815+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:34:13.108815+0000","last_peered":"2024-09-17T15:34:13.108815+0000","last_clean":"2024-09-17T15:34:13.108815+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:34:13.108815+0000","last_undegraded":"2024-09-17T15:34:13.108815+0000","last_fullsized":"2024-09-17T15:34:13.108815+0000","mapping_epoch":33,"log_start":"43'8400","ondisk_log_start":"43'8400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10362880,"num_objects":2530,"num_object_clones":0,"num_object_copies":5060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18440,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18107","reported_seq":18246,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:13.719967+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:34:13.719967+0000","last_peered":"2024-09-17T15:34:13.719967+0000","last_clean":"2024-09-17T15:34:13.719967+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:34:13.719967+0000","last_undegraded":"2024-09-17T15:34:13.719967+0000","last_fullsized":"2024-09-17T15:34:13.719967+0000","mapping_epoch":32,"log_start":"41'8100","ondisk_log_start":"41'8100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10428899,"num_objects":2547,"num_object_clones":0,"num_object_copies":5094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2547,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15089,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18731","reported_seq":19032,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:13.622749+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:34:13.622749+0000","last_peered":"2024-09-17T15:34:13.622749+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:34:13.622749+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'8700","ondisk_log_start":"43'8700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10322403,"num_objects":2521,"num_object_clones":0,"num_object_copies":5042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2521,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15230,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1671},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18337","reported_seq":18656,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:13.714528+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:34:13.714528+0000","last_peered":"2024-09-17T15:34:13.714528+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:34:13.714528+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"42'8300","ondisk_log_start":"42'8300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9958342,"num_objects":2433,"num_object_clones":0,"num_object_copies":4866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2433,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15106,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1656},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18415","reported_seq":18553,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:13.704736+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:34:13.704736+0000","last_peered":"2024-09-17T15:34:13.704736+0000","last_clean":"2024-09-17T15:34:13.704736+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:34:13.704736+0000","last_undegraded":"2024-09-17T15:34:13.704736+0000","last_fullsized":"2024-09-17T15:34:13.704736+0000","mapping_epoch":32,"log_start":"43'8400","ondisk_log_start":"43'8400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10244096,"num_objects":2501,"num_object_clones":0,"num_object_copies":5002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15369,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223906+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.223906+0000","last_peered":"2024-09-17T15:33:11.223906+0000","last_clean":"2024-09-17T15:33:11.223906+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.223906+0000","last_undegraded":"2024-09-17T15:33:11.223906+0000","last_fullsized":"2024-09-17T15:33:11.223906+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18269","reported_seq":26561,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:13.103335+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:34:13.103335+0000","last_peered":"2024-09-17T15:34:13.103335+0000","last_clean":"2024-09-17T15:34:13.103335+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:34:13.103335+0000","last_undegraded":"2024-09-17T15:34:13.103335+0000","last_fullsized":"2024-09-17T15:34:13.103335+0000","mapping_epoch":42,"log_start":"41'8200","ondisk_log_start":"41'8200","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10437091,"num_objects":2549,"num_object_clones":0,"num_object_copies":5098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2549,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16269,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18072","reported_seq":18345,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:13.621566+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:34:13.621566+0000","last_peered":"2024-09-17T15:34:13.621566+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:34:13.621566+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"41'8000","ondisk_log_start":"41'8000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10117120,"num_objects":2470,"num_object_clones":0,"num_object_copies":4940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2470,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14668,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1473}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18055","reported_seq":18274,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:13.158709+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:34:13.158709+0000","last_peered":"2024-09-17T15:34:13.158709+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:34:13.158709+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"41'8000","ondisk_log_start":"41'8000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10093027,"num_objects":2465,"num_object_clones":0,"num_object_copies":4930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2465,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15950,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1568}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18275","reported_seq":26690,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:13.120257+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:34:13.120257+0000","last_peered":"2024-09-17T15:34:13.120257+0000","last_clean":"2024-09-17T15:34:13.120257+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:34:13.120257+0000","last_undegraded":"2024-09-17T15:34:13.120257+0000","last_fullsized":"2024-09-17T15:34:13.120257+0000","mapping_epoch":46,"log_start":"41'8200","ondisk_log_start":"41'8200","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10080256,"num_objects":2461,"num_object_clones":0,"num_object_copies":4922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2461,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16037,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18484","reported_seq":18791,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:13.711759+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:34:13.711759+0000","last_peered":"2024-09-17T15:34:13.711759+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:34:13.711759+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'8400","ondisk_log_start":"43'8400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10166272,"num_objects":2482,"num_object_clones":0,"num_object_copies":4964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15032,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1575}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18285","reported_seq":21219,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:13.719632+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:34:13.719632+0000","last_peered":"2024-09-17T15:34:13.719632+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:34:13.719632+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"41'8200","ondisk_log_start":"41'8200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10186752,"num_objects":2487,"num_object_clones":0,"num_object_copies":4974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2769,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14572,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1391,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-282}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18388","reported_seq":18696,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:13.677068+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:34:13.677068+0000","last_peered":"2024-09-17T15:34:13.677068+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:34:13.677068+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"41'8300","ondisk_log_start":"41'8300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10231808,"num_objects":2498,"num_object_clones":0,"num_object_copies":4996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2498,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15175,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1482}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18162","reported_seq":26546,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:13.090577+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:34:13.090577+0000","last_peered":"2024-09-17T15:34:13.090577+0000","last_clean":"2024-09-17T15:34:13.090577+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:34:13.090577+0000","last_undegraded":"2024-09-17T15:34:13.090577+0000","last_fullsized":"2024-09-17T15:34:13.090577+0000","mapping_epoch":44,"log_start":"41'8100","ondisk_log_start":"41'8100","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10118086,"num_objects":2472,"num_object_clones":0,"num_object_copies":4944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14604,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":163951350,"num_objects":40037,"num_object_clones":0,"num_object_copies":80074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35180,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40037,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":252950,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14280,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":530874368,"data_stored":530739769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160915,"ondisk_log_size":160915,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051694,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":556832,"kb_used_data":214160,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93815008,"statfs":{"total":96636764160,"available":96066568192,"internally_reserved":0,"allocated":219299840,"data_stored":218998242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":350875308},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,4,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724639,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":536244,"kb_used_data":121368,"kb_used_omap":4,"kb_used_meta":414843,"kb_avail":93835596,"statfs":{"total":96636764160,"available":96087650304,"internally_reserved":0,"allocated":124280832,"data_stored":123957092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":424799915},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,6,0,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247922,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610616,"kb_used_data":186084,"kb_used_omap":4,"kb_used_meta":424507,"kb_avail":93761224,"statfs":{"total":96636764160,"available":96011493376,"internally_reserved":0,"allocated":190550016,"data_stored":190248413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":434695848},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,9,0,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":189865984,"data_stored":189829854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":122949632,"data_stored":122891365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":218058752,"data_stored":218018550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:34:20.436 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:34:20.975 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:34:20.978 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:34:21.005 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":338,"stamp":"2024-09-17T15:34:19.774586+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":154288905,"num_objects":37680,"num_object_clones":0,"num_object_copies":75360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35046,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37680,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":255311,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14415,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160577,"ondisk_log_size":160577,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1701800,"kb_used_data":503068,"kb_used_omap":13,"kb_used_meta":1198642,"kb_avail":281413720,"statfs":{"total":289910292480,"available":288167649280,"internally_reserved":0,"allocated":515141632,"data_stored":514214691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1227410431},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,2,10,8,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29442048,"num_objects":-7188,"num_object_clones":0,"num_object_copies":-14376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-342,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7188,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004918"},"pg_stats":[{"pgid":"2.f","version":"47'18325","reported_seq":18585,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:18.642519+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:34:18.642519+0000","last_peered":"2024-09-17T15:34:18.642519+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:34:18.642519+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'8300","ondisk_log_start":"43'8300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9515008,"num_objects":2323,"num_object_clones":0,"num_object_copies":4646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1661}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18508","reported_seq":18620,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:18.183304+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:34:18.183304+0000","last_peered":"2024-09-17T15:34:18.183304+0000","last_clean":"2024-09-17T15:34:18.183304+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:34:18.183304+0000","last_undegraded":"2024-09-17T15:34:18.183304+0000","last_fullsized":"2024-09-17T15:34:18.183304+0000","mapping_epoch":33,"log_start":"43'8500","ondisk_log_start":"43'8500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9929670,"num_objects":2426,"num_object_clones":0,"num_object_copies":4852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2426,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16865,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18508","reported_seq":18730,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:18.173620+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:34:18.173620+0000","last_peered":"2024-09-17T15:34:18.173620+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:34:18.173620+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'8500","ondisk_log_start":"43'8500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9932824,"num_objects":2426,"num_object_clones":0,"num_object_copies":4852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2426,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16665,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1699}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18594","reported_seq":18673,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:18.104160+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:34:18.104160+0000","last_peered":"2024-09-17T15:34:18.104160+0000","last_clean":"2024-09-17T15:34:18.104160+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:34:18.104160+0000","last_undegraded":"2024-09-17T15:34:18.104160+0000","last_fullsized":"2024-09-17T15:34:18.104160+0000","mapping_epoch":33,"log_start":"43'8500","ondisk_log_start":"43'8500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9732096,"num_objects":2376,"num_object_clones":0,"num_object_copies":4752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2376,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18594,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18229","reported_seq":18368,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:18.687623+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:34:18.687623+0000","last_peered":"2024-09-17T15:34:18.687623+0000","last_clean":"2024-09-17T15:34:18.687623+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:34:18.687623+0000","last_undegraded":"2024-09-17T15:34:18.687623+0000","last_fullsized":"2024-09-17T15:34:18.687623+0000","mapping_epoch":32,"log_start":"43'8200","ondisk_log_start":"43'8200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9929187,"num_objects":2425,"num_object_clones":0,"num_object_copies":4850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2425,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15211,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18869","reported_seq":19170,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:18.701426+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:34:18.701426+0000","last_peered":"2024-09-17T15:34:18.701426+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:34:18.701426+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'8800","ondisk_log_start":"43'8800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9757155,"num_objects":2383,"num_object_clones":0,"num_object_copies":4766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2383,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15368,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1809},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18491","reported_seq":18807,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:18.733319+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:34:18.733319+0000","last_peered":"2024-09-17T15:34:18.733319+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:34:18.733319+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'8400","ondisk_log_start":"43'8400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9327558,"num_objects":2279,"num_object_clones":0,"num_object_copies":4558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2279,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15260,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1810},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18574","reported_seq":18712,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:18.681274+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:34:18.681274+0000","last_peered":"2024-09-17T15:34:18.681274+0000","last_clean":"2024-09-17T15:34:18.681274+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:34:18.681274+0000","last_undegraded":"2024-09-17T15:34:18.681274+0000","last_fullsized":"2024-09-17T15:34:18.681274+0000","mapping_epoch":32,"log_start":"43'8500","ondisk_log_start":"43'8500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9592832,"num_objects":2342,"num_object_clones":0,"num_object_copies":4684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15528,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223906+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.223906+0000","last_peered":"2024-09-17T15:33:11.223906+0000","last_clean":"2024-09-17T15:33:11.223906+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.223906+0000","last_undegraded":"2024-09-17T15:33:11.223906+0000","last_fullsized":"2024-09-17T15:33:11.223906+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18418","reported_seq":26710,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:18.113682+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:34:18.113682+0000","last_peered":"2024-09-17T15:34:18.113682+0000","last_clean":"2024-09-17T15:34:18.113682+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:34:18.113682+0000","last_undegraded":"2024-09-17T15:34:18.113682+0000","last_fullsized":"2024-09-17T15:34:18.113682+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9826787,"num_objects":2400,"num_object_clones":0,"num_object_copies":4800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2400,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16418,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18211","reported_seq":18485,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:18.703336+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:34:18.703336+0000","last_peered":"2024-09-17T15:34:18.703336+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:34:18.703336+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'8200","ondisk_log_start":"43'8200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9547776,"num_objects":2331,"num_object_clones":0,"num_object_copies":4662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14807,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1612}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18207","reported_seq":18425,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:18.163715+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:34:18.163715+0000","last_peered":"2024-09-17T15:34:18.163715+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:34:18.163715+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'8200","ondisk_log_start":"43'8200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9470435,"num_objects":2313,"num_object_clones":0,"num_object_copies":4626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2313,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16102,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1720}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18419","reported_seq":26834,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:18.066091+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:34:18.066091+0000","last_peered":"2024-09-17T15:34:18.066091+0000","last_clean":"2024-09-17T15:34:18.066091+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:34:18.066091+0000","last_undegraded":"2024-09-17T15:34:18.066091+0000","last_fullsized":"2024-09-17T15:34:18.066091+0000","mapping_epoch":46,"log_start":"43'8400","ondisk_log_start":"43'8400","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9490432,"num_objects":2317,"num_object_clones":0,"num_object_copies":4634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2317,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16181,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18638","reported_seq":18945,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:18.645515+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:34:18.645515+0000","last_peered":"2024-09-17T15:34:18.645515+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:34:18.645515+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'8600","ondisk_log_start":"43'8600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9535488,"num_objects":2328,"num_object_clones":0,"num_object_copies":4656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15186,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1729}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18433","reported_seq":21636,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:18.759908+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:34:18.759908+0000","last_peered":"2024-09-17T15:34:18.759908+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:34:18.759908+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"43'8400","ondisk_log_start":"43'8400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9580544,"num_objects":2339,"num_object_clones":0,"num_object_copies":4678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14720,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1525,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-296}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18540","reported_seq":18848,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:18.765953+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:34:18.765953+0000","last_peered":"2024-09-17T15:34:18.765953+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:34:18.765953+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'8500","ondisk_log_start":"43'8500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9609216,"num_objects":2346,"num_object_clones":0,"num_object_copies":4692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15327,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1634}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18310","reported_seq":26694,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:18.153262+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:34:18.153262+0000","last_peered":"2024-09-17T15:34:18.153262+0000","last_clean":"2024-09-17T15:34:18.153262+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:34:18.153262+0000","last_undegraded":"2024-09-17T15:34:18.153262+0000","last_fullsized":"2024-09-17T15:34:18.153262+0000","mapping_epoch":44,"log_start":"43'8300","ondisk_log_start":"43'8300","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9511878,"num_objects":2324,"num_object_clones":0,"num_object_copies":4648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14752,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":154288886,"num_objects":37678,"num_object_clones":0,"num_object_copies":75356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35046,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37678,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":255309,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14414,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":511610880,"data_stored":511476281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160574,"ondisk_log_size":160574,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051695,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":556428,"kb_used_data":209596,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93815412,"statfs":{"total":96636764160,"available":96066981888,"internally_reserved":0,"allocated":214626304,"data_stored":214324706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":355135148},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,2,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724640,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537480,"kb_used_data":114272,"kb_used_omap":4,"kb_used_meta":423163,"kb_avail":93834360,"statfs":{"total":96636764160,"available":96086384640,"internally_reserved":0,"allocated":117014528,"data_stored":116690788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":433319595},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,1,4,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247923,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":607892,"kb_used_data":179200,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93763948,"statfs":{"total":96636764160,"available":96014282752,"internally_reserved":0,"allocated":183500800,"data_stored":183199197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":438955688},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,7,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":182751232,"data_stored":182715102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":115658752,"data_stored":115600485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":213200896,"data_stored":213160694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:34:21.007 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:34:21.590 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:34:21.593 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:34:21.612 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":338,"stamp":"2024-09-17T15:34:19.774586+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":154288905,"num_objects":37680,"num_object_clones":0,"num_object_copies":75360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35046,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37680,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":255311,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14415,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160577,"ondisk_log_size":160577,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1701800,"kb_used_data":503068,"kb_used_omap":13,"kb_used_meta":1198642,"kb_avail":281413720,"statfs":{"total":289910292480,"available":288167649280,"internally_reserved":0,"allocated":515141632,"data_stored":514214691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1227410431},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,2,10,8,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29442048,"num_objects":-7188,"num_object_clones":0,"num_object_copies":-14376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-342,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7188,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004918"},"pg_stats":[{"pgid":"2.f","version":"47'18325","reported_seq":18585,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:18.642519+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:34:18.642519+0000","last_peered":"2024-09-17T15:34:18.642519+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:34:18.642519+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'8300","ondisk_log_start":"43'8300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9515008,"num_objects":2323,"num_object_clones":0,"num_object_copies":4646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1661}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18508","reported_seq":18620,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:18.183304+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:34:18.183304+0000","last_peered":"2024-09-17T15:34:18.183304+0000","last_clean":"2024-09-17T15:34:18.183304+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:34:18.183304+0000","last_undegraded":"2024-09-17T15:34:18.183304+0000","last_fullsized":"2024-09-17T15:34:18.183304+0000","mapping_epoch":33,"log_start":"43'8500","ondisk_log_start":"43'8500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9929670,"num_objects":2426,"num_object_clones":0,"num_object_copies":4852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2426,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16865,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18508","reported_seq":18730,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:18.173620+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:34:18.173620+0000","last_peered":"2024-09-17T15:34:18.173620+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:34:18.173620+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'8500","ondisk_log_start":"43'8500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9932824,"num_objects":2426,"num_object_clones":0,"num_object_copies":4852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2426,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16665,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1699}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18594","reported_seq":18673,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:18.104160+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:34:18.104160+0000","last_peered":"2024-09-17T15:34:18.104160+0000","last_clean":"2024-09-17T15:34:18.104160+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:34:18.104160+0000","last_undegraded":"2024-09-17T15:34:18.104160+0000","last_fullsized":"2024-09-17T15:34:18.104160+0000","mapping_epoch":33,"log_start":"43'8500","ondisk_log_start":"43'8500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9732096,"num_objects":2376,"num_object_clones":0,"num_object_copies":4752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2376,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18594,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18229","reported_seq":18368,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:18.687623+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:34:18.687623+0000","last_peered":"2024-09-17T15:34:18.687623+0000","last_clean":"2024-09-17T15:34:18.687623+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:34:18.687623+0000","last_undegraded":"2024-09-17T15:34:18.687623+0000","last_fullsized":"2024-09-17T15:34:18.687623+0000","mapping_epoch":32,"log_start":"43'8200","ondisk_log_start":"43'8200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9929187,"num_objects":2425,"num_object_clones":0,"num_object_copies":4850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2425,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15211,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18869","reported_seq":19170,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:18.701426+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:34:18.701426+0000","last_peered":"2024-09-17T15:34:18.701426+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:34:18.701426+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'8800","ondisk_log_start":"43'8800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9757155,"num_objects":2383,"num_object_clones":0,"num_object_copies":4766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2383,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15368,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1809},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18491","reported_seq":18807,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:18.733319+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:34:18.733319+0000","last_peered":"2024-09-17T15:34:18.733319+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:34:18.733319+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'8400","ondisk_log_start":"43'8400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9327558,"num_objects":2279,"num_object_clones":0,"num_object_copies":4558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2279,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15260,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1810},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18574","reported_seq":18712,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:18.681274+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:34:18.681274+0000","last_peered":"2024-09-17T15:34:18.681274+0000","last_clean":"2024-09-17T15:34:18.681274+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:34:18.681274+0000","last_undegraded":"2024-09-17T15:34:18.681274+0000","last_fullsized":"2024-09-17T15:34:18.681274+0000","mapping_epoch":32,"log_start":"43'8500","ondisk_log_start":"43'8500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9592832,"num_objects":2342,"num_object_clones":0,"num_object_copies":4684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15528,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223906+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.223906+0000","last_peered":"2024-09-17T15:33:11.223906+0000","last_clean":"2024-09-17T15:33:11.223906+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.223906+0000","last_undegraded":"2024-09-17T15:33:11.223906+0000","last_fullsized":"2024-09-17T15:33:11.223906+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18418","reported_seq":26710,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:18.113682+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:34:18.113682+0000","last_peered":"2024-09-17T15:34:18.113682+0000","last_clean":"2024-09-17T15:34:18.113682+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:34:18.113682+0000","last_undegraded":"2024-09-17T15:34:18.113682+0000","last_fullsized":"2024-09-17T15:34:18.113682+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9826787,"num_objects":2400,"num_object_clones":0,"num_object_copies":4800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2400,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16418,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18211","reported_seq":18485,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:18.703336+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:34:18.703336+0000","last_peered":"2024-09-17T15:34:18.703336+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:34:18.703336+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'8200","ondisk_log_start":"43'8200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9547776,"num_objects":2331,"num_object_clones":0,"num_object_copies":4662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14807,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1612}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18207","reported_seq":18425,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:18.163715+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:34:18.163715+0000","last_peered":"2024-09-17T15:34:18.163715+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:34:18.163715+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'8200","ondisk_log_start":"43'8200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9470435,"num_objects":2313,"num_object_clones":0,"num_object_copies":4626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2313,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16102,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1720}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18419","reported_seq":26834,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:18.066091+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:34:18.066091+0000","last_peered":"2024-09-17T15:34:18.066091+0000","last_clean":"2024-09-17T15:34:18.066091+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:34:18.066091+0000","last_undegraded":"2024-09-17T15:34:18.066091+0000","last_fullsized":"2024-09-17T15:34:18.066091+0000","mapping_epoch":46,"log_start":"43'8400","ondisk_log_start":"43'8400","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9490432,"num_objects":2317,"num_object_clones":0,"num_object_copies":4634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2317,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16181,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18638","reported_seq":18945,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:18.645515+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:34:18.645515+0000","last_peered":"2024-09-17T15:34:18.645515+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:34:18.645515+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'8600","ondisk_log_start":"43'8600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9535488,"num_objects":2328,"num_object_clones":0,"num_object_copies":4656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15186,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1729}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18433","reported_seq":21636,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:18.759908+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:34:18.759908+0000","last_peered":"2024-09-17T15:34:18.759908+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:34:18.759908+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"43'8400","ondisk_log_start":"43'8400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9580544,"num_objects":2339,"num_object_clones":0,"num_object_copies":4678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14720,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1525,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-296}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18540","reported_seq":18848,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:18.765953+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:34:18.765953+0000","last_peered":"2024-09-17T15:34:18.765953+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:34:18.765953+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'8500","ondisk_log_start":"43'8500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9609216,"num_objects":2346,"num_object_clones":0,"num_object_copies":4692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15327,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1634}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18310","reported_seq":26694,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:18.153262+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:34:18.153262+0000","last_peered":"2024-09-17T15:34:18.153262+0000","last_clean":"2024-09-17T15:34:18.153262+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:34:18.153262+0000","last_undegraded":"2024-09-17T15:34:18.153262+0000","last_fullsized":"2024-09-17T15:34:18.153262+0000","mapping_epoch":44,"log_start":"43'8300","ondisk_log_start":"43'8300","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9511878,"num_objects":2324,"num_object_clones":0,"num_object_copies":4648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14752,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":154288886,"num_objects":37678,"num_object_clones":0,"num_object_copies":75356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35046,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37678,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":255309,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14414,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":511610880,"data_stored":511476281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160574,"ondisk_log_size":160574,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051695,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":556428,"kb_used_data":209596,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93815412,"statfs":{"total":96636764160,"available":96066981888,"internally_reserved":0,"allocated":214626304,"data_stored":214324706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":355135148},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,2,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724640,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537480,"kb_used_data":114272,"kb_used_omap":4,"kb_used_meta":423163,"kb_avail":93834360,"statfs":{"total":96636764160,"available":96086384640,"internally_reserved":0,"allocated":117014528,"data_stored":116690788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":433319595},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,1,4,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247923,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":607892,"kb_used_data":179200,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93763948,"statfs":{"total":96636764160,"available":96014282752,"internally_reserved":0,"allocated":183500800,"data_stored":183199197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":438955688},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,7,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":182751232,"data_stored":182715102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":115658752,"data_stored":115600485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":213200896,"data_stored":213160694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:34:21.613 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:34:21.613 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'18325", 'reported_seq': 18585, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:18.642519+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:34:18.642519+0000', 'last_peered': '2024-09-17T15:34:18.642519+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:34:18.642519+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'8300", 'ondisk_log_start': "43'8300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9515008, 'num_objects': 2323, 'num_object_clones': 0, 'num_object_copies': 4646, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2323, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18325, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1661}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:21.614 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:34:21.614 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'18508", 'reported_seq': 18730, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:18.173620+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:34:18.173620+0000', 'last_peered': '2024-09-17T15:34:18.173620+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:34:18.173620+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "43'8500", 'ondisk_log_start': "43'8500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9932824, 'num_objects': 2426, 'num_object_clones': 0, 'num_object_copies': 4852, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2426, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16665, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1699}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:34:21.614 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:34:21.614 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'18869", 'reported_seq': 19170, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:18.701426+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:34:18.701426+0000', 'last_peered': '2024-09-17T15:34:18.701426+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:34:18.701426+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "43'8800", 'ondisk_log_start': "43'8800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9757155, 'num_objects': 2383, 'num_object_clones': 0, 'num_object_copies': 4766, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2383, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15368, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1809}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:21.614 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:34:21.615 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'18491", 'reported_seq': 18807, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:18.733319+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:34:18.733319+0000', 'last_peered': '2024-09-17T15:34:18.733319+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:34:18.733319+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "43'8400", 'ondisk_log_start': "43'8400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9327558, 'num_objects': 2279, 'num_object_clones': 0, 'num_object_copies': 4558, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4089, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2279, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 15260, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 85, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1810}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:21.615 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:34:21.615 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'18211", 'reported_seq': 18485, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:18.703336+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:34:18.703336+0000', 'last_peered': '2024-09-17T15:34:18.703336+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:34:18.703336+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'8200", 'ondisk_log_start': "43'8200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9547776, 'num_objects': 2331, 'num_object_clones': 0, 'num_object_copies': 4662, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2331, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14807, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1612}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:21.615 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:34:21.615 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'18207", 'reported_seq': 18425, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:18.163715+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:34:18.163715+0000', 'last_peered': '2024-09-17T15:34:18.163715+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:34:18.163715+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "43'8200", 'ondisk_log_start': "43'8200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9470435, 'num_objects': 2313, 'num_object_clones': 0, 'num_object_copies': 4626, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4034, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2313, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16102, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -1720}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:34:21.616 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:34:21.616 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'18638", 'reported_seq': 18945, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:18.645515+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:34:18.645515+0000', 'last_peered': '2024-09-17T15:34:18.645515+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:34:18.645515+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "43'8600", 'ondisk_log_start': "43'8600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9535488, 'num_objects': 2328, 'num_object_clones': 0, 'num_object_copies': 4656, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2328, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15186, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1729}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:21.616 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:34:21.616 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'18433", 'reported_seq': 21636, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:18.759908+0000', 'last_change': '2024-09-17T15:33:09.763500+0000', 'last_active': '2024-09-17T15:34:18.759908+0000', 'last_peered': '2024-09-17T15:34:18.759908+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:34:18.759908+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "43'8400", 'ondisk_log_start': "43'8400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9580544, 'num_objects': 2339, 'num_object_clones': 0, 'num_object_copies': 4678, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2635, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2339, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14720, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1525, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -296}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:21.616 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:34:21.616 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'18540", 'reported_seq': 18848, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:18.765953+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:34:18.765953+0000', 'last_peered': '2024-09-17T15:34:18.765953+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:34:18.765953+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'8500", 'ondisk_log_start': "43'8500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9609216, 'num_objects': 2346, 'num_object_clones': 0, 'num_object_copies': 4692, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2346, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15327, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1634}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:21.616 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:34:22.299 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:34:22.299 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35046/75360 objects degraded (46.505%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":599,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":37680,"data_bytes":154288905,"bytes_used":1742643200,"bytes_avail":288167649280,"bytes_total":289910292480,"degraded_objects":35046,"degraded_total":75360,"degraded_ratio":0.46504777070063696,"recovering_objects_per_sec":19,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":235,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":395},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:34:22.322 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:34:22.322 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:34:22.865 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:34:22.867 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:34:22.894 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":339,"stamp":"2024-09-17T15:34:21.777119+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":154288905,"num_objects":37680,"num_object_clones":0,"num_object_copies":75360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35046,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37680,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":255311,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14415,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160577,"ondisk_log_size":160577,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1701800,"kb_used_data":503068,"kb_used_omap":13,"kb_used_meta":1198642,"kb_avail":281413720,"statfs":{"total":289910292480,"available":288167649280,"internally_reserved":0,"allocated":515141632,"data_stored":514214691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1227410431},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,2,10,8,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19456000,"num_objects":-4750,"num_object_clones":0,"num_object_copies":-9500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-235,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4750,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4750,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":235,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006634"},"pg_stats":[{"pgid":"2.f","version":"47'18325","reported_seq":18585,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:18.642519+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:34:18.642519+0000","last_peered":"2024-09-17T15:34:18.642519+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:34:18.642519+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'8300","ondisk_log_start":"43'8300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9515008,"num_objects":2323,"num_object_clones":0,"num_object_copies":4646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1661}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18508","reported_seq":18620,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:18.183304+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:34:18.183304+0000","last_peered":"2024-09-17T15:34:18.183304+0000","last_clean":"2024-09-17T15:34:18.183304+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:34:18.183304+0000","last_undegraded":"2024-09-17T15:34:18.183304+0000","last_fullsized":"2024-09-17T15:34:18.183304+0000","mapping_epoch":33,"log_start":"43'8500","ondisk_log_start":"43'8500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9929670,"num_objects":2426,"num_object_clones":0,"num_object_copies":4852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2426,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16865,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18508","reported_seq":18730,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:18.173620+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:34:18.173620+0000","last_peered":"2024-09-17T15:34:18.173620+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:34:18.173620+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'8500","ondisk_log_start":"43'8500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9932824,"num_objects":2426,"num_object_clones":0,"num_object_copies":4852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2426,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16665,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1699}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18594","reported_seq":18673,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:18.104160+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:34:18.104160+0000","last_peered":"2024-09-17T15:34:18.104160+0000","last_clean":"2024-09-17T15:34:18.104160+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:34:18.104160+0000","last_undegraded":"2024-09-17T15:34:18.104160+0000","last_fullsized":"2024-09-17T15:34:18.104160+0000","mapping_epoch":33,"log_start":"43'8500","ondisk_log_start":"43'8500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9732096,"num_objects":2376,"num_object_clones":0,"num_object_copies":4752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2376,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18594,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18229","reported_seq":18368,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:18.687623+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:34:18.687623+0000","last_peered":"2024-09-17T15:34:18.687623+0000","last_clean":"2024-09-17T15:34:18.687623+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:34:18.687623+0000","last_undegraded":"2024-09-17T15:34:18.687623+0000","last_fullsized":"2024-09-17T15:34:18.687623+0000","mapping_epoch":32,"log_start":"43'8200","ondisk_log_start":"43'8200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9929187,"num_objects":2425,"num_object_clones":0,"num_object_copies":4850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2425,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15211,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18869","reported_seq":19170,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:18.701426+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:34:18.701426+0000","last_peered":"2024-09-17T15:34:18.701426+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:34:18.701426+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'8800","ondisk_log_start":"43'8800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9757155,"num_objects":2383,"num_object_clones":0,"num_object_copies":4766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2383,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15368,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1809},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18491","reported_seq":18807,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:18.733319+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:34:18.733319+0000","last_peered":"2024-09-17T15:34:18.733319+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:34:18.733319+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'8400","ondisk_log_start":"43'8400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9327558,"num_objects":2279,"num_object_clones":0,"num_object_copies":4558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2279,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15260,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1810},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18574","reported_seq":18712,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:18.681274+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:34:18.681274+0000","last_peered":"2024-09-17T15:34:18.681274+0000","last_clean":"2024-09-17T15:34:18.681274+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:34:18.681274+0000","last_undegraded":"2024-09-17T15:34:18.681274+0000","last_fullsized":"2024-09-17T15:34:18.681274+0000","mapping_epoch":32,"log_start":"43'8500","ondisk_log_start":"43'8500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9592832,"num_objects":2342,"num_object_clones":0,"num_object_copies":4684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15528,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223906+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.223906+0000","last_peered":"2024-09-17T15:33:11.223906+0000","last_clean":"2024-09-17T15:33:11.223906+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.223906+0000","last_undegraded":"2024-09-17T15:33:11.223906+0000","last_fullsized":"2024-09-17T15:33:11.223906+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18418","reported_seq":26710,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:18.113682+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:34:18.113682+0000","last_peered":"2024-09-17T15:34:18.113682+0000","last_clean":"2024-09-17T15:34:18.113682+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:34:18.113682+0000","last_undegraded":"2024-09-17T15:34:18.113682+0000","last_fullsized":"2024-09-17T15:34:18.113682+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9826787,"num_objects":2400,"num_object_clones":0,"num_object_copies":4800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2400,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16418,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18211","reported_seq":18485,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:18.703336+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:34:18.703336+0000","last_peered":"2024-09-17T15:34:18.703336+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:34:18.703336+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'8200","ondisk_log_start":"43'8200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9547776,"num_objects":2331,"num_object_clones":0,"num_object_copies":4662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14807,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1612}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18207","reported_seq":18425,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:18.163715+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:34:18.163715+0000","last_peered":"2024-09-17T15:34:18.163715+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:34:18.163715+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'8200","ondisk_log_start":"43'8200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9470435,"num_objects":2313,"num_object_clones":0,"num_object_copies":4626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2313,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16102,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1720}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18419","reported_seq":26834,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:18.066091+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:34:18.066091+0000","last_peered":"2024-09-17T15:34:18.066091+0000","last_clean":"2024-09-17T15:34:18.066091+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:34:18.066091+0000","last_undegraded":"2024-09-17T15:34:18.066091+0000","last_fullsized":"2024-09-17T15:34:18.066091+0000","mapping_epoch":46,"log_start":"43'8400","ondisk_log_start":"43'8400","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9490432,"num_objects":2317,"num_object_clones":0,"num_object_copies":4634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2317,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16181,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18638","reported_seq":18945,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:18.645515+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:34:18.645515+0000","last_peered":"2024-09-17T15:34:18.645515+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:34:18.645515+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'8600","ondisk_log_start":"43'8600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9535488,"num_objects":2328,"num_object_clones":0,"num_object_copies":4656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15186,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1729}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18433","reported_seq":21636,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:18.759908+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:34:18.759908+0000","last_peered":"2024-09-17T15:34:18.759908+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:34:18.759908+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"43'8400","ondisk_log_start":"43'8400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9580544,"num_objects":2339,"num_object_clones":0,"num_object_copies":4678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14720,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1525,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-296}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18540","reported_seq":18848,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:18.765953+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:34:18.765953+0000","last_peered":"2024-09-17T15:34:18.765953+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:34:18.765953+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'8500","ondisk_log_start":"43'8500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9609216,"num_objects":2346,"num_object_clones":0,"num_object_copies":4692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15327,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1634}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18310","reported_seq":26694,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:18.153262+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:34:18.153262+0000","last_peered":"2024-09-17T15:34:18.153262+0000","last_clean":"2024-09-17T15:34:18.153262+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:34:18.153262+0000","last_undegraded":"2024-09-17T15:34:18.153262+0000","last_fullsized":"2024-09-17T15:34:18.153262+0000","mapping_epoch":44,"log_start":"43'8300","ondisk_log_start":"43'8300","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9511878,"num_objects":2324,"num_object_clones":0,"num_object_copies":4648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14752,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":154288886,"num_objects":37678,"num_object_clones":0,"num_object_copies":75356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35046,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37678,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":255309,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14414,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":511610880,"data_stored":511476281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160574,"ondisk_log_size":160574,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051695,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":556428,"kb_used_data":209596,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93815412,"statfs":{"total":96636764160,"available":96066981888,"internally_reserved":0,"allocated":214626304,"data_stored":214324706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":355135148},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,2,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724640,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537480,"kb_used_data":114272,"kb_used_omap":4,"kb_used_meta":423163,"kb_avail":93834360,"statfs":{"total":96636764160,"available":96086384640,"internally_reserved":0,"allocated":117014528,"data_stored":116690788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":433319595},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,1,4,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247923,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":607892,"kb_used_data":179200,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93763948,"statfs":{"total":96636764160,"available":96014282752,"internally_reserved":0,"allocated":183500800,"data_stored":183199197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":438955688},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,7,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":182751232,"data_stored":182715102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":115658752,"data_stored":115600485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":213200896,"data_stored":213160694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:34:25.897 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:34:26.473 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:34:26.476 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:34:26.499 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":341,"stamp":"2024-09-17T15:34:25.778562+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":144278281,"num_objects":35236,"num_object_clones":0,"num_object_copies":70472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34937,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35236,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":257755,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14524,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160821,"ondisk_log_size":160821,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1700720,"kb_used_data":482540,"kb_used_omap":13,"kb_used_meta":1218098,"kb_avail":281414800,"statfs":{"total":289910292480,"available":288168755200,"internally_reserved":0,"allocated":494120960,"data_stored":493194019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1247333375},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,2,14,5,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19673088,"num_objects":-4803,"num_object_clones":0,"num_object_copies":-9606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-243,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4803,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006408"},"pg_stats":[{"pgid":"2.f","version":"47'18473","reported_seq":18733,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:23.777262+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:34:23.777262+0000","last_peered":"2024-09-17T15:34:23.777262+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:34:23.777262+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'8400","ondisk_log_start":"43'8400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8908800,"num_objects":2175,"num_object_clones":0,"num_object_copies":4350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18473,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1809}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18651","reported_seq":18762,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:23.151037+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:34:23.151037+0000","last_peered":"2024-09-17T15:34:23.151037+0000","last_clean":"2024-09-17T15:34:23.151037+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:34:23.151037+0000","last_undegraded":"2024-09-17T15:34:23.151037+0000","last_fullsized":"2024-09-17T15:34:23.151037+0000","mapping_epoch":33,"log_start":"43'8600","ondisk_log_start":"43'8600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9343942,"num_objects":2283,"num_object_clones":0,"num_object_copies":4566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2283,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17008,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18651","reported_seq":18873,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:23.137711+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:34:23.137711+0000","last_peered":"2024-09-17T15:34:23.137711+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:34:23.137711+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'8600","ondisk_log_start":"43'8600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9347096,"num_objects":2283,"num_object_clones":0,"num_object_copies":4566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2283,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16808,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1842}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18750","reported_seq":18830,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:23.116379+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:34:23.116379+0000","last_peered":"2024-09-17T15:34:23.116379+0000","last_clean":"2024-09-17T15:34:23.116379+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:34:23.116379+0000","last_undegraded":"2024-09-17T15:34:23.116379+0000","last_fullsized":"2024-09-17T15:34:23.116379+0000","mapping_epoch":33,"log_start":"43'8700","ondisk_log_start":"43'8700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9093120,"num_objects":2220,"num_object_clones":0,"num_object_copies":4440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18750,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18412","reported_seq":18551,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:23.768561+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:34:23.768561+0000","last_peered":"2024-09-17T15:34:23.768561+0000","last_clean":"2024-09-17T15:34:23.768561+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:34:23.768561+0000","last_undegraded":"2024-09-17T15:34:23.768561+0000","last_fullsized":"2024-09-17T15:34:23.768561+0000","mapping_epoch":32,"log_start":"43'8400","ondisk_log_start":"43'8400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9179619,"num_objects":2242,"num_object_clones":0,"num_object_copies":4484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2242,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15394,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19013","reported_seq":19314,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:23.645542+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:34:23.645542+0000","last_peered":"2024-09-17T15:34:23.645542+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:34:23.645542+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'9000","ondisk_log_start":"43'9000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9167331,"num_objects":2239,"num_object_clones":0,"num_object_copies":4478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2239,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15512,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1953},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18645","reported_seq":18963,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:23.785162+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:34:23.785162+0000","last_peered":"2024-09-17T15:34:23.785162+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:34:23.785162+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'8600","ondisk_log_start":"43'8600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8696774,"num_objects":2125,"num_object_clones":0,"num_object_copies":4250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2125,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15414,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1964},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18734","reported_seq":18872,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:23.750499+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:34:23.750499+0000","last_peered":"2024-09-17T15:34:23.750499+0000","last_clean":"2024-09-17T15:34:23.750499+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:34:23.750499+0000","last_undegraded":"2024-09-17T15:34:23.750499+0000","last_fullsized":"2024-09-17T15:34:23.750499+0000","mapping_epoch":32,"log_start":"43'8700","ondisk_log_start":"43'8700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8937472,"num_objects":2182,"num_object_clones":0,"num_object_copies":4364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2182,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15688,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.224142+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.224142+0000","last_peered":"2024-09-17T15:33:11.224142+0000","last_clean":"2024-09-17T15:33:11.224142+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.224142+0000","last_undegraded":"2024-09-17T15:33:11.224142+0000","last_fullsized":"2024-09-17T15:33:11.224142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18573","reported_seq":26865,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:23.210140+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:34:23.210140+0000","last_peered":"2024-09-17T15:34:23.210140+0000","last_clean":"2024-09-17T15:34:23.210140+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:34:23.210140+0000","last_undegraded":"2024-09-17T15:34:23.210140+0000","last_fullsized":"2024-09-17T15:34:23.210140+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9191907,"num_objects":2245,"num_object_clones":0,"num_object_copies":4490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2245,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16573,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18360","reported_seq":18634,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:23.754391+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:34:23.754391+0000","last_peered":"2024-09-17T15:34:23.754391+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:34:23.754391+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'8300","ondisk_log_start":"43'8300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8937472,"num_objects":2182,"num_object_clones":0,"num_object_copies":4364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2182,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14956,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1761}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18344","reported_seq":18563,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:23.113544+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:34:23.113544+0000","last_peered":"2024-09-17T15:34:23.113544+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:34:23.113544+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'8300","ondisk_log_start":"43'8300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8909283,"num_objects":2176,"num_object_clones":0,"num_object_copies":4352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2176,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16239,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1857}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18565","reported_seq":26980,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:23.172244+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:34:23.172244+0000","last_peered":"2024-09-17T15:34:23.172244+0000","last_clean":"2024-09-17T15:34:23.172244+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:34:23.172244+0000","last_undegraded":"2024-09-17T15:34:23.172244+0000","last_fullsized":"2024-09-17T15:34:23.172244+0000","mapping_epoch":46,"log_start":"43'8500","ondisk_log_start":"43'8500","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8892416,"num_objects":2171,"num_object_clones":0,"num_object_copies":4342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2171,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16327,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18776","reported_seq":19081,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:23.749622+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:34:23.749622+0000","last_peered":"2024-09-17T15:34:23.749622+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:34:23.749622+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'8700","ondisk_log_start":"43'8700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8970240,"num_objects":2190,"num_object_clones":0,"num_object_copies":4380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15324,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1867}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18600","reported_seq":22017,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:23.779171+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:34:23.779171+0000","last_peered":"2024-09-17T15:34:23.779171+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:34:23.779171+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"43'8500","ondisk_log_start":"43'8500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8896512,"num_objects":2172,"num_object_clones":0,"num_object_copies":4344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2172,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14887,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1634,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-354}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18704","reported_seq":19012,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:23.751128+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:34:23.751128+0000","last_peered":"2024-09-17T15:34:23.751128+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:34:23.751128+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'8700","ondisk_log_start":"43'8700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8937472,"num_objects":2182,"num_object_clones":0,"num_object_copies":4364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2182,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15491,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1798}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18467","reported_seq":26851,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:23.217797+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:34:23.217797+0000","last_peered":"2024-09-17T15:34:23.217797+0000","last_clean":"2024-09-17T15:34:23.217797+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:34:23.217797+0000","last_undegraded":"2024-09-17T15:34:23.217797+0000","last_fullsized":"2024-09-17T15:34:23.217797+0000","mapping_epoch":44,"log_start":"43'8400","ondisk_log_start":"43'8400","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8868806,"num_objects":2167,"num_object_clones":0,"num_object_copies":4334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14909,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":144278262,"num_objects":35234,"num_object_clones":0,"num_object_copies":70468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34937,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35234,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":257753,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14523,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":491524096,"data_stored":491389497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160818,"ondisk_log_size":160818,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051696,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":538152,"kb_used_data":203800,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93833688,"statfs":{"total":96636764160,"available":96085696512,"internally_reserved":0,"allocated":208691200,"data_stored":208389602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":342355628},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724641,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":553816,"kb_used_data":107008,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93818024,"statfs":{"total":96636764160,"available":96069656576,"internally_reserved":0,"allocated":109576192,"data_stored":109252452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":457502379},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,6,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247924,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":608752,"kb_used_data":171732,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93763088,"statfs":{"total":96636764160,"available":96013402112,"internally_reserved":0,"allocated":175853568,"data_stored":175551965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":447475368},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,6,2,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":175067136,"data_stored":175031006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":108240896,"data_stored":108182629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":208216064,"data_stored":208175862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:34:26.500 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:34:27.088 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:34:27.090 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:34:27.112 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":341,"stamp":"2024-09-17T15:34:25.778562+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":144278281,"num_objects":35236,"num_object_clones":0,"num_object_copies":70472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34937,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35236,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":257755,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14524,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160821,"ondisk_log_size":160821,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1700720,"kb_used_data":482540,"kb_used_omap":13,"kb_used_meta":1218098,"kb_avail":281414800,"statfs":{"total":289910292480,"available":288168755200,"internally_reserved":0,"allocated":494120960,"data_stored":493194019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1247333375},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,2,14,5,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19673088,"num_objects":-4803,"num_object_clones":0,"num_object_copies":-9606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-243,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4803,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006408"},"pg_stats":[{"pgid":"2.f","version":"47'18473","reported_seq":18733,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:23.777262+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:34:23.777262+0000","last_peered":"2024-09-17T15:34:23.777262+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:34:23.777262+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'8400","ondisk_log_start":"43'8400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8908800,"num_objects":2175,"num_object_clones":0,"num_object_copies":4350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18473,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1809}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18651","reported_seq":18762,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:23.151037+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:34:23.151037+0000","last_peered":"2024-09-17T15:34:23.151037+0000","last_clean":"2024-09-17T15:34:23.151037+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:34:23.151037+0000","last_undegraded":"2024-09-17T15:34:23.151037+0000","last_fullsized":"2024-09-17T15:34:23.151037+0000","mapping_epoch":33,"log_start":"43'8600","ondisk_log_start":"43'8600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9343942,"num_objects":2283,"num_object_clones":0,"num_object_copies":4566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2283,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17008,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18651","reported_seq":18873,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:23.137711+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:34:23.137711+0000","last_peered":"2024-09-17T15:34:23.137711+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:34:23.137711+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'8600","ondisk_log_start":"43'8600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9347096,"num_objects":2283,"num_object_clones":0,"num_object_copies":4566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2283,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16808,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1842}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18750","reported_seq":18830,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:23.116379+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:34:23.116379+0000","last_peered":"2024-09-17T15:34:23.116379+0000","last_clean":"2024-09-17T15:34:23.116379+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:34:23.116379+0000","last_undegraded":"2024-09-17T15:34:23.116379+0000","last_fullsized":"2024-09-17T15:34:23.116379+0000","mapping_epoch":33,"log_start":"43'8700","ondisk_log_start":"43'8700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9093120,"num_objects":2220,"num_object_clones":0,"num_object_copies":4440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18750,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18412","reported_seq":18551,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:23.768561+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:34:23.768561+0000","last_peered":"2024-09-17T15:34:23.768561+0000","last_clean":"2024-09-17T15:34:23.768561+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:34:23.768561+0000","last_undegraded":"2024-09-17T15:34:23.768561+0000","last_fullsized":"2024-09-17T15:34:23.768561+0000","mapping_epoch":32,"log_start":"43'8400","ondisk_log_start":"43'8400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9179619,"num_objects":2242,"num_object_clones":0,"num_object_copies":4484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2242,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15394,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19013","reported_seq":19314,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:23.645542+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:34:23.645542+0000","last_peered":"2024-09-17T15:34:23.645542+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:34:23.645542+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'9000","ondisk_log_start":"43'9000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9167331,"num_objects":2239,"num_object_clones":0,"num_object_copies":4478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2239,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15512,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1953},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18645","reported_seq":18963,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:23.785162+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:34:23.785162+0000","last_peered":"2024-09-17T15:34:23.785162+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:34:23.785162+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'8600","ondisk_log_start":"43'8600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8696774,"num_objects":2125,"num_object_clones":0,"num_object_copies":4250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2125,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15414,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1964},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18734","reported_seq":18872,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:23.750499+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:34:23.750499+0000","last_peered":"2024-09-17T15:34:23.750499+0000","last_clean":"2024-09-17T15:34:23.750499+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:34:23.750499+0000","last_undegraded":"2024-09-17T15:34:23.750499+0000","last_fullsized":"2024-09-17T15:34:23.750499+0000","mapping_epoch":32,"log_start":"43'8700","ondisk_log_start":"43'8700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8937472,"num_objects":2182,"num_object_clones":0,"num_object_copies":4364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2182,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15688,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.224142+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.224142+0000","last_peered":"2024-09-17T15:33:11.224142+0000","last_clean":"2024-09-17T15:33:11.224142+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.224142+0000","last_undegraded":"2024-09-17T15:33:11.224142+0000","last_fullsized":"2024-09-17T15:33:11.224142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18573","reported_seq":26865,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:23.210140+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:34:23.210140+0000","last_peered":"2024-09-17T15:34:23.210140+0000","last_clean":"2024-09-17T15:34:23.210140+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:34:23.210140+0000","last_undegraded":"2024-09-17T15:34:23.210140+0000","last_fullsized":"2024-09-17T15:34:23.210140+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9191907,"num_objects":2245,"num_object_clones":0,"num_object_copies":4490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2245,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16573,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18360","reported_seq":18634,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:23.754391+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:34:23.754391+0000","last_peered":"2024-09-17T15:34:23.754391+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:34:23.754391+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'8300","ondisk_log_start":"43'8300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8937472,"num_objects":2182,"num_object_clones":0,"num_object_copies":4364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2182,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14956,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1761}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18344","reported_seq":18563,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:23.113544+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:34:23.113544+0000","last_peered":"2024-09-17T15:34:23.113544+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:34:23.113544+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'8300","ondisk_log_start":"43'8300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8909283,"num_objects":2176,"num_object_clones":0,"num_object_copies":4352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2176,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16239,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1857}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18565","reported_seq":26980,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:23.172244+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:34:23.172244+0000","last_peered":"2024-09-17T15:34:23.172244+0000","last_clean":"2024-09-17T15:34:23.172244+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:34:23.172244+0000","last_undegraded":"2024-09-17T15:34:23.172244+0000","last_fullsized":"2024-09-17T15:34:23.172244+0000","mapping_epoch":46,"log_start":"43'8500","ondisk_log_start":"43'8500","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8892416,"num_objects":2171,"num_object_clones":0,"num_object_copies":4342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2171,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16327,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18776","reported_seq":19081,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:23.749622+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:34:23.749622+0000","last_peered":"2024-09-17T15:34:23.749622+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:34:23.749622+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'8700","ondisk_log_start":"43'8700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8970240,"num_objects":2190,"num_object_clones":0,"num_object_copies":4380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15324,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1867}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18600","reported_seq":22017,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:23.779171+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:34:23.779171+0000","last_peered":"2024-09-17T15:34:23.779171+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:34:23.779171+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"43'8500","ondisk_log_start":"43'8500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8896512,"num_objects":2172,"num_object_clones":0,"num_object_copies":4344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2172,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14887,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1634,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-354}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18704","reported_seq":19012,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:23.751128+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:34:23.751128+0000","last_peered":"2024-09-17T15:34:23.751128+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:34:23.751128+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'8700","ondisk_log_start":"43'8700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8937472,"num_objects":2182,"num_object_clones":0,"num_object_copies":4364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2182,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15491,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1798}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18467","reported_seq":26851,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:23.217797+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:34:23.217797+0000","last_peered":"2024-09-17T15:34:23.217797+0000","last_clean":"2024-09-17T15:34:23.217797+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:34:23.217797+0000","last_undegraded":"2024-09-17T15:34:23.217797+0000","last_fullsized":"2024-09-17T15:34:23.217797+0000","mapping_epoch":44,"log_start":"43'8400","ondisk_log_start":"43'8400","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8868806,"num_objects":2167,"num_object_clones":0,"num_object_copies":4334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14909,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":144278262,"num_objects":35234,"num_object_clones":0,"num_object_copies":70468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34937,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35234,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":257753,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14523,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":491524096,"data_stored":491389497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160818,"ondisk_log_size":160818,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051696,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":538152,"kb_used_data":203800,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93833688,"statfs":{"total":96636764160,"available":96085696512,"internally_reserved":0,"allocated":208691200,"data_stored":208389602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":342355628},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724641,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":553816,"kb_used_data":107008,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93818024,"statfs":{"total":96636764160,"available":96069656576,"internally_reserved":0,"allocated":109576192,"data_stored":109252452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":457502379},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,6,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247924,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":608752,"kb_used_data":171732,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93763088,"statfs":{"total":96636764160,"available":96013402112,"internally_reserved":0,"allocated":175853568,"data_stored":175551965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":447475368},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,6,2,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":175067136,"data_stored":175031006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":108240896,"data_stored":108182629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":208216064,"data_stored":208175862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:34:27.113 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:34:27.113 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'18473", 'reported_seq': 18733, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:23.777262+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:34:23.777262+0000', 'last_peered': '2024-09-17T15:34:23.777262+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:34:23.777262+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'8400", 'ondisk_log_start': "43'8400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8908800, 'num_objects': 2175, 'num_object_clones': 0, 'num_object_copies': 4350, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2175, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18473, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1809}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:27.113 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:34:27.113 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'18651", 'reported_seq': 18873, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:23.137711+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:34:23.137711+0000', 'last_peered': '2024-09-17T15:34:23.137711+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:34:23.137711+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "43'8600", 'ondisk_log_start': "43'8600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9347096, 'num_objects': 2283, 'num_object_clones': 0, 'num_object_copies': 4566, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2283, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16808, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1842}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:34:27.113 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:34:27.113 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'19013", 'reported_seq': 19314, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:23.645542+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:34:23.645542+0000', 'last_peered': '2024-09-17T15:34:23.645542+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:34:23.645542+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "43'9000", 'ondisk_log_start': "43'9000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9167331, 'num_objects': 2239, 'num_object_clones': 0, 'num_object_copies': 4478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2239, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15512, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1953}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:27.114 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:34:27.114 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'18645", 'reported_seq': 18963, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:23.785162+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:34:23.785162+0000', 'last_peered': '2024-09-17T15:34:23.785162+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:34:23.785162+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "43'8600", 'ondisk_log_start': "43'8600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8696774, 'num_objects': 2125, 'num_object_clones': 0, 'num_object_copies': 4250, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4089, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2125, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 15414, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 85, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1964}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:27.114 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:34:27.114 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'18360", 'reported_seq': 18634, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:23.754391+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:34:23.754391+0000', 'last_peered': '2024-09-17T15:34:23.754391+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:34:23.754391+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'8300", 'ondisk_log_start': "43'8300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8937472, 'num_objects': 2182, 'num_object_clones': 0, 'num_object_copies': 4364, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2182, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14956, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1761}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:27.114 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:34:27.114 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'18344", 'reported_seq': 18563, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:23.113544+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:34:23.113544+0000', 'last_peered': '2024-09-17T15:34:23.113544+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:34:23.113544+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "43'8300", 'ondisk_log_start': "43'8300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8909283, 'num_objects': 2176, 'num_object_clones': 0, 'num_object_copies': 4352, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4034, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2176, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16239, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -1857}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:34:27.114 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:34:27.114 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'18776", 'reported_seq': 19081, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:23.749622+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:34:23.749622+0000', 'last_peered': '2024-09-17T15:34:23.749622+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:34:23.749622+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "43'8700", 'ondisk_log_start': "43'8700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8970240, 'num_objects': 2190, 'num_object_clones': 0, 'num_object_copies': 4380, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2190, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15324, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1867}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:27.115 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:34:27.115 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'18600", 'reported_seq': 22017, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:23.779171+0000', 'last_change': '2024-09-17T15:33:09.763500+0000', 'last_active': '2024-09-17T15:34:23.779171+0000', 'last_peered': '2024-09-17T15:34:23.779171+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:34:23.779171+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "43'8500", 'ondisk_log_start': "43'8500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8896512, 'num_objects': 2172, 'num_object_clones': 0, 'num_object_copies': 4344, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2172, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14887, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1634, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -354}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:27.115 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:34:27.115 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'18704", 'reported_seq': 19012, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:23.751128+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:34:23.751128+0000', 'last_peered': '2024-09-17T15:34:23.751128+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:34:23.751128+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'8700", 'ondisk_log_start': "43'8700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8937472, 'num_objects': 2182, 'num_object_clones': 0, 'num_object_copies': 4364, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2182, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15491, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1798}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:27.116 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:34:27.813 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:34:27.813 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34937/70472 objects degraded (49.576%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":604,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":35236,"data_bytes":144278281,"bytes_used":1741537280,"bytes_avail":288168755200,"bytes_total":289910292480,"degraded_objects":34937,"degraded_total":70472,"degraded_ratio":0.49575718015665798,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":243,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":400},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:34:27.836 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:34:27.836 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:34:28.396 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:34:28.399 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:34:28.423 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":342,"stamp":"2024-09-17T15:34:27.779408+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":144278281,"num_objects":35236,"num_object_clones":0,"num_object_copies":70472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34937,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35236,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":257755,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14524,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160821,"ondisk_log_size":160821,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1700720,"kb_used_data":482540,"kb_used_omap":13,"kb_used_meta":1218098,"kb_avail":281414800,"statfs":{"total":289910292480,"available":288168755200,"internally_reserved":0,"allocated":494120960,"data_stored":493194019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1247333375},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,2,14,5,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19673088,"num_objects":-4803,"num_object_clones":0,"num_object_copies":-9606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-243,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4803,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006397"},"pg_stats":[{"pgid":"2.f","version":"47'18473","reported_seq":18733,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:23.777262+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:34:23.777262+0000","last_peered":"2024-09-17T15:34:23.777262+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:34:23.777262+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'8400","ondisk_log_start":"43'8400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8908800,"num_objects":2175,"num_object_clones":0,"num_object_copies":4350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18473,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1809}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18651","reported_seq":18762,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:23.151037+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:34:23.151037+0000","last_peered":"2024-09-17T15:34:23.151037+0000","last_clean":"2024-09-17T15:34:23.151037+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:34:23.151037+0000","last_undegraded":"2024-09-17T15:34:23.151037+0000","last_fullsized":"2024-09-17T15:34:23.151037+0000","mapping_epoch":33,"log_start":"43'8600","ondisk_log_start":"43'8600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9343942,"num_objects":2283,"num_object_clones":0,"num_object_copies":4566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2283,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17008,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18651","reported_seq":18873,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:23.137711+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:34:23.137711+0000","last_peered":"2024-09-17T15:34:23.137711+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:34:23.137711+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'8600","ondisk_log_start":"43'8600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9347096,"num_objects":2283,"num_object_clones":0,"num_object_copies":4566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2283,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16808,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1842}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18750","reported_seq":18830,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:23.116379+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:34:23.116379+0000","last_peered":"2024-09-17T15:34:23.116379+0000","last_clean":"2024-09-17T15:34:23.116379+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:34:23.116379+0000","last_undegraded":"2024-09-17T15:34:23.116379+0000","last_fullsized":"2024-09-17T15:34:23.116379+0000","mapping_epoch":33,"log_start":"43'8700","ondisk_log_start":"43'8700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9093120,"num_objects":2220,"num_object_clones":0,"num_object_copies":4440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18750,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18412","reported_seq":18551,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:23.768561+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:34:23.768561+0000","last_peered":"2024-09-17T15:34:23.768561+0000","last_clean":"2024-09-17T15:34:23.768561+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:34:23.768561+0000","last_undegraded":"2024-09-17T15:34:23.768561+0000","last_fullsized":"2024-09-17T15:34:23.768561+0000","mapping_epoch":32,"log_start":"43'8400","ondisk_log_start":"43'8400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9179619,"num_objects":2242,"num_object_clones":0,"num_object_copies":4484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2242,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15394,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19013","reported_seq":19314,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:23.645542+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:34:23.645542+0000","last_peered":"2024-09-17T15:34:23.645542+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:34:23.645542+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'9000","ondisk_log_start":"43'9000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9167331,"num_objects":2239,"num_object_clones":0,"num_object_copies":4478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2239,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15512,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1953},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18645","reported_seq":18963,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:23.785162+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:34:23.785162+0000","last_peered":"2024-09-17T15:34:23.785162+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:34:23.785162+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'8600","ondisk_log_start":"43'8600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8696774,"num_objects":2125,"num_object_clones":0,"num_object_copies":4250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2125,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15414,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1964},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18734","reported_seq":18872,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:23.750499+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:34:23.750499+0000","last_peered":"2024-09-17T15:34:23.750499+0000","last_clean":"2024-09-17T15:34:23.750499+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:34:23.750499+0000","last_undegraded":"2024-09-17T15:34:23.750499+0000","last_fullsized":"2024-09-17T15:34:23.750499+0000","mapping_epoch":32,"log_start":"43'8700","ondisk_log_start":"43'8700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8937472,"num_objects":2182,"num_object_clones":0,"num_object_copies":4364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2182,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15688,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.224142+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.224142+0000","last_peered":"2024-09-17T15:33:11.224142+0000","last_clean":"2024-09-17T15:33:11.224142+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.224142+0000","last_undegraded":"2024-09-17T15:33:11.224142+0000","last_fullsized":"2024-09-17T15:33:11.224142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18573","reported_seq":26865,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:23.210140+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:34:23.210140+0000","last_peered":"2024-09-17T15:34:23.210140+0000","last_clean":"2024-09-17T15:34:23.210140+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:34:23.210140+0000","last_undegraded":"2024-09-17T15:34:23.210140+0000","last_fullsized":"2024-09-17T15:34:23.210140+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9191907,"num_objects":2245,"num_object_clones":0,"num_object_copies":4490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2245,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16573,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18360","reported_seq":18634,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:23.754391+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:34:23.754391+0000","last_peered":"2024-09-17T15:34:23.754391+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:34:23.754391+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'8300","ondisk_log_start":"43'8300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8937472,"num_objects":2182,"num_object_clones":0,"num_object_copies":4364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2182,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14956,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1761}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18344","reported_seq":18563,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:23.113544+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:34:23.113544+0000","last_peered":"2024-09-17T15:34:23.113544+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:34:23.113544+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'8300","ondisk_log_start":"43'8300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8909283,"num_objects":2176,"num_object_clones":0,"num_object_copies":4352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2176,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16239,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1857}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18565","reported_seq":26980,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:23.172244+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:34:23.172244+0000","last_peered":"2024-09-17T15:34:23.172244+0000","last_clean":"2024-09-17T15:34:23.172244+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:34:23.172244+0000","last_undegraded":"2024-09-17T15:34:23.172244+0000","last_fullsized":"2024-09-17T15:34:23.172244+0000","mapping_epoch":46,"log_start":"43'8500","ondisk_log_start":"43'8500","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8892416,"num_objects":2171,"num_object_clones":0,"num_object_copies":4342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2171,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16327,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18776","reported_seq":19081,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:23.749622+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:34:23.749622+0000","last_peered":"2024-09-17T15:34:23.749622+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:34:23.749622+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'8700","ondisk_log_start":"43'8700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8970240,"num_objects":2190,"num_object_clones":0,"num_object_copies":4380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15324,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1867}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18600","reported_seq":22017,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:23.779171+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:34:23.779171+0000","last_peered":"2024-09-17T15:34:23.779171+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:34:23.779171+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"43'8500","ondisk_log_start":"43'8500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8896512,"num_objects":2172,"num_object_clones":0,"num_object_copies":4344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2172,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14887,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1634,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-354}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18704","reported_seq":19012,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:23.751128+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:34:23.751128+0000","last_peered":"2024-09-17T15:34:23.751128+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:34:23.751128+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'8700","ondisk_log_start":"43'8700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8937472,"num_objects":2182,"num_object_clones":0,"num_object_copies":4364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2182,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15491,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1798}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18467","reported_seq":26851,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:23.217797+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:34:23.217797+0000","last_peered":"2024-09-17T15:34:23.217797+0000","last_clean":"2024-09-17T15:34:23.217797+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:34:23.217797+0000","last_undegraded":"2024-09-17T15:34:23.217797+0000","last_fullsized":"2024-09-17T15:34:23.217797+0000","mapping_epoch":44,"log_start":"43'8400","ondisk_log_start":"43'8400","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8868806,"num_objects":2167,"num_object_clones":0,"num_object_copies":4334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14909,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":144278262,"num_objects":35234,"num_object_clones":0,"num_object_copies":70468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34937,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35234,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":257753,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14523,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":491524096,"data_stored":491389497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160818,"ondisk_log_size":160818,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051696,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":538152,"kb_used_data":203800,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93833688,"statfs":{"total":96636764160,"available":96085696512,"internally_reserved":0,"allocated":208691200,"data_stored":208389602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":342355628},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724641,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":553816,"kb_used_data":107008,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93818024,"statfs":{"total":96636764160,"available":96069656576,"internally_reserved":0,"allocated":109576192,"data_stored":109252452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":457502379},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,6,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247924,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":608752,"kb_used_data":171732,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93763088,"statfs":{"total":96636764160,"available":96013402112,"internally_reserved":0,"allocated":175853568,"data_stored":175551965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":447475368},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,6,2,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":175067136,"data_stored":175031006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":108240896,"data_stored":108182629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":208216064,"data_stored":208175862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:34:31.426 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:34:32.014 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:34:32.015 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:34:32.041 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":344,"stamp":"2024-09-17T15:34:31.781859+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":134226697,"num_objects":32782,"num_object_clones":0,"num_object_copies":65564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34830,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32782,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":260209,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14631,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160875,"ondisk_log_size":160875,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1681004,"kb_used_data":462828,"kb_used_omap":13,"kb_used_meta":1218098,"kb_avail":281434516,"statfs":{"total":289910292480,"available":288188944384,"internally_reserved":0,"allocated":473935872,"data_stored":473008931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1247333375},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,0,1,3,9,10,14,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20062208,"num_objects":-4898,"num_object_clones":0,"num_object_copies":-9796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-216,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4898,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007272"},"pg_stats":[{"pgid":"2.f","version":"47'18637","reported_seq":18896,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:28.665090+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:34:28.665090+0000","last_peered":"2024-09-17T15:34:28.665090+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:34:28.665090+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'8600","ondisk_log_start":"43'8600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8237056,"num_objects":2011,"num_object_clones":0,"num_object_copies":4022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18637,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18795","reported_seq":18908,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:28.079256+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:34:28.079256+0000","last_peered":"2024-09-17T15:34:28.079256+0000","last_clean":"2024-09-17T15:34:28.079256+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:34:28.079256+0000","last_undegraded":"2024-09-17T15:34:28.079256+0000","last_fullsized":"2024-09-17T15:34:28.079256+0000","mapping_epoch":33,"log_start":"43'8700","ondisk_log_start":"43'8700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8754118,"num_objects":2139,"num_object_clones":0,"num_object_copies":4278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2139,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17152,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18790","reported_seq":19012,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:28.122061+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:34:28.122061+0000","last_peered":"2024-09-17T15:34:28.122061+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:34:28.122061+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'8700","ondisk_log_start":"43'8700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8777752,"num_objects":2144,"num_object_clones":0,"num_object_copies":4288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2144,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16947,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1981}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18919","reported_seq":18999,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:28.167431+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:34:28.167431+0000","last_peered":"2024-09-17T15:34:28.167431+0000","last_clean":"2024-09-17T15:34:28.167431+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:34:28.167431+0000","last_undegraded":"2024-09-17T15:34:28.167431+0000","last_fullsized":"2024-09-17T15:34:28.167431+0000","mapping_epoch":33,"log_start":"43'8900","ondisk_log_start":"43'8900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8400896,"num_objects":2051,"num_object_clones":0,"num_object_copies":4102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18919,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18578","reported_seq":18717,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:28.707599+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:34:28.707599+0000","last_peered":"2024-09-17T15:34:28.707599+0000","last_clean":"2024-09-17T15:34:28.707599+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:34:28.707599+0000","last_undegraded":"2024-09-17T15:34:28.707599+0000","last_fullsized":"2024-09-17T15:34:28.707599+0000","mapping_epoch":32,"log_start":"43'8500","ondisk_log_start":"43'8500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8499683,"num_objects":2076,"num_object_clones":0,"num_object_copies":4152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2076,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15560,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19179","reported_seq":19480,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:28.782448+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:34:28.782448+0000","last_peered":"2024-09-17T15:34:28.782448+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:34:28.782448+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'9100","ondisk_log_start":"43'9100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8487395,"num_objects":2073,"num_object_clones":0,"num_object_copies":4146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2073,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15678,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2119},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18797","reported_seq":19114,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:28.792713+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:34:28.792713+0000","last_peered":"2024-09-17T15:34:28.792713+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:34:28.792713+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'8700","ondisk_log_start":"43'8700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8074182,"num_objects":1973,"num_object_clones":0,"num_object_copies":3946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1973,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15566,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2116},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18878","reported_seq":19016,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:28.743874+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:34:28.743874+0000","last_peered":"2024-09-17T15:34:28.743874+0000","last_clean":"2024-09-17T15:34:28.743874+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:34:28.743874+0000","last_undegraded":"2024-09-17T15:34:28.743874+0000","last_fullsized":"2024-09-17T15:34:28.743874+0000","mapping_epoch":32,"log_start":"43'8800","ondisk_log_start":"43'8800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8347648,"num_objects":2038,"num_object_clones":0,"num_object_copies":4076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15832,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.224142+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.224142+0000","last_peered":"2024-09-17T15:33:11.224142+0000","last_clean":"2024-09-17T15:33:11.224142+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.224142+0000","last_undegraded":"2024-09-17T15:33:11.224142+0000","last_fullsized":"2024-09-17T15:33:11.224142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18739","reported_seq":27031,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:28.253373+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:34:28.253373+0000","last_peered":"2024-09-17T15:34:28.253373+0000","last_clean":"2024-09-17T15:34:28.253373+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:34:28.253373+0000","last_undegraded":"2024-09-17T15:34:28.253373+0000","last_fullsized":"2024-09-17T15:34:28.253373+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8511971,"num_objects":2079,"num_object_clones":0,"num_object_copies":4158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2079,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16739,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18509","reported_seq":18783,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:28.698837+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:34:28.698837+0000","last_peered":"2024-09-17T15:34:28.698837+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:34:28.698837+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'8500","ondisk_log_start":"43'8500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8327168,"num_objects":2033,"num_object_clones":0,"num_object_copies":4066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15105,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1910}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18489","reported_seq":18708,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:28.189931+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:34:28.189931+0000","last_peered":"2024-09-17T15:34:28.189931+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:34:28.189931+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'8400","ondisk_log_start":"43'8400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8315363,"num_objects":2031,"num_object_clones":0,"num_object_copies":4062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2031,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16384,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2002}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18716","reported_seq":27131,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:28.146252+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:34:28.146252+0000","last_peered":"2024-09-17T15:34:28.146252+0000","last_clean":"2024-09-17T15:34:28.146252+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:34:28.146252+0000","last_undegraded":"2024-09-17T15:34:28.146252+0000","last_fullsized":"2024-09-17T15:34:28.146252+0000","mapping_epoch":46,"log_start":"43'8700","ondisk_log_start":"43'8700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8273920,"num_objects":2020,"num_object_clones":0,"num_object_copies":4040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16478,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18920","reported_seq":19227,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:28.769892+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:34:28.769892+0000","last_peered":"2024-09-17T15:34:28.769892+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:34:28.769892+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'8900","ondisk_log_start":"43'8900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8380416,"num_objects":2046,"num_object_clones":0,"num_object_copies":4092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2046,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15468,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2011}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18770","reported_seq":22405,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:28.783824+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:34:28.783824+0000","last_peered":"2024-09-17T15:34:28.783824+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:34:28.783824+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"43'8700","ondisk_log_start":"43'8700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8200192,"num_objects":2002,"num_object_clones":0,"num_object_copies":4004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2419,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15057,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1741,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-417}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18843","reported_seq":19151,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:28.695511+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:34:28.695511+0000","last_peered":"2024-09-17T15:34:28.695511+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:34:28.695511+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'8800","ondisk_log_start":"43'8800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8368128,"num_objects":2043,"num_object_clones":0,"num_object_copies":4086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15630,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1937}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18613","reported_seq":26997,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:28.214967+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:34:28.214967+0000","last_peered":"2024-09-17T15:34:28.214967+0000","last_clean":"2024-09-17T15:34:28.214967+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:34:28.214967+0000","last_undegraded":"2024-09-17T15:34:28.214967+0000","last_fullsized":"2024-09-17T15:34:28.214967+0000","mapping_epoch":44,"log_start":"43'8600","ondisk_log_start":"43'8600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8270790,"num_objects":2021,"num_object_clones":0,"num_object_copies":4042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15055,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":134226678,"num_objects":32780,"num_object_clones":0,"num_object_copies":65560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34830,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32780,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":260207,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14630,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":471343104,"data_stored":471208505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160872,"ondisk_log_size":160872,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051697,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537452,"kb_used_data":198936,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93834388,"statfs":{"total":96636764160,"available":96086413312,"internally_reserved":0,"allocated":203710464,"data_stored":203408866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":346615468},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,1,2,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724642,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":554664,"kb_used_data":99544,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93817176,"statfs":{"total":96636764160,"available":96068788224,"internally_reserved":0,"allocated":101933056,"data_stored":101609316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":466022059},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,3,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247925,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":588888,"kb_used_data":164348,"kb_used_omap":4,"kb_used_meta":424507,"kb_avail":93782952,"statfs":{"total":96636764160,"available":96033742848,"internally_reserved":0,"allocated":168292352,"data_stored":167990749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":434695848},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,0,4,5,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":167489536,"data_stored":167453406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":100655104,"data_stored":100596837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":203198464,"data_stored":203158262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:34:32.043 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:34:32.601 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:34:32.601 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:34:32.628 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":344,"stamp":"2024-09-17T15:34:31.781859+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":134226697,"num_objects":32782,"num_object_clones":0,"num_object_copies":65564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34830,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32782,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":260209,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14631,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160875,"ondisk_log_size":160875,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1681004,"kb_used_data":462828,"kb_used_omap":13,"kb_used_meta":1218098,"kb_avail":281434516,"statfs":{"total":289910292480,"available":288188944384,"internally_reserved":0,"allocated":473935872,"data_stored":473008931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1247333375},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,0,1,3,9,10,14,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20062208,"num_objects":-4898,"num_object_clones":0,"num_object_copies":-9796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-216,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4898,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007272"},"pg_stats":[{"pgid":"2.f","version":"47'18637","reported_seq":18896,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:28.665090+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:34:28.665090+0000","last_peered":"2024-09-17T15:34:28.665090+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:34:28.665090+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'8600","ondisk_log_start":"43'8600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8237056,"num_objects":2011,"num_object_clones":0,"num_object_copies":4022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18637,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18795","reported_seq":18908,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:28.079256+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:34:28.079256+0000","last_peered":"2024-09-17T15:34:28.079256+0000","last_clean":"2024-09-17T15:34:28.079256+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:34:28.079256+0000","last_undegraded":"2024-09-17T15:34:28.079256+0000","last_fullsized":"2024-09-17T15:34:28.079256+0000","mapping_epoch":33,"log_start":"43'8700","ondisk_log_start":"43'8700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8754118,"num_objects":2139,"num_object_clones":0,"num_object_copies":4278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2139,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17152,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18790","reported_seq":19012,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:28.122061+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:34:28.122061+0000","last_peered":"2024-09-17T15:34:28.122061+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:34:28.122061+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'8700","ondisk_log_start":"43'8700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8777752,"num_objects":2144,"num_object_clones":0,"num_object_copies":4288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2144,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16947,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1981}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18919","reported_seq":18999,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:28.167431+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:34:28.167431+0000","last_peered":"2024-09-17T15:34:28.167431+0000","last_clean":"2024-09-17T15:34:28.167431+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:34:28.167431+0000","last_undegraded":"2024-09-17T15:34:28.167431+0000","last_fullsized":"2024-09-17T15:34:28.167431+0000","mapping_epoch":33,"log_start":"43'8900","ondisk_log_start":"43'8900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8400896,"num_objects":2051,"num_object_clones":0,"num_object_copies":4102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18919,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18578","reported_seq":18717,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:28.707599+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:34:28.707599+0000","last_peered":"2024-09-17T15:34:28.707599+0000","last_clean":"2024-09-17T15:34:28.707599+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:34:28.707599+0000","last_undegraded":"2024-09-17T15:34:28.707599+0000","last_fullsized":"2024-09-17T15:34:28.707599+0000","mapping_epoch":32,"log_start":"43'8500","ondisk_log_start":"43'8500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8499683,"num_objects":2076,"num_object_clones":0,"num_object_copies":4152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2076,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15560,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19179","reported_seq":19480,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:28.782448+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:34:28.782448+0000","last_peered":"2024-09-17T15:34:28.782448+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:34:28.782448+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'9100","ondisk_log_start":"43'9100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8487395,"num_objects":2073,"num_object_clones":0,"num_object_copies":4146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2073,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15678,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2119},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18797","reported_seq":19114,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:28.792713+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:34:28.792713+0000","last_peered":"2024-09-17T15:34:28.792713+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:34:28.792713+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'8700","ondisk_log_start":"43'8700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8074182,"num_objects":1973,"num_object_clones":0,"num_object_copies":3946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1973,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15566,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2116},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18878","reported_seq":19016,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:28.743874+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:34:28.743874+0000","last_peered":"2024-09-17T15:34:28.743874+0000","last_clean":"2024-09-17T15:34:28.743874+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:34:28.743874+0000","last_undegraded":"2024-09-17T15:34:28.743874+0000","last_fullsized":"2024-09-17T15:34:28.743874+0000","mapping_epoch":32,"log_start":"43'8800","ondisk_log_start":"43'8800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8347648,"num_objects":2038,"num_object_clones":0,"num_object_copies":4076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15832,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.224142+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.224142+0000","last_peered":"2024-09-17T15:33:11.224142+0000","last_clean":"2024-09-17T15:33:11.224142+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.224142+0000","last_undegraded":"2024-09-17T15:33:11.224142+0000","last_fullsized":"2024-09-17T15:33:11.224142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18739","reported_seq":27031,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:28.253373+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:34:28.253373+0000","last_peered":"2024-09-17T15:34:28.253373+0000","last_clean":"2024-09-17T15:34:28.253373+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:34:28.253373+0000","last_undegraded":"2024-09-17T15:34:28.253373+0000","last_fullsized":"2024-09-17T15:34:28.253373+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8511971,"num_objects":2079,"num_object_clones":0,"num_object_copies":4158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2079,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16739,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18509","reported_seq":18783,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:28.698837+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:34:28.698837+0000","last_peered":"2024-09-17T15:34:28.698837+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:34:28.698837+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'8500","ondisk_log_start":"43'8500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8327168,"num_objects":2033,"num_object_clones":0,"num_object_copies":4066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15105,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1910}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18489","reported_seq":18708,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:28.189931+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:34:28.189931+0000","last_peered":"2024-09-17T15:34:28.189931+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:34:28.189931+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'8400","ondisk_log_start":"43'8400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8315363,"num_objects":2031,"num_object_clones":0,"num_object_copies":4062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2031,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16384,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2002}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18716","reported_seq":27131,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:28.146252+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:34:28.146252+0000","last_peered":"2024-09-17T15:34:28.146252+0000","last_clean":"2024-09-17T15:34:28.146252+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:34:28.146252+0000","last_undegraded":"2024-09-17T15:34:28.146252+0000","last_fullsized":"2024-09-17T15:34:28.146252+0000","mapping_epoch":46,"log_start":"43'8700","ondisk_log_start":"43'8700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8273920,"num_objects":2020,"num_object_clones":0,"num_object_copies":4040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16478,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18920","reported_seq":19227,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:28.769892+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:34:28.769892+0000","last_peered":"2024-09-17T15:34:28.769892+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:34:28.769892+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'8900","ondisk_log_start":"43'8900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8380416,"num_objects":2046,"num_object_clones":0,"num_object_copies":4092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2046,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15468,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2011}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18770","reported_seq":22405,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:28.783824+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:34:28.783824+0000","last_peered":"2024-09-17T15:34:28.783824+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:34:28.783824+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"43'8700","ondisk_log_start":"43'8700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8200192,"num_objects":2002,"num_object_clones":0,"num_object_copies":4004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2419,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15057,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1741,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-417}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18843","reported_seq":19151,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:28.695511+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:34:28.695511+0000","last_peered":"2024-09-17T15:34:28.695511+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:34:28.695511+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'8800","ondisk_log_start":"43'8800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8368128,"num_objects":2043,"num_object_clones":0,"num_object_copies":4086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15630,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1937}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18613","reported_seq":26997,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:28.214967+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:34:28.214967+0000","last_peered":"2024-09-17T15:34:28.214967+0000","last_clean":"2024-09-17T15:34:28.214967+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:34:28.214967+0000","last_undegraded":"2024-09-17T15:34:28.214967+0000","last_fullsized":"2024-09-17T15:34:28.214967+0000","mapping_epoch":44,"log_start":"43'8600","ondisk_log_start":"43'8600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8270790,"num_objects":2021,"num_object_clones":0,"num_object_copies":4042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15055,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":134226678,"num_objects":32780,"num_object_clones":0,"num_object_copies":65560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34830,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32780,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":260207,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14630,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":471343104,"data_stored":471208505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160872,"ondisk_log_size":160872,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051697,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537452,"kb_used_data":198936,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93834388,"statfs":{"total":96636764160,"available":96086413312,"internally_reserved":0,"allocated":203710464,"data_stored":203408866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":346615468},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,1,2,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724642,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":554664,"kb_used_data":99544,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93817176,"statfs":{"total":96636764160,"available":96068788224,"internally_reserved":0,"allocated":101933056,"data_stored":101609316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":466022059},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,3,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247925,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":588888,"kb_used_data":164348,"kb_used_omap":4,"kb_used_meta":424507,"kb_avail":93782952,"statfs":{"total":96636764160,"available":96033742848,"internally_reserved":0,"allocated":168292352,"data_stored":167990749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":434695848},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,0,4,5,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":167489536,"data_stored":167453406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":100655104,"data_stored":100596837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":203198464,"data_stored":203158262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:34:32.630 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:34:32.630 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'18637", 'reported_seq': 18896, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:28.665090+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:34:28.665090+0000', 'last_peered': '2024-09-17T15:34:28.665090+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:34:28.665090+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'8600", 'ondisk_log_start': "43'8600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8237056, 'num_objects': 2011, 'num_object_clones': 0, 'num_object_copies': 4022, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2011, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18637, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1973}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:32.630 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:34:32.630 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'18790", 'reported_seq': 19012, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:28.122061+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:34:28.122061+0000', 'last_peered': '2024-09-17T15:34:28.122061+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:34:28.122061+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "43'8700", 'ondisk_log_start': "43'8700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8777752, 'num_objects': 2144, 'num_object_clones': 0, 'num_object_copies': 4288, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2144, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16947, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1981}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:34:32.630 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:34:32.630 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'19179", 'reported_seq': 19480, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:28.782448+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:34:28.782448+0000', 'last_peered': '2024-09-17T15:34:28.782448+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:34:28.782448+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "43'9100", 'ondisk_log_start': "43'9100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8487395, 'num_objects': 2073, 'num_object_clones': 0, 'num_object_copies': 4146, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2073, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15678, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2119}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:32.630 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:34:32.630 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'18797", 'reported_seq': 19114, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:28.792713+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:34:28.792713+0000', 'last_peered': '2024-09-17T15:34:28.792713+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:34:28.792713+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "43'8700", 'ondisk_log_start': "43'8700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8074182, 'num_objects': 1973, 'num_object_clones': 0, 'num_object_copies': 3946, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4089, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1973, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 15566, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 85, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2116}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:32.631 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:34:32.631 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'18509", 'reported_seq': 18783, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:28.698837+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:34:28.698837+0000', 'last_peered': '2024-09-17T15:34:28.698837+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:34:28.698837+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'8500", 'ondisk_log_start': "43'8500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8327168, 'num_objects': 2033, 'num_object_clones': 0, 'num_object_copies': 4066, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2033, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15105, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1910}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:32.631 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:34:32.631 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'18489", 'reported_seq': 18708, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:28.189931+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:34:28.189931+0000', 'last_peered': '2024-09-17T15:34:28.189931+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:34:28.189931+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "43'8400", 'ondisk_log_start': "43'8400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8315363, 'num_objects': 2031, 'num_object_clones': 0, 'num_object_copies': 4062, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4034, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2031, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16384, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -2002}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:34:32.631 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:34:32.631 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'18920", 'reported_seq': 19227, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:28.769892+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:34:28.769892+0000', 'last_peered': '2024-09-17T15:34:28.769892+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:34:28.769892+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "43'8900", 'ondisk_log_start': "43'8900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8380416, 'num_objects': 2046, 'num_object_clones': 0, 'num_object_copies': 4092, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2046, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15468, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2011}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:32.631 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:34:32.631 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'18770", 'reported_seq': 22405, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:28.783824+0000', 'last_change': '2024-09-17T15:33:09.763500+0000', 'last_active': '2024-09-17T15:34:28.783824+0000', 'last_peered': '2024-09-17T15:34:28.783824+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:34:28.783824+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "43'8700", 'ondisk_log_start': "43'8700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8200192, 'num_objects': 2002, 'num_object_clones': 0, 'num_object_copies': 4004, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2419, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2002, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15057, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1741, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -417}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:32.631 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:34:32.631 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'18843", 'reported_seq': 19151, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:28.695511+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:34:28.695511+0000', 'last_peered': '2024-09-17T15:34:28.695511+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:34:28.695511+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'8800", 'ondisk_log_start': "43'8800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8368128, 'num_objects': 2043, 'num_object_clones': 0, 'num_object_copies': 4086, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2043, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15630, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1937}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:32.632 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:34:33.377 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:34:33.377 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34830/65564 objects degraded (53.124%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":610,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":32782,"data_bytes":134226697,"bytes_used":1721348096,"bytes_avail":288188944384,"bytes_total":289910292480,"degraded_objects":34830,"degraded_total":65564,"degraded_ratio":0.53123665426148492,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":216,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":407},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:34:33.404 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:34:33.405 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:34:33.969 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:34:33.973 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:34:33.992 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2024-09-17T15:34:33.785274+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":129790729,"num_objects":31699,"num_object_clones":0,"num_object_copies":63398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34830,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31699,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":261292,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14631,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160958,"ondisk_log_size":160958,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1660448,"kb_used_data":450584,"kb_used_omap":13,"kb_used_meta":1209778,"kb_avail":281455072,"statfs":{"total":289910292480,"available":288209993728,"internally_reserved":0,"allocated":461398016,"data_stored":460471075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1238813695},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,2,9,9,9,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24498176,"num_objects":-5981,"num_object_clones":0,"num_object_copies":-11962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-216,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5981,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008154"},"pg_stats":[{"pgid":"2.f","version":"47'18637","reported_seq":18896,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:28.665090+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:34:28.665090+0000","last_peered":"2024-09-17T15:34:28.665090+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:34:28.665090+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'8600","ondisk_log_start":"43'8600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8237056,"num_objects":2011,"num_object_clones":0,"num_object_copies":4022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18637,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18958","reported_seq":19071,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:33.121762+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:34:33.121762+0000","last_peered":"2024-09-17T15:34:33.121762+0000","last_clean":"2024-09-17T15:34:33.121762+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:34:33.121762+0000","last_undegraded":"2024-09-17T15:34:33.121762+0000","last_fullsized":"2024-09-17T15:34:33.121762+0000","mapping_epoch":33,"log_start":"43'8900","ondisk_log_start":"43'8900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8086470,"num_objects":1976,"num_object_clones":0,"num_object_copies":3952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1976,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17315,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18932","reported_seq":19151,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:33.157332+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:34:33.157332+0000","last_peered":"2024-09-17T15:34:33.157332+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:34:33.157332+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'8900","ondisk_log_start":"43'8900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8196120,"num_objects":2002,"num_object_clones":0,"num_object_copies":4004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2002,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17089,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2123}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19069","reported_seq":19149,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:33.157994+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:34:33.157994+0000","last_peered":"2024-09-17T15:34:33.157994+0000","last_clean":"2024-09-17T15:34:33.157994+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:34:33.157994+0000","last_undegraded":"2024-09-17T15:34:33.157994+0000","last_fullsized":"2024-09-17T15:34:33.157994+0000","mapping_epoch":33,"log_start":"43'9000","ondisk_log_start":"43'9000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7786496,"num_objects":1901,"num_object_clones":0,"num_object_copies":3802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19069,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18578","reported_seq":18717,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:28.707599+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:34:28.707599+0000","last_peered":"2024-09-17T15:34:28.707599+0000","last_clean":"2024-09-17T15:34:28.707599+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:34:28.707599+0000","last_undegraded":"2024-09-17T15:34:28.707599+0000","last_fullsized":"2024-09-17T15:34:28.707599+0000","mapping_epoch":32,"log_start":"43'8500","ondisk_log_start":"43'8500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8499683,"num_objects":2076,"num_object_clones":0,"num_object_copies":4152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2076,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15560,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19179","reported_seq":19480,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:28.782448+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:34:28.782448+0000","last_peered":"2024-09-17T15:34:28.782448+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:34:28.782448+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'9100","ondisk_log_start":"43'9100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8487395,"num_objects":2073,"num_object_clones":0,"num_object_copies":4146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2073,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15678,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2119},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18797","reported_seq":19114,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:28.792713+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:34:28.792713+0000","last_peered":"2024-09-17T15:34:28.792713+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:34:28.792713+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'8700","ondisk_log_start":"43'8700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8074182,"num_objects":1973,"num_object_clones":0,"num_object_copies":3946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1973,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15566,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2116},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18878","reported_seq":19016,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:28.743874+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:34:28.743874+0000","last_peered":"2024-09-17T15:34:28.743874+0000","last_clean":"2024-09-17T15:34:28.743874+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:34:28.743874+0000","last_undegraded":"2024-09-17T15:34:28.743874+0000","last_fullsized":"2024-09-17T15:34:28.743874+0000","mapping_epoch":32,"log_start":"43'8800","ondisk_log_start":"43'8800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8347648,"num_objects":2038,"num_object_clones":0,"num_object_copies":4076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15832,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.224142+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.224142+0000","last_peered":"2024-09-17T15:33:11.224142+0000","last_clean":"2024-09-17T15:33:11.224142+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.224142+0000","last_undegraded":"2024-09-17T15:33:11.224142+0000","last_fullsized":"2024-09-17T15:33:11.224142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18884","reported_seq":27176,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:33.318152+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:34:33.318152+0000","last_peered":"2024-09-17T15:34:33.318152+0000","last_clean":"2024-09-17T15:34:33.318152+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:34:33.318152+0000","last_undegraded":"2024-09-17T15:34:33.318152+0000","last_fullsized":"2024-09-17T15:34:33.318152+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7918051,"num_objects":1934,"num_object_clones":0,"num_object_copies":3868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1934,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16884,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18509","reported_seq":18783,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:28.698837+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:34:28.698837+0000","last_peered":"2024-09-17T15:34:28.698837+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:34:28.698837+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'8500","ondisk_log_start":"43'8500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8327168,"num_objects":2033,"num_object_clones":0,"num_object_copies":4066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15105,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1910}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18660","reported_seq":18879,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:33.111855+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:34:33.111855+0000","last_peered":"2024-09-17T15:34:33.111855+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:34:33.111855+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'8600","ondisk_log_start":"43'8600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7614947,"num_objects":1860,"num_object_clones":0,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16555,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2173}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18862","reported_seq":27277,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:33.171133+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:34:33.171133+0000","last_peered":"2024-09-17T15:34:33.171133+0000","last_clean":"2024-09-17T15:34:33.171133+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:34:33.171133+0000","last_undegraded":"2024-09-17T15:34:33.171133+0000","last_fullsized":"2024-09-17T15:34:33.171133+0000","mapping_epoch":46,"log_start":"43'8800","ondisk_log_start":"43'8800","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7675904,"num_objects":1874,"num_object_clones":0,"num_object_copies":3748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16624,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18920","reported_seq":19227,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:28.769892+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:34:28.769892+0000","last_peered":"2024-09-17T15:34:28.769892+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:34:28.769892+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'8900","ondisk_log_start":"43'8900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8380416,"num_objects":2046,"num_object_clones":0,"num_object_copies":4092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2046,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15468,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2011}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18770","reported_seq":22405,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:28.783824+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:34:28.783824+0000","last_peered":"2024-09-17T15:34:28.783824+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:34:28.783824+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"43'8700","ondisk_log_start":"43'8700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8200192,"num_objects":2002,"num_object_clones":0,"num_object_copies":4004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2419,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15057,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1741,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-417}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18843","reported_seq":19151,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:28.695511+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:34:28.695511+0000","last_peered":"2024-09-17T15:34:28.695511+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:34:28.695511+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'8800","ondisk_log_start":"43'8800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8368128,"num_objects":2043,"num_object_clones":0,"num_object_copies":4086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15630,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1937}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18779","reported_seq":27163,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:33.314078+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:34:33.314078+0000","last_peered":"2024-09-17T15:34:33.314078+0000","last_clean":"2024-09-17T15:34:33.314078+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:34:33.314078+0000","last_undegraded":"2024-09-17T15:34:33.314078+0000","last_fullsized":"2024-09-17T15:34:33.314078+0000","mapping_epoch":44,"log_start":"43'8700","ondisk_log_start":"43'8700","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7590854,"num_objects":1855,"num_object_clones":0,"num_object_copies":3710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15221,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":129790710,"num_objects":31697,"num_object_clones":0,"num_object_copies":63394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34830,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31697,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":261290,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14630,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":458584064,"data_stored":458449465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160955,"ondisk_log_size":160955,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051698,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":536728,"kb_used_data":194048,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93835112,"statfs":{"total":96636764160,"available":96087154688,"internally_reserved":0,"allocated":198705152,"data_stored":198403554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":350875308},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724643,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":534832,"kb_used_data":92188,"kb_used_omap":4,"kb_used_meta":442619,"kb_avail":93837008,"statfs":{"total":96636764160,"available":96089096192,"internally_reserved":0,"allocated":94400512,"data_stored":94076772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":453242539},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,3,3,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247925,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":588888,"kb_used_data":164348,"kb_used_omap":4,"kb_used_meta":424507,"kb_avail":93782952,"statfs":{"total":96636764160,"available":96033742848,"internally_reserved":0,"allocated":168292352,"data_stored":167990749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":434695848},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,0,4,5,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":167489536,"data_stored":167453406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":92942336,"data_stored":92884069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":198152192,"data_stored":198111990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:34:36.995 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:34:37.557 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:34:37.558 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:34:37.587 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2024-09-17T15:34:35.786267+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":124056329,"num_objects":30299,"num_object_clones":0,"num_object_copies":60598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34717,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30299,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":262692,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14744,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160858,"ondisk_log_size":160858,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1661332,"kb_used_data":443160,"kb_used_omap":13,"kb_used_meta":1218098,"kb_avail":281454188,"statfs":{"total":289910292480,"available":288209088512,"internally_reserved":0,"allocated":453795840,"data_stored":452868899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1247333375},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,3,9,12,5,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25985024,"num_objects":-6344,"num_object_clones":0,"num_object_copies":-12688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-329,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6344,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":329,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008554"},"pg_stats":[{"pgid":"2.f","version":"47'18780","reported_seq":19040,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:33.802297+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:34:33.802297+0000","last_peered":"2024-09-17T15:34:33.802297+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:34:33.802297+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'8700","ondisk_log_start":"43'8700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7651328,"num_objects":1868,"num_object_clones":0,"num_object_copies":3736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18780,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2116}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18958","reported_seq":19071,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:33.121762+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:34:33.121762+0000","last_peered":"2024-09-17T15:34:33.121762+0000","last_clean":"2024-09-17T15:34:33.121762+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:34:33.121762+0000","last_undegraded":"2024-09-17T15:34:33.121762+0000","last_fullsized":"2024-09-17T15:34:33.121762+0000","mapping_epoch":33,"log_start":"43'8900","ondisk_log_start":"43'8900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8086470,"num_objects":1976,"num_object_clones":0,"num_object_copies":3952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1976,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17315,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18932","reported_seq":19151,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:33.157332+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:34:33.157332+0000","last_peered":"2024-09-17T15:34:33.157332+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:34:33.157332+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'8900","ondisk_log_start":"43'8900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8196120,"num_objects":2002,"num_object_clones":0,"num_object_copies":4004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2002,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17089,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2123}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19069","reported_seq":19149,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:33.157994+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:34:33.157994+0000","last_peered":"2024-09-17T15:34:33.157994+0000","last_clean":"2024-09-17T15:34:33.157994+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:34:33.157994+0000","last_undegraded":"2024-09-17T15:34:33.157994+0000","last_fullsized":"2024-09-17T15:34:33.157994+0000","mapping_epoch":33,"log_start":"43'9000","ondisk_log_start":"43'9000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7786496,"num_objects":1901,"num_object_clones":0,"num_object_copies":3802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19069,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18727","reported_seq":18866,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:33.781152+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:34:33.781152+0000","last_peered":"2024-09-17T15:34:33.781152+0000","last_clean":"2024-09-17T15:34:33.781152+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:34:33.781152+0000","last_undegraded":"2024-09-17T15:34:33.781152+0000","last_fullsized":"2024-09-17T15:34:33.781152+0000","mapping_epoch":32,"log_start":"43'8700","ondisk_log_start":"43'8700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7889379,"num_objects":1927,"num_object_clones":0,"num_object_copies":3854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1927,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15709,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19334","reported_seq":19635,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:33.821120+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:34:33.821120+0000","last_peered":"2024-09-17T15:34:33.821120+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:34:33.821120+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'9300","ondisk_log_start":"43'9300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7852515,"num_objects":1918,"num_object_clones":0,"num_object_copies":3836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1918,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15833,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2274},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18955","reported_seq":19274,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:33.743983+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:34:33.743983+0000","last_peered":"2024-09-17T15:34:33.743983+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:34:33.743983+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'8900","ondisk_log_start":"43'8900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7427014,"num_objects":1815,"num_object_clones":0,"num_object_copies":3630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1815,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15724,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2274},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19041","reported_seq":19179,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:33.755506+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:34:33.755506+0000","last_peered":"2024-09-17T15:34:33.755506+0000","last_clean":"2024-09-17T15:34:33.755506+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:34:33.755506+0000","last_undegraded":"2024-09-17T15:34:33.755506+0000","last_fullsized":"2024-09-17T15:34:33.755506+0000","mapping_epoch":32,"log_start":"43'9000","ondisk_log_start":"43'9000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7680000,"num_objects":1875,"num_object_clones":0,"num_object_copies":3750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15995,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.224142+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.224142+0000","last_peered":"2024-09-17T15:33:11.224142+0000","last_clean":"2024-09-17T15:33:11.224142+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.224142+0000","last_undegraded":"2024-09-17T15:33:11.224142+0000","last_fullsized":"2024-09-17T15:33:11.224142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18884","reported_seq":27176,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:33.318152+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:34:33.318152+0000","last_peered":"2024-09-17T15:34:33.318152+0000","last_clean":"2024-09-17T15:34:33.318152+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:34:33.318152+0000","last_undegraded":"2024-09-17T15:34:33.318152+0000","last_fullsized":"2024-09-17T15:34:33.318152+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7918051,"num_objects":1934,"num_object_clones":0,"num_object_copies":3868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1934,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16884,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18653","reported_seq":18925,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:33.834272+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:34:33.834272+0000","last_peered":"2024-09-17T15:34:33.834272+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:34:33.834272+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'8600","ondisk_log_start":"43'8600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7737344,"num_objects":1889,"num_object_clones":0,"num_object_copies":3778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15249,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2054}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18660","reported_seq":18879,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:33.111855+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:34:33.111855+0000","last_peered":"2024-09-17T15:34:33.111855+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:34:33.111855+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'8600","ondisk_log_start":"43'8600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7614947,"num_objects":1860,"num_object_clones":0,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16555,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2173}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18862","reported_seq":27277,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:33.171133+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:34:33.171133+0000","last_peered":"2024-09-17T15:34:33.171133+0000","last_clean":"2024-09-17T15:34:33.171133+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:34:33.171133+0000","last_undegraded":"2024-09-17T15:34:33.171133+0000","last_fullsized":"2024-09-17T15:34:33.171133+0000","mapping_epoch":46,"log_start":"43'8800","ondisk_log_start":"43'8800","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7675904,"num_objects":1874,"num_object_clones":0,"num_object_copies":3748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16624,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19073","reported_seq":19376,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:33.836994+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:34:33.836994+0000","last_peered":"2024-09-17T15:34:33.836994+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:34:33.836994+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'9000","ondisk_log_start":"43'9000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7753728,"num_objects":1893,"num_object_clones":0,"num_object_copies":3786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15621,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18941","reported_seq":22802,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:33.800313+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:34:33.800313+0000","last_peered":"2024-09-17T15:34:33.800313+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:34:33.800313+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"43'8900","ondisk_log_start":"43'8900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7499776,"num_objects":1831,"num_object_clones":0,"num_object_copies":3662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1831,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15228,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1854,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-475}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19007","reported_seq":19315,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:33.696415+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:34:33.696415+0000","last_peered":"2024-09-17T15:34:33.696415+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:34:33.696415+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'9000","ondisk_log_start":"43'9000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7696384,"num_objects":1879,"num_object_clones":0,"num_object_copies":3758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1879,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15794,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2101}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18779","reported_seq":27163,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:33.314078+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:34:33.314078+0000","last_peered":"2024-09-17T15:34:33.314078+0000","last_clean":"2024-09-17T15:34:33.314078+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:34:33.314078+0000","last_undegraded":"2024-09-17T15:34:33.314078+0000","last_fullsized":"2024-09-17T15:34:33.314078+0000","mapping_epoch":44,"log_start":"43'8700","ondisk_log_start":"43'8700","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7590854,"num_objects":1855,"num_object_clones":0,"num_object_copies":3710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15221,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":124056310,"num_objects":30297,"num_object_clones":0,"num_object_copies":60594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34717,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30297,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":262690,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14743,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":450969600,"data_stored":450835001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051698,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":536728,"kb_used_data":194048,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93835112,"statfs":{"total":96636764160,"available":96087154688,"internally_reserved":0,"allocated":198705152,"data_stored":198403554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":350875308},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724643,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":534832,"kb_used_data":92188,"kb_used_omap":4,"kb_used_meta":442619,"kb_avail":93837008,"statfs":{"total":96636764160,"available":96089096192,"internally_reserved":0,"allocated":94400512,"data_stored":94076772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":453242539},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,3,3,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247926,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589772,"kb_used_data":156924,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93782068,"statfs":{"total":96636764160,"available":96032837632,"internally_reserved":0,"allocated":160690176,"data_stored":160388573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":443215528},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,1,4,8,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":159875072,"data_stored":159838942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":92942336,"data_stored":92884069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":198152192,"data_stored":198111990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:34:37.589 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:34:38.142 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:34:38.142 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:34:38.168 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2024-09-17T15:34:37.787095+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":124056329,"num_objects":30299,"num_object_clones":0,"num_object_copies":60598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34717,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30299,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":262692,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14744,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160858,"ondisk_log_size":160858,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1661332,"kb_used_data":443160,"kb_used_omap":13,"kb_used_meta":1218098,"kb_avail":281454188,"statfs":{"total":289910292480,"available":288209088512,"internally_reserved":0,"allocated":453795840,"data_stored":452868899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1247333375},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,3,9,12,5,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20221952,"num_objects":-4937,"num_object_clones":0,"num_object_copies":-9874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-220,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4937,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4937,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":220,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008533"},"pg_stats":[{"pgid":"2.f","version":"47'18780","reported_seq":19040,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:33.802297+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:34:33.802297+0000","last_peered":"2024-09-17T15:34:33.802297+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:34:33.802297+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'8700","ondisk_log_start":"43'8700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7651328,"num_objects":1868,"num_object_clones":0,"num_object_copies":3736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18780,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2116}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18958","reported_seq":19071,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:33.121762+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:34:33.121762+0000","last_peered":"2024-09-17T15:34:33.121762+0000","last_clean":"2024-09-17T15:34:33.121762+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:34:33.121762+0000","last_undegraded":"2024-09-17T15:34:33.121762+0000","last_fullsized":"2024-09-17T15:34:33.121762+0000","mapping_epoch":33,"log_start":"43'8900","ondisk_log_start":"43'8900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8086470,"num_objects":1976,"num_object_clones":0,"num_object_copies":3952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1976,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17315,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18932","reported_seq":19151,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:33.157332+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:34:33.157332+0000","last_peered":"2024-09-17T15:34:33.157332+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:34:33.157332+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'8900","ondisk_log_start":"43'8900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8196120,"num_objects":2002,"num_object_clones":0,"num_object_copies":4004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2002,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17089,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2123}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19069","reported_seq":19149,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:33.157994+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:34:33.157994+0000","last_peered":"2024-09-17T15:34:33.157994+0000","last_clean":"2024-09-17T15:34:33.157994+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:34:33.157994+0000","last_undegraded":"2024-09-17T15:34:33.157994+0000","last_fullsized":"2024-09-17T15:34:33.157994+0000","mapping_epoch":33,"log_start":"43'9000","ondisk_log_start":"43'9000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7786496,"num_objects":1901,"num_object_clones":0,"num_object_copies":3802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19069,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18727","reported_seq":18866,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:33.781152+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:34:33.781152+0000","last_peered":"2024-09-17T15:34:33.781152+0000","last_clean":"2024-09-17T15:34:33.781152+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:34:33.781152+0000","last_undegraded":"2024-09-17T15:34:33.781152+0000","last_fullsized":"2024-09-17T15:34:33.781152+0000","mapping_epoch":32,"log_start":"43'8700","ondisk_log_start":"43'8700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7889379,"num_objects":1927,"num_object_clones":0,"num_object_copies":3854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1927,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15709,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19334","reported_seq":19635,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:33.821120+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:34:33.821120+0000","last_peered":"2024-09-17T15:34:33.821120+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:34:33.821120+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'9300","ondisk_log_start":"43'9300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7852515,"num_objects":1918,"num_object_clones":0,"num_object_copies":3836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1918,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15833,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2274},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18955","reported_seq":19274,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:33.743983+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:34:33.743983+0000","last_peered":"2024-09-17T15:34:33.743983+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:34:33.743983+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'8900","ondisk_log_start":"43'8900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7427014,"num_objects":1815,"num_object_clones":0,"num_object_copies":3630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1815,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15724,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2274},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19041","reported_seq":19179,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:33.755506+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:34:33.755506+0000","last_peered":"2024-09-17T15:34:33.755506+0000","last_clean":"2024-09-17T15:34:33.755506+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:34:33.755506+0000","last_undegraded":"2024-09-17T15:34:33.755506+0000","last_fullsized":"2024-09-17T15:34:33.755506+0000","mapping_epoch":32,"log_start":"43'9000","ondisk_log_start":"43'9000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7680000,"num_objects":1875,"num_object_clones":0,"num_object_copies":3750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15995,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.224142+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.224142+0000","last_peered":"2024-09-17T15:33:11.224142+0000","last_clean":"2024-09-17T15:33:11.224142+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.224142+0000","last_undegraded":"2024-09-17T15:33:11.224142+0000","last_fullsized":"2024-09-17T15:33:11.224142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18884","reported_seq":27176,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:33.318152+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:34:33.318152+0000","last_peered":"2024-09-17T15:34:33.318152+0000","last_clean":"2024-09-17T15:34:33.318152+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:34:33.318152+0000","last_undegraded":"2024-09-17T15:34:33.318152+0000","last_fullsized":"2024-09-17T15:34:33.318152+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7918051,"num_objects":1934,"num_object_clones":0,"num_object_copies":3868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1934,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16884,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18653","reported_seq":18925,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:33.834272+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:34:33.834272+0000","last_peered":"2024-09-17T15:34:33.834272+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:34:33.834272+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'8600","ondisk_log_start":"43'8600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7737344,"num_objects":1889,"num_object_clones":0,"num_object_copies":3778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15249,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2054}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18660","reported_seq":18879,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:33.111855+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:34:33.111855+0000","last_peered":"2024-09-17T15:34:33.111855+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:34:33.111855+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'8600","ondisk_log_start":"43'8600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7614947,"num_objects":1860,"num_object_clones":0,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16555,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2173}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18862","reported_seq":27277,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:33.171133+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:34:33.171133+0000","last_peered":"2024-09-17T15:34:33.171133+0000","last_clean":"2024-09-17T15:34:33.171133+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:34:33.171133+0000","last_undegraded":"2024-09-17T15:34:33.171133+0000","last_fullsized":"2024-09-17T15:34:33.171133+0000","mapping_epoch":46,"log_start":"43'8800","ondisk_log_start":"43'8800","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7675904,"num_objects":1874,"num_object_clones":0,"num_object_copies":3748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16624,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19073","reported_seq":19376,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:33.836994+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:34:33.836994+0000","last_peered":"2024-09-17T15:34:33.836994+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:34:33.836994+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'9000","ondisk_log_start":"43'9000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7753728,"num_objects":1893,"num_object_clones":0,"num_object_copies":3786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15621,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18941","reported_seq":22802,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:33.800313+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:34:33.800313+0000","last_peered":"2024-09-17T15:34:33.800313+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:34:33.800313+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"43'8900","ondisk_log_start":"43'8900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7499776,"num_objects":1831,"num_object_clones":0,"num_object_copies":3662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1831,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15228,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1854,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-475}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19007","reported_seq":19315,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:33.696415+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:34:33.696415+0000","last_peered":"2024-09-17T15:34:33.696415+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:34:33.696415+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'9000","ondisk_log_start":"43'9000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7696384,"num_objects":1879,"num_object_clones":0,"num_object_copies":3758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1879,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15794,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2101}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18779","reported_seq":27163,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:33.314078+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:34:33.314078+0000","last_peered":"2024-09-17T15:34:33.314078+0000","last_clean":"2024-09-17T15:34:33.314078+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:34:33.314078+0000","last_undegraded":"2024-09-17T15:34:33.314078+0000","last_fullsized":"2024-09-17T15:34:33.314078+0000","mapping_epoch":44,"log_start":"43'8700","ondisk_log_start":"43'8700","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7590854,"num_objects":1855,"num_object_clones":0,"num_object_copies":3710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15221,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":124056310,"num_objects":30297,"num_object_clones":0,"num_object_copies":60594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34717,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30297,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":262690,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14743,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":450969600,"data_stored":450835001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051698,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":536728,"kb_used_data":194048,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93835112,"statfs":{"total":96636764160,"available":96087154688,"internally_reserved":0,"allocated":198705152,"data_stored":198403554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":350875308},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724643,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":534832,"kb_used_data":92188,"kb_used_omap":4,"kb_used_meta":442619,"kb_avail":93837008,"statfs":{"total":96636764160,"available":96089096192,"internally_reserved":0,"allocated":94400512,"data_stored":94076772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":453242539},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,3,3,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247926,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589772,"kb_used_data":156924,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93782068,"statfs":{"total":96636764160,"available":96032837632,"internally_reserved":0,"allocated":160690176,"data_stored":160388573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":443215528},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,1,4,8,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":159875072,"data_stored":159838942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":92942336,"data_stored":92884069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":198152192,"data_stored":198111990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:34:38.169 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:34:38.170 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'18780", 'reported_seq': 19040, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:33.802297+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:34:33.802297+0000', 'last_peered': '2024-09-17T15:34:33.802297+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:34:33.802297+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'8700", 'ondisk_log_start': "43'8700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7651328, 'num_objects': 1868, 'num_object_clones': 0, 'num_object_copies': 3736, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1868, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18780, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2116}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:38.170 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:34:38.170 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'18932", 'reported_seq': 19151, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:33.157332+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:34:33.157332+0000', 'last_peered': '2024-09-17T15:34:33.157332+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:34:33.157332+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "43'8900", 'ondisk_log_start': "43'8900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8196120, 'num_objects': 2002, 'num_object_clones': 0, 'num_object_copies': 4004, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2002, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17089, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2123}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:34:38.170 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:34:38.170 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'19334", 'reported_seq': 19635, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:33.821120+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:34:33.821120+0000', 'last_peered': '2024-09-17T15:34:33.821120+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:34:33.821120+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "43'9300", 'ondisk_log_start': "43'9300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7852515, 'num_objects': 1918, 'num_object_clones': 0, 'num_object_copies': 3836, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1918, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15833, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2274}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:38.170 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:34:38.170 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'18955", 'reported_seq': 19274, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:33.743983+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:34:33.743983+0000', 'last_peered': '2024-09-17T15:34:33.743983+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:34:33.743983+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "43'8900", 'ondisk_log_start': "43'8900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7427014, 'num_objects': 1815, 'num_object_clones': 0, 'num_object_copies': 3630, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4089, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1815, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 15724, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 85, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2274}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:38.170 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:34:38.170 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'18653", 'reported_seq': 18925, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:33.834272+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:34:33.834272+0000', 'last_peered': '2024-09-17T15:34:33.834272+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:34:33.834272+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'8600", 'ondisk_log_start': "43'8600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7737344, 'num_objects': 1889, 'num_object_clones': 0, 'num_object_copies': 3778, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1889, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15249, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2054}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:38.170 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:34:38.171 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'18660", 'reported_seq': 18879, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:33.111855+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:34:33.111855+0000', 'last_peered': '2024-09-17T15:34:33.111855+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:34:33.111855+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "43'8600", 'ondisk_log_start': "43'8600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7614947, 'num_objects': 1860, 'num_object_clones': 0, 'num_object_copies': 3720, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4034, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1860, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16555, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -2173}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:34:38.171 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:34:38.171 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'19073", 'reported_seq': 19376, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:33.836994+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:34:33.836994+0000', 'last_peered': '2024-09-17T15:34:33.836994+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:34:33.836994+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "43'9000", 'ondisk_log_start': "43'9000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7753728, 'num_objects': 1893, 'num_object_clones': 0, 'num_object_copies': 3786, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1893, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15621, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:38.171 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:34:38.171 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'18941", 'reported_seq': 22802, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:33.800313+0000', 'last_change': '2024-09-17T15:33:09.763500+0000', 'last_active': '2024-09-17T15:34:33.800313+0000', 'last_peered': '2024-09-17T15:34:33.800313+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:34:33.800313+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "43'8900", 'ondisk_log_start': "43'8900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7499776, 'num_objects': 1831, 'num_object_clones': 0, 'num_object_copies': 3662, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2306, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1831, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15228, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1854, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -475}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:38.171 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:34:38.171 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'19007", 'reported_seq': 19315, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:33.696415+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:34:33.696415+0000', 'last_peered': '2024-09-17T15:34:33.696415+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:34:33.696415+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'9000", 'ondisk_log_start': "43'9000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7696384, 'num_objects': 1879, 'num_object_clones': 0, 'num_object_copies': 3758, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1879, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15794, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2101}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:38.172 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:34:38.851 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:34:38.851 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34717/60598 objects degraded (57.291%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":615,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":30299,"data_bytes":124056329,"bytes_used":1701203968,"bytes_avail":288209088512,"bytes_total":289910292480,"degraded_objects":34717,"degraded_total":60598,"degraded_ratio":0.57290669659064652,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":220,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":411},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:34:38.878 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:34:38.879 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:34:39.435 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:34:39.436 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:34:39.472 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2024-09-17T15:34:37.787095+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":124056329,"num_objects":30299,"num_object_clones":0,"num_object_copies":60598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34717,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30299,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":262692,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14744,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160858,"ondisk_log_size":160858,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1661332,"kb_used_data":443160,"kb_used_omap":13,"kb_used_meta":1218098,"kb_avail":281454188,"statfs":{"total":289910292480,"available":288209088512,"internally_reserved":0,"allocated":453795840,"data_stored":452868899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1247333375},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,3,9,12,5,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20221952,"num_objects":-4937,"num_object_clones":0,"num_object_copies":-9874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-220,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4937,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4937,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":220,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008533"},"pg_stats":[{"pgid":"2.f","version":"47'18780","reported_seq":19040,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:33.802297+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:34:33.802297+0000","last_peered":"2024-09-17T15:34:33.802297+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:34:33.802297+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'8700","ondisk_log_start":"43'8700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7651328,"num_objects":1868,"num_object_clones":0,"num_object_copies":3736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18780,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2116}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18958","reported_seq":19071,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:33.121762+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:34:33.121762+0000","last_peered":"2024-09-17T15:34:33.121762+0000","last_clean":"2024-09-17T15:34:33.121762+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:34:33.121762+0000","last_undegraded":"2024-09-17T15:34:33.121762+0000","last_fullsized":"2024-09-17T15:34:33.121762+0000","mapping_epoch":33,"log_start":"43'8900","ondisk_log_start":"43'8900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8086470,"num_objects":1976,"num_object_clones":0,"num_object_copies":3952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1976,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17315,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18932","reported_seq":19151,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:33.157332+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:34:33.157332+0000","last_peered":"2024-09-17T15:34:33.157332+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:34:33.157332+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'8900","ondisk_log_start":"43'8900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8196120,"num_objects":2002,"num_object_clones":0,"num_object_copies":4004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2002,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17089,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2123}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19069","reported_seq":19149,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:33.157994+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:34:33.157994+0000","last_peered":"2024-09-17T15:34:33.157994+0000","last_clean":"2024-09-17T15:34:33.157994+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:34:33.157994+0000","last_undegraded":"2024-09-17T15:34:33.157994+0000","last_fullsized":"2024-09-17T15:34:33.157994+0000","mapping_epoch":33,"log_start":"43'9000","ondisk_log_start":"43'9000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7786496,"num_objects":1901,"num_object_clones":0,"num_object_copies":3802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19069,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18727","reported_seq":18866,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:33.781152+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:34:33.781152+0000","last_peered":"2024-09-17T15:34:33.781152+0000","last_clean":"2024-09-17T15:34:33.781152+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:34:33.781152+0000","last_undegraded":"2024-09-17T15:34:33.781152+0000","last_fullsized":"2024-09-17T15:34:33.781152+0000","mapping_epoch":32,"log_start":"43'8700","ondisk_log_start":"43'8700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7889379,"num_objects":1927,"num_object_clones":0,"num_object_copies":3854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1927,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15709,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19334","reported_seq":19635,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:33.821120+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:34:33.821120+0000","last_peered":"2024-09-17T15:34:33.821120+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:34:33.821120+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'9300","ondisk_log_start":"43'9300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7852515,"num_objects":1918,"num_object_clones":0,"num_object_copies":3836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1918,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15833,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2274},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18955","reported_seq":19274,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:33.743983+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:34:33.743983+0000","last_peered":"2024-09-17T15:34:33.743983+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:34:33.743983+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'8900","ondisk_log_start":"43'8900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7427014,"num_objects":1815,"num_object_clones":0,"num_object_copies":3630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1815,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15724,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2274},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19041","reported_seq":19179,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:33.755506+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:34:33.755506+0000","last_peered":"2024-09-17T15:34:33.755506+0000","last_clean":"2024-09-17T15:34:33.755506+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:34:33.755506+0000","last_undegraded":"2024-09-17T15:34:33.755506+0000","last_fullsized":"2024-09-17T15:34:33.755506+0000","mapping_epoch":32,"log_start":"43'9000","ondisk_log_start":"43'9000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7680000,"num_objects":1875,"num_object_clones":0,"num_object_copies":3750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15995,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.224142+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.224142+0000","last_peered":"2024-09-17T15:33:11.224142+0000","last_clean":"2024-09-17T15:33:11.224142+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.224142+0000","last_undegraded":"2024-09-17T15:33:11.224142+0000","last_fullsized":"2024-09-17T15:33:11.224142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18884","reported_seq":27176,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:33.318152+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:34:33.318152+0000","last_peered":"2024-09-17T15:34:33.318152+0000","last_clean":"2024-09-17T15:34:33.318152+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:34:33.318152+0000","last_undegraded":"2024-09-17T15:34:33.318152+0000","last_fullsized":"2024-09-17T15:34:33.318152+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7918051,"num_objects":1934,"num_object_clones":0,"num_object_copies":3868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1934,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16884,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18653","reported_seq":18925,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:33.834272+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:34:33.834272+0000","last_peered":"2024-09-17T15:34:33.834272+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:34:33.834272+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'8600","ondisk_log_start":"43'8600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7737344,"num_objects":1889,"num_object_clones":0,"num_object_copies":3778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15249,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2054}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18660","reported_seq":18879,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:33.111855+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:34:33.111855+0000","last_peered":"2024-09-17T15:34:33.111855+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:34:33.111855+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'8600","ondisk_log_start":"43'8600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7614947,"num_objects":1860,"num_object_clones":0,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16555,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2173}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18862","reported_seq":27277,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:33.171133+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:34:33.171133+0000","last_peered":"2024-09-17T15:34:33.171133+0000","last_clean":"2024-09-17T15:34:33.171133+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:34:33.171133+0000","last_undegraded":"2024-09-17T15:34:33.171133+0000","last_fullsized":"2024-09-17T15:34:33.171133+0000","mapping_epoch":46,"log_start":"43'8800","ondisk_log_start":"43'8800","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7675904,"num_objects":1874,"num_object_clones":0,"num_object_copies":3748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16624,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19073","reported_seq":19376,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:33.836994+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:34:33.836994+0000","last_peered":"2024-09-17T15:34:33.836994+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:34:33.836994+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'9000","ondisk_log_start":"43'9000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7753728,"num_objects":1893,"num_object_clones":0,"num_object_copies":3786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15621,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18941","reported_seq":22802,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:33.800313+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:34:33.800313+0000","last_peered":"2024-09-17T15:34:33.800313+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:34:33.800313+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"43'8900","ondisk_log_start":"43'8900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7499776,"num_objects":1831,"num_object_clones":0,"num_object_copies":3662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1831,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15228,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1854,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-475}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19007","reported_seq":19315,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:33.696415+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:34:33.696415+0000","last_peered":"2024-09-17T15:34:33.696415+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:34:33.696415+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'9000","ondisk_log_start":"43'9000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7696384,"num_objects":1879,"num_object_clones":0,"num_object_copies":3758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1879,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15794,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2101}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18779","reported_seq":27163,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:33.314078+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:34:33.314078+0000","last_peered":"2024-09-17T15:34:33.314078+0000","last_clean":"2024-09-17T15:34:33.314078+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:34:33.314078+0000","last_undegraded":"2024-09-17T15:34:33.314078+0000","last_fullsized":"2024-09-17T15:34:33.314078+0000","mapping_epoch":44,"log_start":"43'8700","ondisk_log_start":"43'8700","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7590854,"num_objects":1855,"num_object_clones":0,"num_object_copies":3710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15221,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":124056310,"num_objects":30297,"num_object_clones":0,"num_object_copies":60594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34717,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30297,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":262690,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14743,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":450969600,"data_stored":450835001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051698,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":536728,"kb_used_data":194048,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93835112,"statfs":{"total":96636764160,"available":96087154688,"internally_reserved":0,"allocated":198705152,"data_stored":198403554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":350875308},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724643,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":534832,"kb_used_data":92188,"kb_used_omap":4,"kb_used_meta":442619,"kb_avail":93837008,"statfs":{"total":96636764160,"available":96089096192,"internally_reserved":0,"allocated":94400512,"data_stored":94076772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":453242539},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,3,3,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247926,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589772,"kb_used_data":156924,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93782068,"statfs":{"total":96636764160,"available":96032837632,"internally_reserved":0,"allocated":160690176,"data_stored":160388573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":443215528},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,1,4,8,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":159875072,"data_stored":159838942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":92942336,"data_stored":92884069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":198152192,"data_stored":198111990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:34:42.474 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:34:43.010 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:34:43.012 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:34:43.039 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":349,"stamp":"2024-09-17T15:34:41.789119+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":113951497,"num_objects":27832,"num_object_clones":0,"num_object_copies":55664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34600,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27832,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":265159,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14861,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160825,"ondisk_log_size":160825,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1658488,"kb_used_data":423668,"kb_used_omap":13,"kb_used_meta":1234738,"kb_avail":281457032,"statfs":{"total":289910292480,"available":288212000768,"internally_reserved":0,"allocated":433836032,"data_stored":432909091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1264372735},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,4,1,3,8,10,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20275200,"num_objects":-4950,"num_object_clones":0,"num_object_copies":-9900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-230,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4950,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008953"},"pg_stats":[{"pgid":"2.f","version":"47'18937","reported_seq":19196,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:38.843662+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:34:38.843662+0000","last_peered":"2024-09-17T15:34:38.843662+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:34:38.843662+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'8900","ondisk_log_start":"43'8900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7008256,"num_objects":1711,"num_object_clones":0,"num_object_copies":3422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18937,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2273}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19114","reported_seq":19227,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:38.158619+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:34:38.158619+0000","last_peered":"2024-09-17T15:34:38.158619+0000","last_clean":"2024-09-17T15:34:38.158619+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:34:38.158619+0000","last_undegraded":"2024-09-17T15:34:38.158619+0000","last_fullsized":"2024-09-17T15:34:38.158619+0000","mapping_epoch":33,"log_start":"43'9100","ondisk_log_start":"43'9100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7447494,"num_objects":1820,"num_object_clones":0,"num_object_copies":3640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1820,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17471,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19081","reported_seq":19303,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:38.175839+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:34:38.175839+0000","last_peered":"2024-09-17T15:34:38.175839+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:34:38.175839+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'9000","ondisk_log_start":"43'9000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7585816,"num_objects":1853,"num_object_clones":0,"num_object_copies":3706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1853,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17238,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2272}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19213","reported_seq":19293,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:38.197276+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:34:38.197276+0000","last_peered":"2024-09-17T15:34:38.197276+0000","last_clean":"2024-09-17T15:34:38.197276+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:34:38.197276+0000","last_undegraded":"2024-09-17T15:34:38.197276+0000","last_fullsized":"2024-09-17T15:34:38.197276+0000","mapping_epoch":33,"log_start":"43'9200","ondisk_log_start":"43'9200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7196672,"num_objects":1757,"num_object_clones":0,"num_object_copies":3514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19213,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18890","reported_seq":19029,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:38.813363+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:34:38.813363+0000","last_peered":"2024-09-17T15:34:38.813363+0000","last_clean":"2024-09-17T15:34:38.813363+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:34:38.813363+0000","last_undegraded":"2024-09-17T15:34:38.813363+0000","last_fullsized":"2024-09-17T15:34:38.813363+0000","mapping_epoch":32,"log_start":"43'8800","ondisk_log_start":"43'8800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7221731,"num_objects":1764,"num_object_clones":0,"num_object_copies":3528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1764,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15872,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19496","reported_seq":19797,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:38.821166+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:34:38.821166+0000","last_peered":"2024-09-17T15:34:38.821166+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:34:38.821166+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'9400","ondisk_log_start":"43'9400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7188963,"num_objects":1756,"num_object_clones":0,"num_object_copies":3512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1756,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15995,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2436},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19094","reported_seq":19411,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:38.869937+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:34:38.869937+0000","last_peered":"2024-09-17T15:34:38.869937+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:34:38.869937+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'9000","ondisk_log_start":"43'9000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6857670,"num_objects":1676,"num_object_clones":0,"num_object_copies":3352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1676,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15863,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2413},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19209","reported_seq":19347,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:38.842403+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:34:38.842403+0000","last_peered":"2024-09-17T15:34:38.842403+0000","last_clean":"2024-09-17T15:34:38.842403+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:34:38.842403+0000","last_undegraded":"2024-09-17T15:34:38.842403+0000","last_fullsized":"2024-09-17T15:34:38.842403+0000","mapping_epoch":32,"log_start":"43'9200","ondisk_log_start":"43'9200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6991872,"num_objects":1707,"num_object_clones":0,"num_object_copies":3414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16163,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.224142+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.224142+0000","last_peered":"2024-09-17T15:33:11.224142+0000","last_clean":"2024-09-17T15:33:11.224142+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.224142+0000","last_undegraded":"2024-09-17T15:33:11.224142+0000","last_fullsized":"2024-09-17T15:33:11.224142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19056","reported_seq":27348,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:38.426314+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:34:38.426314+0000","last_peered":"2024-09-17T15:34:38.426314+0000","last_clean":"2024-09-17T15:34:38.426314+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:34:38.426314+0000","last_undegraded":"2024-09-17T15:34:38.426314+0000","last_fullsized":"2024-09-17T15:34:38.426314+0000","mapping_epoch":42,"log_start":"43'9000","ondisk_log_start":"43'9000","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7213539,"num_objects":1762,"num_object_clones":0,"num_object_copies":3524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1762,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17056,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18810","reported_seq":19085,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:38.833102+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:34:38.833102+0000","last_peered":"2024-09-17T15:34:38.833102+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:34:38.833102+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'8800","ondisk_log_start":"43'8800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7094272,"num_objects":1732,"num_object_clones":0,"num_object_copies":3464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15406,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2211}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18824","reported_seq":19043,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:38.138401+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:34:38.138401+0000","last_peered":"2024-09-17T15:34:38.138401+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:34:38.138401+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'8800","ondisk_log_start":"43'8800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6943203,"num_objects":1696,"num_object_clones":0,"num_object_copies":3392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1696,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16719,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2337}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18997","reported_seq":27412,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:38.164156+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:34:38.164156+0000","last_peered":"2024-09-17T15:34:38.164156+0000","last_clean":"2024-09-17T15:34:38.164156+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:34:38.164156+0000","last_undegraded":"2024-09-17T15:34:38.164156+0000","last_fullsized":"2024-09-17T15:34:38.164156+0000","mapping_epoch":46,"log_start":"43'8900","ondisk_log_start":"43'8900","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7122944,"num_objects":1739,"num_object_clones":0,"num_object_copies":3478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16759,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19234","reported_seq":19541,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:38.775234+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:34:38.775234+0000","last_peered":"2024-09-17T15:34:38.775234+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:34:38.775234+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'9200","ondisk_log_start":"43'9200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7094272,"num_objects":1732,"num_object_clones":0,"num_object_copies":3464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15782,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2325}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19098","reported_seq":23192,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:38.882081+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:34:38.882081+0000","last_peered":"2024-09-17T15:34:38.882081+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:34:38.882081+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"43'9000","ondisk_log_start":"43'9000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6856704,"num_objects":1674,"num_object_clones":0,"num_object_copies":3348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2189,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15385,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1971,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19153","reported_seq":19461,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:38.853379+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:34:38.853379+0000","last_peered":"2024-09-17T15:34:38.853379+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:34:38.853379+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'9100","ondisk_log_start":"43'9100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7098368,"num_objects":1733,"num_object_clones":0,"num_object_copies":3466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1733,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15940,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2247}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18916","reported_seq":27300,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:38.407771+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:34:38.407771+0000","last_peered":"2024-09-17T15:34:38.407771+0000","last_clean":"2024-09-17T15:34:38.407771+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:34:38.407771+0000","last_undegraded":"2024-09-17T15:34:38.407771+0000","last_fullsized":"2024-09-17T15:34:38.407771+0000","mapping_epoch":44,"log_start":"43'8900","ondisk_log_start":"43'8900","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7029702,"num_objects":1718,"num_object_clones":0,"num_object_copies":3436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15358,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":113951478,"num_objects":27830,"num_object_clones":0,"num_object_copies":55660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34600,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27830,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":265157,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14860,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430649344,"data_stored":430514745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160822,"ondisk_log_size":160822,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051699,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535944,"kb_used_data":189104,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93835896,"statfs":{"total":96636764160,"available":96087957504,"internally_reserved":0,"allocated":193642496,"data_stored":193340898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":355135148},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724644,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535868,"kb_used_data":84904,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93835972,"statfs":{"total":96636764160,"available":96088035328,"internally_reserved":0,"allocated":86941696,"data_stored":86617956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":461762219},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,5,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247927,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":586676,"kb_used_data":149660,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93785164,"statfs":{"total":96636764160,"available":96036007936,"internally_reserved":0,"allocated":153251840,"data_stored":152950237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":447475368},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,3,3,5,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":152268800,"data_stored":152232670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":85348352,"data_stored":85290085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":193032192,"data_stored":192991990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:34:43.041 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:34:43.627 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:34:43.627 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:34:43.660 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":349,"stamp":"2024-09-17T15:34:41.789119+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":113951497,"num_objects":27832,"num_object_clones":0,"num_object_copies":55664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34600,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27832,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":265159,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14861,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160825,"ondisk_log_size":160825,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1658488,"kb_used_data":423668,"kb_used_omap":13,"kb_used_meta":1234738,"kb_avail":281457032,"statfs":{"total":289910292480,"available":288212000768,"internally_reserved":0,"allocated":433836032,"data_stored":432909091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1264372735},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,4,1,3,8,10,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20275200,"num_objects":-4950,"num_object_clones":0,"num_object_copies":-9900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-230,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4950,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008953"},"pg_stats":[{"pgid":"2.f","version":"47'18937","reported_seq":19196,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:38.843662+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:34:38.843662+0000","last_peered":"2024-09-17T15:34:38.843662+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:34:38.843662+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'8900","ondisk_log_start":"43'8900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7008256,"num_objects":1711,"num_object_clones":0,"num_object_copies":3422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18937,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2273}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19114","reported_seq":19227,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:38.158619+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:34:38.158619+0000","last_peered":"2024-09-17T15:34:38.158619+0000","last_clean":"2024-09-17T15:34:38.158619+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:34:38.158619+0000","last_undegraded":"2024-09-17T15:34:38.158619+0000","last_fullsized":"2024-09-17T15:34:38.158619+0000","mapping_epoch":33,"log_start":"43'9100","ondisk_log_start":"43'9100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7447494,"num_objects":1820,"num_object_clones":0,"num_object_copies":3640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1820,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17471,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19081","reported_seq":19303,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:38.175839+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:34:38.175839+0000","last_peered":"2024-09-17T15:34:38.175839+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:34:38.175839+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'9000","ondisk_log_start":"43'9000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7585816,"num_objects":1853,"num_object_clones":0,"num_object_copies":3706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1853,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17238,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2272}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19213","reported_seq":19293,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:38.197276+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:34:38.197276+0000","last_peered":"2024-09-17T15:34:38.197276+0000","last_clean":"2024-09-17T15:34:38.197276+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:34:38.197276+0000","last_undegraded":"2024-09-17T15:34:38.197276+0000","last_fullsized":"2024-09-17T15:34:38.197276+0000","mapping_epoch":33,"log_start":"43'9200","ondisk_log_start":"43'9200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7196672,"num_objects":1757,"num_object_clones":0,"num_object_copies":3514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19213,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18890","reported_seq":19029,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:38.813363+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:34:38.813363+0000","last_peered":"2024-09-17T15:34:38.813363+0000","last_clean":"2024-09-17T15:34:38.813363+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:34:38.813363+0000","last_undegraded":"2024-09-17T15:34:38.813363+0000","last_fullsized":"2024-09-17T15:34:38.813363+0000","mapping_epoch":32,"log_start":"43'8800","ondisk_log_start":"43'8800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7221731,"num_objects":1764,"num_object_clones":0,"num_object_copies":3528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1764,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15872,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19496","reported_seq":19797,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:38.821166+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:34:38.821166+0000","last_peered":"2024-09-17T15:34:38.821166+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:34:38.821166+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'9400","ondisk_log_start":"43'9400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7188963,"num_objects":1756,"num_object_clones":0,"num_object_copies":3512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1756,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15995,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2436},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19094","reported_seq":19411,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:38.869937+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:34:38.869937+0000","last_peered":"2024-09-17T15:34:38.869937+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:34:38.869937+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'9000","ondisk_log_start":"43'9000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6857670,"num_objects":1676,"num_object_clones":0,"num_object_copies":3352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1676,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15863,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2413},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19209","reported_seq":19347,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:38.842403+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:34:38.842403+0000","last_peered":"2024-09-17T15:34:38.842403+0000","last_clean":"2024-09-17T15:34:38.842403+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:34:38.842403+0000","last_undegraded":"2024-09-17T15:34:38.842403+0000","last_fullsized":"2024-09-17T15:34:38.842403+0000","mapping_epoch":32,"log_start":"43'9200","ondisk_log_start":"43'9200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6991872,"num_objects":1707,"num_object_clones":0,"num_object_copies":3414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16163,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.224142+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.224142+0000","last_peered":"2024-09-17T15:33:11.224142+0000","last_clean":"2024-09-17T15:33:11.224142+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.224142+0000","last_undegraded":"2024-09-17T15:33:11.224142+0000","last_fullsized":"2024-09-17T15:33:11.224142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19056","reported_seq":27348,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:38.426314+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:34:38.426314+0000","last_peered":"2024-09-17T15:34:38.426314+0000","last_clean":"2024-09-17T15:34:38.426314+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:34:38.426314+0000","last_undegraded":"2024-09-17T15:34:38.426314+0000","last_fullsized":"2024-09-17T15:34:38.426314+0000","mapping_epoch":42,"log_start":"43'9000","ondisk_log_start":"43'9000","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7213539,"num_objects":1762,"num_object_clones":0,"num_object_copies":3524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1762,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17056,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18810","reported_seq":19085,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:38.833102+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:34:38.833102+0000","last_peered":"2024-09-17T15:34:38.833102+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:34:38.833102+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'8800","ondisk_log_start":"43'8800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7094272,"num_objects":1732,"num_object_clones":0,"num_object_copies":3464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15406,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2211}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18824","reported_seq":19043,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:38.138401+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:34:38.138401+0000","last_peered":"2024-09-17T15:34:38.138401+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:34:38.138401+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'8800","ondisk_log_start":"43'8800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6943203,"num_objects":1696,"num_object_clones":0,"num_object_copies":3392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1696,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16719,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2337}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":87,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223472+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223472+0000","last_peered":"2024-09-17T15:33:11.223472+0000","last_clean":"2024-09-17T15:33:11.223472+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223472+0000","last_undegraded":"2024-09-17T15:33:11.223472+0000","last_fullsized":"2024-09-17T15:33:11.223472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18997","reported_seq":27412,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:38.164156+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:34:38.164156+0000","last_peered":"2024-09-17T15:34:38.164156+0000","last_clean":"2024-09-17T15:34:38.164156+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:34:38.164156+0000","last_undegraded":"2024-09-17T15:34:38.164156+0000","last_fullsized":"2024-09-17T15:34:38.164156+0000","mapping_epoch":46,"log_start":"43'8900","ondisk_log_start":"43'8900","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7122944,"num_objects":1739,"num_object_clones":0,"num_object_copies":3478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16759,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19234","reported_seq":19541,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:38.775234+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:34:38.775234+0000","last_peered":"2024-09-17T15:34:38.775234+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:34:38.775234+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'9200","ondisk_log_start":"43'9200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7094272,"num_objects":1732,"num_object_clones":0,"num_object_copies":3464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15782,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2325}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223771+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223771+0000","last_peered":"2024-09-17T15:33:11.223771+0000","last_clean":"2024-09-17T15:33:11.223771+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223771+0000","last_undegraded":"2024-09-17T15:33:11.223771+0000","last_fullsized":"2024-09-17T15:33:11.223771+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19098","reported_seq":23192,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:38.882081+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:34:38.882081+0000","last_peered":"2024-09-17T15:34:38.882081+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:34:38.882081+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"43'9000","ondisk_log_start":"43'9000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6856704,"num_objects":1674,"num_object_clones":0,"num_object_copies":3348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2189,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15385,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1971,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19153","reported_seq":19461,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:38.853379+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:34:38.853379+0000","last_peered":"2024-09-17T15:34:38.853379+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:34:38.853379+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'9100","ondisk_log_start":"43'9100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7098368,"num_objects":1733,"num_object_clones":0,"num_object_copies":3466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1733,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15940,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2247}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18916","reported_seq":27300,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:38.407771+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:34:38.407771+0000","last_peered":"2024-09-17T15:34:38.407771+0000","last_clean":"2024-09-17T15:34:38.407771+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:34:38.407771+0000","last_undegraded":"2024-09-17T15:34:38.407771+0000","last_fullsized":"2024-09-17T15:34:38.407771+0000","mapping_epoch":44,"log_start":"43'8900","ondisk_log_start":"43'8900","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7029702,"num_objects":1718,"num_object_clones":0,"num_object_copies":3436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15358,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":113951478,"num_objects":27830,"num_object_clones":0,"num_object_copies":55660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34600,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27830,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":265157,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14860,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430649344,"data_stored":430514745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160822,"ondisk_log_size":160822,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051699,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535944,"kb_used_data":189104,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93835896,"statfs":{"total":96636764160,"available":96087957504,"internally_reserved":0,"allocated":193642496,"data_stored":193340898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":355135148},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724644,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535868,"kb_used_data":84904,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93835972,"statfs":{"total":96636764160,"available":96088035328,"internally_reserved":0,"allocated":86941696,"data_stored":86617956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":461762219},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,5,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247927,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":586676,"kb_used_data":149660,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93785164,"statfs":{"total":96636764160,"available":96036007936,"internally_reserved":0,"allocated":153251840,"data_stored":152950237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":447475368},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,3,3,5,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":152268800,"data_stored":152232670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":85348352,"data_stored":85290085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":193032192,"data_stored":192991990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:34:43.661 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:34:43.661 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'18937", 'reported_seq': 19196, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:38.843662+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:34:38.843662+0000', 'last_peered': '2024-09-17T15:34:38.843662+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:34:38.843662+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'8900", 'ondisk_log_start': "43'8900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7008256, 'num_objects': 1711, 'num_object_clones': 0, 'num_object_copies': 3422, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1711, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18937, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2273}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:43.661 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:34:43.661 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'19081", 'reported_seq': 19303, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:38.175839+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:34:38.175839+0000', 'last_peered': '2024-09-17T15:34:38.175839+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:34:38.175839+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "43'9000", 'ondisk_log_start': "43'9000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7585816, 'num_objects': 1853, 'num_object_clones': 0, 'num_object_copies': 3706, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1853, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17238, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2272}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:34:43.662 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:34:43.662 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'19496", 'reported_seq': 19797, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:38.821166+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:34:38.821166+0000', 'last_peered': '2024-09-17T15:34:38.821166+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:34:38.821166+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "43'9400", 'ondisk_log_start': "43'9400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7188963, 'num_objects': 1756, 'num_object_clones': 0, 'num_object_copies': 3512, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1756, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15995, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2436}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:43.662 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:34:43.662 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'19094", 'reported_seq': 19411, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:38.869937+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:34:38.869937+0000', 'last_peered': '2024-09-17T15:34:38.869937+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:34:38.869937+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "43'9000", 'ondisk_log_start': "43'9000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6857670, 'num_objects': 1676, 'num_object_clones': 0, 'num_object_copies': 3352, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4089, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1676, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 15863, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 85, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2413}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:43.662 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:34:43.662 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'18810", 'reported_seq': 19085, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:38.833102+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:34:38.833102+0000', 'last_peered': '2024-09-17T15:34:38.833102+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:34:38.833102+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'8800", 'ondisk_log_start': "43'8800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7094272, 'num_objects': 1732, 'num_object_clones': 0, 'num_object_copies': 3464, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1732, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15406, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2211}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:43.662 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:34:43.662 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'18824", 'reported_seq': 19043, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:38.138401+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:34:38.138401+0000', 'last_peered': '2024-09-17T15:34:38.138401+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:34:38.138401+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "43'8800", 'ondisk_log_start': "43'8800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6943203, 'num_objects': 1696, 'num_object_clones': 0, 'num_object_copies': 3392, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4034, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1696, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16719, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -2337}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:34:43.662 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:34:43.662 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'19234", 'reported_seq': 19541, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:38.775234+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:34:38.775234+0000', 'last_peered': '2024-09-17T15:34:38.775234+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:34:38.775234+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "43'9200", 'ondisk_log_start': "43'9200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7094272, 'num_objects': 1732, 'num_object_clones': 0, 'num_object_copies': 3464, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1732, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15782, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2325}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:43.663 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:34:43.663 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'19098", 'reported_seq': 23192, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:38.882081+0000', 'last_change': '2024-09-17T15:33:09.763500+0000', 'last_active': '2024-09-17T15:34:38.882081+0000', 'last_peered': '2024-09-17T15:34:38.882081+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:34:38.882081+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "43'9000", 'ondisk_log_start': "43'9000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6856704, 'num_objects': 1674, 'num_object_clones': 0, 'num_object_copies': 3348, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2189, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1674, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15385, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1971, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -515}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:43.663 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:34:43.663 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'19153", 'reported_seq': 19461, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:38.853379+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:34:38.853379+0000', 'last_peered': '2024-09-17T15:34:38.853379+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:34:38.853379+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'9100", 'ondisk_log_start': "43'9100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7098368, 'num_objects': 1733, 'num_object_clones': 0, 'num_object_copies': 3466, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1733, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15940, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2247}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:43.663 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:34:44.355 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:34:44.355 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34600/53518 objects degraded (64.651%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":621,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":26759,"data_bytes":109556489,"bytes_used":1677123584,"bytes_avail":288233168896,"bytes_total":289910292480,"degraded_objects":34600,"degraded_total":53518,"degraded_ratio":0.64651145409021249,"recovering_objects_per_sec":19,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":230,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":501},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:34:44.390 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:34:44.390 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:34:44.977 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:34:44.978 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:34:45.003 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":350,"stamp":"2024-09-17T15:34:43.789665+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":109556489,"num_objects":26759,"num_object_clones":0,"num_object_copies":53518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34600,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26759,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":266232,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14861,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160898,"ondisk_log_size":160898,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1637816,"kb_used_data":411324,"kb_used_omap":13,"kb_used_meta":1226418,"kb_avail":281477704,"statfs":{"total":289910292480,"available":288233168896,"internally_reserved":0,"allocated":421195776,"data_stored":420268835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1255853055},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8,3,1,3,8,12,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24670208,"num_objects":-6023,"num_object_clones":0,"num_object_copies":-12046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-230,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6023,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6023,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007806"},"pg_stats":[{"pgid":"2.f","version":"47'18937","reported_seq":19196,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:38.843662+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:34:38.843662+0000","last_peered":"2024-09-17T15:34:38.843662+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:34:38.843662+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'8900","ondisk_log_start":"43'8900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7008256,"num_objects":1711,"num_object_clones":0,"num_object_copies":3422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18937,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2273}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19274","reported_seq":19387,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:43.212667+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:34:43.212667+0000","last_peered":"2024-09-17T15:34:43.212667+0000","last_clean":"2024-09-17T15:34:43.212667+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:34:43.212667+0000","last_undegraded":"2024-09-17T15:34:43.212667+0000","last_fullsized":"2024-09-17T15:34:43.212667+0000","mapping_epoch":33,"log_start":"43'9200","ondisk_log_start":"43'9200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6792134,"num_objects":1660,"num_object_clones":0,"num_object_copies":3320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1660,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17631,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19255","reported_seq":19473,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:43.125196+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:34:43.125196+0000","last_peered":"2024-09-17T15:34:43.125196+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:34:43.125196+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'9200","ondisk_log_start":"43'9200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6873112,"num_objects":1679,"num_object_clones":0,"num_object_copies":3358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1679,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17412,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2446}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19367","reported_seq":19447,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:43.227775+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:34:43.227775+0000","last_peered":"2024-09-17T15:34:43.227775+0000","last_clean":"2024-09-17T15:34:43.227775+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:34:43.227775+0000","last_undegraded":"2024-09-17T15:34:43.227775+0000","last_fullsized":"2024-09-17T15:34:43.227775+0000","mapping_epoch":33,"log_start":"43'9300","ondisk_log_start":"43'9300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6565888,"num_objects":1603,"num_object_clones":0,"num_object_copies":3206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1603,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19367,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18890","reported_seq":19029,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:38.813363+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:34:38.813363+0000","last_peered":"2024-09-17T15:34:38.813363+0000","last_clean":"2024-09-17T15:34:38.813363+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:34:38.813363+0000","last_undegraded":"2024-09-17T15:34:38.813363+0000","last_fullsized":"2024-09-17T15:34:38.813363+0000","mapping_epoch":32,"log_start":"43'8800","ondisk_log_start":"43'8800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7221731,"num_objects":1764,"num_object_clones":0,"num_object_copies":3528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1764,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15872,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19496","reported_seq":19797,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:38.821166+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:34:38.821166+0000","last_peered":"2024-09-17T15:34:38.821166+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:34:38.821166+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'9400","ondisk_log_start":"43'9400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7188963,"num_objects":1756,"num_object_clones":0,"num_object_copies":3512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1756,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15995,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2436},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19094","reported_seq":19411,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:38.869937+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:34:38.869937+0000","last_peered":"2024-09-17T15:34:38.869937+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:34:38.869937+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'9000","ondisk_log_start":"43'9000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6857670,"num_objects":1676,"num_object_clones":0,"num_object_copies":3352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1676,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15863,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2413},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19209","reported_seq":19347,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:38.842403+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:34:38.842403+0000","last_peered":"2024-09-17T15:34:38.842403+0000","last_clean":"2024-09-17T15:34:38.842403+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:34:38.842403+0000","last_undegraded":"2024-09-17T15:34:38.842403+0000","last_fullsized":"2024-09-17T15:34:38.842403+0000","mapping_epoch":32,"log_start":"43'9200","ondisk_log_start":"43'9200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6991872,"num_objects":1707,"num_object_clones":0,"num_object_copies":3414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16163,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.224142+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.224142+0000","last_peered":"2024-09-17T15:33:11.224142+0000","last_clean":"2024-09-17T15:33:11.224142+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.224142+0000","last_undegraded":"2024-09-17T15:33:11.224142+0000","last_fullsized":"2024-09-17T15:33:11.224142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19216","reported_seq":27508,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:43.539257+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:34:43.539257+0000","last_peered":"2024-09-17T15:34:43.539257+0000","last_clean":"2024-09-17T15:34:43.539257+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:34:43.539257+0000","last_undegraded":"2024-09-17T15:34:43.539257+0000","last_fullsized":"2024-09-17T15:34:43.539257+0000","mapping_epoch":42,"log_start":"43'9200","ondisk_log_start":"43'9200","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6558179,"num_objects":1602,"num_object_clones":0,"num_object_copies":3204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1602,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17216,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18810","reported_seq":19085,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:38.833102+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:34:38.833102+0000","last_peered":"2024-09-17T15:34:38.833102+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:34:38.833102+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'8800","ondisk_log_start":"43'8800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7094272,"num_objects":1732,"num_object_clones":0,"num_object_copies":3464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15406,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2211}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18967","reported_seq":19186,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:43.133956+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:34:43.133956+0000","last_peered":"2024-09-17T15:34:43.133956+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:34:43.133956+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'8900","ondisk_log_start":"43'8900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6357475,"num_objects":1553,"num_object_clones":0,"num_object_copies":3106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1553,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16862,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2480}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":88,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223926+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223926+0000","last_peered":"2024-09-17T15:33:11.223926+0000","last_clean":"2024-09-17T15:33:11.223926+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223926+0000","last_undegraded":"2024-09-17T15:33:11.223926+0000","last_fullsized":"2024-09-17T15:33:11.223926+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19143","reported_seq":27558,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:43.225832+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:34:43.225832+0000","last_peered":"2024-09-17T15:34:43.225832+0000","last_clean":"2024-09-17T15:34:43.225832+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:34:43.225832+0000","last_undegraded":"2024-09-17T15:34:43.225832+0000","last_fullsized":"2024-09-17T15:34:43.225832+0000","mapping_epoch":46,"log_start":"43'9100","ondisk_log_start":"43'9100","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6524928,"num_objects":1593,"num_object_clones":0,"num_object_copies":3186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16905,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19234","reported_seq":19541,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:38.775234+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:34:38.775234+0000","last_peered":"2024-09-17T15:34:38.775234+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:34:38.775234+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'9200","ondisk_log_start":"43'9200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7094272,"num_objects":1732,"num_object_clones":0,"num_object_copies":3464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15782,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2325}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223809+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223809+0000","last_peered":"2024-09-17T15:33:11.223809+0000","last_clean":"2024-09-17T15:33:11.223809+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223809+0000","last_undegraded":"2024-09-17T15:33:11.223809+0000","last_fullsized":"2024-09-17T15:33:11.223809+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19098","reported_seq":23192,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:38.882081+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:34:38.882081+0000","last_peered":"2024-09-17T15:34:38.882081+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:34:38.882081+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"43'9000","ondisk_log_start":"43'9000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6856704,"num_objects":1674,"num_object_clones":0,"num_object_copies":3348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2189,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15385,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1971,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19153","reported_seq":19461,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:38.853379+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:34:38.853379+0000","last_peered":"2024-09-17T15:34:38.853379+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:34:38.853379+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'9100","ondisk_log_start":"43'9100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7098368,"num_objects":1733,"num_object_clones":0,"num_object_copies":3466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1733,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15940,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2247}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19052","reported_seq":27436,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:43.502576+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:34:43.502576+0000","last_peered":"2024-09-17T15:34:43.502576+0000","last_clean":"2024-09-17T15:34:43.502576+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:34:43.502576+0000","last_undegraded":"2024-09-17T15:34:43.502576+0000","last_fullsized":"2024-09-17T15:34:43.502576+0000","mapping_epoch":44,"log_start":"43'9000","ondisk_log_start":"43'9000","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6472646,"num_objects":1582,"num_object_clones":0,"num_object_copies":3164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15494,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":109556470,"num_objects":26757,"num_object_clones":0,"num_object_copies":53514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34600,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26757,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":266230,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14860,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417988608,"data_stored":417854009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160895,"ondisk_log_size":160895,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051700,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518516,"kb_used_data":184164,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93853324,"statfs":{"total":96636764160,"available":96105803776,"internally_reserved":0,"allocated":188583936,"data_stored":188282338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":342355628},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1,0,0,0,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724645,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":532624,"kb_used_data":77500,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93839216,"statfs":{"total":96636764160,"available":96091357184,"internally_reserved":0,"allocated":79360000,"data_stored":79036260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":466022059},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,5,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247927,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":586676,"kb_used_data":149660,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93785164,"statfs":{"total":96636764160,"available":96036007936,"internally_reserved":0,"allocated":153251840,"data_stored":152950237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":447475368},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,3,3,5,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":152268800,"data_stored":152232670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77721600,"data_stored":77663333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":187998208,"data_stored":187958006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:34:48.005 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:34:48.549 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:34:48.550 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:34:48.584 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":352,"stamp":"2024-09-17T15:34:47.791301+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":103826185,"num_objects":25360,"num_object_clones":0,"num_object_copies":50720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34475,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25360,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":267631,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14986,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160797,"ondisk_log_size":160797,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1636084,"kb_used_data":404016,"kb_used_omap":13,"kb_used_meta":1231986,"kb_avail":281479436,"statfs":{"total":289910292480,"available":288234942464,"internally_reserved":0,"allocated":413712384,"data_stored":412785443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1261554687},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,3,1,2,7,12,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20230144,"num_objects":-4939,"num_object_clones":0,"num_object_copies":-9878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-242,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4939,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4939,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005034"},"pg_stats":[{"pgid":"2.f","version":"47'19106","reported_seq":19365,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:43.853312+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:34:43.853312+0000","last_peered":"2024-09-17T15:34:43.853312+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:34:43.853312+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'9100","ondisk_log_start":"43'9100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6316032,"num_objects":1542,"num_object_clones":0,"num_object_copies":3084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19106,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19274","reported_seq":19387,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:43.212667+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:34:43.212667+0000","last_peered":"2024-09-17T15:34:43.212667+0000","last_clean":"2024-09-17T15:34:43.212667+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:34:43.212667+0000","last_undegraded":"2024-09-17T15:34:43.212667+0000","last_fullsized":"2024-09-17T15:34:43.212667+0000","mapping_epoch":33,"log_start":"43'9200","ondisk_log_start":"43'9200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6792134,"num_objects":1660,"num_object_clones":0,"num_object_copies":3320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1660,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17631,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19255","reported_seq":19473,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:43.125196+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:34:43.125196+0000","last_peered":"2024-09-17T15:34:43.125196+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:34:43.125196+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'9200","ondisk_log_start":"43'9200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6873112,"num_objects":1679,"num_object_clones":0,"num_object_copies":3358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1679,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17412,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2446}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19367","reported_seq":19447,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:43.227775+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:34:43.227775+0000","last_peered":"2024-09-17T15:34:43.227775+0000","last_clean":"2024-09-17T15:34:43.227775+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:34:43.227775+0000","last_undegraded":"2024-09-17T15:34:43.227775+0000","last_fullsized":"2024-09-17T15:34:43.227775+0000","mapping_epoch":33,"log_start":"43'9300","ondisk_log_start":"43'9300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6565888,"num_objects":1603,"num_object_clones":0,"num_object_copies":3206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1603,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19367,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19045","reported_seq":19183,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:43.919168+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:34:43.919168+0000","last_peered":"2024-09-17T15:34:43.919168+0000","last_clean":"2024-09-17T15:34:43.919168+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:34:43.919168+0000","last_undegraded":"2024-09-17T15:34:43.919168+0000","last_fullsized":"2024-09-17T15:34:43.919168+0000","mapping_epoch":32,"log_start":"43'9000","ondisk_log_start":"43'9000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6586851,"num_objects":1609,"num_object_clones":0,"num_object_copies":3218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1609,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16027,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19657","reported_seq":19958,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:43.857675+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:34:43.857675+0000","last_peered":"2024-09-17T15:34:43.857675+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:34:43.857675+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'9600","ondisk_log_start":"43'9600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6529507,"num_objects":1595,"num_object_clones":0,"num_object_copies":3190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1595,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16156,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2597},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19237","reported_seq":19556,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:43.838250+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:34:43.838250+0000","last_peered":"2024-09-17T15:34:43.838250+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:34:43.838250+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'9200","ondisk_log_start":"43'9200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6271942,"num_objects":1533,"num_object_clones":0,"num_object_copies":3066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1533,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16006,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2556},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19357","reported_seq":19495,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:43.896057+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:34:43.896057+0000","last_peered":"2024-09-17T15:34:43.896057+0000","last_clean":"2024-09-17T15:34:43.896057+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:34:43.896057+0000","last_undegraded":"2024-09-17T15:34:43.896057+0000","last_fullsized":"2024-09-17T15:34:43.896057+0000","mapping_epoch":32,"log_start":"43'9300","ondisk_log_start":"43'9300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6385664,"num_objects":1559,"num_object_clones":0,"num_object_copies":3118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1559,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16311,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.224142+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.224142+0000","last_peered":"2024-09-17T15:33:11.224142+0000","last_clean":"2024-09-17T15:33:11.224142+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.224142+0000","last_undegraded":"2024-09-17T15:33:11.224142+0000","last_fullsized":"2024-09-17T15:33:11.224142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19216","reported_seq":27508,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:43.539257+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:34:43.539257+0000","last_peered":"2024-09-17T15:34:43.539257+0000","last_clean":"2024-09-17T15:34:43.539257+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:34:43.539257+0000","last_undegraded":"2024-09-17T15:34:43.539257+0000","last_fullsized":"2024-09-17T15:34:43.539257+0000","mapping_epoch":42,"log_start":"43'9200","ondisk_log_start":"43'9200","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6558179,"num_objects":1602,"num_object_clones":0,"num_object_copies":3204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1602,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17216,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18970","reported_seq":19245,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:43.847531+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:34:43.847531+0000","last_peered":"2024-09-17T15:34:43.847531+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:34:43.847531+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'8900","ondisk_log_start":"43'8900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6438912,"num_objects":1572,"num_object_clones":0,"num_object_copies":3144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15566,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2371}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18967","reported_seq":19186,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:43.133956+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:34:43.133956+0000","last_peered":"2024-09-17T15:34:43.133956+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:34:43.133956+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'8900","ondisk_log_start":"43'8900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6357475,"num_objects":1553,"num_object_clones":0,"num_object_copies":3106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1553,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16862,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2480}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":88,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223926+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223926+0000","last_peered":"2024-09-17T15:33:11.223926+0000","last_clean":"2024-09-17T15:33:11.223926+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223926+0000","last_undegraded":"2024-09-17T15:33:11.223926+0000","last_fullsized":"2024-09-17T15:33:11.223926+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19143","reported_seq":27558,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:43.225832+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:34:43.225832+0000","last_peered":"2024-09-17T15:34:43.225832+0000","last_clean":"2024-09-17T15:34:43.225832+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:34:43.225832+0000","last_undegraded":"2024-09-17T15:34:43.225832+0000","last_fullsized":"2024-09-17T15:34:43.225832+0000","mapping_epoch":46,"log_start":"43'9100","ondisk_log_start":"43'9100","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6524928,"num_objects":1593,"num_object_clones":0,"num_object_copies":3186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16905,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19375","reported_seq":19682,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:43.878249+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:34:43.878249+0000","last_peered":"2024-09-17T15:34:43.878249+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:34:43.878249+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'9300","ondisk_log_start":"43'9300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6516736,"num_objects":1591,"num_object_clones":0,"num_object_copies":3182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15923,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2466}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223809+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223809+0000","last_peered":"2024-09-17T15:33:11.223809+0000","last_clean":"2024-09-17T15:33:11.223809+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223809+0000","last_undegraded":"2024-09-17T15:33:11.223809+0000","last_fullsized":"2024-09-17T15:33:11.223809+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19255","reported_seq":23600,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:43.887961+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:34:43.887961+0000","last_peered":"2024-09-17T15:34:43.887961+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:34:43.887961+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"43'9200","ondisk_log_start":"43'9200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6213632,"num_objects":1517,"num_object_clones":0,"num_object_copies":3034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2064,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15542,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2096,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-547}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19318","reported_seq":19625,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:43.838008+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:34:43.838008+0000","last_peered":"2024-09-17T15:34:43.838008+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:34:43.838008+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'9300","ondisk_log_start":"43'9300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6422528,"num_objects":1568,"num_object_clones":0,"num_object_copies":3136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16105,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2412}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19052","reported_seq":27436,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:43.502576+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:34:43.502576+0000","last_peered":"2024-09-17T15:34:43.502576+0000","last_clean":"2024-09-17T15:34:43.502576+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:34:43.502576+0000","last_undegraded":"2024-09-17T15:34:43.502576+0000","last_fullsized":"2024-09-17T15:34:43.502576+0000","mapping_epoch":44,"log_start":"43'9000","ondisk_log_start":"43'9000","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6472646,"num_objects":1582,"num_object_clones":0,"num_object_copies":3164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15494,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":103826166,"num_objects":25358,"num_object_clones":0,"num_object_copies":50716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34475,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25358,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":267629,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14985,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":410431488,"data_stored":410296889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160794,"ondisk_log_size":160794,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051700,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518516,"kb_used_data":184164,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93853324,"statfs":{"total":96636764160,"available":96105803776,"internally_reserved":0,"allocated":188583936,"data_stored":188282338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":342355628},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1,0,0,0,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724645,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":532624,"kb_used_data":77500,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93839216,"statfs":{"total":96636764160,"available":96091357184,"internally_reserved":0,"allocated":79360000,"data_stored":79036260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":466022059},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,5,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247928,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":584944,"kb_used_data":142352,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93786896,"statfs":{"total":96636764160,"available":96037781504,"internally_reserved":0,"allocated":145768448,"data_stored":145466845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":453177000},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,5,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":144711680,"data_stored":144675550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77721600,"data_stored":77663333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":187998208,"data_stored":187958006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:34:48.585 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:34:49.157 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:34:49.158 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:34:49.182 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":352,"stamp":"2024-09-17T15:34:47.791301+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":103826185,"num_objects":25360,"num_object_clones":0,"num_object_copies":50720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34475,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25360,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":267631,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14986,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160797,"ondisk_log_size":160797,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1636084,"kb_used_data":404016,"kb_used_omap":13,"kb_used_meta":1231986,"kb_avail":281479436,"statfs":{"total":289910292480,"available":288234942464,"internally_reserved":0,"allocated":413712384,"data_stored":412785443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1261554687},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,3,1,2,7,12,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20230144,"num_objects":-4939,"num_object_clones":0,"num_object_copies":-9878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-242,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4939,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4939,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005034"},"pg_stats":[{"pgid":"2.f","version":"47'19106","reported_seq":19365,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:43.853312+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:34:43.853312+0000","last_peered":"2024-09-17T15:34:43.853312+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:34:43.853312+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'9100","ondisk_log_start":"43'9100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6316032,"num_objects":1542,"num_object_clones":0,"num_object_copies":3084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19106,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19274","reported_seq":19387,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:43.212667+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:34:43.212667+0000","last_peered":"2024-09-17T15:34:43.212667+0000","last_clean":"2024-09-17T15:34:43.212667+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:34:43.212667+0000","last_undegraded":"2024-09-17T15:34:43.212667+0000","last_fullsized":"2024-09-17T15:34:43.212667+0000","mapping_epoch":33,"log_start":"43'9200","ondisk_log_start":"43'9200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6792134,"num_objects":1660,"num_object_clones":0,"num_object_copies":3320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1660,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17631,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19255","reported_seq":19473,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:43.125196+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:34:43.125196+0000","last_peered":"2024-09-17T15:34:43.125196+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:34:43.125196+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'9200","ondisk_log_start":"43'9200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6873112,"num_objects":1679,"num_object_clones":0,"num_object_copies":3358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1679,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17412,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2446}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19367","reported_seq":19447,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:43.227775+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:34:43.227775+0000","last_peered":"2024-09-17T15:34:43.227775+0000","last_clean":"2024-09-17T15:34:43.227775+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:34:43.227775+0000","last_undegraded":"2024-09-17T15:34:43.227775+0000","last_fullsized":"2024-09-17T15:34:43.227775+0000","mapping_epoch":33,"log_start":"43'9300","ondisk_log_start":"43'9300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6565888,"num_objects":1603,"num_object_clones":0,"num_object_copies":3206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1603,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19367,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19045","reported_seq":19183,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:43.919168+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:34:43.919168+0000","last_peered":"2024-09-17T15:34:43.919168+0000","last_clean":"2024-09-17T15:34:43.919168+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:34:43.919168+0000","last_undegraded":"2024-09-17T15:34:43.919168+0000","last_fullsized":"2024-09-17T15:34:43.919168+0000","mapping_epoch":32,"log_start":"43'9000","ondisk_log_start":"43'9000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6586851,"num_objects":1609,"num_object_clones":0,"num_object_copies":3218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1609,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16027,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19657","reported_seq":19958,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:43.857675+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:34:43.857675+0000","last_peered":"2024-09-17T15:34:43.857675+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:34:43.857675+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'9600","ondisk_log_start":"43'9600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6529507,"num_objects":1595,"num_object_clones":0,"num_object_copies":3190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1595,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16156,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2597},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19237","reported_seq":19556,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:43.838250+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:34:43.838250+0000","last_peered":"2024-09-17T15:34:43.838250+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:34:43.838250+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'9200","ondisk_log_start":"43'9200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6271942,"num_objects":1533,"num_object_clones":0,"num_object_copies":3066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1533,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16006,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2556},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19357","reported_seq":19495,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:43.896057+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:34:43.896057+0000","last_peered":"2024-09-17T15:34:43.896057+0000","last_clean":"2024-09-17T15:34:43.896057+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:34:43.896057+0000","last_undegraded":"2024-09-17T15:34:43.896057+0000","last_fullsized":"2024-09-17T15:34:43.896057+0000","mapping_epoch":32,"log_start":"43'9300","ondisk_log_start":"43'9300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6385664,"num_objects":1559,"num_object_clones":0,"num_object_copies":3118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1559,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16311,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.224142+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.224142+0000","last_peered":"2024-09-17T15:33:11.224142+0000","last_clean":"2024-09-17T15:33:11.224142+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.224142+0000","last_undegraded":"2024-09-17T15:33:11.224142+0000","last_fullsized":"2024-09-17T15:33:11.224142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19216","reported_seq":27508,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:43.539257+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:34:43.539257+0000","last_peered":"2024-09-17T15:34:43.539257+0000","last_clean":"2024-09-17T15:34:43.539257+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:34:43.539257+0000","last_undegraded":"2024-09-17T15:34:43.539257+0000","last_fullsized":"2024-09-17T15:34:43.539257+0000","mapping_epoch":42,"log_start":"43'9200","ondisk_log_start":"43'9200","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6558179,"num_objects":1602,"num_object_clones":0,"num_object_copies":3204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1602,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17216,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18970","reported_seq":19245,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:43.847531+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:34:43.847531+0000","last_peered":"2024-09-17T15:34:43.847531+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:34:43.847531+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'8900","ondisk_log_start":"43'8900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6438912,"num_objects":1572,"num_object_clones":0,"num_object_copies":3144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15566,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2371}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18967","reported_seq":19186,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:43.133956+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:34:43.133956+0000","last_peered":"2024-09-17T15:34:43.133956+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:34:43.133956+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'8900","ondisk_log_start":"43'8900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6357475,"num_objects":1553,"num_object_clones":0,"num_object_copies":3106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1553,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16862,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2480}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":88,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223926+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223926+0000","last_peered":"2024-09-17T15:33:11.223926+0000","last_clean":"2024-09-17T15:33:11.223926+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223926+0000","last_undegraded":"2024-09-17T15:33:11.223926+0000","last_fullsized":"2024-09-17T15:33:11.223926+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19143","reported_seq":27558,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:43.225832+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:34:43.225832+0000","last_peered":"2024-09-17T15:34:43.225832+0000","last_clean":"2024-09-17T15:34:43.225832+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:34:43.225832+0000","last_undegraded":"2024-09-17T15:34:43.225832+0000","last_fullsized":"2024-09-17T15:34:43.225832+0000","mapping_epoch":46,"log_start":"43'9100","ondisk_log_start":"43'9100","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6524928,"num_objects":1593,"num_object_clones":0,"num_object_copies":3186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16905,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19375","reported_seq":19682,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:43.878249+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:34:43.878249+0000","last_peered":"2024-09-17T15:34:43.878249+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:34:43.878249+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'9300","ondisk_log_start":"43'9300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6516736,"num_objects":1591,"num_object_clones":0,"num_object_copies":3182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15923,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2466}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223809+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223809+0000","last_peered":"2024-09-17T15:33:11.223809+0000","last_clean":"2024-09-17T15:33:11.223809+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223809+0000","last_undegraded":"2024-09-17T15:33:11.223809+0000","last_fullsized":"2024-09-17T15:33:11.223809+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19255","reported_seq":23600,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:43.887961+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:34:43.887961+0000","last_peered":"2024-09-17T15:34:43.887961+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:34:43.887961+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"43'9200","ondisk_log_start":"43'9200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6213632,"num_objects":1517,"num_object_clones":0,"num_object_copies":3034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2064,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15542,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2096,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-547}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19318","reported_seq":19625,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:43.838008+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:34:43.838008+0000","last_peered":"2024-09-17T15:34:43.838008+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:34:43.838008+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'9300","ondisk_log_start":"43'9300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6422528,"num_objects":1568,"num_object_clones":0,"num_object_copies":3136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16105,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2412}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223881+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223881+0000","last_peered":"2024-09-17T15:33:11.223881+0000","last_clean":"2024-09-17T15:33:11.223881+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223881+0000","last_undegraded":"2024-09-17T15:33:11.223881+0000","last_fullsized":"2024-09-17T15:33:11.223881+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19052","reported_seq":27436,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:43.502576+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:34:43.502576+0000","last_peered":"2024-09-17T15:34:43.502576+0000","last_clean":"2024-09-17T15:34:43.502576+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:34:43.502576+0000","last_undegraded":"2024-09-17T15:34:43.502576+0000","last_fullsized":"2024-09-17T15:34:43.502576+0000","mapping_epoch":44,"log_start":"43'9000","ondisk_log_start":"43'9000","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6472646,"num_objects":1582,"num_object_clones":0,"num_object_copies":3164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15494,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":103826166,"num_objects":25358,"num_object_clones":0,"num_object_copies":50716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34475,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25358,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":267629,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14985,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":410431488,"data_stored":410296889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160794,"ondisk_log_size":160794,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051700,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518516,"kb_used_data":184164,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93853324,"statfs":{"total":96636764160,"available":96105803776,"internally_reserved":0,"allocated":188583936,"data_stored":188282338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":342355628},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1,0,0,0,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724645,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":532624,"kb_used_data":77500,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93839216,"statfs":{"total":96636764160,"available":96091357184,"internally_reserved":0,"allocated":79360000,"data_stored":79036260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":466022059},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,5,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247928,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":584944,"kb_used_data":142352,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93786896,"statfs":{"total":96636764160,"available":96037781504,"internally_reserved":0,"allocated":145768448,"data_stored":145466845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":453177000},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,5,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":144711680,"data_stored":144675550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77721600,"data_stored":77663333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":187998208,"data_stored":187958006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:34:49.183 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:34:49.183 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'19106", 'reported_seq': 19365, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:43.853312+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:34:43.853312+0000', 'last_peered': '2024-09-17T15:34:43.853312+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:34:43.853312+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'9100", 'ondisk_log_start': "43'9100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6316032, 'num_objects': 1542, 'num_object_clones': 0, 'num_object_copies': 3084, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1542, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19106, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:49.183 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:34:49.183 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'19255", 'reported_seq': 19473, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:43.125196+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:34:43.125196+0000', 'last_peered': '2024-09-17T15:34:43.125196+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:34:43.125196+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "43'9200", 'ondisk_log_start': "43'9200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6873112, 'num_objects': 1679, 'num_object_clones': 0, 'num_object_copies': 3358, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1679, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17412, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2446}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:34:49.183 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:34:49.184 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'19657", 'reported_seq': 19958, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:43.857675+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:34:43.857675+0000', 'last_peered': '2024-09-17T15:34:43.857675+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:34:43.857675+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "43'9600", 'ondisk_log_start': "43'9600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6529507, 'num_objects': 1595, 'num_object_clones': 0, 'num_object_copies': 3190, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1595, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16156, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2597}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:49.184 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:34:49.184 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'19237", 'reported_seq': 19556, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:43.838250+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:34:43.838250+0000', 'last_peered': '2024-09-17T15:34:43.838250+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:34:43.838250+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "43'9200", 'ondisk_log_start': "43'9200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6271942, 'num_objects': 1533, 'num_object_clones': 0, 'num_object_copies': 3066, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4089, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1533, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 16006, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 85, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2556}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:49.184 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:34:49.184 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'18970", 'reported_seq': 19245, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:43.847531+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:34:43.847531+0000', 'last_peered': '2024-09-17T15:34:43.847531+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:34:43.847531+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'8900", 'ondisk_log_start': "43'8900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6438912, 'num_objects': 1572, 'num_object_clones': 0, 'num_object_copies': 3144, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1572, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15566, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2371}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:49.184 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:34:49.184 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'18967", 'reported_seq': 19186, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:43.133956+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:34:43.133956+0000', 'last_peered': '2024-09-17T15:34:43.133956+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:34:43.133956+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "43'8900", 'ondisk_log_start': "43'8900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6357475, 'num_objects': 1553, 'num_object_clones': 0, 'num_object_copies': 3106, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4034, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1553, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16862, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -2480}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:34:49.185 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:34:49.185 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'19375", 'reported_seq': 19682, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:43.878249+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:34:43.878249+0000', 'last_peered': '2024-09-17T15:34:43.878249+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:34:43.878249+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "43'9300", 'ondisk_log_start': "43'9300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6516736, 'num_objects': 1591, 'num_object_clones': 0, 'num_object_copies': 3182, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1591, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15923, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2466}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:49.185 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:34:49.185 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'19255", 'reported_seq': 23600, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:43.887961+0000', 'last_change': '2024-09-17T15:33:09.763500+0000', 'last_active': '2024-09-17T15:34:43.887961+0000', 'last_peered': '2024-09-17T15:34:43.887961+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:34:43.887961+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "43'9200", 'ondisk_log_start': "43'9200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6213632, 'num_objects': 1517, 'num_object_clones': 0, 'num_object_copies': 3034, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2064, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1517, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15542, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2096, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -547}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:49.185 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:34:49.185 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'19318", 'reported_seq': 19625, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:43.838008+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:34:43.838008+0000', 'last_peered': '2024-09-17T15:34:43.838008+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:34:43.838008+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'9300", 'ondisk_log_start': "43'9300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6422528, 'num_objects': 1568, 'num_object_clones': 0, 'num_object_copies': 3136, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1568, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16105, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2412}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:49.185 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:34:50.020 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:34:50.020 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34475/50720 objects degraded (67.971%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":627,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":25360,"data_bytes":103826185,"bytes_used":1675350016,"bytes_avail":288234942464,"bytes_total":289910292480,"degraded_objects":34475,"degraded_total":50720,"degraded_ratio":0.67971214511041012,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":242,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":411},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:34:50.040 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:34:50.041 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:34:50.594 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:34:50.594 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:34:50.626 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":353,"stamp":"2024-09-17T15:34:49.792092+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":93823753,"num_objects":22918,"num_object_clones":0,"num_object_copies":45836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34347,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22918,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":270073,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15114,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160639,"ondisk_log_size":160639,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1638920,"kb_used_data":384644,"kb_used_omap":13,"kb_used_meta":1254194,"kb_avail":281476600,"statfs":{"total":289910292480,"available":288232038400,"internally_reserved":0,"allocated":393875456,"data_stored":392948515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1284295679},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,5,9,6,8,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-30232576,"num_objects":-7381,"num_object_clones":0,"num_object_copies":-14762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-370,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7381,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":370,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004997"},"pg_stats":[{"pgid":"2.f","version":"47'19253","reported_seq":19513,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:48.967079+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:34:48.967079+0000","last_peered":"2024-09-17T15:34:48.967079+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:34:48.967079+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'9200","ondisk_log_start":"43'9200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5713920,"num_objects":1395,"num_object_clones":0,"num_object_copies":2790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19253,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2589}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19434","reported_seq":19547,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:48.193746+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:34:48.193746+0000","last_peered":"2024-09-17T15:34:48.193746+0000","last_clean":"2024-09-17T15:34:48.193746+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:34:48.193746+0000","last_undegraded":"2024-09-17T15:34:48.193746+0000","last_fullsized":"2024-09-17T15:34:48.193746+0000","mapping_epoch":33,"log_start":"43'9400","ondisk_log_start":"43'9400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6136774,"num_objects":1500,"num_object_clones":0,"num_object_copies":3000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1500,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17791,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19399","reported_seq":19621,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:48.247988+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:34:48.247988+0000","last_peered":"2024-09-17T15:34:48.247988+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:34:48.247988+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'9300","ondisk_log_start":"43'9300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6283288,"num_objects":1535,"num_object_clones":0,"num_object_copies":3070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1535,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17556,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2590}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19520","reported_seq":19600,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:48.267494+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:34:48.267494+0000","last_peered":"2024-09-17T15:34:48.267494+0000","last_clean":"2024-09-17T15:34:48.267494+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:34:48.267494+0000","last_undegraded":"2024-09-17T15:34:48.267494+0000","last_fullsized":"2024-09-17T15:34:48.267494+0000","mapping_epoch":33,"log_start":"43'9500","ondisk_log_start":"43'9500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5939200,"num_objects":1450,"num_object_clones":0,"num_object_copies":2900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19520,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19209","reported_seq":19348,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:48.840218+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:34:48.840218+0000","last_peered":"2024-09-17T15:34:48.840218+0000","last_clean":"2024-09-17T15:34:48.840218+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:34:48.840218+0000","last_undegraded":"2024-09-17T15:34:48.840218+0000","last_fullsized":"2024-09-17T15:34:48.840218+0000","mapping_epoch":32,"log_start":"43'9200","ondisk_log_start":"43'9200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5915107,"num_objects":1445,"num_object_clones":0,"num_object_copies":2890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1445,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16191,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19816","reported_seq":20114,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:48.922837+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:34:48.922837+0000","last_peered":"2024-09-17T15:34:48.922837+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:34:48.922837+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'9800","ondisk_log_start":"43'9800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5878243,"num_objects":1436,"num_object_clones":0,"num_object_copies":2872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1436,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16315,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2756},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19375","reported_seq":19694,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:48.864868+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:34:48.864868+0000","last_peered":"2024-09-17T15:34:48.864868+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:34:48.864868+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'9300","ondisk_log_start":"43'9300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5706694,"num_objects":1395,"num_object_clones":0,"num_object_copies":2790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1395,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16144,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2694},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19512","reported_seq":19650,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:48.858417+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:34:48.858417+0000","last_peered":"2024-09-17T15:34:48.858417+0000","last_clean":"2024-09-17T15:34:48.858417+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:34:48.858417+0000","last_undegraded":"2024-09-17T15:34:48.858417+0000","last_fullsized":"2024-09-17T15:34:48.858417+0000","mapping_epoch":32,"log_start":"43'9500","ondisk_log_start":"43'9500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5750784,"num_objects":1404,"num_object_clones":0,"num_object_copies":2808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16466,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.224142+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.224142+0000","last_peered":"2024-09-17T15:33:11.224142+0000","last_clean":"2024-09-17T15:33:11.224142+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.224142+0000","last_undegraded":"2024-09-17T15:33:11.224142+0000","last_fullsized":"2024-09-17T15:33:11.224142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19364","reported_seq":27657,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:48.475707+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:34:48.475707+0000","last_peered":"2024-09-17T15:34:48.475707+0000","last_clean":"2024-09-17T15:34:48.475707+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:34:48.475707+0000","last_undegraded":"2024-09-17T15:34:48.475707+0000","last_fullsized":"2024-09-17T15:34:48.475707+0000","mapping_epoch":42,"log_start":"43'9300","ondisk_log_start":"43'9300","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5951971,"num_objects":1454,"num_object_clones":0,"num_object_copies":2908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1454,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17364,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19124","reported_seq":19399,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:48.926835+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:34:48.926835+0000","last_peered":"2024-09-17T15:34:48.926835+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:34:48.926835+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'9100","ondisk_log_start":"43'9100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5808128,"num_objects":1418,"num_object_clones":0,"num_object_copies":2836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15720,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2525}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19113","reported_seq":19332,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:48.265369+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:34:48.265369+0000","last_peered":"2024-09-17T15:34:48.265369+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:34:48.265369+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'9100","ondisk_log_start":"43'9100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5759459,"num_objects":1407,"num_object_clones":0,"num_object_copies":2814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1407,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17008,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2626}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":88,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223926+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223926+0000","last_peered":"2024-09-17T15:33:11.223926+0000","last_clean":"2024-09-17T15:33:11.223926+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223926+0000","last_undegraded":"2024-09-17T15:33:11.223926+0000","last_fullsized":"2024-09-17T15:33:11.223926+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19307","reported_seq":27722,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:48.282506+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:34:48.282506+0000","last_peered":"2024-09-17T15:34:48.282506+0000","last_clean":"2024-09-17T15:34:48.282506+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:34:48.282506+0000","last_undegraded":"2024-09-17T15:34:48.282506+0000","last_fullsized":"2024-09-17T15:34:48.282506+0000","mapping_epoch":46,"log_start":"43'9300","ondisk_log_start":"43'9300","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5853184,"num_objects":1429,"num_object_clones":0,"num_object_copies":2858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17069,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19546","reported_seq":19851,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:48.980619+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:34:48.980619+0000","last_peered":"2024-09-17T15:34:48.980619+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:34:48.980619+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'9500","ondisk_log_start":"43'9500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5816320,"num_objects":1420,"num_object_clones":0,"num_object_copies":2840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16094,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2637}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223809+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223809+0000","last_peered":"2024-09-17T15:33:11.223809+0000","last_clean":"2024-09-17T15:33:11.223809+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223809+0000","last_undegraded":"2024-09-17T15:33:11.223809+0000","last_fullsized":"2024-09-17T15:33:11.223809+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19402","reported_seq":24002,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:48.917177+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:34:48.917177+0000","last_peered":"2024-09-17T15:34:48.917177+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:34:48.917177+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"43'9400","ondisk_log_start":"43'9400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5611520,"num_objects":1370,"num_object_clones":0,"num_object_copies":2740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1936,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15689,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2224,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19469","reported_seq":19777,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:48.897355+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:34:48.897355+0000","last_peered":"2024-09-17T15:34:48.897355+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:34:48.897355+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'9400","ondisk_log_start":"43'9400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5804032,"num_objects":1417,"num_object_clones":0,"num_object_copies":2834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1417,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16256,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2563}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223917+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223917+0000","last_peered":"2024-09-17T15:33:11.223917+0000","last_clean":"2024-09-17T15:33:11.223917+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223917+0000","last_undegraded":"2024-09-17T15:33:11.223917+0000","last_fullsized":"2024-09-17T15:33:11.223917+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19193","reported_seq":27577,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:48.549527+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:34:48.549527+0000","last_peered":"2024-09-17T15:34:48.549527+0000","last_clean":"2024-09-17T15:34:48.549527+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:34:48.549527+0000","last_undegraded":"2024-09-17T15:34:48.549527+0000","last_fullsized":"2024-09-17T15:34:48.549527+0000","mapping_epoch":44,"log_start":"43'9100","ondisk_log_start":"43'9100","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5895110,"num_objects":1441,"num_object_clones":0,"num_object_copies":2882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1441,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15635,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":93823734,"num_objects":22916,"num_object_clones":0,"num_object_copies":45832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34347,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22916,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":270071,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15113,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":390365184,"data_stored":390230585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160636,"ondisk_log_size":160636,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051701,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535908,"kb_used_data":179340,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93835932,"statfs":{"total":96636764160,"available":96087994368,"internally_reserved":0,"allocated":183644160,"data_stored":183342562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":365096620},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,1,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724646,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517176,"kb_used_data":70372,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93854664,"statfs":{"total":96636764160,"available":96107175936,"internally_reserved":0,"allocated":72060928,"data_stored":71737188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":457502379},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,5,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247929,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":585836,"kb_used_data":134932,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93786004,"statfs":{"total":96636764160,"available":96036868096,"internally_reserved":0,"allocated":138170368,"data_stored":137868765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":461696680},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,2,1,3,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":137121792,"data_stored":137085662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":70332416,"data_stored":70274149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":182910976,"data_stored":182870774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:34:53.628 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:34:54.210 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:34:54.211 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:34:54.241 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2024-09-17T15:34:53.793588+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":89338633,"num_objects":21823,"num_object_clones":0,"num_object_copies":43646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34347,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21823,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":271168,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15114,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160734,"ondisk_log_size":160734,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1635296,"kb_used_data":372712,"kb_used_omap":13,"kb_used_meta":1262514,"kb_avail":281480224,"statfs":{"total":289910292480,"available":288235749376,"internally_reserved":0,"allocated":381657088,"data_stored":380730147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1292815359},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,3,5,3,13,14,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24612864,"num_objects":-6009,"num_object_clones":0,"num_object_copies":-12018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-253,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6009,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":253,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004469"},"pg_stats":[{"pgid":"2.f","version":"47'19253","reported_seq":19513,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:48.967079+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:34:48.967079+0000","last_peered":"2024-09-17T15:34:48.967079+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:34:48.967079+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'9200","ondisk_log_start":"43'9200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5713920,"num_objects":1395,"num_object_clones":0,"num_object_copies":2790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19253,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2589}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19591","reported_seq":19704,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:53.290581+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:34:53.290581+0000","last_peered":"2024-09-17T15:34:53.290581+0000","last_clean":"2024-09-17T15:34:53.290581+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:34:53.290581+0000","last_undegraded":"2024-09-17T15:34:53.290581+0000","last_fullsized":"2024-09-17T15:34:53.290581+0000","mapping_epoch":33,"log_start":"43'9500","ondisk_log_start":"43'9500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5493702,"num_objects":1343,"num_object_clones":0,"num_object_copies":2686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1343,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17948,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19563","reported_seq":19786,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:53.258596+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:34:53.258596+0000","last_peered":"2024-09-17T15:34:53.258596+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:34:53.258596+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'9500","ondisk_log_start":"43'9500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5611544,"num_objects":1371,"num_object_clones":0,"num_object_copies":2742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1371,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17720,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2754}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19688","reported_seq":19768,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:53.244268+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:34:53.244268+0000","last_peered":"2024-09-17T15:34:53.244268+0000","last_clean":"2024-09-17T15:34:53.244268+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:34:53.244268+0000","last_undegraded":"2024-09-17T15:34:53.244268+0000","last_fullsized":"2024-09-17T15:34:53.244268+0000","mapping_epoch":33,"log_start":"43'9600","ondisk_log_start":"43'9600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5251072,"num_objects":1282,"num_object_clones":0,"num_object_copies":2564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19688,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19209","reported_seq":19348,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:48.840218+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:34:48.840218+0000","last_peered":"2024-09-17T15:34:48.840218+0000","last_clean":"2024-09-17T15:34:48.840218+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:34:48.840218+0000","last_undegraded":"2024-09-17T15:34:48.840218+0000","last_fullsized":"2024-09-17T15:34:48.840218+0000","mapping_epoch":32,"log_start":"43'9200","ondisk_log_start":"43'9200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5915107,"num_objects":1445,"num_object_clones":0,"num_object_copies":2890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1445,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16191,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19816","reported_seq":20114,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:48.922837+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:34:48.922837+0000","last_peered":"2024-09-17T15:34:48.922837+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:34:48.922837+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'9800","ondisk_log_start":"43'9800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5878243,"num_objects":1436,"num_object_clones":0,"num_object_copies":2872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1436,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16315,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2756},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19375","reported_seq":19694,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:48.864868+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:34:48.864868+0000","last_peered":"2024-09-17T15:34:48.864868+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:34:48.864868+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'9300","ondisk_log_start":"43'9300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5706694,"num_objects":1395,"num_object_clones":0,"num_object_copies":2790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1395,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16144,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2694},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19512","reported_seq":19650,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:48.858417+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:34:48.858417+0000","last_peered":"2024-09-17T15:34:48.858417+0000","last_clean":"2024-09-17T15:34:48.858417+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:34:48.858417+0000","last_undegraded":"2024-09-17T15:34:48.858417+0000","last_fullsized":"2024-09-17T15:34:48.858417+0000","mapping_epoch":32,"log_start":"43'9500","ondisk_log_start":"43'9500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5750784,"num_objects":1404,"num_object_clones":0,"num_object_copies":2808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16466,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.224142+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.224142+0000","last_peered":"2024-09-17T15:33:11.224142+0000","last_clean":"2024-09-17T15:33:11.224142+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.224142+0000","last_undegraded":"2024-09-17T15:33:11.224142+0000","last_fullsized":"2024-09-17T15:33:11.224142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19531","reported_seq":27824,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:53.592354+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:34:53.592354+0000","last_peered":"2024-09-17T15:34:53.592354+0000","last_clean":"2024-09-17T15:34:53.592354+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:34:53.592354+0000","last_undegraded":"2024-09-17T15:34:53.592354+0000","last_fullsized":"2024-09-17T15:34:53.592354+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5267939,"num_objects":1287,"num_object_clones":0,"num_object_copies":2574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1287,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17531,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19124","reported_seq":19399,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:48.926835+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:34:48.926835+0000","last_peered":"2024-09-17T15:34:48.926835+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:34:48.926835+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'9100","ondisk_log_start":"43'9100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5808128,"num_objects":1418,"num_object_clones":0,"num_object_copies":2836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15720,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2525}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19265","reported_seq":19479,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:53.233837+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:34:53.233837+0000","last_peered":"2024-09-17T15:34:53.233837+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:34:53.233837+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'9200","ondisk_log_start":"43'9200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5136867,"num_objects":1255,"num_object_clones":0,"num_object_copies":2510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1255,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17160,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2778}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":88,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223926+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223926+0000","last_peered":"2024-09-17T15:33:11.223926+0000","last_clean":"2024-09-17T15:33:11.223926+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223926+0000","last_undegraded":"2024-09-17T15:33:11.223926+0000","last_fullsized":"2024-09-17T15:33:11.223926+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19455","reported_seq":27870,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:53.316221+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:34:53.316221+0000","last_peered":"2024-09-17T15:34:53.316221+0000","last_clean":"2024-09-17T15:34:53.316221+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:34:53.316221+0000","last_undegraded":"2024-09-17T15:34:53.316221+0000","last_fullsized":"2024-09-17T15:34:53.316221+0000","mapping_epoch":46,"log_start":"43'9400","ondisk_log_start":"43'9400","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5246976,"num_objects":1281,"num_object_clones":0,"num_object_copies":2562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17217,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19546","reported_seq":19851,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:48.980619+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:34:48.980619+0000","last_peered":"2024-09-17T15:34:48.980619+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:34:48.980619+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'9500","ondisk_log_start":"43'9500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5816320,"num_objects":1420,"num_object_clones":0,"num_object_copies":2840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16094,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2637}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223809+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223809+0000","last_peered":"2024-09-17T15:33:11.223809+0000","last_clean":"2024-09-17T15:33:11.223809+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223809+0000","last_undegraded":"2024-09-17T15:33:11.223809+0000","last_fullsized":"2024-09-17T15:33:11.223809+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19402","reported_seq":24002,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:48.917177+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:34:48.917177+0000","last_peered":"2024-09-17T15:34:48.917177+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:34:48.917177+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"43'9400","ondisk_log_start":"43'9400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5611520,"num_objects":1370,"num_object_clones":0,"num_object_copies":2740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1936,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15689,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2224,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19469","reported_seq":19777,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:48.897355+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:34:48.897355+0000","last_peered":"2024-09-17T15:34:48.897355+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:34:48.897355+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'9400","ondisk_log_start":"43'9400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5804032,"num_objects":1417,"num_object_clones":0,"num_object_copies":2834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1417,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16256,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2563}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223917+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223917+0000","last_peered":"2024-09-17T15:33:11.223917+0000","last_clean":"2024-09-17T15:33:11.223917+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223917+0000","last_undegraded":"2024-09-17T15:33:11.223917+0000","last_fullsized":"2024-09-17T15:33:11.223917+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19332","reported_seq":27711,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:53.603928+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:34:53.603928+0000","last_peered":"2024-09-17T15:34:53.603928+0000","last_clean":"2024-09-17T15:34:53.603928+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:34:53.603928+0000","last_undegraded":"2024-09-17T15:34:53.603928+0000","last_fullsized":"2024-09-17T15:34:53.603928+0000","mapping_epoch":44,"log_start":"43'9300","ondisk_log_start":"43'9300","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5325766,"num_objects":1302,"num_object_clones":0,"num_object_copies":2604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15774,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":89338614,"num_objects":21821,"num_object_clones":0,"num_object_copies":43642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34347,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21821,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":271166,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15113,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":377724928,"data_stored":377590329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160731,"ondisk_log_size":160731,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051702,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535188,"kb_used_data":174468,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93836652,"statfs":{"total":96636764160,"available":96088731648,"internally_reserved":0,"allocated":178655232,"data_stored":178353634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":369356460},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,6,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724647,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":514272,"kb_used_data":63312,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93857568,"statfs":{"total":96636764160,"available":96110149632,"internally_reserved":0,"allocated":64831488,"data_stored":64507748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":461762219},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,2,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247929,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":585836,"kb_used_data":134932,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93786004,"statfs":{"total":96636764160,"available":96036868096,"internally_reserved":0,"allocated":138170368,"data_stored":137868765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":461696680},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,2,1,3,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":137121792,"data_stored":137085662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":62881792,"data_stored":62823525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":177721344,"data_stored":177681142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:34:54.243 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:34:54.810 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:34:54.812 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:34:54.832 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2024-09-17T15:34:53.793588+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":89338633,"num_objects":21823,"num_object_clones":0,"num_object_copies":43646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34347,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21823,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":271168,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15114,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160734,"ondisk_log_size":160734,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1635296,"kb_used_data":372712,"kb_used_omap":13,"kb_used_meta":1262514,"kb_avail":281480224,"statfs":{"total":289910292480,"available":288235749376,"internally_reserved":0,"allocated":381657088,"data_stored":380730147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1292815359},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,3,5,3,13,14,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24612864,"num_objects":-6009,"num_object_clones":0,"num_object_copies":-12018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-253,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6009,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":253,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004469"},"pg_stats":[{"pgid":"2.f","version":"47'19253","reported_seq":19513,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:48.967079+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:34:48.967079+0000","last_peered":"2024-09-17T15:34:48.967079+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:34:48.967079+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'9200","ondisk_log_start":"43'9200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5713920,"num_objects":1395,"num_object_clones":0,"num_object_copies":2790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19253,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2589}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19591","reported_seq":19704,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:53.290581+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:34:53.290581+0000","last_peered":"2024-09-17T15:34:53.290581+0000","last_clean":"2024-09-17T15:34:53.290581+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:34:53.290581+0000","last_undegraded":"2024-09-17T15:34:53.290581+0000","last_fullsized":"2024-09-17T15:34:53.290581+0000","mapping_epoch":33,"log_start":"43'9500","ondisk_log_start":"43'9500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5493702,"num_objects":1343,"num_object_clones":0,"num_object_copies":2686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1343,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17948,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19563","reported_seq":19786,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:53.258596+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:34:53.258596+0000","last_peered":"2024-09-17T15:34:53.258596+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:34:53.258596+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'9500","ondisk_log_start":"43'9500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5611544,"num_objects":1371,"num_object_clones":0,"num_object_copies":2742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1371,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17720,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2754}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19688","reported_seq":19768,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:53.244268+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:34:53.244268+0000","last_peered":"2024-09-17T15:34:53.244268+0000","last_clean":"2024-09-17T15:34:53.244268+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:34:53.244268+0000","last_undegraded":"2024-09-17T15:34:53.244268+0000","last_fullsized":"2024-09-17T15:34:53.244268+0000","mapping_epoch":33,"log_start":"43'9600","ondisk_log_start":"43'9600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5251072,"num_objects":1282,"num_object_clones":0,"num_object_copies":2564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19688,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19209","reported_seq":19348,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:48.840218+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:34:48.840218+0000","last_peered":"2024-09-17T15:34:48.840218+0000","last_clean":"2024-09-17T15:34:48.840218+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:34:48.840218+0000","last_undegraded":"2024-09-17T15:34:48.840218+0000","last_fullsized":"2024-09-17T15:34:48.840218+0000","mapping_epoch":32,"log_start":"43'9200","ondisk_log_start":"43'9200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5915107,"num_objects":1445,"num_object_clones":0,"num_object_copies":2890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1445,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16191,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19816","reported_seq":20114,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:48.922837+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:34:48.922837+0000","last_peered":"2024-09-17T15:34:48.922837+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:34:48.922837+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'9800","ondisk_log_start":"43'9800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5878243,"num_objects":1436,"num_object_clones":0,"num_object_copies":2872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1436,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16315,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2756},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19375","reported_seq":19694,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:48.864868+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:34:48.864868+0000","last_peered":"2024-09-17T15:34:48.864868+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:34:48.864868+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'9300","ondisk_log_start":"43'9300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5706694,"num_objects":1395,"num_object_clones":0,"num_object_copies":2790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1395,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16144,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2694},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19512","reported_seq":19650,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:48.858417+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:34:48.858417+0000","last_peered":"2024-09-17T15:34:48.858417+0000","last_clean":"2024-09-17T15:34:48.858417+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:34:48.858417+0000","last_undegraded":"2024-09-17T15:34:48.858417+0000","last_fullsized":"2024-09-17T15:34:48.858417+0000","mapping_epoch":32,"log_start":"43'9500","ondisk_log_start":"43'9500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5750784,"num_objects":1404,"num_object_clones":0,"num_object_copies":2808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16466,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.224142+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.224142+0000","last_peered":"2024-09-17T15:33:11.224142+0000","last_clean":"2024-09-17T15:33:11.224142+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.224142+0000","last_undegraded":"2024-09-17T15:33:11.224142+0000","last_fullsized":"2024-09-17T15:33:11.224142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19531","reported_seq":27824,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:53.592354+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:34:53.592354+0000","last_peered":"2024-09-17T15:34:53.592354+0000","last_clean":"2024-09-17T15:34:53.592354+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:34:53.592354+0000","last_undegraded":"2024-09-17T15:34:53.592354+0000","last_fullsized":"2024-09-17T15:34:53.592354+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5267939,"num_objects":1287,"num_object_clones":0,"num_object_copies":2574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1287,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17531,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19124","reported_seq":19399,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:48.926835+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:34:48.926835+0000","last_peered":"2024-09-17T15:34:48.926835+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:34:48.926835+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'9100","ondisk_log_start":"43'9100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5808128,"num_objects":1418,"num_object_clones":0,"num_object_copies":2836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15720,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2525}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19265","reported_seq":19479,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:53.233837+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:34:53.233837+0000","last_peered":"2024-09-17T15:34:53.233837+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:34:53.233837+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'9200","ondisk_log_start":"43'9200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5136867,"num_objects":1255,"num_object_clones":0,"num_object_copies":2510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1255,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17160,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2778}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":88,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223926+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223926+0000","last_peered":"2024-09-17T15:33:11.223926+0000","last_clean":"2024-09-17T15:33:11.223926+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223926+0000","last_undegraded":"2024-09-17T15:33:11.223926+0000","last_fullsized":"2024-09-17T15:33:11.223926+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19455","reported_seq":27870,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:53.316221+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:34:53.316221+0000","last_peered":"2024-09-17T15:34:53.316221+0000","last_clean":"2024-09-17T15:34:53.316221+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:34:53.316221+0000","last_undegraded":"2024-09-17T15:34:53.316221+0000","last_fullsized":"2024-09-17T15:34:53.316221+0000","mapping_epoch":46,"log_start":"43'9400","ondisk_log_start":"43'9400","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5246976,"num_objects":1281,"num_object_clones":0,"num_object_copies":2562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17217,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19546","reported_seq":19851,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:48.980619+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:34:48.980619+0000","last_peered":"2024-09-17T15:34:48.980619+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:34:48.980619+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'9500","ondisk_log_start":"43'9500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5816320,"num_objects":1420,"num_object_clones":0,"num_object_copies":2840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16094,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2637}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223809+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223809+0000","last_peered":"2024-09-17T15:33:11.223809+0000","last_clean":"2024-09-17T15:33:11.223809+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223809+0000","last_undegraded":"2024-09-17T15:33:11.223809+0000","last_fullsized":"2024-09-17T15:33:11.223809+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19402","reported_seq":24002,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:48.917177+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:34:48.917177+0000","last_peered":"2024-09-17T15:34:48.917177+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:34:48.917177+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"43'9400","ondisk_log_start":"43'9400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5611520,"num_objects":1370,"num_object_clones":0,"num_object_copies":2740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1936,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15689,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2224,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19469","reported_seq":19777,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:48.897355+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:34:48.897355+0000","last_peered":"2024-09-17T15:34:48.897355+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:34:48.897355+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'9400","ondisk_log_start":"43'9400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5804032,"num_objects":1417,"num_object_clones":0,"num_object_copies":2834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1417,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16256,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2563}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223917+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223917+0000","last_peered":"2024-09-17T15:33:11.223917+0000","last_clean":"2024-09-17T15:33:11.223917+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223917+0000","last_undegraded":"2024-09-17T15:33:11.223917+0000","last_fullsized":"2024-09-17T15:33:11.223917+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19332","reported_seq":27711,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:53.603928+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:34:53.603928+0000","last_peered":"2024-09-17T15:34:53.603928+0000","last_clean":"2024-09-17T15:34:53.603928+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:34:53.603928+0000","last_undegraded":"2024-09-17T15:34:53.603928+0000","last_fullsized":"2024-09-17T15:34:53.603928+0000","mapping_epoch":44,"log_start":"43'9300","ondisk_log_start":"43'9300","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5325766,"num_objects":1302,"num_object_clones":0,"num_object_copies":2604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15774,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":89338614,"num_objects":21821,"num_object_clones":0,"num_object_copies":43642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34347,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21821,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":271166,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15113,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":377724928,"data_stored":377590329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160731,"ondisk_log_size":160731,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051702,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535188,"kb_used_data":174468,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93836652,"statfs":{"total":96636764160,"available":96088731648,"internally_reserved":0,"allocated":178655232,"data_stored":178353634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":369356460},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,6,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724647,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":514272,"kb_used_data":63312,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93857568,"statfs":{"total":96636764160,"available":96110149632,"internally_reserved":0,"allocated":64831488,"data_stored":64507748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":461762219},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,2,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247929,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":585836,"kb_used_data":134932,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93786004,"statfs":{"total":96636764160,"available":96036868096,"internally_reserved":0,"allocated":138170368,"data_stored":137868765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":461696680},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,2,1,3,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":137121792,"data_stored":137085662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":62881792,"data_stored":62823525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":177721344,"data_stored":177681142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:34:54.833 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:34:54.834 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'19253", 'reported_seq': 19513, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:48.967079+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:34:48.967079+0000', 'last_peered': '2024-09-17T15:34:48.967079+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:34:48.967079+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'9200", 'ondisk_log_start': "43'9200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5713920, 'num_objects': 1395, 'num_object_clones': 0, 'num_object_copies': 2790, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1395, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19253, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2589}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:54.834 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:34:54.834 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'19563", 'reported_seq': 19786, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:53.258596+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:34:53.258596+0000', 'last_peered': '2024-09-17T15:34:53.258596+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:34:53.258596+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "43'9500", 'ondisk_log_start': "43'9500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5611544, 'num_objects': 1371, 'num_object_clones': 0, 'num_object_copies': 2742, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1371, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17720, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2754}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:34:54.834 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:34:54.834 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'19816", 'reported_seq': 20114, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:48.922837+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:34:48.922837+0000', 'last_peered': '2024-09-17T15:34:48.922837+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:34:48.922837+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "43'9800", 'ondisk_log_start': "43'9800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5878243, 'num_objects': 1436, 'num_object_clones': 0, 'num_object_copies': 2872, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1436, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16315, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2756}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:54.834 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:34:54.835 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'19375", 'reported_seq': 19694, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:48.864868+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:34:48.864868+0000', 'last_peered': '2024-09-17T15:34:48.864868+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:34:48.864868+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "43'9300", 'ondisk_log_start': "43'9300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5706694, 'num_objects': 1395, 'num_object_clones': 0, 'num_object_copies': 2790, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4089, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1395, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 16144, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 85, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2694}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:54.835 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:34:54.835 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'19124", 'reported_seq': 19399, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:48.926835+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:34:48.926835+0000', 'last_peered': '2024-09-17T15:34:48.926835+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:34:48.926835+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'9100", 'ondisk_log_start': "43'9100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5808128, 'num_objects': 1418, 'num_object_clones': 0, 'num_object_copies': 2836, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1418, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15720, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2525}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:54.835 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:34:54.835 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'19265", 'reported_seq': 19479, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:53.233837+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:34:53.233837+0000', 'last_peered': '2024-09-17T15:34:53.233837+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:34:53.233837+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "43'9200", 'ondisk_log_start': "43'9200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5136867, 'num_objects': 1255, 'num_object_clones': 0, 'num_object_copies': 2510, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4034, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1255, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17160, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -2778}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:34:54.835 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:34:54.835 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'19546", 'reported_seq': 19851, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:48.980619+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:34:48.980619+0000', 'last_peered': '2024-09-17T15:34:48.980619+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:34:48.980619+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "43'9500", 'ondisk_log_start': "43'9500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5816320, 'num_objects': 1420, 'num_object_clones': 0, 'num_object_copies': 2840, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1420, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16094, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2637}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:54.835 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:34:54.835 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'19402", 'reported_seq': 24002, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:48.917177+0000', 'last_change': '2024-09-17T15:33:09.763500+0000', 'last_active': '2024-09-17T15:34:48.917177+0000', 'last_peered': '2024-09-17T15:34:48.917177+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:34:48.917177+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "43'9400", 'ondisk_log_start': "43'9400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5611520, 'num_objects': 1370, 'num_object_clones': 0, 'num_object_copies': 2740, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1936, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1370, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15689, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2224, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -566}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:54.835 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:34:54.836 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'19469", 'reported_seq': 19777, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:48.897355+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:34:48.897355+0000', 'last_peered': '2024-09-17T15:34:48.897355+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:34:48.897355+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'9400", 'ondisk_log_start': "43'9400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5804032, 'num_objects': 1417, 'num_object_clones': 0, 'num_object_copies': 2834, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1417, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16256, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2563}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:34:54.836 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:34:55.543 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:34:55.543 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34347/43646 objects degraded (78.694%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":632,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":21823,"data_bytes":89338633,"bytes_used":1674543104,"bytes_avail":288235749376,"bytes_total":289910292480,"degraded_objects":34347,"degraded_total":43646,"degraded_ratio":0.78694496631993771,"recovering_objects_per_sec":21,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":253,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":500},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:34:55.573 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:34:55.574 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:34:56.146 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:34:56.149 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:34:56.169 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":356,"stamp":"2024-09-17T15:34:55.794446+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":83641097,"num_objects":20432,"num_object_clones":0,"num_object_copies":40864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34220,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20432,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":272559,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15241,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1632152,"kb_used_data":365408,"kb_used_omap":13,"kb_used_meta":1266674,"kb_avail":281483368,"statfs":{"total":289910292480,"available":288238968832,"internally_reserved":0,"allocated":374177792,"data_stored":373250851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1297075199},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,3,3,16,15,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25915392,"num_objects":-6327,"num_object_clones":0,"num_object_copies":-12654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-380,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6327,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":380,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004780"},"pg_stats":[{"pgid":"2.f","version":"47'19397","reported_seq":19657,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:54.010686+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:34:54.010686+0000","last_peered":"2024-09-17T15:34:54.010686+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:34:54.010686+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'9300","ondisk_log_start":"43'9300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5124096,"num_objects":1251,"num_object_clones":0,"num_object_copies":2502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1251,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19397,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2733}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19591","reported_seq":19704,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:53.290581+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:34:53.290581+0000","last_peered":"2024-09-17T15:34:53.290581+0000","last_clean":"2024-09-17T15:34:53.290581+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:34:53.290581+0000","last_undegraded":"2024-09-17T15:34:53.290581+0000","last_fullsized":"2024-09-17T15:34:53.290581+0000","mapping_epoch":33,"log_start":"43'9500","ondisk_log_start":"43'9500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5493702,"num_objects":1343,"num_object_clones":0,"num_object_copies":2686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1343,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17948,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19563","reported_seq":19786,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:53.258596+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:34:53.258596+0000","last_peered":"2024-09-17T15:34:53.258596+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:34:53.258596+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'9500","ondisk_log_start":"43'9500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5611544,"num_objects":1371,"num_object_clones":0,"num_object_copies":2742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1371,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17720,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2754}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19688","reported_seq":19768,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:53.244268+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:34:53.244268+0000","last_peered":"2024-09-17T15:34:53.244268+0000","last_clean":"2024-09-17T15:34:53.244268+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:34:53.244268+0000","last_undegraded":"2024-09-17T15:34:53.244268+0000","last_fullsized":"2024-09-17T15:34:53.244268+0000","mapping_epoch":33,"log_start":"43'9600","ondisk_log_start":"43'9600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5251072,"num_objects":1282,"num_object_clones":0,"num_object_copies":2564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19688,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19383","reported_seq":19522,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:53.958470+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:34:53.958470+0000","last_peered":"2024-09-17T15:34:53.958470+0000","last_clean":"2024-09-17T15:34:53.958470+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:34:53.958470+0000","last_undegraded":"2024-09-17T15:34:53.958470+0000","last_fullsized":"2024-09-17T15:34:53.958470+0000","mapping_epoch":32,"log_start":"43'9300","ondisk_log_start":"43'9300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5202403,"num_objects":1271,"num_object_clones":0,"num_object_copies":2542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1271,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16365,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19986","reported_seq":20287,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:53.993108+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:34:53.993108+0000","last_peered":"2024-09-17T15:34:53.993108+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:34:53.993108+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'9900","ondisk_log_start":"43'9900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5181923,"num_objects":1266,"num_object_clones":0,"num_object_copies":2532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1266,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16485,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2926},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19510","reported_seq":19829,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:53.989078+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:34:53.989078+0000","last_peered":"2024-09-17T15:34:53.989078+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:34:53.989078+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'9500","ondisk_log_start":"43'9500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5153734,"num_objects":1260,"num_object_clones":0,"num_object_copies":2520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1260,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16279,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2829},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19665","reported_seq":19803,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:53.975680+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:34:53.975680+0000","last_peered":"2024-09-17T15:34:53.975680+0000","last_clean":"2024-09-17T15:34:53.975680+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:34:53.975680+0000","last_undegraded":"2024-09-17T15:34:53.975680+0000","last_fullsized":"2024-09-17T15:34:53.975680+0000","mapping_epoch":32,"log_start":"43'9600","ondisk_log_start":"43'9600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5124096,"num_objects":1251,"num_object_clones":0,"num_object_copies":2502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1251,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16619,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.224142+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.224142+0000","last_peered":"2024-09-17T15:33:11.224142+0000","last_clean":"2024-09-17T15:33:11.224142+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.224142+0000","last_undegraded":"2024-09-17T15:33:11.224142+0000","last_fullsized":"2024-09-17T15:33:11.224142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19531","reported_seq":27824,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:53.592354+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:34:53.592354+0000","last_peered":"2024-09-17T15:34:53.592354+0000","last_clean":"2024-09-17T15:34:53.592354+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:34:53.592354+0000","last_undegraded":"2024-09-17T15:34:53.592354+0000","last_fullsized":"2024-09-17T15:34:53.592354+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5267939,"num_objects":1287,"num_object_clones":0,"num_object_copies":2574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1287,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17531,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19280","reported_seq":19555,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:54.030022+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:34:54.030022+0000","last_peered":"2024-09-17T15:34:54.030022+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:34:54.030022+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'9200","ondisk_log_start":"43'9200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5169152,"num_objects":1262,"num_object_clones":0,"num_object_copies":2524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15876,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2681}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19265","reported_seq":19479,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:53.233837+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:34:53.233837+0000","last_peered":"2024-09-17T15:34:53.233837+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:34:53.233837+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'9200","ondisk_log_start":"43'9200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5136867,"num_objects":1255,"num_object_clones":0,"num_object_copies":2510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1255,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17160,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2778}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":88,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223926+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223926+0000","last_peered":"2024-09-17T15:33:11.223926+0000","last_clean":"2024-09-17T15:33:11.223926+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223926+0000","last_undegraded":"2024-09-17T15:33:11.223926+0000","last_fullsized":"2024-09-17T15:33:11.223926+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19455","reported_seq":27870,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:53.316221+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:34:53.316221+0000","last_peered":"2024-09-17T15:34:53.316221+0000","last_clean":"2024-09-17T15:34:53.316221+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:34:53.316221+0000","last_undegraded":"2024-09-17T15:34:53.316221+0000","last_fullsized":"2024-09-17T15:34:53.316221+0000","mapping_epoch":46,"log_start":"43'9400","ondisk_log_start":"43'9400","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5246976,"num_objects":1281,"num_object_clones":0,"num_object_copies":2562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17217,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19700","reported_seq":20004,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:54.008580+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:34:54.008580+0000","last_peered":"2024-09-17T15:34:54.008580+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:34:54.008580+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'9600","ondisk_log_start":"43'9600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5185536,"num_objects":1266,"num_object_clones":0,"num_object_copies":2532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16248,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2791}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223809+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223809+0000","last_peered":"2024-09-17T15:33:11.223809+0000","last_clean":"2024-09-17T15:33:11.223809+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223809+0000","last_undegraded":"2024-09-17T15:33:11.223809+0000","last_fullsized":"2024-09-17T15:33:11.223809+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19548","reported_seq":24403,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:53.992790+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:34:53.992790+0000","last_peered":"2024-09-17T15:34:53.992790+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:34:53.992790+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"43'9500","ondisk_log_start":"43'9500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5013504,"num_objects":1224,"num_object_clones":0,"num_object_copies":2448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1809,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15835,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2351,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-585}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19628","reported_seq":19936,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:53.958588+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:34:53.958588+0000","last_peered":"2024-09-17T15:34:53.958588+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:34:53.958588+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'9600","ondisk_log_start":"43'9600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5152768,"num_objects":1258,"num_object_clones":0,"num_object_copies":2516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16415,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2722}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223917+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223917+0000","last_peered":"2024-09-17T15:33:11.223917+0000","last_clean":"2024-09-17T15:33:11.223917+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223917+0000","last_undegraded":"2024-09-17T15:33:11.223917+0000","last_fullsized":"2024-09-17T15:33:11.223917+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19332","reported_seq":27711,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:53.603928+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:34:53.603928+0000","last_peered":"2024-09-17T15:34:53.603928+0000","last_clean":"2024-09-17T15:34:53.603928+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:34:53.603928+0000","last_undegraded":"2024-09-17T15:34:53.603928+0000","last_fullsized":"2024-09-17T15:34:53.603928+0000","mapping_epoch":44,"log_start":"43'9300","ondisk_log_start":"43'9300","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5325766,"num_objects":1302,"num_object_clones":0,"num_object_copies":2604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15774,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":83641078,"num_objects":20430,"num_object_clones":0,"num_object_copies":40860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34220,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20430,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":272557,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15240,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":370098176,"data_stored":369963577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051702,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535188,"kb_used_data":174468,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93836652,"statfs":{"total":96636764160,"available":96088731648,"internally_reserved":0,"allocated":178655232,"data_stored":178353634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":369356460},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,6,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724647,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":514272,"kb_used_data":63312,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93857568,"statfs":{"total":96636764160,"available":96110149632,"internally_reserved":0,"allocated":64831488,"data_stored":64507748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":461762219},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,2,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247930,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":582692,"kb_used_data":127628,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93789148,"statfs":{"total":96636764160,"available":96040087552,"internally_reserved":0,"allocated":130691072,"data_stored":130389469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":465956520},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,6,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":129495040,"data_stored":129458910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":62881792,"data_stored":62823525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":177721344,"data_stored":177681142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:34:59.171 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:34:59.747 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:34:59.747 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:34:59.774 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":357,"stamp":"2024-09-17T15:34:57.795221+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":83641097,"num_objects":20432,"num_object_clones":0,"num_object_copies":40864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34220,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20432,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":272559,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15241,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1632152,"kb_used_data":365408,"kb_used_omap":13,"kb_used_meta":1266674,"kb_avail":281483368,"statfs":{"total":289910292480,"available":288238968832,"internally_reserved":0,"allocated":374177792,"data_stored":373250851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1297075199},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,3,3,16,15,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20185088,"num_objects":-4928,"num_object_clones":0,"num_object_copies":-9856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-255,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4928,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4928,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":255,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004734"},"pg_stats":[{"pgid":"2.f","version":"47'19397","reported_seq":19657,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:54.010686+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:34:54.010686+0000","last_peered":"2024-09-17T15:34:54.010686+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:34:54.010686+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'9300","ondisk_log_start":"43'9300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5124096,"num_objects":1251,"num_object_clones":0,"num_object_copies":2502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1251,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19397,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2733}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19591","reported_seq":19704,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:53.290581+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:34:53.290581+0000","last_peered":"2024-09-17T15:34:53.290581+0000","last_clean":"2024-09-17T15:34:53.290581+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:34:53.290581+0000","last_undegraded":"2024-09-17T15:34:53.290581+0000","last_fullsized":"2024-09-17T15:34:53.290581+0000","mapping_epoch":33,"log_start":"43'9500","ondisk_log_start":"43'9500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5493702,"num_objects":1343,"num_object_clones":0,"num_object_copies":2686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1343,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17948,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19563","reported_seq":19786,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:53.258596+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:34:53.258596+0000","last_peered":"2024-09-17T15:34:53.258596+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:34:53.258596+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'9500","ondisk_log_start":"43'9500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5611544,"num_objects":1371,"num_object_clones":0,"num_object_copies":2742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1371,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17720,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2754}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19688","reported_seq":19768,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:53.244268+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:34:53.244268+0000","last_peered":"2024-09-17T15:34:53.244268+0000","last_clean":"2024-09-17T15:34:53.244268+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:34:53.244268+0000","last_undegraded":"2024-09-17T15:34:53.244268+0000","last_fullsized":"2024-09-17T15:34:53.244268+0000","mapping_epoch":33,"log_start":"43'9600","ondisk_log_start":"43'9600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5251072,"num_objects":1282,"num_object_clones":0,"num_object_copies":2564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19688,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19383","reported_seq":19522,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:53.958470+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:34:53.958470+0000","last_peered":"2024-09-17T15:34:53.958470+0000","last_clean":"2024-09-17T15:34:53.958470+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:34:53.958470+0000","last_undegraded":"2024-09-17T15:34:53.958470+0000","last_fullsized":"2024-09-17T15:34:53.958470+0000","mapping_epoch":32,"log_start":"43'9300","ondisk_log_start":"43'9300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5202403,"num_objects":1271,"num_object_clones":0,"num_object_copies":2542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1271,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16365,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19986","reported_seq":20287,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:53.993108+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:34:53.993108+0000","last_peered":"2024-09-17T15:34:53.993108+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:34:53.993108+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'9900","ondisk_log_start":"43'9900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5181923,"num_objects":1266,"num_object_clones":0,"num_object_copies":2532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1266,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16485,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2926},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19510","reported_seq":19829,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:53.989078+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:34:53.989078+0000","last_peered":"2024-09-17T15:34:53.989078+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:34:53.989078+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'9500","ondisk_log_start":"43'9500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5153734,"num_objects":1260,"num_object_clones":0,"num_object_copies":2520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1260,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16279,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2829},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19665","reported_seq":19803,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:53.975680+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:34:53.975680+0000","last_peered":"2024-09-17T15:34:53.975680+0000","last_clean":"2024-09-17T15:34:53.975680+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:34:53.975680+0000","last_undegraded":"2024-09-17T15:34:53.975680+0000","last_fullsized":"2024-09-17T15:34:53.975680+0000","mapping_epoch":32,"log_start":"43'9600","ondisk_log_start":"43'9600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5124096,"num_objects":1251,"num_object_clones":0,"num_object_copies":2502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1251,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16619,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.224142+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.224142+0000","last_peered":"2024-09-17T15:33:11.224142+0000","last_clean":"2024-09-17T15:33:11.224142+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.224142+0000","last_undegraded":"2024-09-17T15:33:11.224142+0000","last_fullsized":"2024-09-17T15:33:11.224142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19531","reported_seq":27824,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:53.592354+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:34:53.592354+0000","last_peered":"2024-09-17T15:34:53.592354+0000","last_clean":"2024-09-17T15:34:53.592354+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:34:53.592354+0000","last_undegraded":"2024-09-17T15:34:53.592354+0000","last_fullsized":"2024-09-17T15:34:53.592354+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5267939,"num_objects":1287,"num_object_clones":0,"num_object_copies":2574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1287,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17531,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19280","reported_seq":19555,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:54.030022+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:34:54.030022+0000","last_peered":"2024-09-17T15:34:54.030022+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:34:54.030022+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'9200","ondisk_log_start":"43'9200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5169152,"num_objects":1262,"num_object_clones":0,"num_object_copies":2524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15876,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2681}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19265","reported_seq":19479,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:53.233837+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:34:53.233837+0000","last_peered":"2024-09-17T15:34:53.233837+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:34:53.233837+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'9200","ondisk_log_start":"43'9200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5136867,"num_objects":1255,"num_object_clones":0,"num_object_copies":2510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1255,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17160,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2778}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":88,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223926+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223926+0000","last_peered":"2024-09-17T15:33:11.223926+0000","last_clean":"2024-09-17T15:33:11.223926+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223926+0000","last_undegraded":"2024-09-17T15:33:11.223926+0000","last_fullsized":"2024-09-17T15:33:11.223926+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19455","reported_seq":27870,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:53.316221+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:34:53.316221+0000","last_peered":"2024-09-17T15:34:53.316221+0000","last_clean":"2024-09-17T15:34:53.316221+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:34:53.316221+0000","last_undegraded":"2024-09-17T15:34:53.316221+0000","last_fullsized":"2024-09-17T15:34:53.316221+0000","mapping_epoch":46,"log_start":"43'9400","ondisk_log_start":"43'9400","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5246976,"num_objects":1281,"num_object_clones":0,"num_object_copies":2562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17217,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19700","reported_seq":20004,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:54.008580+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:34:54.008580+0000","last_peered":"2024-09-17T15:34:54.008580+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:34:54.008580+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'9600","ondisk_log_start":"43'9600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5185536,"num_objects":1266,"num_object_clones":0,"num_object_copies":2532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16248,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2791}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223809+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223809+0000","last_peered":"2024-09-17T15:33:11.223809+0000","last_clean":"2024-09-17T15:33:11.223809+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223809+0000","last_undegraded":"2024-09-17T15:33:11.223809+0000","last_fullsized":"2024-09-17T15:33:11.223809+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19548","reported_seq":24403,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:53.992790+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:34:53.992790+0000","last_peered":"2024-09-17T15:34:53.992790+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:34:53.992790+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"43'9500","ondisk_log_start":"43'9500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5013504,"num_objects":1224,"num_object_clones":0,"num_object_copies":2448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1809,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15835,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2351,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-585}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19628","reported_seq":19936,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:53.958588+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:34:53.958588+0000","last_peered":"2024-09-17T15:34:53.958588+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:34:53.958588+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'9600","ondisk_log_start":"43'9600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5152768,"num_objects":1258,"num_object_clones":0,"num_object_copies":2516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16415,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2722}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223917+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223917+0000","last_peered":"2024-09-17T15:33:11.223917+0000","last_clean":"2024-09-17T15:33:11.223917+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223917+0000","last_undegraded":"2024-09-17T15:33:11.223917+0000","last_fullsized":"2024-09-17T15:33:11.223917+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19332","reported_seq":27711,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:53.603928+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:34:53.603928+0000","last_peered":"2024-09-17T15:34:53.603928+0000","last_clean":"2024-09-17T15:34:53.603928+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:34:53.603928+0000","last_undegraded":"2024-09-17T15:34:53.603928+0000","last_fullsized":"2024-09-17T15:34:53.603928+0000","mapping_epoch":44,"log_start":"43'9300","ondisk_log_start":"43'9300","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5325766,"num_objects":1302,"num_object_clones":0,"num_object_copies":2604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15774,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":83641078,"num_objects":20430,"num_object_clones":0,"num_object_copies":40860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34220,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20430,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":272557,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15240,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":370098176,"data_stored":369963577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051702,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535188,"kb_used_data":174468,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93836652,"statfs":{"total":96636764160,"available":96088731648,"internally_reserved":0,"allocated":178655232,"data_stored":178353634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":369356460},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,6,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724647,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":514272,"kb_used_data":63312,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93857568,"statfs":{"total":96636764160,"available":96110149632,"internally_reserved":0,"allocated":64831488,"data_stored":64507748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":461762219},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,2,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247930,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":582692,"kb_used_data":127628,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93789148,"statfs":{"total":96636764160,"available":96040087552,"internally_reserved":0,"allocated":130691072,"data_stored":130389469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":465956520},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,6,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":129495040,"data_stored":129458910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":62881792,"data_stored":62823525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":177721344,"data_stored":177681142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:34:59.775 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:35:00.331 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:35:00.331 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:35:00.369 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":358,"stamp":"2024-09-17T15:34:59.795960+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":73556745,"num_objects":17970,"num_object_clones":0,"num_object_copies":35940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34082,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17970,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":275021,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15379,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160787,"ondisk_log_size":160787,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1599900,"kb_used_data":345640,"kb_used_omap":13,"kb_used_meta":1254194,"kb_avail":281515620,"statfs":{"total":289910292480,"available":288271994880,"internally_reserved":0,"allocated":353935360,"data_stored":353008419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1284295679},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,6,8,9,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-30269440,"num_objects":-7390,"num_object_clones":0,"num_object_copies":-14780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-393,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7390,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":393,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004659"},"pg_stats":[{"pgid":"2.f","version":"47'19552","reported_seq":19812,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:59.053621+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:34:59.053621+0000","last_peered":"2024-09-17T15:34:59.053621+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:34:59.053621+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'9500","ondisk_log_start":"43'9500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4489216,"num_objects":1096,"num_object_clones":0,"num_object_copies":2192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19552,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2888}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19741","reported_seq":19854,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:58.308174+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:34:58.308174+0000","last_peered":"2024-09-17T15:34:58.308174+0000","last_clean":"2024-09-17T15:34:58.308174+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:34:58.308174+0000","last_undegraded":"2024-09-17T15:34:58.308174+0000","last_fullsized":"2024-09-17T15:34:58.308174+0000","mapping_epoch":33,"log_start":"43'9700","ondisk_log_start":"43'9700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4879302,"num_objects":1193,"num_object_clones":0,"num_object_copies":2386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1193,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18098,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19733","reported_seq":19956,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:58.315401+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:34:58.315401+0000","last_peered":"2024-09-17T15:34:58.315401+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:34:58.315401+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'9700","ondisk_log_start":"43'9700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4915224,"num_objects":1201,"num_object_clones":0,"num_object_copies":2402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1201,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17890,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2924}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19871","reported_seq":19951,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:58.273255+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:34:58.273255+0000","last_peered":"2024-09-17T15:34:58.273255+0000","last_clean":"2024-09-17T15:34:58.273255+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:34:58.273255+0000","last_undegraded":"2024-09-17T15:34:58.273255+0000","last_fullsized":"2024-09-17T15:34:58.273255+0000","mapping_epoch":33,"log_start":"43'9800","ondisk_log_start":"43'9800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4501504,"num_objects":1099,"num_object_clones":0,"num_object_copies":2198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1099,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19871,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19527","reported_seq":19667,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:59.080869+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:34:59.080869+0000","last_peered":"2024-09-17T15:34:59.080869+0000","last_clean":"2024-09-17T15:34:59.080869+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:34:59.080869+0000","last_undegraded":"2024-09-17T15:34:59.080869+0000","last_fullsized":"2024-09-17T15:34:59.080869+0000","mapping_epoch":32,"log_start":"43'9500","ondisk_log_start":"43'9500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4612579,"num_objects":1127,"num_object_clones":0,"num_object_copies":2254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1127,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16509,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'20136","reported_seq":20434,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:58.900769+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:34:58.900769+0000","last_peered":"2024-09-17T15:34:58.900769+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:34:58.900769+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'10100","ondisk_log_start":"43'10100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4567523,"num_objects":1116,"num_object_clones":0,"num_object_copies":2232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1116,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16635,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3076},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19677","reported_seq":19996,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:59.012535+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:34:59.012535+0000","last_peered":"2024-09-17T15:34:59.012535+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:34:59.012535+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'9600","ondisk_log_start":"43'9600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4469702,"num_objects":1093,"num_object_clones":0,"num_object_copies":2186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1093,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16446,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2996},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19827","reported_seq":19965,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:59.033409+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:34:59.033409+0000","last_peered":"2024-09-17T15:34:59.033409+0000","last_clean":"2024-09-17T15:34:59.033409+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:34:59.033409+0000","last_undegraded":"2024-09-17T15:34:59.033409+0000","last_fullsized":"2024-09-17T15:34:59.033409+0000","mapping_epoch":32,"log_start":"43'9800","ondisk_log_start":"43'9800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4460544,"num_objects":1089,"num_object_clones":0,"num_object_copies":2178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16781,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.224142+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.224142+0000","last_peered":"2024-09-17T15:33:11.224142+0000","last_clean":"2024-09-17T15:33:11.224142+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.224142+0000","last_undegraded":"2024-09-17T15:33:11.224142+0000","last_fullsized":"2024-09-17T15:33:11.224142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19667","reported_seq":27960,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:58.634469+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:34:58.634469+0000","last_peered":"2024-09-17T15:34:58.634469+0000","last_clean":"2024-09-17T15:34:58.634469+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:34:58.634469+0000","last_undegraded":"2024-09-17T15:34:58.634469+0000","last_fullsized":"2024-09-17T15:34:58.634469+0000","mapping_epoch":42,"log_start":"43'9600","ondisk_log_start":"43'9600","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4710883,"num_objects":1151,"num_object_clones":0,"num_object_copies":2302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1151,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17667,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19428","reported_seq":19703,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:59.021807+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:34:59.021807+0000","last_peered":"2024-09-17T15:34:59.021807+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:34:59.021807+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'9400","ondisk_log_start":"43'9400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4562944,"num_objects":1114,"num_object_clones":0,"num_object_copies":2228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16024,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2829}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19405","reported_seq":19624,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:58.255925+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:34:58.255925+0000","last_peered":"2024-09-17T15:34:58.255925+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:34:58.255925+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'9400","ondisk_log_start":"43'9400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4563427,"num_objects":1115,"num_object_clones":0,"num_object_copies":2230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1115,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17300,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2918}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":88,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223926+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223926+0000","last_peered":"2024-09-17T15:33:11.223926+0000","last_clean":"2024-09-17T15:33:11.223926+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223926+0000","last_undegraded":"2024-09-17T15:33:11.223926+0000","last_fullsized":"2024-09-17T15:33:11.223926+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19593","reported_seq":28008,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:58.248602+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:34:58.248602+0000","last_peered":"2024-09-17T15:34:58.248602+0000","last_clean":"2024-09-17T15:34:58.248602+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:34:58.248602+0000","last_undegraded":"2024-09-17T15:34:58.248602+0000","last_fullsized":"2024-09-17T15:34:58.248602+0000","mapping_epoch":46,"log_start":"43'9500","ondisk_log_start":"43'9500","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4681728,"num_objects":1143,"num_object_clones":0,"num_object_copies":2286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1143,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17355,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19848","reported_seq":20156,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:59.052083+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:34:59.052083+0000","last_peered":"2024-09-17T15:34:59.052083+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:34:59.052083+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'9800","ondisk_log_start":"43'9800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4579328,"num_objects":1118,"num_object_clones":0,"num_object_copies":2236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16396,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223809+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223809+0000","last_peered":"2024-09-17T15:33:11.223809+0000","last_clean":"2024-09-17T15:33:11.223809+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223809+0000","last_undegraded":"2024-09-17T15:33:11.223809+0000","last_fullsized":"2024-09-17T15:33:11.223809+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19703","reported_seq":24832,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:59.084377+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:34:59.084377+0000","last_peered":"2024-09-17T15:34:59.084377+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:34:59.084377+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"43'9700","ondisk_log_start":"43'9700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4378624,"num_objects":1069,"num_object_clones":0,"num_object_copies":2138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1671,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15990,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2489,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-602}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19783","reported_seq":20090,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:59.041818+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:34:59.041818+0000","last_peered":"2024-09-17T15:34:59.041818+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:34:59.041818+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'9700","ondisk_log_start":"43'9700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4517888,"num_objects":1103,"num_object_clones":0,"num_object_copies":2206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16570,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223917+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223917+0000","last_peered":"2024-09-17T15:33:11.223917+0000","last_clean":"2024-09-17T15:33:11.223917+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223917+0000","last_undegraded":"2024-09-17T15:33:11.223917+0000","last_fullsized":"2024-09-17T15:33:11.223917+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19493","reported_seq":27876,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:58.634685+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:34:58.634685+0000","last_peered":"2024-09-17T15:34:58.634685+0000","last_clean":"2024-09-17T15:34:58.634685+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:34:58.634685+0000","last_undegraded":"2024-09-17T15:34:58.634685+0000","last_fullsized":"2024-09-17T15:34:58.634685+0000","mapping_epoch":44,"log_start":"43'9400","ondisk_log_start":"43'9400","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4666310,"num_objects":1141,"num_object_clones":0,"num_object_copies":2282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1141,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15935,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":73556726,"num_objects":17968,"num_object_clones":0,"num_object_copies":35936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34082,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17968,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":275019,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15378,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":349937664,"data_stored":349803065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160784,"ondisk_log_size":160784,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051703,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":538356,"kb_used_data":169312,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93833484,"statfs":{"total":96636764160,"available":96085487616,"internally_reserved":0,"allocated":173375488,"data_stored":173073890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":377876140},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724648,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":498668,"kb_used_data":56028,"kb_used_omap":4,"kb_used_meta":442619,"kb_avail":93873172,"statfs":{"total":96636764160,"available":96126128128,"internally_reserved":0,"allocated":57372672,"data_stored":57048932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":453242539},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,4,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247931,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562876,"kb_used_data":120300,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93808964,"statfs":{"total":96636764160,"available":96060379136,"internally_reserved":0,"allocated":123187200,"data_stored":122885597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":453177000},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,4,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":121823232,"data_stored":121787102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":55451648,"data_stored":55393381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":172662784,"data_stored":172622582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:35:00.371 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:35:00.371 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'19552", 'reported_seq': 19812, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:59.053621+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:34:59.053621+0000', 'last_peered': '2024-09-17T15:34:59.053621+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:34:59.053621+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'9500", 'ondisk_log_start': "43'9500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4489216, 'num_objects': 1096, 'num_object_clones': 0, 'num_object_copies': 2192, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1096, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19552, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2888}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:00.371 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:35:00.371 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'19733", 'reported_seq': 19956, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:58.315401+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:34:58.315401+0000', 'last_peered': '2024-09-17T15:34:58.315401+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:34:58.315401+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "43'9700", 'ondisk_log_start': "43'9700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4915224, 'num_objects': 1201, 'num_object_clones': 0, 'num_object_copies': 2402, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1201, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17890, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2924}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:35:00.371 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:35:00.371 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'20136", 'reported_seq': 20434, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:58.900769+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:34:58.900769+0000', 'last_peered': '2024-09-17T15:34:58.900769+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:34:58.900769+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "43'10100", 'ondisk_log_start': "43'10100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4567523, 'num_objects': 1116, 'num_object_clones': 0, 'num_object_copies': 2232, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1116, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16635, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3076}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:00.371 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:35:00.371 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'19677", 'reported_seq': 19996, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:59.012535+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:34:59.012535+0000', 'last_peered': '2024-09-17T15:34:59.012535+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:34:59.012535+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "43'9600", 'ondisk_log_start': "43'9600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4469702, 'num_objects': 1093, 'num_object_clones': 0, 'num_object_copies': 2186, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4089, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1093, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 16446, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 85, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2996}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:00.371 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:35:00.372 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'19428", 'reported_seq': 19703, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:59.021807+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:34:59.021807+0000', 'last_peered': '2024-09-17T15:34:59.021807+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:34:59.021807+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'9400", 'ondisk_log_start': "43'9400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4562944, 'num_objects': 1114, 'num_object_clones': 0, 'num_object_copies': 2228, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1114, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16024, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2829}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:00.372 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:35:00.372 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'19405", 'reported_seq': 19624, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:58.255925+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:34:58.255925+0000', 'last_peered': '2024-09-17T15:34:58.255925+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:34:58.255925+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "43'9400", 'ondisk_log_start': "43'9400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4563427, 'num_objects': 1115, 'num_object_clones': 0, 'num_object_copies': 2230, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4034, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1115, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17300, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -2918}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:35:00.372 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:35:00.372 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'19848", 'reported_seq': 20156, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:59.052083+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:34:59.052083+0000', 'last_peered': '2024-09-17T15:34:59.052083+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:34:59.052083+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "43'9800", 'ondisk_log_start': "43'9800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4579328, 'num_objects': 1118, 'num_object_clones': 0, 'num_object_copies': 2236, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1118, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16396, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2939}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:00.372 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:35:00.372 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'19703", 'reported_seq': 24832, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:59.084377+0000', 'last_change': '2024-09-17T15:33:09.763500+0000', 'last_active': '2024-09-17T15:34:59.084377+0000', 'last_peered': '2024-09-17T15:34:59.084377+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:34:59.084377+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "43'9700", 'ondisk_log_start': "43'9700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4378624, 'num_objects': 1069, 'num_object_clones': 0, 'num_object_copies': 2138, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1671, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1069, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15990, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2489, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -602}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:00.372 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:35:00.372 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'19783", 'reported_seq': 20090, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:34:59.041818+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:34:59.041818+0000', 'last_peered': '2024-09-17T15:34:59.041818+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:34:59.041818+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'9700", 'ondisk_log_start': "43'9700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4517888, 'num_objects': 1103, 'num_object_clones': 0, 'num_object_copies': 2206, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1103, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16570, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2877}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:00.373 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:35:01.054 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:35:01.054 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34082/35940 objects degraded (94.830%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":638,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":17970,"data_bytes":73556745,"bytes_used":1638297600,"bytes_avail":288271994880,"bytes_total":289910292480,"degraded_objects":34082,"degraded_total":35940,"degraded_ratio":0.94830272676683369,"recovering_objects_per_sec":32,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":393,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":615},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:35:01.079 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:35:01.079 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:35:01.632 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:35:01.634 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:35:01.657 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":358,"stamp":"2024-09-17T15:34:59.795960+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":73556745,"num_objects":17970,"num_object_clones":0,"num_object_copies":35940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34082,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17970,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":275021,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15379,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160787,"ondisk_log_size":160787,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1599900,"kb_used_data":345640,"kb_used_omap":13,"kb_used_meta":1254194,"kb_avail":281515620,"statfs":{"total":289910292480,"available":288271994880,"internally_reserved":0,"allocated":353935360,"data_stored":353008419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1284295679},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,6,8,9,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-30269440,"num_objects":-7390,"num_object_clones":0,"num_object_copies":-14780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-393,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7390,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":393,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004659"},"pg_stats":[{"pgid":"2.f","version":"47'19552","reported_seq":19812,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:59.053621+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:34:59.053621+0000","last_peered":"2024-09-17T15:34:59.053621+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:34:59.053621+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'9500","ondisk_log_start":"43'9500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4489216,"num_objects":1096,"num_object_clones":0,"num_object_copies":2192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19552,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2888}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19741","reported_seq":19854,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:58.308174+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:34:58.308174+0000","last_peered":"2024-09-17T15:34:58.308174+0000","last_clean":"2024-09-17T15:34:58.308174+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:34:58.308174+0000","last_undegraded":"2024-09-17T15:34:58.308174+0000","last_fullsized":"2024-09-17T15:34:58.308174+0000","mapping_epoch":33,"log_start":"43'9700","ondisk_log_start":"43'9700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4879302,"num_objects":1193,"num_object_clones":0,"num_object_copies":2386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1193,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18098,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19733","reported_seq":19956,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:58.315401+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:34:58.315401+0000","last_peered":"2024-09-17T15:34:58.315401+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:34:58.315401+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'9700","ondisk_log_start":"43'9700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4915224,"num_objects":1201,"num_object_clones":0,"num_object_copies":2402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1201,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17890,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2924}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19871","reported_seq":19951,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:58.273255+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:34:58.273255+0000","last_peered":"2024-09-17T15:34:58.273255+0000","last_clean":"2024-09-17T15:34:58.273255+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:34:58.273255+0000","last_undegraded":"2024-09-17T15:34:58.273255+0000","last_fullsized":"2024-09-17T15:34:58.273255+0000","mapping_epoch":33,"log_start":"43'9800","ondisk_log_start":"43'9800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4501504,"num_objects":1099,"num_object_clones":0,"num_object_copies":2198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1099,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19871,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19527","reported_seq":19667,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:59.080869+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:34:59.080869+0000","last_peered":"2024-09-17T15:34:59.080869+0000","last_clean":"2024-09-17T15:34:59.080869+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:34:59.080869+0000","last_undegraded":"2024-09-17T15:34:59.080869+0000","last_fullsized":"2024-09-17T15:34:59.080869+0000","mapping_epoch":32,"log_start":"43'9500","ondisk_log_start":"43'9500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4612579,"num_objects":1127,"num_object_clones":0,"num_object_copies":2254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1127,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16509,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'20136","reported_seq":20434,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:58.900769+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:34:58.900769+0000","last_peered":"2024-09-17T15:34:58.900769+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:34:58.900769+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'10100","ondisk_log_start":"43'10100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4567523,"num_objects":1116,"num_object_clones":0,"num_object_copies":2232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1116,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16635,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3076},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19677","reported_seq":19996,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:59.012535+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:34:59.012535+0000","last_peered":"2024-09-17T15:34:59.012535+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:34:59.012535+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'9600","ondisk_log_start":"43'9600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4469702,"num_objects":1093,"num_object_clones":0,"num_object_copies":2186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1093,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16446,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2996},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19827","reported_seq":19965,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:59.033409+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:34:59.033409+0000","last_peered":"2024-09-17T15:34:59.033409+0000","last_clean":"2024-09-17T15:34:59.033409+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:34:59.033409+0000","last_undegraded":"2024-09-17T15:34:59.033409+0000","last_fullsized":"2024-09-17T15:34:59.033409+0000","mapping_epoch":32,"log_start":"43'9800","ondisk_log_start":"43'9800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4460544,"num_objects":1089,"num_object_clones":0,"num_object_copies":2178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16781,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.224142+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.224142+0000","last_peered":"2024-09-17T15:33:11.224142+0000","last_clean":"2024-09-17T15:33:11.224142+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.224142+0000","last_undegraded":"2024-09-17T15:33:11.224142+0000","last_fullsized":"2024-09-17T15:33:11.224142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19667","reported_seq":27960,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:58.634469+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:34:58.634469+0000","last_peered":"2024-09-17T15:34:58.634469+0000","last_clean":"2024-09-17T15:34:58.634469+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:34:58.634469+0000","last_undegraded":"2024-09-17T15:34:58.634469+0000","last_fullsized":"2024-09-17T15:34:58.634469+0000","mapping_epoch":42,"log_start":"43'9600","ondisk_log_start":"43'9600","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4710883,"num_objects":1151,"num_object_clones":0,"num_object_copies":2302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1151,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17667,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19428","reported_seq":19703,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:59.021807+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:34:59.021807+0000","last_peered":"2024-09-17T15:34:59.021807+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:34:59.021807+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'9400","ondisk_log_start":"43'9400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4562944,"num_objects":1114,"num_object_clones":0,"num_object_copies":2228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16024,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2829}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19405","reported_seq":19624,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:58.255925+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:34:58.255925+0000","last_peered":"2024-09-17T15:34:58.255925+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:34:58.255925+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'9400","ondisk_log_start":"43'9400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4563427,"num_objects":1115,"num_object_clones":0,"num_object_copies":2230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1115,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17300,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2918}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":88,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223926+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223926+0000","last_peered":"2024-09-17T15:33:11.223926+0000","last_clean":"2024-09-17T15:33:11.223926+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223926+0000","last_undegraded":"2024-09-17T15:33:11.223926+0000","last_fullsized":"2024-09-17T15:33:11.223926+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19593","reported_seq":28008,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:58.248602+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:34:58.248602+0000","last_peered":"2024-09-17T15:34:58.248602+0000","last_clean":"2024-09-17T15:34:58.248602+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:34:58.248602+0000","last_undegraded":"2024-09-17T15:34:58.248602+0000","last_fullsized":"2024-09-17T15:34:58.248602+0000","mapping_epoch":46,"log_start":"43'9500","ondisk_log_start":"43'9500","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4681728,"num_objects":1143,"num_object_clones":0,"num_object_copies":2286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1143,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17355,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19848","reported_seq":20156,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:59.052083+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:34:59.052083+0000","last_peered":"2024-09-17T15:34:59.052083+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:34:59.052083+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'9800","ondisk_log_start":"43'9800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4579328,"num_objects":1118,"num_object_clones":0,"num_object_copies":2236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16396,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223809+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223809+0000","last_peered":"2024-09-17T15:33:11.223809+0000","last_clean":"2024-09-17T15:33:11.223809+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223809+0000","last_undegraded":"2024-09-17T15:33:11.223809+0000","last_fullsized":"2024-09-17T15:33:11.223809+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19703","reported_seq":24832,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:59.084377+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:34:59.084377+0000","last_peered":"2024-09-17T15:34:59.084377+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:34:59.084377+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"43'9700","ondisk_log_start":"43'9700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4378624,"num_objects":1069,"num_object_clones":0,"num_object_copies":2138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1671,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15990,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2489,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-602}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19783","reported_seq":20090,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:59.041818+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:34:59.041818+0000","last_peered":"2024-09-17T15:34:59.041818+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:34:59.041818+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'9700","ondisk_log_start":"43'9700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4517888,"num_objects":1103,"num_object_clones":0,"num_object_copies":2206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16570,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223917+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223917+0000","last_peered":"2024-09-17T15:33:11.223917+0000","last_clean":"2024-09-17T15:33:11.223917+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223917+0000","last_undegraded":"2024-09-17T15:33:11.223917+0000","last_fullsized":"2024-09-17T15:33:11.223917+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223676+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223676+0000","last_peered":"2024-09-17T15:33:11.223676+0000","last_clean":"2024-09-17T15:33:11.223676+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223676+0000","last_undegraded":"2024-09-17T15:33:11.223676+0000","last_fullsized":"2024-09-17T15:33:11.223676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19493","reported_seq":27876,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:58.634685+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:34:58.634685+0000","last_peered":"2024-09-17T15:34:58.634685+0000","last_clean":"2024-09-17T15:34:58.634685+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:34:58.634685+0000","last_undegraded":"2024-09-17T15:34:58.634685+0000","last_fullsized":"2024-09-17T15:34:58.634685+0000","mapping_epoch":44,"log_start":"43'9400","ondisk_log_start":"43'9400","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4666310,"num_objects":1141,"num_object_clones":0,"num_object_copies":2282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1141,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15935,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":73556726,"num_objects":17968,"num_object_clones":0,"num_object_copies":35936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34082,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17968,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":275019,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15378,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":349937664,"data_stored":349803065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160784,"ondisk_log_size":160784,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051703,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":538356,"kb_used_data":169312,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93833484,"statfs":{"total":96636764160,"available":96085487616,"internally_reserved":0,"allocated":173375488,"data_stored":173073890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":377876140},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724648,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":498668,"kb_used_data":56028,"kb_used_omap":4,"kb_used_meta":442619,"kb_avail":93873172,"statfs":{"total":96636764160,"available":96126128128,"internally_reserved":0,"allocated":57372672,"data_stored":57048932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":453242539},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,4,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247931,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562876,"kb_used_data":120300,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93808964,"statfs":{"total":96636764160,"available":96060379136,"internally_reserved":0,"allocated":123187200,"data_stored":122885597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":453177000},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,4,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":121823232,"data_stored":121787102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":55451648,"data_stored":55393381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":172662784,"data_stored":172622582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:35:04.660 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:35:05.237 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:35:05.238 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:35:05.261 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2024-09-17T15:35:03.797596+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":69133065,"num_objects":16890,"num_object_clones":0,"num_object_copies":33780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34082,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16890,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":276101,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15379,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1583308,"kb_used_data":333196,"kb_used_omap":13,"kb_used_meta":1250034,"kb_avail":281532212,"statfs":{"total":289910292480,"available":288288985088,"internally_reserved":0,"allocated":341192704,"data_stored":340265763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1280035839},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,7,5,7,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24690688,"num_objects":-6028,"num_object_clones":0,"num_object_copies":-12056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-265,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6028,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6028,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":265,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004804"},"pg_stats":[{"pgid":"2.f","version":"47'19552","reported_seq":19812,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:59.053621+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:34:59.053621+0000","last_peered":"2024-09-17T15:34:59.053621+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:34:59.053621+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'9500","ondisk_log_start":"43'9500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4489216,"num_objects":1096,"num_object_clones":0,"num_object_copies":2192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19552,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2888}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19893","reported_seq":20006,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:03.314686+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:03.314686+0000","last_peered":"2024-09-17T15:35:03.314686+0000","last_clean":"2024-09-17T15:35:03.314686+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:03.314686+0000","last_undegraded":"2024-09-17T15:35:03.314686+0000","last_fullsized":"2024-09-17T15:35:03.314686+0000","mapping_epoch":33,"log_start":"43'9800","ondisk_log_start":"43'9800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4256710,"num_objects":1041,"num_object_clones":0,"num_object_copies":2082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1041,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18250,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19906","reported_seq":20129,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:03.099102+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:03.099102+0000","last_peered":"2024-09-17T15:35:03.099102+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:03.099102+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'9900","ondisk_log_start":"43'9900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4206616,"num_objects":1028,"num_object_clones":0,"num_object_copies":2056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1028,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18063,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3097}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20013","reported_seq":20093,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:03.321159+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:03.321159+0000","last_peered":"2024-09-17T15:35:03.321159+0000","last_clean":"2024-09-17T15:35:03.321159+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:03.321159+0000","last_undegraded":"2024-09-17T15:35:03.321159+0000","last_fullsized":"2024-09-17T15:35:03.321159+0000","mapping_epoch":33,"log_start":"43'10000","ondisk_log_start":"43'10000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3919872,"num_objects":957,"num_object_clones":0,"num_object_copies":1914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20013,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19527","reported_seq":19667,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:59.080869+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:34:59.080869+0000","last_peered":"2024-09-17T15:34:59.080869+0000","last_clean":"2024-09-17T15:34:59.080869+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:34:59.080869+0000","last_undegraded":"2024-09-17T15:34:59.080869+0000","last_fullsized":"2024-09-17T15:34:59.080869+0000","mapping_epoch":32,"log_start":"43'9500","ondisk_log_start":"43'9500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4612579,"num_objects":1127,"num_object_clones":0,"num_object_copies":2254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1127,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16509,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'20136","reported_seq":20434,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:58.900769+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:34:58.900769+0000","last_peered":"2024-09-17T15:34:58.900769+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:34:58.900769+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'10100","ondisk_log_start":"43'10100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4567523,"num_objects":1116,"num_object_clones":0,"num_object_copies":2232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1116,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16635,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3076},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19677","reported_seq":19996,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:59.012535+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:34:59.012535+0000","last_peered":"2024-09-17T15:34:59.012535+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:34:59.012535+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'9600","ondisk_log_start":"43'9600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4469702,"num_objects":1093,"num_object_clones":0,"num_object_copies":2186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1093,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16446,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2996},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19827","reported_seq":19965,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:34:59.033409+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:34:59.033409+0000","last_peered":"2024-09-17T15:34:59.033409+0000","last_clean":"2024-09-17T15:34:59.033409+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:34:59.033409+0000","last_undegraded":"2024-09-17T15:34:59.033409+0000","last_fullsized":"2024-09-17T15:34:59.033409+0000","mapping_epoch":32,"log_start":"43'9800","ondisk_log_start":"43'9800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4460544,"num_objects":1089,"num_object_clones":0,"num_object_copies":2178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16781,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.224142+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.224142+0000","last_peered":"2024-09-17T15:33:11.224142+0000","last_clean":"2024-09-17T15:33:11.224142+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.224142+0000","last_undegraded":"2024-09-17T15:33:11.224142+0000","last_fullsized":"2024-09-17T15:33:11.224142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19835","reported_seq":28127,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:03.618530+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:03.618530+0000","last_peered":"2024-09-17T15:35:03.618530+0000","last_clean":"2024-09-17T15:35:03.618530+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:03.618530+0000","last_undegraded":"2024-09-17T15:35:03.618530+0000","last_fullsized":"2024-09-17T15:35:03.618530+0000","mapping_epoch":42,"log_start":"43'9800","ondisk_log_start":"43'9800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4022755,"num_objects":983,"num_object_clones":0,"num_object_copies":1966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":983,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17835,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19428","reported_seq":19703,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:59.021807+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:34:59.021807+0000","last_peered":"2024-09-17T15:34:59.021807+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:34:59.021807+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'9400","ondisk_log_start":"43'9400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4562944,"num_objects":1114,"num_object_clones":0,"num_object_copies":2228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16024,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2829}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19548","reported_seq":19767,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:03.287409+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:35:03.287409+0000","last_peered":"2024-09-17T15:35:03.287409+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:35:03.287409+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'9500","ondisk_log_start":"43'9500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3977699,"num_objects":972,"num_object_clones":0,"num_object_copies":1944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":972,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17443,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3061}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":88,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223926+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223926+0000","last_peered":"2024-09-17T15:33:11.223926+0000","last_clean":"2024-09-17T15:33:11.223926+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223926+0000","last_undegraded":"2024-09-17T15:33:11.223926+0000","last_fullsized":"2024-09-17T15:33:11.223926+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19754","reported_seq":28168,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:03.239438+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:03.239438+0000","last_peered":"2024-09-17T15:35:03.239438+0000","last_clean":"2024-09-17T15:35:03.239438+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:03.239438+0000","last_undegraded":"2024-09-17T15:35:03.239438+0000","last_fullsized":"2024-09-17T15:35:03.239438+0000","mapping_epoch":46,"log_start":"43'9700","ondisk_log_start":"43'9700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4022272,"num_objects":982,"num_object_clones":0,"num_object_copies":1964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17516,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19848","reported_seq":20156,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:59.052083+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:34:59.052083+0000","last_peered":"2024-09-17T15:34:59.052083+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:34:59.052083+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'9800","ondisk_log_start":"43'9800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4579328,"num_objects":1118,"num_object_clones":0,"num_object_copies":2236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16396,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223809+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223809+0000","last_peered":"2024-09-17T15:33:11.223809+0000","last_clean":"2024-09-17T15:33:11.223809+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223809+0000","last_undegraded":"2024-09-17T15:33:11.223809+0000","last_fullsized":"2024-09-17T15:33:11.223809+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19703","reported_seq":24832,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:59.084377+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:34:59.084377+0000","last_peered":"2024-09-17T15:34:59.084377+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:34:59.084377+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"43'9700","ondisk_log_start":"43'9700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4378624,"num_objects":1069,"num_object_clones":0,"num_object_copies":2138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1671,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15990,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2489,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-602}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19783","reported_seq":20090,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:34:59.041818+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:34:59.041818+0000","last_peered":"2024-09-17T15:34:59.041818+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:34:59.041818+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'9700","ondisk_log_start":"43'9700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4517888,"num_objects":1103,"num_object_clones":0,"num_object_copies":2206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16570,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223917+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223917+0000","last_peered":"2024-09-17T15:33:11.223917+0000","last_clean":"2024-09-17T15:33:11.223917+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223917+0000","last_undegraded":"2024-09-17T15:33:11.223917+0000","last_fullsized":"2024-09-17T15:33:11.223917+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223714+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223714+0000","last_peered":"2024-09-17T15:33:11.223714+0000","last_clean":"2024-09-17T15:33:11.223714+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223714+0000","last_undegraded":"2024-09-17T15:33:11.223714+0000","last_fullsized":"2024-09-17T15:33:11.223714+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19634","reported_seq":28018,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:03.656444+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:03.656444+0000","last_peered":"2024-09-17T15:35:03.656444+0000","last_clean":"2024-09-17T15:35:03.656444+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:03.656444+0000","last_undegraded":"2024-09-17T15:35:03.656444+0000","last_fullsized":"2024-09-17T15:35:03.656444+0000","mapping_epoch":44,"log_start":"43'9600","ondisk_log_start":"43'9600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4088774,"num_objects":1000,"num_object_clones":0,"num_object_copies":2000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1000,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16076,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":69133046,"num_objects":16888,"num_object_clones":0,"num_object_copies":33776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34082,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16888,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":276099,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15378,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":337276928,"data_stored":337142329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051704,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":520948,"kb_used_data":164388,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93850892,"statfs":{"total":96636764160,"available":96103313408,"internally_reserved":0,"allocated":168333312,"data_stored":168031714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":365096620},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724649,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":499484,"kb_used_data":48508,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93872356,"statfs":{"total":96636764160,"available":96125292544,"internally_reserved":0,"allocated":49672192,"data_stored":49348452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":461762219},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247931,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562876,"kb_used_data":120300,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93808964,"statfs":{"total":96636764160,"available":96060379136,"internally_reserved":0,"allocated":123187200,"data_stored":122885597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":453177000},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,4,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":121823232,"data_stored":121787102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":47820800,"data_stored":47762533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":167632896,"data_stored":167592694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:35:05.263 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:35:05.810 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:35:05.810 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:35:05.834 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":361,"stamp":"2024-09-17T15:35:05.798276+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":63542025,"num_objects":15525,"num_object_clones":0,"num_object_copies":31050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33943,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15525,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":277466,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15518,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160832,"ondisk_log_size":160832,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1584400,"kb_used_data":325960,"kb_used_omap":13,"kb_used_meta":1258354,"kb_avail":281531120,"statfs":{"total":289910292480,"available":288287866880,"internally_reserved":0,"allocated":333783040,"data_stored":332856099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1288555519},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,7,3,4,3,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25796608,"num_objects":-6298,"num_object_clones":0,"num_object_copies":-12596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-404,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6298,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":404,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004687"},"pg_stats":[{"pgid":"2.f","version":"47'19696","reported_seq":19956,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:04.109736+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:04.109736+0000","last_peered":"2024-09-17T15:35:04.109736+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:04.109736+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'9600","ondisk_log_start":"43'9600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3899392,"num_objects":952,"num_object_clones":0,"num_object_copies":1904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19696,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3032}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19893","reported_seq":20006,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:03.314686+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:03.314686+0000","last_peered":"2024-09-17T15:35:03.314686+0000","last_clean":"2024-09-17T15:35:03.314686+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:03.314686+0000","last_undegraded":"2024-09-17T15:35:03.314686+0000","last_fullsized":"2024-09-17T15:35:03.314686+0000","mapping_epoch":33,"log_start":"43'9800","ondisk_log_start":"43'9800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4256710,"num_objects":1041,"num_object_clones":0,"num_object_copies":2082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1041,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18250,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19906","reported_seq":20129,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:03.099102+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:03.099102+0000","last_peered":"2024-09-17T15:35:03.099102+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:03.099102+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'9900","ondisk_log_start":"43'9900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4206616,"num_objects":1028,"num_object_clones":0,"num_object_copies":2056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1028,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18063,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3097}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20013","reported_seq":20093,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:03.321159+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:03.321159+0000","last_peered":"2024-09-17T15:35:03.321159+0000","last_clean":"2024-09-17T15:35:03.321159+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:03.321159+0000","last_undegraded":"2024-09-17T15:35:03.321159+0000","last_fullsized":"2024-09-17T15:35:03.321159+0000","mapping_epoch":33,"log_start":"43'10000","ondisk_log_start":"43'10000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3919872,"num_objects":957,"num_object_clones":0,"num_object_copies":1914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20013,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19691","reported_seq":19831,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:04.072706+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:04.072706+0000","last_peered":"2024-09-17T15:35:04.072706+0000","last_clean":"2024-09-17T15:35:04.072706+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:04.072706+0000","last_undegraded":"2024-09-17T15:35:04.072706+0000","last_fullsized":"2024-09-17T15:35:04.072706+0000","mapping_epoch":32,"log_start":"43'9600","ondisk_log_start":"43'9600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3940835,"num_objects":963,"num_object_clones":0,"num_object_copies":1926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":963,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16673,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'20273","reported_seq":20574,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:04.049927+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:04.049927+0000","last_peered":"2024-09-17T15:35:04.049927+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:04.049927+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'10200","ondisk_log_start":"43'10200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4006371,"num_objects":979,"num_object_clones":0,"num_object_copies":1958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":979,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16772,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3213},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19846","reported_seq":20165,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:03.982911+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:35:03.982911+0000","last_peered":"2024-09-17T15:35:03.982911+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:35:03.982911+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'9800","ondisk_log_start":"43'9800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3777478,"num_objects":924,"num_object_clones":0,"num_object_copies":1848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":924,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16615,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3165},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19974","reported_seq":20112,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:04.094243+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:04.094243+0000","last_peered":"2024-09-17T15:35:04.094243+0000","last_clean":"2024-09-17T15:35:04.094243+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:04.094243+0000","last_undegraded":"2024-09-17T15:35:04.094243+0000","last_fullsized":"2024-09-17T15:35:04.094243+0000","mapping_epoch":32,"log_start":"43'9900","ondisk_log_start":"43'9900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3858432,"num_objects":942,"num_object_clones":0,"num_object_copies":1884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16928,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.224142+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.224142+0000","last_peered":"2024-09-17T15:33:11.224142+0000","last_clean":"2024-09-17T15:33:11.224142+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.224142+0000","last_undegraded":"2024-09-17T15:33:11.224142+0000","last_fullsized":"2024-09-17T15:33:11.224142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19835","reported_seq":28127,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:03.618530+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:03.618530+0000","last_peered":"2024-09-17T15:35:03.618530+0000","last_clean":"2024-09-17T15:35:03.618530+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:03.618530+0000","last_undegraded":"2024-09-17T15:35:03.618530+0000","last_fullsized":"2024-09-17T15:35:03.618530+0000","mapping_epoch":42,"log_start":"43'9800","ondisk_log_start":"43'9800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4022755,"num_objects":983,"num_object_clones":0,"num_object_copies":1966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":983,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17835,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19580","reported_seq":19853,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:04.113819+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:04.113819+0000","last_peered":"2024-09-17T15:35:04.113819+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:04.113819+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'9500","ondisk_log_start":"43'9500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3940352,"num_objects":962,"num_object_clones":0,"num_object_copies":1924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16176,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2981}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19548","reported_seq":19767,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:03.287409+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:35:03.287409+0000","last_peered":"2024-09-17T15:35:03.287409+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:35:03.287409+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'9500","ondisk_log_start":"43'9500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3977699,"num_objects":972,"num_object_clones":0,"num_object_copies":1944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":972,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17443,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3061}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":88,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223926+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223926+0000","last_peered":"2024-09-17T15:33:11.223926+0000","last_clean":"2024-09-17T15:33:11.223926+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223926+0000","last_undegraded":"2024-09-17T15:33:11.223926+0000","last_fullsized":"2024-09-17T15:33:11.223926+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19754","reported_seq":28168,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:03.239438+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:03.239438+0000","last_peered":"2024-09-17T15:35:03.239438+0000","last_clean":"2024-09-17T15:35:03.239438+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:03.239438+0000","last_undegraded":"2024-09-17T15:35:03.239438+0000","last_fullsized":"2024-09-17T15:35:03.239438+0000","mapping_epoch":46,"log_start":"43'9700","ondisk_log_start":"43'9700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4022272,"num_objects":982,"num_object_clones":0,"num_object_copies":1964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17516,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20004","reported_seq":20312,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:04.108869+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:04.108869+0000","last_peered":"2024-09-17T15:35:04.108869+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:04.108869+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10000","ondisk_log_start":"43'10000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3940352,"num_objects":962,"num_object_clones":0,"num_object_copies":1924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16552,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3095}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223809+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223809+0000","last_peered":"2024-09-17T15:33:11.223809+0000","last_clean":"2024-09-17T15:33:11.223809+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223809+0000","last_undegraded":"2024-09-17T15:33:11.223809+0000","last_fullsized":"2024-09-17T15:33:11.223809+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19849","reported_seq":25258,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:04.136668+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:35:04.136668+0000","last_peered":"2024-09-17T15:35:04.136668+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:35:04.136668+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"43'9800","ondisk_log_start":"43'9800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3780608,"num_objects":923,"num_object_clones":0,"num_object_copies":1846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1532,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16136,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2628,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-609}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19933","reported_seq":20241,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:04.081158+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:04.081158+0000","last_peered":"2024-09-17T15:35:04.081158+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:04.081158+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'9900","ondisk_log_start":"43'9900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3903488,"num_objects":953,"num_object_clones":0,"num_object_copies":1906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16720,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3027}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223917+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223917+0000","last_peered":"2024-09-17T15:33:11.223917+0000","last_clean":"2024-09-17T15:33:11.223917+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223917+0000","last_undegraded":"2024-09-17T15:33:11.223917+0000","last_fullsized":"2024-09-17T15:33:11.223917+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223714+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223714+0000","last_peered":"2024-09-17T15:33:11.223714+0000","last_clean":"2024-09-17T15:33:11.223714+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223714+0000","last_undegraded":"2024-09-17T15:33:11.223714+0000","last_fullsized":"2024-09-17T15:33:11.223714+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19634","reported_seq":28018,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:03.656444+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:03.656444+0000","last_peered":"2024-09-17T15:35:03.656444+0000","last_clean":"2024-09-17T15:35:03.656444+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:03.656444+0000","last_undegraded":"2024-09-17T15:35:03.656444+0000","last_fullsized":"2024-09-17T15:35:03.656444+0000","mapping_epoch":44,"log_start":"43'9600","ondisk_log_start":"43'9600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4088774,"num_objects":1000,"num_object_clones":0,"num_object_copies":2000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1000,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16076,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":63542006,"num_objects":15523,"num_object_clones":0,"num_object_copies":31046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33943,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15523,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":277464,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15517,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":329859072,"data_stored":329724473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160829,"ondisk_log_size":160829,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051704,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":520948,"kb_used_data":164388,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93850892,"statfs":{"total":96636764160,"available":96103313408,"internally_reserved":0,"allocated":168333312,"data_stored":168031714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":365096620},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724649,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":499484,"kb_used_data":48508,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93872356,"statfs":{"total":96636764160,"available":96125292544,"internally_reserved":0,"allocated":49672192,"data_stored":49348452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":461762219},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247932,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":563968,"kb_used_data":113064,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93807872,"statfs":{"total":96636764160,"available":96059260928,"internally_reserved":0,"allocated":115777536,"data_stored":115475933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":461696680},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,1,1,2,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":114405376,"data_stored":114369246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":47820800,"data_stored":47762533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":167632896,"data_stored":167592694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:35:05.836 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:35:05.836 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'19696", 'reported_seq': 19956, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:04.109736+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:35:04.109736+0000', 'last_peered': '2024-09-17T15:35:04.109736+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:35:04.109736+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'9600", 'ondisk_log_start': "43'9600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3899392, 'num_objects': 952, 'num_object_clones': 0, 'num_object_copies': 1904, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 952, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19696, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3032}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:05.836 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:35:05.836 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'19906", 'reported_seq': 20129, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:03.099102+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:35:03.099102+0000', 'last_peered': '2024-09-17T15:35:03.099102+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:35:03.099102+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "43'9900", 'ondisk_log_start': "43'9900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4206616, 'num_objects': 1028, 'num_object_clones': 0, 'num_object_copies': 2056, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1028, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18063, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3097}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:35:05.836 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:35:05.836 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'20273", 'reported_seq': 20574, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:04.049927+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:35:04.049927+0000', 'last_peered': '2024-09-17T15:35:04.049927+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:35:04.049927+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "43'10200", 'ondisk_log_start': "43'10200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4006371, 'num_objects': 979, 'num_object_clones': 0, 'num_object_copies': 1958, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 979, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16772, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3213}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:05.837 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:35:05.837 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'19846", 'reported_seq': 20165, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:03.982911+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:35:03.982911+0000', 'last_peered': '2024-09-17T15:35:03.982911+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:35:03.982911+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "43'9800", 'ondisk_log_start': "43'9800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3777478, 'num_objects': 924, 'num_object_clones': 0, 'num_object_copies': 1848, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4089, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 924, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 16615, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 85, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3165}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:05.837 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:35:05.837 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'19580", 'reported_seq': 19853, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:04.113819+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:35:04.113819+0000', 'last_peered': '2024-09-17T15:35:04.113819+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:35:04.113819+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'9500", 'ondisk_log_start': "43'9500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3940352, 'num_objects': 962, 'num_object_clones': 0, 'num_object_copies': 1924, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 962, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16176, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2981}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:05.837 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:35:05.837 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'19548", 'reported_seq': 19767, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:03.287409+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:35:03.287409+0000', 'last_peered': '2024-09-17T15:35:03.287409+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:35:03.287409+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "43'9500", 'ondisk_log_start': "43'9500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3977699, 'num_objects': 972, 'num_object_clones': 0, 'num_object_copies': 1944, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4034, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 972, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17443, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -3061}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:35:05.837 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:35:05.837 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'20004", 'reported_seq': 20312, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:04.108869+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:35:04.108869+0000', 'last_peered': '2024-09-17T15:35:04.108869+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:35:04.108869+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "43'10000", 'ondisk_log_start': "43'10000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3940352, 'num_objects': 962, 'num_object_clones': 0, 'num_object_copies': 1924, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 962, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16552, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3095}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:05.838 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:35:05.838 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'19849", 'reported_seq': 25258, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:04.136668+0000', 'last_change': '2024-09-17T15:33:09.763500+0000', 'last_active': '2024-09-17T15:35:04.136668+0000', 'last_peered': '2024-09-17T15:35:04.136668+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:35:04.136668+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "43'9800", 'ondisk_log_start': "43'9800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3780608, 'num_objects': 923, 'num_object_clones': 0, 'num_object_copies': 1846, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1532, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 923, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16136, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2628, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -609}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:05.838 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:35:05.838 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'19933", 'reported_seq': 20241, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:04.081158+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:35:04.081158+0000', 'last_peered': '2024-09-17T15:35:04.081158+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:35:04.081158+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'9900", 'ondisk_log_start': "43'9900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3903488, 'num_objects': 953, 'num_object_clones': 0, 'num_object_copies': 1906, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 953, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16720, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3027}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:05.838 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:35:06.548 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:35:06.548 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33943/31050 objects degraded (109.317%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":643,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":15525,"data_bytes":63542025,"bytes_used":1622425600,"bytes_avail":288287866880,"bytes_total":289910292480,"degraded_objects":33943,"degraded_total":31050,"degraded_ratio":1.0931723027375202,"recovering_objects_per_sec":33,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":404,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":524},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:35:06.577 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:35:06.577 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:35:07.171 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:35:07.171 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:35:07.195 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":361,"stamp":"2024-09-17T15:35:05.798276+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":63542025,"num_objects":15525,"num_object_clones":0,"num_object_copies":31050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33943,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15525,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":277466,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15518,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160832,"ondisk_log_size":160832,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1584400,"kb_used_data":325960,"kb_used_omap":13,"kb_used_meta":1258354,"kb_avail":281531120,"statfs":{"total":289910292480,"available":288287866880,"internally_reserved":0,"allocated":333783040,"data_stored":332856099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1288555519},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,7,3,4,3,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25796608,"num_objects":-6298,"num_object_clones":0,"num_object_copies":-12596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-404,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6298,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":404,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004687"},"pg_stats":[{"pgid":"2.f","version":"47'19696","reported_seq":19956,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:04.109736+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:04.109736+0000","last_peered":"2024-09-17T15:35:04.109736+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:04.109736+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'9600","ondisk_log_start":"43'9600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3899392,"num_objects":952,"num_object_clones":0,"num_object_copies":1904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19696,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3032}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19893","reported_seq":20006,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:03.314686+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:03.314686+0000","last_peered":"2024-09-17T15:35:03.314686+0000","last_clean":"2024-09-17T15:35:03.314686+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:03.314686+0000","last_undegraded":"2024-09-17T15:35:03.314686+0000","last_fullsized":"2024-09-17T15:35:03.314686+0000","mapping_epoch":33,"log_start":"43'9800","ondisk_log_start":"43'9800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4256710,"num_objects":1041,"num_object_clones":0,"num_object_copies":2082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1041,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18250,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19906","reported_seq":20129,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:03.099102+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:03.099102+0000","last_peered":"2024-09-17T15:35:03.099102+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:03.099102+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'9900","ondisk_log_start":"43'9900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4206616,"num_objects":1028,"num_object_clones":0,"num_object_copies":2056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1028,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18063,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3097}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20013","reported_seq":20093,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:03.321159+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:03.321159+0000","last_peered":"2024-09-17T15:35:03.321159+0000","last_clean":"2024-09-17T15:35:03.321159+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:03.321159+0000","last_undegraded":"2024-09-17T15:35:03.321159+0000","last_fullsized":"2024-09-17T15:35:03.321159+0000","mapping_epoch":33,"log_start":"43'10000","ondisk_log_start":"43'10000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3919872,"num_objects":957,"num_object_clones":0,"num_object_copies":1914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20013,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19691","reported_seq":19831,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:04.072706+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:04.072706+0000","last_peered":"2024-09-17T15:35:04.072706+0000","last_clean":"2024-09-17T15:35:04.072706+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:04.072706+0000","last_undegraded":"2024-09-17T15:35:04.072706+0000","last_fullsized":"2024-09-17T15:35:04.072706+0000","mapping_epoch":32,"log_start":"43'9600","ondisk_log_start":"43'9600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3940835,"num_objects":963,"num_object_clones":0,"num_object_copies":1926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":963,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16673,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'20273","reported_seq":20574,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:04.049927+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:04.049927+0000","last_peered":"2024-09-17T15:35:04.049927+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:04.049927+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'10200","ondisk_log_start":"43'10200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4006371,"num_objects":979,"num_object_clones":0,"num_object_copies":1958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":979,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16772,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3213},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19846","reported_seq":20165,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:03.982911+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:35:03.982911+0000","last_peered":"2024-09-17T15:35:03.982911+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:35:03.982911+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'9800","ondisk_log_start":"43'9800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3777478,"num_objects":924,"num_object_clones":0,"num_object_copies":1848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":924,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16615,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3165},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19974","reported_seq":20112,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:04.094243+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:04.094243+0000","last_peered":"2024-09-17T15:35:04.094243+0000","last_clean":"2024-09-17T15:35:04.094243+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:04.094243+0000","last_undegraded":"2024-09-17T15:35:04.094243+0000","last_fullsized":"2024-09-17T15:35:04.094243+0000","mapping_epoch":32,"log_start":"43'9900","ondisk_log_start":"43'9900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3858432,"num_objects":942,"num_object_clones":0,"num_object_copies":1884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16928,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.224142+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.224142+0000","last_peered":"2024-09-17T15:33:11.224142+0000","last_clean":"2024-09-17T15:33:11.224142+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.224142+0000","last_undegraded":"2024-09-17T15:33:11.224142+0000","last_fullsized":"2024-09-17T15:33:11.224142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19835","reported_seq":28127,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:03.618530+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:03.618530+0000","last_peered":"2024-09-17T15:35:03.618530+0000","last_clean":"2024-09-17T15:35:03.618530+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:03.618530+0000","last_undegraded":"2024-09-17T15:35:03.618530+0000","last_fullsized":"2024-09-17T15:35:03.618530+0000","mapping_epoch":42,"log_start":"43'9800","ondisk_log_start":"43'9800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4022755,"num_objects":983,"num_object_clones":0,"num_object_copies":1966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":983,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17835,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19580","reported_seq":19853,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:04.113819+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:04.113819+0000","last_peered":"2024-09-17T15:35:04.113819+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:04.113819+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'9500","ondisk_log_start":"43'9500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3940352,"num_objects":962,"num_object_clones":0,"num_object_copies":1924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16176,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2981}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19548","reported_seq":19767,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:03.287409+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:35:03.287409+0000","last_peered":"2024-09-17T15:35:03.287409+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:35:03.287409+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'9500","ondisk_log_start":"43'9500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3977699,"num_objects":972,"num_object_clones":0,"num_object_copies":1944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":972,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17443,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3061}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":88,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223926+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223926+0000","last_peered":"2024-09-17T15:33:11.223926+0000","last_clean":"2024-09-17T15:33:11.223926+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223926+0000","last_undegraded":"2024-09-17T15:33:11.223926+0000","last_fullsized":"2024-09-17T15:33:11.223926+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19754","reported_seq":28168,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:03.239438+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:03.239438+0000","last_peered":"2024-09-17T15:35:03.239438+0000","last_clean":"2024-09-17T15:35:03.239438+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:03.239438+0000","last_undegraded":"2024-09-17T15:35:03.239438+0000","last_fullsized":"2024-09-17T15:35:03.239438+0000","mapping_epoch":46,"log_start":"43'9700","ondisk_log_start":"43'9700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4022272,"num_objects":982,"num_object_clones":0,"num_object_copies":1964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17516,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20004","reported_seq":20312,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:04.108869+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:04.108869+0000","last_peered":"2024-09-17T15:35:04.108869+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:04.108869+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10000","ondisk_log_start":"43'10000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3940352,"num_objects":962,"num_object_clones":0,"num_object_copies":1924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16552,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3095}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223809+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223809+0000","last_peered":"2024-09-17T15:33:11.223809+0000","last_clean":"2024-09-17T15:33:11.223809+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223809+0000","last_undegraded":"2024-09-17T15:33:11.223809+0000","last_fullsized":"2024-09-17T15:33:11.223809+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19849","reported_seq":25258,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:04.136668+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:35:04.136668+0000","last_peered":"2024-09-17T15:35:04.136668+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:35:04.136668+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"43'9800","ondisk_log_start":"43'9800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3780608,"num_objects":923,"num_object_clones":0,"num_object_copies":1846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1532,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16136,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2628,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-609}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19933","reported_seq":20241,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:04.081158+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:04.081158+0000","last_peered":"2024-09-17T15:35:04.081158+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:04.081158+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'9900","ondisk_log_start":"43'9900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3903488,"num_objects":953,"num_object_clones":0,"num_object_copies":1906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16720,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3027}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223917+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223917+0000","last_peered":"2024-09-17T15:33:11.223917+0000","last_clean":"2024-09-17T15:33:11.223917+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223917+0000","last_undegraded":"2024-09-17T15:33:11.223917+0000","last_fullsized":"2024-09-17T15:33:11.223917+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223714+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223714+0000","last_peered":"2024-09-17T15:33:11.223714+0000","last_clean":"2024-09-17T15:33:11.223714+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223714+0000","last_undegraded":"2024-09-17T15:33:11.223714+0000","last_fullsized":"2024-09-17T15:33:11.223714+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19634","reported_seq":28018,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:03.656444+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:03.656444+0000","last_peered":"2024-09-17T15:35:03.656444+0000","last_clean":"2024-09-17T15:35:03.656444+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:03.656444+0000","last_undegraded":"2024-09-17T15:35:03.656444+0000","last_fullsized":"2024-09-17T15:35:03.656444+0000","mapping_epoch":44,"log_start":"43'9600","ondisk_log_start":"43'9600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4088774,"num_objects":1000,"num_object_clones":0,"num_object_copies":2000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1000,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16076,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":63542006,"num_objects":15523,"num_object_clones":0,"num_object_copies":31046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33943,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15523,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":277464,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15517,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":329859072,"data_stored":329724473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160829,"ondisk_log_size":160829,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051704,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":520948,"kb_used_data":164388,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93850892,"statfs":{"total":96636764160,"available":96103313408,"internally_reserved":0,"allocated":168333312,"data_stored":168031714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":365096620},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724649,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":499484,"kb_used_data":48508,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93872356,"statfs":{"total":96636764160,"available":96125292544,"internally_reserved":0,"allocated":49672192,"data_stored":49348452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":461762219},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247932,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":563968,"kb_used_data":113064,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93807872,"statfs":{"total":96636764160,"available":96059260928,"internally_reserved":0,"allocated":115777536,"data_stored":115475933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":461696680},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,1,1,2,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":114405376,"data_stored":114369246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":47820800,"data_stored":47762533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":167632896,"data_stored":167592694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:35:10.197 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:35:10.789 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:35:10.792 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:35:10.817 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2024-09-17T15:35:09.799938+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":53515017,"num_objects":13077,"num_object_clones":0,"num_object_copies":26154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33805,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13077,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":279914,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15656,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160980,"ondisk_log_size":160980,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1593956,"kb_used_data":305012,"kb_used_omap":13,"kb_used_meta":1288882,"kb_avail":281521564,"statfs":{"total":289910292480,"available":288278081536,"internally_reserved":0,"allocated":312332288,"data_stored":311405347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1319816191},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,5,6,12,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-30126080,"num_objects":-7355,"num_object_clones":0,"num_object_copies":-14710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-415,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7355,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7355,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":415,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004717"},"pg_stats":[{"pgid":"2.f","version":"47'19844","reported_seq":20104,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:09.157550+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:09.157550+0000","last_peered":"2024-09-17T15:35:09.157550+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:09.157550+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'9800","ondisk_log_start":"43'9800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3293184,"num_objects":804,"num_object_clones":0,"num_object_copies":1608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19844,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3180}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20057","reported_seq":20170,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:08.306757+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:08.306757+0000","last_peered":"2024-09-17T15:35:08.306757+0000","last_clean":"2024-09-17T15:35:08.306757+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:08.306757+0000","last_undegraded":"2024-09-17T15:35:08.306757+0000","last_fullsized":"2024-09-17T15:35:08.306757+0000","mapping_epoch":33,"log_start":"43'10000","ondisk_log_start":"43'10000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3584966,"num_objects":877,"num_object_clones":0,"num_object_copies":1754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":877,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18414,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20068","reported_seq":20291,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:08.324082+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:08.324082+0000","last_peered":"2024-09-17T15:35:08.324082+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:08.324082+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10000","ondisk_log_start":"43'10000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3543064,"num_objects":866,"num_object_clones":0,"num_object_copies":1732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":866,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18225,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3259}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20168","reported_seq":20248,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:08.337668+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:08.337668+0000","last_peered":"2024-09-17T15:35:08.337668+0000","last_clean":"2024-09-17T15:35:08.337668+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:08.337668+0000","last_undegraded":"2024-09-17T15:35:08.337668+0000","last_fullsized":"2024-09-17T15:35:08.337668+0000","mapping_epoch":33,"log_start":"43'10100","ondisk_log_start":"43'10100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3284992,"num_objects":802,"num_object_clones":0,"num_object_copies":1604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20168,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19856","reported_seq":19996,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:09.132844+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:09.132844+0000","last_peered":"2024-09-17T15:35:09.132844+0000","last_clean":"2024-09-17T15:35:09.132844+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:09.132844+0000","last_undegraded":"2024-09-17T15:35:09.132844+0000","last_fullsized":"2024-09-17T15:35:09.132844+0000","mapping_epoch":32,"log_start":"43'9800","ondisk_log_start":"43'9800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3264995,"num_objects":798,"num_object_clones":0,"num_object_copies":1596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":798,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16838,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'20419","reported_seq":20720,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:09.129329+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:09.129329+0000","last_peered":"2024-09-17T15:35:09.129329+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:09.129329+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'10400","ondisk_log_start":"43'10400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3408355,"num_objects":833,"num_object_clones":0,"num_object_copies":1666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":833,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16918,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3359},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19993","reported_seq":20312,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:09.146609+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:35:09.146609+0000","last_peered":"2024-09-17T15:35:09.146609+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:35:09.146609+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'9900","ondisk_log_start":"43'9900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3175366,"num_objects":777,"num_object_clones":0,"num_object_copies":1554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":777,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16762,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3312},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20118","reported_seq":20256,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:09.189968+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:09.189968+0000","last_peered":"2024-09-17T15:35:09.189968+0000","last_clean":"2024-09-17T15:35:09.189968+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:09.189968+0000","last_undegraded":"2024-09-17T15:35:09.189968+0000","last_fullsized":"2024-09-17T15:35:09.189968+0000","mapping_epoch":32,"log_start":"43'10100","ondisk_log_start":"43'10100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3268608,"num_objects":798,"num_object_clones":0,"num_object_copies":1596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17072,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.224142+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.224142+0000","last_peered":"2024-09-17T15:33:11.224142+0000","last_clean":"2024-09-17T15:33:11.224142+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.224142+0000","last_undegraded":"2024-09-17T15:33:11.224142+0000","last_fullsized":"2024-09-17T15:33:11.224142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19988","reported_seq":28281,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:08.708773+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:08.708773+0000","last_peered":"2024-09-17T15:35:08.708773+0000","last_clean":"2024-09-17T15:35:08.708773+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:08.708773+0000","last_undegraded":"2024-09-17T15:35:08.708773+0000","last_fullsized":"2024-09-17T15:35:08.708773+0000","mapping_epoch":42,"log_start":"43'9900","ondisk_log_start":"43'9900","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3396067,"num_objects":830,"num_object_clones":0,"num_object_copies":1660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":830,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17988,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19740","reported_seq":20014,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:09.158282+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:09.158282+0000","last_peered":"2024-09-17T15:35:09.158282+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:09.158282+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'9700","ondisk_log_start":"43'9700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3284992,"num_objects":802,"num_object_clones":0,"num_object_copies":1604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16336,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3141}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19695","reported_seq":19914,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:08.196699+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:35:08.196699+0000","last_peered":"2024-09-17T15:35:08.196699+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:35:08.196699+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'9600","ondisk_log_start":"43'9600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3375587,"num_objects":825,"num_object_clones":0,"num_object_copies":1650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":825,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17590,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3208}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":88,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223926+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223926+0000","last_peered":"2024-09-17T15:33:11.223926+0000","last_clean":"2024-09-17T15:33:11.223926+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223926+0000","last_undegraded":"2024-09-17T15:33:11.223926+0000","last_fullsized":"2024-09-17T15:33:11.223926+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19907","reported_seq":28322,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:08.323967+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:08.323967+0000","last_peered":"2024-09-17T15:35:08.323967+0000","last_clean":"2024-09-17T15:35:08.323967+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:08.323967+0000","last_undegraded":"2024-09-17T15:35:08.323967+0000","last_fullsized":"2024-09-17T15:35:08.323967+0000","mapping_epoch":46,"log_start":"43'9900","ondisk_log_start":"43'9900","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3395584,"num_objects":829,"num_object_clones":0,"num_object_copies":1658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17669,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20142","reported_seq":20450,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:09.181160+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:09.181160+0000","last_peered":"2024-09-17T15:35:09.181160+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:09.181160+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10100","ondisk_log_start":"43'10100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3375104,"num_objects":824,"num_object_clones":0,"num_object_copies":1648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16690,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3233}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223809+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223809+0000","last_peered":"2024-09-17T15:33:11.223809+0000","last_clean":"2024-09-17T15:33:11.223809+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223809+0000","last_undegraded":"2024-09-17T15:33:11.223809+0000","last_fullsized":"2024-09-17T15:33:11.223809+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20000","reported_seq":25684,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:09.189711+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:35:09.189711+0000","last_peered":"2024-09-17T15:35:09.189711+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:35:09.189711+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"43'9900","ondisk_log_start":"43'9900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3162112,"num_objects":772,"num_object_clones":0,"num_object_copies":1544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1394,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16287,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2766,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-622}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20087","reported_seq":20395,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:09.178560+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:09.178560+0000","last_peered":"2024-09-17T15:35:09.178560+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:09.178560+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10000","ondisk_log_start":"43'10000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3272704,"num_objects":799,"num_object_clones":0,"num_object_copies":1598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":799,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16874,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3181}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223917+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223917+0000","last_peered":"2024-09-17T15:33:11.223917+0000","last_clean":"2024-09-17T15:33:11.223917+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223917+0000","last_undegraded":"2024-09-17T15:33:11.223917+0000","last_fullsized":"2024-09-17T15:33:11.223917+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223714+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223714+0000","last_peered":"2024-09-17T15:33:11.223714+0000","last_clean":"2024-09-17T15:33:11.223714+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223714+0000","last_undegraded":"2024-09-17T15:33:11.223714+0000","last_fullsized":"2024-09-17T15:33:11.223714+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19795","reported_seq":28179,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:08.690719+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:08.690719+0000","last_peered":"2024-09-17T15:35:08.690719+0000","last_clean":"2024-09-17T15:35:08.690719+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:08.690719+0000","last_undegraded":"2024-09-17T15:35:08.690719+0000","last_fullsized":"2024-09-17T15:35:08.690719+0000","mapping_epoch":44,"log_start":"43'9700","ondisk_log_start":"43'9700","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3429318,"num_objects":839,"num_object_clones":0,"num_object_copies":1678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16237,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":53514998,"num_objects":13075,"num_object_clones":0,"num_object_copies":26150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33805,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13075,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":279912,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15655,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":309760000,"data_stored":309625401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160977,"ondisk_log_size":160977,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051705,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":520156,"kb_used_data":159436,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93851684,"statfs":{"total":96636764160,"available":96104124416,"internally_reserved":0,"allocated":163262464,"data_stored":162960866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":369356460},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,7],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724650,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":513048,"kb_used_data":39880,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93858792,"statfs":{"total":96636764160,"available":96111403008,"internally_reserved":0,"allocated":40837120,"data_stored":40513380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":484503211},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,2,3,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247933,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":560752,"kb_used_data":105696,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93811088,"statfs":{"total":96636764160,"available":96062554112,"internally_reserved":0,"allocated":108232704,"data_stored":107931101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":465956520},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,3,2,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":106876928,"data_stored":106840798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40361984,"data_stored":40303717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":162521088,"data_stored":162480886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:35:10.818 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:35:11.384 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:35:11.387 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:35:11.415 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2024-09-17T15:35:09.799938+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":53515017,"num_objects":13077,"num_object_clones":0,"num_object_copies":26154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33805,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13077,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":279914,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15656,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160980,"ondisk_log_size":160980,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1593956,"kb_used_data":305012,"kb_used_omap":13,"kb_used_meta":1288882,"kb_avail":281521564,"statfs":{"total":289910292480,"available":288278081536,"internally_reserved":0,"allocated":312332288,"data_stored":311405347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1319816191},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,5,6,12,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-30126080,"num_objects":-7355,"num_object_clones":0,"num_object_copies":-14710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-415,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7355,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7355,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":415,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004717"},"pg_stats":[{"pgid":"2.f","version":"47'19844","reported_seq":20104,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:09.157550+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:09.157550+0000","last_peered":"2024-09-17T15:35:09.157550+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:09.157550+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'9800","ondisk_log_start":"43'9800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3293184,"num_objects":804,"num_object_clones":0,"num_object_copies":1608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19844,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3180}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20057","reported_seq":20170,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:08.306757+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:08.306757+0000","last_peered":"2024-09-17T15:35:08.306757+0000","last_clean":"2024-09-17T15:35:08.306757+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:08.306757+0000","last_undegraded":"2024-09-17T15:35:08.306757+0000","last_fullsized":"2024-09-17T15:35:08.306757+0000","mapping_epoch":33,"log_start":"43'10000","ondisk_log_start":"43'10000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3584966,"num_objects":877,"num_object_clones":0,"num_object_copies":1754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":877,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18414,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20068","reported_seq":20291,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:08.324082+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:08.324082+0000","last_peered":"2024-09-17T15:35:08.324082+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:08.324082+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10000","ondisk_log_start":"43'10000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3543064,"num_objects":866,"num_object_clones":0,"num_object_copies":1732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":866,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18225,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3259}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20168","reported_seq":20248,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:08.337668+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:08.337668+0000","last_peered":"2024-09-17T15:35:08.337668+0000","last_clean":"2024-09-17T15:35:08.337668+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:08.337668+0000","last_undegraded":"2024-09-17T15:35:08.337668+0000","last_fullsized":"2024-09-17T15:35:08.337668+0000","mapping_epoch":33,"log_start":"43'10100","ondisk_log_start":"43'10100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3284992,"num_objects":802,"num_object_clones":0,"num_object_copies":1604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20168,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19856","reported_seq":19996,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:09.132844+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:09.132844+0000","last_peered":"2024-09-17T15:35:09.132844+0000","last_clean":"2024-09-17T15:35:09.132844+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:09.132844+0000","last_undegraded":"2024-09-17T15:35:09.132844+0000","last_fullsized":"2024-09-17T15:35:09.132844+0000","mapping_epoch":32,"log_start":"43'9800","ondisk_log_start":"43'9800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3264995,"num_objects":798,"num_object_clones":0,"num_object_copies":1596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":798,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16838,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'20419","reported_seq":20720,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:09.129329+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:09.129329+0000","last_peered":"2024-09-17T15:35:09.129329+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:09.129329+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'10400","ondisk_log_start":"43'10400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3408355,"num_objects":833,"num_object_clones":0,"num_object_copies":1666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":833,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16918,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3359},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19993","reported_seq":20312,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:09.146609+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:35:09.146609+0000","last_peered":"2024-09-17T15:35:09.146609+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:35:09.146609+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'9900","ondisk_log_start":"43'9900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3175366,"num_objects":777,"num_object_clones":0,"num_object_copies":1554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":777,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16762,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3312},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20118","reported_seq":20256,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:09.189968+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:09.189968+0000","last_peered":"2024-09-17T15:35:09.189968+0000","last_clean":"2024-09-17T15:35:09.189968+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:09.189968+0000","last_undegraded":"2024-09-17T15:35:09.189968+0000","last_fullsized":"2024-09-17T15:35:09.189968+0000","mapping_epoch":32,"log_start":"43'10100","ondisk_log_start":"43'10100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3268608,"num_objects":798,"num_object_clones":0,"num_object_copies":1596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17072,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.224142+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.224142+0000","last_peered":"2024-09-17T15:33:11.224142+0000","last_clean":"2024-09-17T15:33:11.224142+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.224142+0000","last_undegraded":"2024-09-17T15:33:11.224142+0000","last_fullsized":"2024-09-17T15:33:11.224142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19988","reported_seq":28281,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:08.708773+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:08.708773+0000","last_peered":"2024-09-17T15:35:08.708773+0000","last_clean":"2024-09-17T15:35:08.708773+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:08.708773+0000","last_undegraded":"2024-09-17T15:35:08.708773+0000","last_fullsized":"2024-09-17T15:35:08.708773+0000","mapping_epoch":42,"log_start":"43'9900","ondisk_log_start":"43'9900","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3396067,"num_objects":830,"num_object_clones":0,"num_object_copies":1660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":830,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17988,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19740","reported_seq":20014,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:09.158282+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:09.158282+0000","last_peered":"2024-09-17T15:35:09.158282+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:09.158282+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'9700","ondisk_log_start":"43'9700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3284992,"num_objects":802,"num_object_clones":0,"num_object_copies":1604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16336,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3141}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19695","reported_seq":19914,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:08.196699+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:35:08.196699+0000","last_peered":"2024-09-17T15:35:08.196699+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:35:08.196699+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'9600","ondisk_log_start":"43'9600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3375587,"num_objects":825,"num_object_clones":0,"num_object_copies":1650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":825,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17590,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3208}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":88,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223926+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223926+0000","last_peered":"2024-09-17T15:33:11.223926+0000","last_clean":"2024-09-17T15:33:11.223926+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223926+0000","last_undegraded":"2024-09-17T15:33:11.223926+0000","last_fullsized":"2024-09-17T15:33:11.223926+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19907","reported_seq":28322,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:08.323967+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:08.323967+0000","last_peered":"2024-09-17T15:35:08.323967+0000","last_clean":"2024-09-17T15:35:08.323967+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:08.323967+0000","last_undegraded":"2024-09-17T15:35:08.323967+0000","last_fullsized":"2024-09-17T15:35:08.323967+0000","mapping_epoch":46,"log_start":"43'9900","ondisk_log_start":"43'9900","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3395584,"num_objects":829,"num_object_clones":0,"num_object_copies":1658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17669,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20142","reported_seq":20450,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:09.181160+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:09.181160+0000","last_peered":"2024-09-17T15:35:09.181160+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:09.181160+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10100","ondisk_log_start":"43'10100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3375104,"num_objects":824,"num_object_clones":0,"num_object_copies":1648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16690,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3233}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223809+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223809+0000","last_peered":"2024-09-17T15:33:11.223809+0000","last_clean":"2024-09-17T15:33:11.223809+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223809+0000","last_undegraded":"2024-09-17T15:33:11.223809+0000","last_fullsized":"2024-09-17T15:33:11.223809+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20000","reported_seq":25684,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:09.189711+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:35:09.189711+0000","last_peered":"2024-09-17T15:35:09.189711+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:35:09.189711+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"43'9900","ondisk_log_start":"43'9900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3162112,"num_objects":772,"num_object_clones":0,"num_object_copies":1544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1394,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16287,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2766,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-622}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20087","reported_seq":20395,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:09.178560+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:09.178560+0000","last_peered":"2024-09-17T15:35:09.178560+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:09.178560+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10000","ondisk_log_start":"43'10000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3272704,"num_objects":799,"num_object_clones":0,"num_object_copies":1598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":799,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16874,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3181}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223917+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223917+0000","last_peered":"2024-09-17T15:33:11.223917+0000","last_clean":"2024-09-17T15:33:11.223917+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223917+0000","last_undegraded":"2024-09-17T15:33:11.223917+0000","last_fullsized":"2024-09-17T15:33:11.223917+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223714+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223714+0000","last_peered":"2024-09-17T15:33:11.223714+0000","last_clean":"2024-09-17T15:33:11.223714+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223714+0000","last_undegraded":"2024-09-17T15:33:11.223714+0000","last_fullsized":"2024-09-17T15:33:11.223714+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19795","reported_seq":28179,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:08.690719+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:08.690719+0000","last_peered":"2024-09-17T15:35:08.690719+0000","last_clean":"2024-09-17T15:35:08.690719+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:08.690719+0000","last_undegraded":"2024-09-17T15:35:08.690719+0000","last_fullsized":"2024-09-17T15:35:08.690719+0000","mapping_epoch":44,"log_start":"43'9700","ondisk_log_start":"43'9700","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3429318,"num_objects":839,"num_object_clones":0,"num_object_copies":1678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16237,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":53514998,"num_objects":13075,"num_object_clones":0,"num_object_copies":26150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33805,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13075,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":279912,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15655,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":309760000,"data_stored":309625401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160977,"ondisk_log_size":160977,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051705,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":520156,"kb_used_data":159436,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93851684,"statfs":{"total":96636764160,"available":96104124416,"internally_reserved":0,"allocated":163262464,"data_stored":162960866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":369356460},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,7],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724650,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":513048,"kb_used_data":39880,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93858792,"statfs":{"total":96636764160,"available":96111403008,"internally_reserved":0,"allocated":40837120,"data_stored":40513380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":484503211},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,2,3,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247933,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":560752,"kb_used_data":105696,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93811088,"statfs":{"total":96636764160,"available":96062554112,"internally_reserved":0,"allocated":108232704,"data_stored":107931101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":465956520},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,3,2,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":106876928,"data_stored":106840798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40361984,"data_stored":40303717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":162521088,"data_stored":162480886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:35:11.416 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:35:11.416 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'19844", 'reported_seq': 20104, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:09.157550+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:35:09.157550+0000', 'last_peered': '2024-09-17T15:35:09.157550+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:35:09.157550+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'9800", 'ondisk_log_start': "43'9800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3293184, 'num_objects': 804, 'num_object_clones': 0, 'num_object_copies': 1608, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 804, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19844, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3180}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:11.417 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:35:11.417 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'20068", 'reported_seq': 20291, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:08.324082+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:35:08.324082+0000', 'last_peered': '2024-09-17T15:35:08.324082+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:35:08.324082+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "43'10000", 'ondisk_log_start': "43'10000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3543064, 'num_objects': 866, 'num_object_clones': 0, 'num_object_copies': 1732, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 866, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18225, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3259}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:35:11.417 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:35:11.417 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'20419", 'reported_seq': 20720, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:09.129329+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:35:09.129329+0000', 'last_peered': '2024-09-17T15:35:09.129329+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:35:09.129329+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "43'10400", 'ondisk_log_start': "43'10400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3408355, 'num_objects': 833, 'num_object_clones': 0, 'num_object_copies': 1666, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 833, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16918, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3359}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:11.417 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:35:11.417 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'19993", 'reported_seq': 20312, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:09.146609+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:35:09.146609+0000', 'last_peered': '2024-09-17T15:35:09.146609+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:35:09.146609+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "43'9900", 'ondisk_log_start': "43'9900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3175366, 'num_objects': 777, 'num_object_clones': 0, 'num_object_copies': 1554, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4089, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 777, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 16762, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 85, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3312}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:11.417 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:35:11.417 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'19740", 'reported_seq': 20014, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:09.158282+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:35:09.158282+0000', 'last_peered': '2024-09-17T15:35:09.158282+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:35:09.158282+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'9700", 'ondisk_log_start': "43'9700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3284992, 'num_objects': 802, 'num_object_clones': 0, 'num_object_copies': 1604, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 802, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16336, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3141}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:11.418 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:35:11.418 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'19695", 'reported_seq': 19914, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:08.196699+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:35:08.196699+0000', 'last_peered': '2024-09-17T15:35:08.196699+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:35:08.196699+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "43'9600", 'ondisk_log_start': "43'9600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3375587, 'num_objects': 825, 'num_object_clones': 0, 'num_object_copies': 1650, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4034, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 825, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17590, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -3208}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:35:11.418 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:35:11.418 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'20142", 'reported_seq': 20450, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:09.181160+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:35:09.181160+0000', 'last_peered': '2024-09-17T15:35:09.181160+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:35:09.181160+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "43'10100", 'ondisk_log_start': "43'10100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3375104, 'num_objects': 824, 'num_object_clones': 0, 'num_object_copies': 1648, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 824, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16690, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3233}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:11.418 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:35:11.418 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'20000", 'reported_seq': 25684, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:09.189711+0000', 'last_change': '2024-09-17T15:33:09.763500+0000', 'last_active': '2024-09-17T15:35:09.189711+0000', 'last_peered': '2024-09-17T15:35:09.189711+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:35:09.189711+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "43'9900", 'ondisk_log_start': "43'9900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3162112, 'num_objects': 772, 'num_object_clones': 0, 'num_object_copies': 1544, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1394, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 772, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16287, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2766, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -622}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:11.418 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:35:11.418 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'20087", 'reported_seq': 20395, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:09.178560+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:35:09.178560+0000', 'last_peered': '2024-09-17T15:35:09.178560+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:35:09.178560+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'10000", 'ondisk_log_start': "43'10000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3272704, 'num_objects': 799, 'num_object_clones': 0, 'num_object_copies': 1598, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 799, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16874, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3181}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:11.418 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:35:12.125 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:35:12.126 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33805/26154 objects degraded (129.254%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":649,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":13077,"data_bytes":53515017,"bytes_used":1632210944,"bytes_avail":288278081536,"bytes_total":289910292480,"degraded_objects":33805,"degraded_total":26154,"degraded_ratio":1.2925365144910912,"recovering_objects_per_sec":34,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":415,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":612},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:35:12.155 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:35:12.155 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:35:12.694 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:35:12.694 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:35:12.719 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":364,"stamp":"2024-09-17T15:35:11.800419+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":53515017,"num_objects":13077,"num_object_clones":0,"num_object_copies":26154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33805,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13077,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":279914,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15656,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160980,"ondisk_log_size":160980,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1593956,"kb_used_data":305012,"kb_used_omap":13,"kb_used_meta":1288882,"kb_avail":281521564,"statfs":{"total":289910292480,"available":288278081536,"internally_reserved":0,"allocated":312332288,"data_stored":311405347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1319816191},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,5,6,12,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20041728,"num_objects":-4893,"num_object_clones":0,"num_object_copies":-9786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-277,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4893,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":277,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004458"},"pg_stats":[{"pgid":"2.f","version":"47'19844","reported_seq":20104,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:09.157550+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:09.157550+0000","last_peered":"2024-09-17T15:35:09.157550+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:09.157550+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'9800","ondisk_log_start":"43'9800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3293184,"num_objects":804,"num_object_clones":0,"num_object_copies":1608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19844,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3180}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20057","reported_seq":20170,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:08.306757+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:08.306757+0000","last_peered":"2024-09-17T15:35:08.306757+0000","last_clean":"2024-09-17T15:35:08.306757+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:08.306757+0000","last_undegraded":"2024-09-17T15:35:08.306757+0000","last_fullsized":"2024-09-17T15:35:08.306757+0000","mapping_epoch":33,"log_start":"43'10000","ondisk_log_start":"43'10000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3584966,"num_objects":877,"num_object_clones":0,"num_object_copies":1754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":877,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18414,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20068","reported_seq":20291,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:08.324082+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:08.324082+0000","last_peered":"2024-09-17T15:35:08.324082+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:08.324082+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10000","ondisk_log_start":"43'10000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3543064,"num_objects":866,"num_object_clones":0,"num_object_copies":1732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":866,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18225,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3259}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20168","reported_seq":20248,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:08.337668+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:08.337668+0000","last_peered":"2024-09-17T15:35:08.337668+0000","last_clean":"2024-09-17T15:35:08.337668+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:08.337668+0000","last_undegraded":"2024-09-17T15:35:08.337668+0000","last_fullsized":"2024-09-17T15:35:08.337668+0000","mapping_epoch":33,"log_start":"43'10100","ondisk_log_start":"43'10100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3284992,"num_objects":802,"num_object_clones":0,"num_object_copies":1604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20168,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19856","reported_seq":19996,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:09.132844+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:09.132844+0000","last_peered":"2024-09-17T15:35:09.132844+0000","last_clean":"2024-09-17T15:35:09.132844+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:09.132844+0000","last_undegraded":"2024-09-17T15:35:09.132844+0000","last_fullsized":"2024-09-17T15:35:09.132844+0000","mapping_epoch":32,"log_start":"43'9800","ondisk_log_start":"43'9800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3264995,"num_objects":798,"num_object_clones":0,"num_object_copies":1596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":798,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16838,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'20419","reported_seq":20720,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:09.129329+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:09.129329+0000","last_peered":"2024-09-17T15:35:09.129329+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:09.129329+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'10400","ondisk_log_start":"43'10400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3408355,"num_objects":833,"num_object_clones":0,"num_object_copies":1666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":833,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16918,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3359},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19993","reported_seq":20312,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:09.146609+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:35:09.146609+0000","last_peered":"2024-09-17T15:35:09.146609+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:35:09.146609+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'9900","ondisk_log_start":"43'9900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3175366,"num_objects":777,"num_object_clones":0,"num_object_copies":1554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":777,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16762,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3312},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20118","reported_seq":20256,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:09.189968+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:09.189968+0000","last_peered":"2024-09-17T15:35:09.189968+0000","last_clean":"2024-09-17T15:35:09.189968+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:09.189968+0000","last_undegraded":"2024-09-17T15:35:09.189968+0000","last_fullsized":"2024-09-17T15:35:09.189968+0000","mapping_epoch":32,"log_start":"43'10100","ondisk_log_start":"43'10100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3268608,"num_objects":798,"num_object_clones":0,"num_object_copies":1596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17072,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.224142+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.224142+0000","last_peered":"2024-09-17T15:33:11.224142+0000","last_clean":"2024-09-17T15:33:11.224142+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.224142+0000","last_undegraded":"2024-09-17T15:33:11.224142+0000","last_fullsized":"2024-09-17T15:33:11.224142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19988","reported_seq":28281,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:08.708773+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:08.708773+0000","last_peered":"2024-09-17T15:35:08.708773+0000","last_clean":"2024-09-17T15:35:08.708773+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:08.708773+0000","last_undegraded":"2024-09-17T15:35:08.708773+0000","last_fullsized":"2024-09-17T15:35:08.708773+0000","mapping_epoch":42,"log_start":"43'9900","ondisk_log_start":"43'9900","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3396067,"num_objects":830,"num_object_clones":0,"num_object_copies":1660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":830,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17988,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19740","reported_seq":20014,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:09.158282+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:09.158282+0000","last_peered":"2024-09-17T15:35:09.158282+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:09.158282+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'9700","ondisk_log_start":"43'9700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3284992,"num_objects":802,"num_object_clones":0,"num_object_copies":1604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16336,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3141}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19695","reported_seq":19914,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:08.196699+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:35:08.196699+0000","last_peered":"2024-09-17T15:35:08.196699+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:35:08.196699+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'9600","ondisk_log_start":"43'9600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3375587,"num_objects":825,"num_object_clones":0,"num_object_copies":1650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":825,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17590,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3208}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":88,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223926+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223926+0000","last_peered":"2024-09-17T15:33:11.223926+0000","last_clean":"2024-09-17T15:33:11.223926+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223926+0000","last_undegraded":"2024-09-17T15:33:11.223926+0000","last_fullsized":"2024-09-17T15:33:11.223926+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19907","reported_seq":28322,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:08.323967+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:08.323967+0000","last_peered":"2024-09-17T15:35:08.323967+0000","last_clean":"2024-09-17T15:35:08.323967+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:08.323967+0000","last_undegraded":"2024-09-17T15:35:08.323967+0000","last_fullsized":"2024-09-17T15:35:08.323967+0000","mapping_epoch":46,"log_start":"43'9900","ondisk_log_start":"43'9900","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3395584,"num_objects":829,"num_object_clones":0,"num_object_copies":1658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17669,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20142","reported_seq":20450,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:09.181160+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:09.181160+0000","last_peered":"2024-09-17T15:35:09.181160+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:09.181160+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10100","ondisk_log_start":"43'10100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3375104,"num_objects":824,"num_object_clones":0,"num_object_copies":1648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16690,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3233}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223809+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223809+0000","last_peered":"2024-09-17T15:33:11.223809+0000","last_clean":"2024-09-17T15:33:11.223809+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223809+0000","last_undegraded":"2024-09-17T15:33:11.223809+0000","last_fullsized":"2024-09-17T15:33:11.223809+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20000","reported_seq":25684,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:09.189711+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:35:09.189711+0000","last_peered":"2024-09-17T15:35:09.189711+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:35:09.189711+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"43'9900","ondisk_log_start":"43'9900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3162112,"num_objects":772,"num_object_clones":0,"num_object_copies":1544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1394,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16287,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2766,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-622}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20087","reported_seq":20395,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:09.178560+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:09.178560+0000","last_peered":"2024-09-17T15:35:09.178560+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:09.178560+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10000","ondisk_log_start":"43'10000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3272704,"num_objects":799,"num_object_clones":0,"num_object_copies":1598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":799,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16874,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3181}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223917+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223917+0000","last_peered":"2024-09-17T15:33:11.223917+0000","last_clean":"2024-09-17T15:33:11.223917+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223917+0000","last_undegraded":"2024-09-17T15:33:11.223917+0000","last_fullsized":"2024-09-17T15:33:11.223917+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223714+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223714+0000","last_peered":"2024-09-17T15:33:11.223714+0000","last_clean":"2024-09-17T15:33:11.223714+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223714+0000","last_undegraded":"2024-09-17T15:33:11.223714+0000","last_fullsized":"2024-09-17T15:33:11.223714+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19795","reported_seq":28179,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:08.690719+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:08.690719+0000","last_peered":"2024-09-17T15:35:08.690719+0000","last_clean":"2024-09-17T15:35:08.690719+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:08.690719+0000","last_undegraded":"2024-09-17T15:35:08.690719+0000","last_fullsized":"2024-09-17T15:35:08.690719+0000","mapping_epoch":44,"log_start":"43'9700","ondisk_log_start":"43'9700","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3429318,"num_objects":839,"num_object_clones":0,"num_object_copies":1678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16237,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":53514998,"num_objects":13075,"num_object_clones":0,"num_object_copies":26150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33805,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13075,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":279912,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15655,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":309760000,"data_stored":309625401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160977,"ondisk_log_size":160977,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051705,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":520156,"kb_used_data":159436,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93851684,"statfs":{"total":96636764160,"available":96104124416,"internally_reserved":0,"allocated":163262464,"data_stored":162960866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":369356460},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,7],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724650,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":513048,"kb_used_data":39880,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93858792,"statfs":{"total":96636764160,"available":96111403008,"internally_reserved":0,"allocated":40837120,"data_stored":40513380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":484503211},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,2,3,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247933,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":560752,"kb_used_data":105696,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93811088,"statfs":{"total":96636764160,"available":96062554112,"internally_reserved":0,"allocated":108232704,"data_stored":107931101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":465956520},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,3,2,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":106876928,"data_stored":106840798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40361984,"data_stored":40303717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":162521088,"data_stored":162480886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:35:15.721 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:35:16.287 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:35:16.287 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:35:16.318 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":366,"stamp":"2024-09-17T15:35:15.801540+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":43471625,"num_objects":10625,"num_object_clones":0,"num_object_copies":21250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33655,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10625,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":282366,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15806,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160732,"ondisk_log_size":160732,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1561868,"kb_used_data":285376,"kb_used_omap":13,"kb_used_meta":1276402,"kb_avail":281553652,"statfs":{"total":289910292480,"available":288310939648,"internally_reserved":0,"allocated":292225024,"data_stored":291298083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1307036671},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,9,6,6,6,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25661440,"num_objects":-6265,"num_object_clones":0,"num_object_copies":-12530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-427,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6265,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6265,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":427,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003944"},"pg_stats":[{"pgid":"2.f","version":"47'19985","reported_seq":20245,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:14.201543+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:14.201543+0000","last_peered":"2024-09-17T15:35:14.201543+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:14.201543+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'9900","ondisk_log_start":"43'9900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2715648,"num_objects":663,"num_object_clones":0,"num_object_copies":1326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19985,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3321}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20222","reported_seq":20335,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:13.358239+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:13.358239+0000","last_peered":"2024-09-17T15:35:13.358239+0000","last_clean":"2024-09-17T15:35:13.358239+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:13.358239+0000","last_undegraded":"2024-09-17T15:35:13.358239+0000","last_fullsized":"2024-09-17T15:35:13.358239+0000","mapping_epoch":33,"log_start":"43'10200","ondisk_log_start":"43'10200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2909126,"num_objects":712,"num_object_clones":0,"num_object_copies":1424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":712,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18579,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20239","reported_seq":20462,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:13.371432+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:13.371432+0000","last_peered":"2024-09-17T15:35:13.371432+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:13.371432+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10200","ondisk_log_start":"43'10200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2842648,"num_objects":695,"num_object_clones":0,"num_object_copies":1390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":695,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18396,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3430}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20301","reported_seq":20381,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:13.286177+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:13.286177+0000","last_peered":"2024-09-17T15:35:13.286177+0000","last_clean":"2024-09-17T15:35:13.286177+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:13.286177+0000","last_undegraded":"2024-09-17T15:35:13.286177+0000","last_fullsized":"2024-09-17T15:35:13.286177+0000","mapping_epoch":33,"log_start":"43'10300","ondisk_log_start":"43'10300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2740224,"num_objects":669,"num_object_clones":0,"num_object_copies":1338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":669,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20301,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20006","reported_seq":20146,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:14.160744+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:14.160744+0000","last_peered":"2024-09-17T15:35:14.160744+0000","last_clean":"2024-09-17T15:35:14.160744+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:14.160744+0000","last_undegraded":"2024-09-17T15:35:14.160744+0000","last_fullsized":"2024-09-17T15:35:14.160744+0000","mapping_epoch":32,"log_start":"43'10000","ondisk_log_start":"43'10000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2650595,"num_objects":648,"num_object_clones":0,"num_object_copies":1296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":648,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16988,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'20578","reported_seq":20879,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:14.164800+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:14.164800+0000","last_peered":"2024-09-17T15:35:14.164800+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:14.164800+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2757091,"num_objects":674,"num_object_clones":0,"num_object_copies":1348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":674,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17077,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3518},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20138","reported_seq":20457,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:14.161520+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:35:14.161520+0000","last_peered":"2024-09-17T15:35:14.161520+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:35:14.161520+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'10100","ondisk_log_start":"43'10100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2581446,"num_objects":632,"num_object_clones":0,"num_object_copies":1264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":632,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16907,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3457},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20282","reported_seq":20420,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:14.167896+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:14.167896+0000","last_peered":"2024-09-17T15:35:14.167896+0000","last_clean":"2024-09-17T15:35:14.167896+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:14.167896+0000","last_undegraded":"2024-09-17T15:35:14.167896+0000","last_fullsized":"2024-09-17T15:35:14.167896+0000","mapping_epoch":32,"log_start":"43'10200","ondisk_log_start":"43'10200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2596864,"num_objects":634,"num_object_clones":0,"num_object_copies":1268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17236,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.224142+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.224142+0000","last_peered":"2024-09-17T15:33:11.224142+0000","last_clean":"2024-09-17T15:33:11.224142+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.224142+0000","last_undegraded":"2024-09-17T15:33:11.224142+0000","last_fullsized":"2024-09-17T15:33:11.224142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20149","reported_seq":28442,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:13.774542+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:13.774542+0000","last_peered":"2024-09-17T15:35:13.774542+0000","last_clean":"2024-09-17T15:35:13.774542+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:13.774542+0000","last_undegraded":"2024-09-17T15:35:13.774542+0000","last_fullsized":"2024-09-17T15:35:13.774542+0000","mapping_epoch":42,"log_start":"43'10100","ondisk_log_start":"43'10100","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2736611,"num_objects":669,"num_object_clones":0,"num_object_copies":1338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":669,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18149,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19900","reported_seq":20174,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:14.200287+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:14.200287+0000","last_peered":"2024-09-17T15:35:14.200287+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:14.200287+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'9800","ondisk_log_start":"43'9800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2629632,"num_objects":642,"num_object_clones":0,"num_object_copies":1284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":642,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16496,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3301}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19839","reported_seq":20056,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:13.338215+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:35:13.338215+0000","last_peered":"2024-09-17T15:35:13.338215+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:35:13.338215+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'9800","ondisk_log_start":"43'9800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2785763,"num_objects":681,"num_object_clones":0,"num_object_copies":1362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":681,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17734,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3352}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":88,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223926+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223926+0000","last_peered":"2024-09-17T15:33:11.223926+0000","last_clean":"2024-09-17T15:33:11.223926+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223926+0000","last_undegraded":"2024-09-17T15:33:11.223926+0000","last_fullsized":"2024-09-17T15:33:11.223926+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20067","reported_seq":28482,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:13.249274+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:13.249274+0000","last_peered":"2024-09-17T15:35:13.249274+0000","last_clean":"2024-09-17T15:35:13.249274+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:13.249274+0000","last_undegraded":"2024-09-17T15:35:13.249274+0000","last_fullsized":"2024-09-17T15:35:13.249274+0000","mapping_epoch":46,"log_start":"43'10000","ondisk_log_start":"43'10000","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2740224,"num_objects":669,"num_object_clones":0,"num_object_copies":1338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":669,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17829,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20301","reported_seq":20609,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:14.201325+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:14.201325+0000","last_peered":"2024-09-17T15:35:14.201325+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:14.201325+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10300","ondisk_log_start":"43'10300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2723840,"num_objects":665,"num_object_clones":0,"num_object_copies":1330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16849,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3392}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223809+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223809+0000","last_peered":"2024-09-17T15:33:11.223809+0000","last_clean":"2024-09-17T15:33:11.223809+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223809+0000","last_undegraded":"2024-09-17T15:33:11.223809+0000","last_fullsized":"2024-09-17T15:33:11.223809+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20138","reported_seq":26123,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:14.218533+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:35:14.218533+0000","last_peered":"2024-09-17T15:35:14.218533+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:35:14.218533+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"43'10100","ondisk_log_start":"43'10100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2596864,"num_objects":634,"num_object_clones":0,"num_object_copies":1268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1244,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2916,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-610}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20241","reported_seq":20549,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:14.172170+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:14.172170+0000","last_peered":"2024-09-17T15:35:14.172170+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:14.172170+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10200","ondisk_log_start":"43'10200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2641920,"num_objects":645,"num_object_clones":0,"num_object_copies":1290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":645,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17028,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223917+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223917+0000","last_peered":"2024-09-17T15:33:11.223917+0000","last_clean":"2024-09-17T15:33:11.223917+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223917+0000","last_undegraded":"2024-09-17T15:33:11.223917+0000","last_fullsized":"2024-09-17T15:33:11.223917+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223714+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223714+0000","last_peered":"2024-09-17T15:33:11.223714+0000","last_clean":"2024-09-17T15:33:11.223714+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223714+0000","last_undegraded":"2024-09-17T15:33:11.223714+0000","last_fullsized":"2024-09-17T15:33:11.223714+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19943","reported_seq":28328,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:13.774226+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:13.774226+0000","last_peered":"2024-09-17T15:35:13.774226+0000","last_clean":"2024-09-17T15:35:13.774226+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:13.774226+0000","last_undegraded":"2024-09-17T15:35:13.774226+0000","last_fullsized":"2024-09-17T15:35:13.774226+0000","mapping_epoch":44,"log_start":"43'9900","ondisk_log_start":"43'9900","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2823110,"num_objects":691,"num_object_clones":0,"num_object_copies":1382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16385,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":43471606,"num_objects":10623,"num_object_clones":0,"num_object_copies":21246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33655,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10623,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":282364,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15805,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":289701888,"data_stored":289567289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160729,"ondisk_log_size":160729,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051706,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519252,"kb_used_data":154372,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93852588,"statfs":{"total":96636764160,"available":96105050112,"internally_reserved":0,"allocated":158076928,"data_stored":157775330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724651,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497432,"kb_used_data":32556,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93874408,"statfs":{"total":96636764160,"available":96127393792,"internally_reserved":0,"allocated":33337344,"data_stored":33013604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":475983531},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247934,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545184,"kb_used_data":98448,"kb_used_omap":4,"kb_used_meta":446715,"kb_avail":93826656,"statfs":{"total":96636764160,"available":96078495744,"internally_reserved":0,"allocated":100810752,"data_stored":100509149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":457436840},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,0,4,4,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":99450880,"data_stored":99414750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32870400,"data_stored":32812133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":157380608,"data_stored":157340406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:35:16.319 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:35:16.889 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:35:16.890 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:35:16.909 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":366,"stamp":"2024-09-17T15:35:15.801540+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":43471625,"num_objects":10625,"num_object_clones":0,"num_object_copies":21250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33655,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10625,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":282366,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15806,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160732,"ondisk_log_size":160732,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1561868,"kb_used_data":285376,"kb_used_omap":13,"kb_used_meta":1276402,"kb_avail":281553652,"statfs":{"total":289910292480,"available":288310939648,"internally_reserved":0,"allocated":292225024,"data_stored":291298083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1307036671},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,9,6,6,6,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25661440,"num_objects":-6265,"num_object_clones":0,"num_object_copies":-12530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-427,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6265,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6265,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":427,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003944"},"pg_stats":[{"pgid":"2.f","version":"47'19985","reported_seq":20245,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:14.201543+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:14.201543+0000","last_peered":"2024-09-17T15:35:14.201543+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:14.201543+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'9900","ondisk_log_start":"43'9900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2715648,"num_objects":663,"num_object_clones":0,"num_object_copies":1326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19985,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3321}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20222","reported_seq":20335,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:13.358239+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:13.358239+0000","last_peered":"2024-09-17T15:35:13.358239+0000","last_clean":"2024-09-17T15:35:13.358239+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:13.358239+0000","last_undegraded":"2024-09-17T15:35:13.358239+0000","last_fullsized":"2024-09-17T15:35:13.358239+0000","mapping_epoch":33,"log_start":"43'10200","ondisk_log_start":"43'10200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2909126,"num_objects":712,"num_object_clones":0,"num_object_copies":1424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":712,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18579,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20239","reported_seq":20462,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:13.371432+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:13.371432+0000","last_peered":"2024-09-17T15:35:13.371432+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:13.371432+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10200","ondisk_log_start":"43'10200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2842648,"num_objects":695,"num_object_clones":0,"num_object_copies":1390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":695,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18396,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3430}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20301","reported_seq":20381,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:13.286177+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:13.286177+0000","last_peered":"2024-09-17T15:35:13.286177+0000","last_clean":"2024-09-17T15:35:13.286177+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:13.286177+0000","last_undegraded":"2024-09-17T15:35:13.286177+0000","last_fullsized":"2024-09-17T15:35:13.286177+0000","mapping_epoch":33,"log_start":"43'10300","ondisk_log_start":"43'10300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2740224,"num_objects":669,"num_object_clones":0,"num_object_copies":1338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":669,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20301,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20006","reported_seq":20146,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:14.160744+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:14.160744+0000","last_peered":"2024-09-17T15:35:14.160744+0000","last_clean":"2024-09-17T15:35:14.160744+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:14.160744+0000","last_undegraded":"2024-09-17T15:35:14.160744+0000","last_fullsized":"2024-09-17T15:35:14.160744+0000","mapping_epoch":32,"log_start":"43'10000","ondisk_log_start":"43'10000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2650595,"num_objects":648,"num_object_clones":0,"num_object_copies":1296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":648,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16988,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'20578","reported_seq":20879,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:14.164800+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:14.164800+0000","last_peered":"2024-09-17T15:35:14.164800+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:14.164800+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2757091,"num_objects":674,"num_object_clones":0,"num_object_copies":1348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":674,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17077,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3518},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20138","reported_seq":20457,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:14.161520+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:35:14.161520+0000","last_peered":"2024-09-17T15:35:14.161520+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:35:14.161520+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'10100","ondisk_log_start":"43'10100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2581446,"num_objects":632,"num_object_clones":0,"num_object_copies":1264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":632,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16907,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3457},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20282","reported_seq":20420,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:14.167896+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:14.167896+0000","last_peered":"2024-09-17T15:35:14.167896+0000","last_clean":"2024-09-17T15:35:14.167896+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:14.167896+0000","last_undegraded":"2024-09-17T15:35:14.167896+0000","last_fullsized":"2024-09-17T15:35:14.167896+0000","mapping_epoch":32,"log_start":"43'10200","ondisk_log_start":"43'10200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2596864,"num_objects":634,"num_object_clones":0,"num_object_copies":1268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17236,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.224142+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.224142+0000","last_peered":"2024-09-17T15:33:11.224142+0000","last_clean":"2024-09-17T15:33:11.224142+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.224142+0000","last_undegraded":"2024-09-17T15:33:11.224142+0000","last_fullsized":"2024-09-17T15:33:11.224142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20149","reported_seq":28442,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:13.774542+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:13.774542+0000","last_peered":"2024-09-17T15:35:13.774542+0000","last_clean":"2024-09-17T15:35:13.774542+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:13.774542+0000","last_undegraded":"2024-09-17T15:35:13.774542+0000","last_fullsized":"2024-09-17T15:35:13.774542+0000","mapping_epoch":42,"log_start":"43'10100","ondisk_log_start":"43'10100","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2736611,"num_objects":669,"num_object_clones":0,"num_object_copies":1338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":669,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18149,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19900","reported_seq":20174,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:14.200287+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:14.200287+0000","last_peered":"2024-09-17T15:35:14.200287+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:14.200287+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'9800","ondisk_log_start":"43'9800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2629632,"num_objects":642,"num_object_clones":0,"num_object_copies":1284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":642,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16496,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3301}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19839","reported_seq":20056,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:13.338215+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:35:13.338215+0000","last_peered":"2024-09-17T15:35:13.338215+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:35:13.338215+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'9800","ondisk_log_start":"43'9800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2785763,"num_objects":681,"num_object_clones":0,"num_object_copies":1362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":681,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17734,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3352}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":88,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223926+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223926+0000","last_peered":"2024-09-17T15:33:11.223926+0000","last_clean":"2024-09-17T15:33:11.223926+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223926+0000","last_undegraded":"2024-09-17T15:33:11.223926+0000","last_fullsized":"2024-09-17T15:33:11.223926+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20067","reported_seq":28482,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:13.249274+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:13.249274+0000","last_peered":"2024-09-17T15:35:13.249274+0000","last_clean":"2024-09-17T15:35:13.249274+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:13.249274+0000","last_undegraded":"2024-09-17T15:35:13.249274+0000","last_fullsized":"2024-09-17T15:35:13.249274+0000","mapping_epoch":46,"log_start":"43'10000","ondisk_log_start":"43'10000","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2740224,"num_objects":669,"num_object_clones":0,"num_object_copies":1338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":669,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17829,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20301","reported_seq":20609,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:14.201325+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:14.201325+0000","last_peered":"2024-09-17T15:35:14.201325+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:14.201325+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10300","ondisk_log_start":"43'10300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2723840,"num_objects":665,"num_object_clones":0,"num_object_copies":1330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16849,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3392}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223809+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223809+0000","last_peered":"2024-09-17T15:33:11.223809+0000","last_clean":"2024-09-17T15:33:11.223809+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223809+0000","last_undegraded":"2024-09-17T15:33:11.223809+0000","last_fullsized":"2024-09-17T15:33:11.223809+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20138","reported_seq":26123,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:14.218533+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:35:14.218533+0000","last_peered":"2024-09-17T15:35:14.218533+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:35:14.218533+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"43'10100","ondisk_log_start":"43'10100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2596864,"num_objects":634,"num_object_clones":0,"num_object_copies":1268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1244,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2916,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-610}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20241","reported_seq":20549,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:14.172170+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:14.172170+0000","last_peered":"2024-09-17T15:35:14.172170+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:14.172170+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10200","ondisk_log_start":"43'10200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2641920,"num_objects":645,"num_object_clones":0,"num_object_copies":1290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":645,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17028,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223917+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223917+0000","last_peered":"2024-09-17T15:33:11.223917+0000","last_clean":"2024-09-17T15:33:11.223917+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223917+0000","last_undegraded":"2024-09-17T15:33:11.223917+0000","last_fullsized":"2024-09-17T15:33:11.223917+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223714+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223714+0000","last_peered":"2024-09-17T15:33:11.223714+0000","last_clean":"2024-09-17T15:33:11.223714+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223714+0000","last_undegraded":"2024-09-17T15:33:11.223714+0000","last_fullsized":"2024-09-17T15:33:11.223714+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19943","reported_seq":28328,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:13.774226+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:13.774226+0000","last_peered":"2024-09-17T15:35:13.774226+0000","last_clean":"2024-09-17T15:35:13.774226+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:13.774226+0000","last_undegraded":"2024-09-17T15:35:13.774226+0000","last_fullsized":"2024-09-17T15:35:13.774226+0000","mapping_epoch":44,"log_start":"43'9900","ondisk_log_start":"43'9900","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2823110,"num_objects":691,"num_object_clones":0,"num_object_copies":1382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16385,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":43471606,"num_objects":10623,"num_object_clones":0,"num_object_copies":21246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33655,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10623,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":282364,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15805,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":289701888,"data_stored":289567289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160729,"ondisk_log_size":160729,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051706,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519252,"kb_used_data":154372,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93852588,"statfs":{"total":96636764160,"available":96105050112,"internally_reserved":0,"allocated":158076928,"data_stored":157775330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724651,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497432,"kb_used_data":32556,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93874408,"statfs":{"total":96636764160,"available":96127393792,"internally_reserved":0,"allocated":33337344,"data_stored":33013604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":475983531},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247934,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545184,"kb_used_data":98448,"kb_used_omap":4,"kb_used_meta":446715,"kb_avail":93826656,"statfs":{"total":96636764160,"available":96078495744,"internally_reserved":0,"allocated":100810752,"data_stored":100509149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":457436840},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,0,4,4,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":99450880,"data_stored":99414750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32870400,"data_stored":32812133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":157380608,"data_stored":157340406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:35:16.910 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:35:16.910 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'19985", 'reported_seq': 20245, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:14.201543+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:35:14.201543+0000', 'last_peered': '2024-09-17T15:35:14.201543+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:35:14.201543+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'9900", 'ondisk_log_start': "43'9900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2715648, 'num_objects': 663, 'num_object_clones': 0, 'num_object_copies': 1326, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 663, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19985, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3321}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:16.910 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:35:16.911 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'20239", 'reported_seq': 20462, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:13.371432+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:35:13.371432+0000', 'last_peered': '2024-09-17T15:35:13.371432+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:35:13.371432+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "43'10200", 'ondisk_log_start': "43'10200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2842648, 'num_objects': 695, 'num_object_clones': 0, 'num_object_copies': 1390, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 695, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18396, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3430}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:35:16.911 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:35:16.911 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'20578", 'reported_seq': 20879, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:14.164800+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:35:14.164800+0000', 'last_peered': '2024-09-17T15:35:14.164800+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:35:14.164800+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2757091, 'num_objects': 674, 'num_object_clones': 0, 'num_object_copies': 1348, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 674, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17077, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3518}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:16.911 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:35:16.911 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'20138", 'reported_seq': 20457, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:14.161520+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:35:14.161520+0000', 'last_peered': '2024-09-17T15:35:14.161520+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:35:14.161520+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "43'10100", 'ondisk_log_start': "43'10100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2581446, 'num_objects': 632, 'num_object_clones': 0, 'num_object_copies': 1264, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4089, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 632, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 16907, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 85, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3457}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:16.911 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:35:16.912 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'19900", 'reported_seq': 20174, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:14.200287+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:35:14.200287+0000', 'last_peered': '2024-09-17T15:35:14.200287+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:35:14.200287+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'9800", 'ondisk_log_start': "43'9800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2629632, 'num_objects': 642, 'num_object_clones': 0, 'num_object_copies': 1284, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 642, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16496, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3301}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:16.912 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:35:16.912 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'19839", 'reported_seq': 20056, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:13.338215+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:35:13.338215+0000', 'last_peered': '2024-09-17T15:35:13.338215+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:35:13.338215+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "43'9800", 'ondisk_log_start': "43'9800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2785763, 'num_objects': 681, 'num_object_clones': 0, 'num_object_copies': 1362, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4034, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 681, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17734, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -3352}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:35:16.912 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:35:16.912 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'20301", 'reported_seq': 20609, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:14.201325+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:35:14.201325+0000', 'last_peered': '2024-09-17T15:35:14.201325+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:35:14.201325+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "43'10300", 'ondisk_log_start': "43'10300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2723840, 'num_objects': 665, 'num_object_clones': 0, 'num_object_copies': 1330, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 665, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16849, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3392}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:16.912 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:35:16.912 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'20138", 'reported_seq': 26123, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:14.218533+0000', 'last_change': '2024-09-17T15:33:09.763500+0000', 'last_active': '2024-09-17T15:35:14.218533+0000', 'last_peered': '2024-09-17T15:35:14.218533+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:35:14.218533+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "43'10100", 'ondisk_log_start': "43'10100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2596864, 'num_objects': 634, 'num_object_clones': 0, 'num_object_copies': 1268, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1244, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 634, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2916, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -610}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:16.912 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:35:16.912 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'20241", 'reported_seq': 20549, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:14.172170+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:35:14.172170+0000', 'last_peered': '2024-09-17T15:35:14.172170+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:35:14.172170+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'10200", 'ondisk_log_start': "43'10200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2641920, 'num_objects': 645, 'num_object_clones': 0, 'num_object_copies': 1290, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 645, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17028, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3335}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:16.913 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:35:17.609 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:35:17.609 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33655/21250 objects degraded (158.376%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":654,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":10625,"data_bytes":43471625,"bytes_used":1599352832,"bytes_avail":288310939648,"bytes_total":289910292480,"degraded_objects":33655,"degraded_total":21250,"degraded_ratio":1.583764705882353,"recovering_objects_per_sec":35,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":427,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":521},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:35:17.637 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:35:17.637 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:35:18.224 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:35:18.225 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:35:18.253 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":367,"stamp":"2024-09-17T15:35:17.802448+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":43471625,"num_objects":10625,"num_object_clones":0,"num_object_copies":21250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33655,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10625,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":282366,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15806,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160732,"ondisk_log_size":160732,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1561868,"kb_used_data":285376,"kb_used_omap":13,"kb_used_meta":1276402,"kb_avail":281553652,"statfs":{"total":289910292480,"available":288310939648,"internally_reserved":0,"allocated":292225024,"data_stored":291298083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1307036671},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,9,6,6,6,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20070400,"num_objects":-4900,"num_object_clones":0,"num_object_copies":-9800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-288,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4900,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004172"},"pg_stats":[{"pgid":"2.f","version":"47'19985","reported_seq":20245,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:14.201543+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:14.201543+0000","last_peered":"2024-09-17T15:35:14.201543+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:14.201543+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'9900","ondisk_log_start":"43'9900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2715648,"num_objects":663,"num_object_clones":0,"num_object_copies":1326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19985,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3321}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20222","reported_seq":20335,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:13.358239+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:13.358239+0000","last_peered":"2024-09-17T15:35:13.358239+0000","last_clean":"2024-09-17T15:35:13.358239+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:13.358239+0000","last_undegraded":"2024-09-17T15:35:13.358239+0000","last_fullsized":"2024-09-17T15:35:13.358239+0000","mapping_epoch":33,"log_start":"43'10200","ondisk_log_start":"43'10200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2909126,"num_objects":712,"num_object_clones":0,"num_object_copies":1424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":712,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18579,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20239","reported_seq":20462,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:13.371432+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:13.371432+0000","last_peered":"2024-09-17T15:35:13.371432+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:13.371432+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10200","ondisk_log_start":"43'10200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2842648,"num_objects":695,"num_object_clones":0,"num_object_copies":1390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":695,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18396,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3430}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20301","reported_seq":20381,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:13.286177+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:13.286177+0000","last_peered":"2024-09-17T15:35:13.286177+0000","last_clean":"2024-09-17T15:35:13.286177+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:13.286177+0000","last_undegraded":"2024-09-17T15:35:13.286177+0000","last_fullsized":"2024-09-17T15:35:13.286177+0000","mapping_epoch":33,"log_start":"43'10300","ondisk_log_start":"43'10300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2740224,"num_objects":669,"num_object_clones":0,"num_object_copies":1338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":669,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20301,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20006","reported_seq":20146,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:14.160744+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:14.160744+0000","last_peered":"2024-09-17T15:35:14.160744+0000","last_clean":"2024-09-17T15:35:14.160744+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:14.160744+0000","last_undegraded":"2024-09-17T15:35:14.160744+0000","last_fullsized":"2024-09-17T15:35:14.160744+0000","mapping_epoch":32,"log_start":"43'10000","ondisk_log_start":"43'10000","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2650595,"num_objects":648,"num_object_clones":0,"num_object_copies":1296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":648,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16988,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'20578","reported_seq":20879,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:14.164800+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:14.164800+0000","last_peered":"2024-09-17T15:35:14.164800+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:14.164800+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2757091,"num_objects":674,"num_object_clones":0,"num_object_copies":1348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":674,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17077,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3518},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20138","reported_seq":20457,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:14.161520+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:35:14.161520+0000","last_peered":"2024-09-17T15:35:14.161520+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:35:14.161520+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'10100","ondisk_log_start":"43'10100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2581446,"num_objects":632,"num_object_clones":0,"num_object_copies":1264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":632,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16907,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3457},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20282","reported_seq":20420,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:14.167896+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:14.167896+0000","last_peered":"2024-09-17T15:35:14.167896+0000","last_clean":"2024-09-17T15:35:14.167896+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:14.167896+0000","last_undegraded":"2024-09-17T15:35:14.167896+0000","last_fullsized":"2024-09-17T15:35:14.167896+0000","mapping_epoch":32,"log_start":"43'10200","ondisk_log_start":"43'10200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2596864,"num_objects":634,"num_object_clones":0,"num_object_copies":1268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17236,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.224142+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.224142+0000","last_peered":"2024-09-17T15:33:11.224142+0000","last_clean":"2024-09-17T15:33:11.224142+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.224142+0000","last_undegraded":"2024-09-17T15:33:11.224142+0000","last_fullsized":"2024-09-17T15:33:11.224142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20149","reported_seq":28442,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:13.774542+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:13.774542+0000","last_peered":"2024-09-17T15:35:13.774542+0000","last_clean":"2024-09-17T15:35:13.774542+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:13.774542+0000","last_undegraded":"2024-09-17T15:35:13.774542+0000","last_fullsized":"2024-09-17T15:35:13.774542+0000","mapping_epoch":42,"log_start":"43'10100","ondisk_log_start":"43'10100","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2736611,"num_objects":669,"num_object_clones":0,"num_object_copies":1338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":669,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18149,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19900","reported_seq":20174,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:14.200287+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:14.200287+0000","last_peered":"2024-09-17T15:35:14.200287+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:14.200287+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'9800","ondisk_log_start":"43'9800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2629632,"num_objects":642,"num_object_clones":0,"num_object_copies":1284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":642,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16496,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3301}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19839","reported_seq":20056,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:13.338215+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:35:13.338215+0000","last_peered":"2024-09-17T15:35:13.338215+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:35:13.338215+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'9800","ondisk_log_start":"43'9800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2785763,"num_objects":681,"num_object_clones":0,"num_object_copies":1362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":681,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17734,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3352}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":88,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223926+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223926+0000","last_peered":"2024-09-17T15:33:11.223926+0000","last_clean":"2024-09-17T15:33:11.223926+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223926+0000","last_undegraded":"2024-09-17T15:33:11.223926+0000","last_fullsized":"2024-09-17T15:33:11.223926+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20067","reported_seq":28482,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:13.249274+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:13.249274+0000","last_peered":"2024-09-17T15:35:13.249274+0000","last_clean":"2024-09-17T15:35:13.249274+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:13.249274+0000","last_undegraded":"2024-09-17T15:35:13.249274+0000","last_fullsized":"2024-09-17T15:35:13.249274+0000","mapping_epoch":46,"log_start":"43'10000","ondisk_log_start":"43'10000","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2740224,"num_objects":669,"num_object_clones":0,"num_object_copies":1338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":669,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17829,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20301","reported_seq":20609,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:14.201325+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:14.201325+0000","last_peered":"2024-09-17T15:35:14.201325+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:14.201325+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10300","ondisk_log_start":"43'10300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2723840,"num_objects":665,"num_object_clones":0,"num_object_copies":1330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16849,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3392}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223809+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223809+0000","last_peered":"2024-09-17T15:33:11.223809+0000","last_clean":"2024-09-17T15:33:11.223809+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223809+0000","last_undegraded":"2024-09-17T15:33:11.223809+0000","last_fullsized":"2024-09-17T15:33:11.223809+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20138","reported_seq":26123,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:14.218533+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:35:14.218533+0000","last_peered":"2024-09-17T15:35:14.218533+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:35:14.218533+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"43'10100","ondisk_log_start":"43'10100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2596864,"num_objects":634,"num_object_clones":0,"num_object_copies":1268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1244,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2916,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-610}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20241","reported_seq":20549,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:14.172170+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:14.172170+0000","last_peered":"2024-09-17T15:35:14.172170+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:14.172170+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10200","ondisk_log_start":"43'10200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2641920,"num_objects":645,"num_object_clones":0,"num_object_copies":1290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":645,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17028,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223917+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223917+0000","last_peered":"2024-09-17T15:33:11.223917+0000","last_clean":"2024-09-17T15:33:11.223917+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223917+0000","last_undegraded":"2024-09-17T15:33:11.223917+0000","last_fullsized":"2024-09-17T15:33:11.223917+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223714+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223714+0000","last_peered":"2024-09-17T15:33:11.223714+0000","last_clean":"2024-09-17T15:33:11.223714+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223714+0000","last_undegraded":"2024-09-17T15:33:11.223714+0000","last_fullsized":"2024-09-17T15:33:11.223714+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19943","reported_seq":28328,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:13.774226+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:13.774226+0000","last_peered":"2024-09-17T15:35:13.774226+0000","last_clean":"2024-09-17T15:35:13.774226+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:13.774226+0000","last_undegraded":"2024-09-17T15:35:13.774226+0000","last_fullsized":"2024-09-17T15:35:13.774226+0000","mapping_epoch":44,"log_start":"43'9900","ondisk_log_start":"43'9900","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2823110,"num_objects":691,"num_object_clones":0,"num_object_copies":1382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16385,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":43471606,"num_objects":10623,"num_object_clones":0,"num_object_copies":21246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33655,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10623,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":282364,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15805,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":289701888,"data_stored":289567289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160729,"ondisk_log_size":160729,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051706,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519252,"kb_used_data":154372,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93852588,"statfs":{"total":96636764160,"available":96105050112,"internally_reserved":0,"allocated":158076928,"data_stored":157775330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724651,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497432,"kb_used_data":32556,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93874408,"statfs":{"total":96636764160,"available":96127393792,"internally_reserved":0,"allocated":33337344,"data_stored":33013604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":475983531},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247934,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545184,"kb_used_data":98448,"kb_used_omap":4,"kb_used_meta":446715,"kb_avail":93826656,"statfs":{"total":96636764160,"available":96078495744,"internally_reserved":0,"allocated":100810752,"data_stored":100509149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":457436840},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,0,4,4,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":99450880,"data_stored":99414750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32870400,"data_stored":32812133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":157380608,"data_stored":157340406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:35:21.258 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:35:21.832 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:35:21.833 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:35:21.860 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":369,"stamp":"2024-09-17T15:35:21.805672+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":33563401,"num_objects":8206,"num_object_clones":0,"num_object_copies":16412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33505,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8206,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":284785,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15956,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160651,"ondisk_log_size":160651,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1538392,"kb_used_data":266064,"kb_used_omap":13,"kb_used_meta":1272242,"kb_avail":281577128,"statfs":{"total":289910292480,"available":288334979072,"internally_reserved":0,"allocated":272449536,"data_stored":271522595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1302776831},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,4,8,12,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19951616,"num_objects":-4871,"num_object_clones":0,"num_object_copies":-9742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-300,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4871,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4871,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005734"},"pg_stats":[{"pgid":"2.f","version":"47'20136","reported_seq":20396,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:19.043440+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:19.043440+0000","last_peered":"2024-09-17T15:35:19.043440+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:19.043440+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10100","ondisk_log_start":"43'10100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":512,"num_object_clones":0,"num_object_copies":1024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20136,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3472}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20384","reported_seq":20497,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:18.303847+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:18.303847+0000","last_peered":"2024-09-17T15:35:18.303847+0000","last_clean":"2024-09-17T15:35:18.303847+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:18.303847+0000","last_undegraded":"2024-09-17T15:35:18.303847+0000","last_fullsized":"2024-09-17T15:35:18.303847+0000","mapping_epoch":33,"log_start":"43'10300","ondisk_log_start":"43'10300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2245574,"num_objects":550,"num_object_clones":0,"num_object_copies":1100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":550,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18741,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20376","reported_seq":20599,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:18.403981+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:18.403981+0000","last_peered":"2024-09-17T15:35:18.403981+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:18.403981+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10300","ondisk_log_start":"43'10300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2281496,"num_objects":558,"num_object_clones":0,"num_object_copies":1116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":558,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18533,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3567}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20452","reported_seq":20532,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:18.407692+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:18.407692+0000","last_peered":"2024-09-17T15:35:18.407692+0000","last_clean":"2024-09-17T15:35:18.407692+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:18.407692+0000","last_undegraded":"2024-09-17T15:35:18.407692+0000","last_fullsized":"2024-09-17T15:35:18.407692+0000","mapping_epoch":33,"log_start":"43'10400","ondisk_log_start":"43'10400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2121728,"num_objects":518,"num_object_clones":0,"num_object_copies":1036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20452,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20139","reported_seq":20279,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:19.155916+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:19.155916+0000","last_peered":"2024-09-17T15:35:19.155916+0000","last_clean":"2024-09-17T15:35:19.155916+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:19.155916+0000","last_undegraded":"2024-09-17T15:35:19.155916+0000","last_fullsized":"2024-09-17T15:35:19.155916+0000","mapping_epoch":32,"log_start":"43'10100","ondisk_log_start":"43'10100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2105827,"num_objects":515,"num_object_clones":0,"num_object_copies":1030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":515,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17121,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'20734","reported_seq":21035,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:19.195568+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:19.195568+0000","last_peered":"2024-09-17T15:35:19.195568+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:19.195568+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2118115,"num_objects":518,"num_object_clones":0,"num_object_copies":1036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":518,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17233,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3674},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20291","reported_seq":20610,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:19.182959+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:35:19.182959+0000","last_peered":"2024-09-17T15:35:19.182959+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:35:19.182959+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'10200","ondisk_log_start":"43'10200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1954758,"num_objects":479,"num_object_clones":0,"num_object_copies":958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":479,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17060,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3610},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20420","reported_seq":20556,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:19.235478+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:19.235478+0000","last_peered":"2024-09-17T15:35:19.235478+0000","last_clean":"2024-09-17T15:35:19.235478+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:19.235478+0000","last_undegraded":"2024-09-17T15:35:19.235478+0000","last_fullsized":"2024-09-17T15:35:19.235478+0000","mapping_epoch":32,"log_start":"43'10400","ondisk_log_start":"43'10400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":496,"num_object_clones":0,"num_object_copies":992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17374,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.224142+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.224142+0000","last_peered":"2024-09-17T15:33:11.224142+0000","last_clean":"2024-09-17T15:33:11.224142+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.224142+0000","last_undegraded":"2024-09-17T15:33:11.224142+0000","last_fullsized":"2024-09-17T15:33:11.224142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20301","reported_seq":28594,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:18.884456+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:18.884456+0000","last_peered":"2024-09-17T15:35:18.884456+0000","last_clean":"2024-09-17T15:35:18.884456+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:18.884456+0000","last_undegraded":"2024-09-17T15:35:18.884456+0000","last_fullsized":"2024-09-17T15:35:18.884456+0000","mapping_epoch":42,"log_start":"43'10300","ondisk_log_start":"43'10300","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2114019,"num_objects":517,"num_object_clones":0,"num_object_copies":1034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":517,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18301,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20046","reported_seq":20321,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:19.164644+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:19.164644+0000","last_peered":"2024-09-17T15:35:19.164644+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:19.164644+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10000","ondisk_log_start":"43'10000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":496,"num_object_clones":0,"num_object_copies":992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16642,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3447}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20013","reported_seq":20230,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:18.382784+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:35:18.382784+0000","last_peered":"2024-09-17T15:35:18.382784+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:35:18.382784+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'10000","ondisk_log_start":"43'10000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2073059,"num_objects":507,"num_object_clones":0,"num_object_copies":1014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":507,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3526}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":88,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223926+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223926+0000","last_peered":"2024-09-17T15:33:11.223926+0000","last_clean":"2024-09-17T15:33:11.223926+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223926+0000","last_undegraded":"2024-09-17T15:33:11.223926+0000","last_fullsized":"2024-09-17T15:33:11.223926+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20208","reported_seq":28623,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:18.306905+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:18.306905+0000","last_peered":"2024-09-17T15:35:18.306905+0000","last_clean":"2024-09-17T15:35:18.306905+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:18.306905+0000","last_undegraded":"2024-09-17T15:35:18.306905+0000","last_fullsized":"2024-09-17T15:35:18.306905+0000","mapping_epoch":46,"log_start":"43'10200","ondisk_log_start":"43'10200","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2162688,"num_objects":528,"num_object_clones":0,"num_object_copies":1056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17970,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20464","reported_seq":20772,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:19.208089+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:19.208089+0000","last_peered":"2024-09-17T15:35:19.208089+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:19.208089+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10400","ondisk_log_start":"43'10400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2056192,"num_objects":502,"num_object_clones":0,"num_object_copies":1004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17012,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223809+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223809+0000","last_peered":"2024-09-17T15:33:11.223809+0000","last_clean":"2024-09-17T15:33:11.223809+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223809+0000","last_undegraded":"2024-09-17T15:33:11.223809+0000","last_fullsized":"2024-09-17T15:33:11.223809+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20270","reported_seq":26555,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:19.236605+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:35:19.236605+0000","last_peered":"2024-09-17T15:35:19.236605+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:35:19.236605+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"43'10200","ondisk_log_start":"43'10200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2056192,"num_objects":502,"num_object_clones":0,"num_object_copies":1004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1094,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16557,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3066,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-592}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20407","reported_seq":20714,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:19.150684+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:19.150684+0000","last_peered":"2024-09-17T15:35:19.150684+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:19.150684+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10400","ondisk_log_start":"43'10400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1961984,"num_objects":479,"num_object_clones":0,"num_object_copies":958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17194,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3501}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223917+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223917+0000","last_peered":"2024-09-17T15:33:11.223917+0000","last_clean":"2024-09-17T15:33:11.223917+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223917+0000","last_undegraded":"2024-09-17T15:33:11.223917+0000","last_fullsized":"2024-09-17T15:33:11.223917+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223714+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223714+0000","last_peered":"2024-09-17T15:33:11.223714+0000","last_clean":"2024-09-17T15:33:11.223714+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223714+0000","last_undegraded":"2024-09-17T15:33:11.223714+0000","last_fullsized":"2024-09-17T15:33:11.223714+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20107","reported_seq":28492,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:18.858503+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:18.858503+0000","last_peered":"2024-09-17T15:35:18.858503+0000","last_clean":"2024-09-17T15:35:18.858503+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:18.858503+0000","last_undegraded":"2024-09-17T15:35:18.858503+0000","last_fullsized":"2024-09-17T15:35:18.858503+0000","mapping_epoch":44,"log_start":"43'10100","ondisk_log_start":"43'10100","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2151366,"num_objects":527,"num_object_clones":0,"num_object_copies":1054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":527,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16549,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":33563382,"num_objects":8204,"num_object_clones":0,"num_object_copies":16408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33505,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8204,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":284783,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15955,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":269869056,"data_stored":269734457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160648,"ondisk_log_size":160648,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051707,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502060,"kb_used_data":149652,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93869780,"statfs":{"total":96636764160,"available":96122654720,"internally_reserved":0,"allocated":153243648,"data_stored":152942050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,1,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724652,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":494204,"kb_used_data":25184,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93877636,"statfs":{"total":96636764160,"available":96130699264,"internally_reserved":0,"allocated":25788416,"data_stored":25464676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480243371},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,3,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247935,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":542128,"kb_used_data":91228,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93829712,"statfs":{"total":96636764160,"available":96081625088,"internally_reserved":0,"allocated":93417472,"data_stored":93115869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":461696680},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,4,3,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":92037120,"data_stored":92000990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":25243648,"data_stored":25185381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":152588288,"data_stored":152548086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:35:21.861 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:35:22.431 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:35:22.434 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:35:22.455 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":369,"stamp":"2024-09-17T15:35:21.805672+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":33563401,"num_objects":8206,"num_object_clones":0,"num_object_copies":16412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33505,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8206,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":284785,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15956,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160651,"ondisk_log_size":160651,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1538392,"kb_used_data":266064,"kb_used_omap":13,"kb_used_meta":1272242,"kb_avail":281577128,"statfs":{"total":289910292480,"available":288334979072,"internally_reserved":0,"allocated":272449536,"data_stored":271522595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1302776831},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,4,8,12,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19951616,"num_objects":-4871,"num_object_clones":0,"num_object_copies":-9742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-300,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4871,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4871,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005734"},"pg_stats":[{"pgid":"2.f","version":"47'20136","reported_seq":20396,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:19.043440+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:19.043440+0000","last_peered":"2024-09-17T15:35:19.043440+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:19.043440+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10100","ondisk_log_start":"43'10100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":512,"num_object_clones":0,"num_object_copies":1024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20136,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3472}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20384","reported_seq":20497,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:18.303847+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:18.303847+0000","last_peered":"2024-09-17T15:35:18.303847+0000","last_clean":"2024-09-17T15:35:18.303847+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:18.303847+0000","last_undegraded":"2024-09-17T15:35:18.303847+0000","last_fullsized":"2024-09-17T15:35:18.303847+0000","mapping_epoch":33,"log_start":"43'10300","ondisk_log_start":"43'10300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2245574,"num_objects":550,"num_object_clones":0,"num_object_copies":1100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":550,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18741,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20376","reported_seq":20599,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:18.403981+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:18.403981+0000","last_peered":"2024-09-17T15:35:18.403981+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:18.403981+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10300","ondisk_log_start":"43'10300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2281496,"num_objects":558,"num_object_clones":0,"num_object_copies":1116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":558,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18533,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3567}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20452","reported_seq":20532,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:18.407692+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:18.407692+0000","last_peered":"2024-09-17T15:35:18.407692+0000","last_clean":"2024-09-17T15:35:18.407692+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:18.407692+0000","last_undegraded":"2024-09-17T15:35:18.407692+0000","last_fullsized":"2024-09-17T15:35:18.407692+0000","mapping_epoch":33,"log_start":"43'10400","ondisk_log_start":"43'10400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2121728,"num_objects":518,"num_object_clones":0,"num_object_copies":1036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20452,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20139","reported_seq":20279,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:19.155916+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:19.155916+0000","last_peered":"2024-09-17T15:35:19.155916+0000","last_clean":"2024-09-17T15:35:19.155916+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:19.155916+0000","last_undegraded":"2024-09-17T15:35:19.155916+0000","last_fullsized":"2024-09-17T15:35:19.155916+0000","mapping_epoch":32,"log_start":"43'10100","ondisk_log_start":"43'10100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2105827,"num_objects":515,"num_object_clones":0,"num_object_copies":1030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":515,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17121,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'20734","reported_seq":21035,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:19.195568+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:19.195568+0000","last_peered":"2024-09-17T15:35:19.195568+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:19.195568+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2118115,"num_objects":518,"num_object_clones":0,"num_object_copies":1036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":518,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17233,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3674},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20291","reported_seq":20610,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:19.182959+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:35:19.182959+0000","last_peered":"2024-09-17T15:35:19.182959+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:35:19.182959+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'10200","ondisk_log_start":"43'10200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1954758,"num_objects":479,"num_object_clones":0,"num_object_copies":958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":479,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17060,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3610},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20420","reported_seq":20556,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:19.235478+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:19.235478+0000","last_peered":"2024-09-17T15:35:19.235478+0000","last_clean":"2024-09-17T15:35:19.235478+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:19.235478+0000","last_undegraded":"2024-09-17T15:35:19.235478+0000","last_fullsized":"2024-09-17T15:35:19.235478+0000","mapping_epoch":32,"log_start":"43'10400","ondisk_log_start":"43'10400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":496,"num_object_clones":0,"num_object_copies":992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17374,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.224142+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.224142+0000","last_peered":"2024-09-17T15:33:11.224142+0000","last_clean":"2024-09-17T15:33:11.224142+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.224142+0000","last_undegraded":"2024-09-17T15:33:11.224142+0000","last_fullsized":"2024-09-17T15:33:11.224142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20301","reported_seq":28594,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:18.884456+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:18.884456+0000","last_peered":"2024-09-17T15:35:18.884456+0000","last_clean":"2024-09-17T15:35:18.884456+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:18.884456+0000","last_undegraded":"2024-09-17T15:35:18.884456+0000","last_fullsized":"2024-09-17T15:35:18.884456+0000","mapping_epoch":42,"log_start":"43'10300","ondisk_log_start":"43'10300","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2114019,"num_objects":517,"num_object_clones":0,"num_object_copies":1034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":517,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18301,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20046","reported_seq":20321,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:19.164644+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:19.164644+0000","last_peered":"2024-09-17T15:35:19.164644+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:19.164644+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10000","ondisk_log_start":"43'10000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":496,"num_object_clones":0,"num_object_copies":992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16642,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3447}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20013","reported_seq":20230,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:18.382784+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:35:18.382784+0000","last_peered":"2024-09-17T15:35:18.382784+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:35:18.382784+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'10000","ondisk_log_start":"43'10000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2073059,"num_objects":507,"num_object_clones":0,"num_object_copies":1014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":507,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3526}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":88,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223926+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223926+0000","last_peered":"2024-09-17T15:33:11.223926+0000","last_clean":"2024-09-17T15:33:11.223926+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223926+0000","last_undegraded":"2024-09-17T15:33:11.223926+0000","last_fullsized":"2024-09-17T15:33:11.223926+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20208","reported_seq":28623,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:18.306905+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:18.306905+0000","last_peered":"2024-09-17T15:35:18.306905+0000","last_clean":"2024-09-17T15:35:18.306905+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:18.306905+0000","last_undegraded":"2024-09-17T15:35:18.306905+0000","last_fullsized":"2024-09-17T15:35:18.306905+0000","mapping_epoch":46,"log_start":"43'10200","ondisk_log_start":"43'10200","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2162688,"num_objects":528,"num_object_clones":0,"num_object_copies":1056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17970,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20464","reported_seq":20772,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:19.208089+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:19.208089+0000","last_peered":"2024-09-17T15:35:19.208089+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:19.208089+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10400","ondisk_log_start":"43'10400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2056192,"num_objects":502,"num_object_clones":0,"num_object_copies":1004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17012,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223809+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223809+0000","last_peered":"2024-09-17T15:33:11.223809+0000","last_clean":"2024-09-17T15:33:11.223809+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223809+0000","last_undegraded":"2024-09-17T15:33:11.223809+0000","last_fullsized":"2024-09-17T15:33:11.223809+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20270","reported_seq":26555,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:19.236605+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:35:19.236605+0000","last_peered":"2024-09-17T15:35:19.236605+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:35:19.236605+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"43'10200","ondisk_log_start":"43'10200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2056192,"num_objects":502,"num_object_clones":0,"num_object_copies":1004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1094,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16557,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3066,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-592}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20407","reported_seq":20714,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:19.150684+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:19.150684+0000","last_peered":"2024-09-17T15:35:19.150684+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:19.150684+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10400","ondisk_log_start":"43'10400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1961984,"num_objects":479,"num_object_clones":0,"num_object_copies":958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17194,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3501}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223917+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223917+0000","last_peered":"2024-09-17T15:33:11.223917+0000","last_clean":"2024-09-17T15:33:11.223917+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223917+0000","last_undegraded":"2024-09-17T15:33:11.223917+0000","last_fullsized":"2024-09-17T15:33:11.223917+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223714+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223714+0000","last_peered":"2024-09-17T15:33:11.223714+0000","last_clean":"2024-09-17T15:33:11.223714+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223714+0000","last_undegraded":"2024-09-17T15:33:11.223714+0000","last_fullsized":"2024-09-17T15:33:11.223714+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20107","reported_seq":28492,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:18.858503+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:18.858503+0000","last_peered":"2024-09-17T15:35:18.858503+0000","last_clean":"2024-09-17T15:35:18.858503+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:18.858503+0000","last_undegraded":"2024-09-17T15:35:18.858503+0000","last_fullsized":"2024-09-17T15:35:18.858503+0000","mapping_epoch":44,"log_start":"43'10100","ondisk_log_start":"43'10100","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2151366,"num_objects":527,"num_object_clones":0,"num_object_copies":1054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":527,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16549,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":33563382,"num_objects":8204,"num_object_clones":0,"num_object_copies":16408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33505,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8204,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":284783,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15955,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":269869056,"data_stored":269734457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160648,"ondisk_log_size":160648,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051707,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502060,"kb_used_data":149652,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93869780,"statfs":{"total":96636764160,"available":96122654720,"internally_reserved":0,"allocated":153243648,"data_stored":152942050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,1,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724652,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":494204,"kb_used_data":25184,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93877636,"statfs":{"total":96636764160,"available":96130699264,"internally_reserved":0,"allocated":25788416,"data_stored":25464676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480243371},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,3,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247935,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":542128,"kb_used_data":91228,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93829712,"statfs":{"total":96636764160,"available":96081625088,"internally_reserved":0,"allocated":93417472,"data_stored":93115869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":461696680},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,4,3,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":92037120,"data_stored":92000990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":25243648,"data_stored":25185381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":152588288,"data_stored":152548086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:35:22.456 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:35:22.456 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'20136", 'reported_seq': 20396, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:19.043440+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:35:19.043440+0000', 'last_peered': '2024-09-17T15:35:19.043440+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:35:19.043440+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'10100", 'ondisk_log_start': "43'10100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2097152, 'num_objects': 512, 'num_object_clones': 0, 'num_object_copies': 1024, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 512, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20136, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3472}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:22.456 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:35:22.456 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'20376", 'reported_seq': 20599, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:18.403981+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:35:18.403981+0000', 'last_peered': '2024-09-17T15:35:18.403981+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:35:18.403981+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "43'10300", 'ondisk_log_start': "43'10300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2281496, 'num_objects': 558, 'num_object_clones': 0, 'num_object_copies': 1116, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 558, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18533, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3567}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:35:22.456 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:35:22.456 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'20734", 'reported_seq': 21035, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:19.195568+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:35:19.195568+0000', 'last_peered': '2024-09-17T15:35:19.195568+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:35:19.195568+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "43'10700", 'ondisk_log_start': "43'10700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2118115, 'num_objects': 518, 'num_object_clones': 0, 'num_object_copies': 1036, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 518, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17233, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3674}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:22.457 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:35:22.457 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'20291", 'reported_seq': 20610, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:19.182959+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:35:19.182959+0000', 'last_peered': '2024-09-17T15:35:19.182959+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:35:19.182959+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "43'10200", 'ondisk_log_start': "43'10200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1954758, 'num_objects': 479, 'num_object_clones': 0, 'num_object_copies': 958, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4089, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 479, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17060, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 85, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3610}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:22.457 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:35:22.457 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20046", 'reported_seq': 20321, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:19.164644+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:35:19.164644+0000', 'last_peered': '2024-09-17T15:35:19.164644+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:35:19.164644+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10000", 'ondisk_log_start': "43'10000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2031616, 'num_objects': 496, 'num_object_clones': 0, 'num_object_copies': 992, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 496, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16642, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3447}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:22.457 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:35:22.457 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'20013", 'reported_seq': 20230, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:18.382784+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:35:18.382784+0000', 'last_peered': '2024-09-17T15:35:18.382784+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:35:18.382784+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "43'10000", 'ondisk_log_start': "43'10000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2073059, 'num_objects': 507, 'num_object_clones': 0, 'num_object_copies': 1014, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4034, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 507, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17908, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -3526}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:35:22.457 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:35:22.457 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'20464", 'reported_seq': 20772, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:19.208089+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:35:19.208089+0000', 'last_peered': '2024-09-17T15:35:19.208089+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:35:19.208089+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "43'10400", 'ondisk_log_start': "43'10400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2056192, 'num_objects': 502, 'num_object_clones': 0, 'num_object_copies': 1004, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 502, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17012, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3555}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:22.457 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:35:22.457 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'20270", 'reported_seq': 26555, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:19.236605+0000', 'last_change': '2024-09-17T15:33:09.763500+0000', 'last_active': '2024-09-17T15:35:19.236605+0000', 'last_peered': '2024-09-17T15:35:19.236605+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:35:19.236605+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "43'10200", 'ondisk_log_start': "43'10200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2056192, 'num_objects': 502, 'num_object_clones': 0, 'num_object_copies': 1004, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1094, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 502, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16557, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3066, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -592}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:22.458 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:35:22.458 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'20407", 'reported_seq': 20714, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:19.150684+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:35:19.150684+0000', 'last_peered': '2024-09-17T15:35:19.150684+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:35:19.150684+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'10400", 'ondisk_log_start': "43'10400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1961984, 'num_objects': 479, 'num_object_clones': 0, 'num_object_copies': 958, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 479, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17194, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3501}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:22.458 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:35:23.142 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:35:23.142 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33505/16412 objects degraded (204.149%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":660,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":8206,"data_bytes":33563401,"bytes_used":1575313408,"bytes_avail":288334979072,"bytes_total":289910292480,"degraded_objects":33505,"degraded_total":16412,"degraded_ratio":2.0414940287594443,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":405},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:35:23.170 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:35:23.171 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:35:23.728 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:35:23.728 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:35:23.759 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":369,"stamp":"2024-09-17T15:35:21.805672+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":33563401,"num_objects":8206,"num_object_clones":0,"num_object_copies":16412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33505,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8206,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":284785,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15956,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160651,"ondisk_log_size":160651,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1538392,"kb_used_data":266064,"kb_used_omap":13,"kb_used_meta":1272242,"kb_avail":281577128,"statfs":{"total":289910292480,"available":288334979072,"internally_reserved":0,"allocated":272449536,"data_stored":271522595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1302776831},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,4,8,12,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19951616,"num_objects":-4871,"num_object_clones":0,"num_object_copies":-9742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-300,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4871,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4871,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005734"},"pg_stats":[{"pgid":"2.f","version":"47'20136","reported_seq":20396,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:19.043440+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:19.043440+0000","last_peered":"2024-09-17T15:35:19.043440+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:19.043440+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10100","ondisk_log_start":"43'10100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":512,"num_object_clones":0,"num_object_copies":1024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20136,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3472}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20384","reported_seq":20497,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:18.303847+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:18.303847+0000","last_peered":"2024-09-17T15:35:18.303847+0000","last_clean":"2024-09-17T15:35:18.303847+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:18.303847+0000","last_undegraded":"2024-09-17T15:35:18.303847+0000","last_fullsized":"2024-09-17T15:35:18.303847+0000","mapping_epoch":33,"log_start":"43'10300","ondisk_log_start":"43'10300","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2245574,"num_objects":550,"num_object_clones":0,"num_object_copies":1100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":550,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18741,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20376","reported_seq":20599,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:18.403981+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:18.403981+0000","last_peered":"2024-09-17T15:35:18.403981+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:18.403981+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10300","ondisk_log_start":"43'10300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2281496,"num_objects":558,"num_object_clones":0,"num_object_copies":1116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":558,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18533,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3567}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20452","reported_seq":20532,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:18.407692+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:18.407692+0000","last_peered":"2024-09-17T15:35:18.407692+0000","last_clean":"2024-09-17T15:35:18.407692+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:18.407692+0000","last_undegraded":"2024-09-17T15:35:18.407692+0000","last_fullsized":"2024-09-17T15:35:18.407692+0000","mapping_epoch":33,"log_start":"43'10400","ondisk_log_start":"43'10400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2121728,"num_objects":518,"num_object_clones":0,"num_object_copies":1036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20452,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20139","reported_seq":20279,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:19.155916+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:19.155916+0000","last_peered":"2024-09-17T15:35:19.155916+0000","last_clean":"2024-09-17T15:35:19.155916+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:19.155916+0000","last_undegraded":"2024-09-17T15:35:19.155916+0000","last_fullsized":"2024-09-17T15:35:19.155916+0000","mapping_epoch":32,"log_start":"43'10100","ondisk_log_start":"43'10100","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2105827,"num_objects":515,"num_object_clones":0,"num_object_copies":1030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":515,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17121,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'20734","reported_seq":21035,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:19.195568+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:19.195568+0000","last_peered":"2024-09-17T15:35:19.195568+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:19.195568+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2118115,"num_objects":518,"num_object_clones":0,"num_object_copies":1036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":518,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17233,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3674},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20291","reported_seq":20610,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:19.182959+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:35:19.182959+0000","last_peered":"2024-09-17T15:35:19.182959+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:35:19.182959+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'10200","ondisk_log_start":"43'10200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1954758,"num_objects":479,"num_object_clones":0,"num_object_copies":958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":479,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17060,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3610},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20420","reported_seq":20556,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:19.235478+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:19.235478+0000","last_peered":"2024-09-17T15:35:19.235478+0000","last_clean":"2024-09-17T15:35:19.235478+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:19.235478+0000","last_undegraded":"2024-09-17T15:35:19.235478+0000","last_fullsized":"2024-09-17T15:35:19.235478+0000","mapping_epoch":32,"log_start":"43'10400","ondisk_log_start":"43'10400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":496,"num_object_clones":0,"num_object_copies":992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17374,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.224142+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.224142+0000","last_peered":"2024-09-17T15:33:11.224142+0000","last_clean":"2024-09-17T15:33:11.224142+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.224142+0000","last_undegraded":"2024-09-17T15:33:11.224142+0000","last_fullsized":"2024-09-17T15:33:11.224142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20301","reported_seq":28594,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:18.884456+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:18.884456+0000","last_peered":"2024-09-17T15:35:18.884456+0000","last_clean":"2024-09-17T15:35:18.884456+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:18.884456+0000","last_undegraded":"2024-09-17T15:35:18.884456+0000","last_fullsized":"2024-09-17T15:35:18.884456+0000","mapping_epoch":42,"log_start":"43'10300","ondisk_log_start":"43'10300","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2114019,"num_objects":517,"num_object_clones":0,"num_object_copies":1034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":517,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18301,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20046","reported_seq":20321,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:19.164644+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:19.164644+0000","last_peered":"2024-09-17T15:35:19.164644+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:19.164644+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10000","ondisk_log_start":"43'10000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":496,"num_object_clones":0,"num_object_copies":992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16642,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3447}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20013","reported_seq":20230,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:18.382784+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:35:18.382784+0000","last_peered":"2024-09-17T15:35:18.382784+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:35:18.382784+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'10000","ondisk_log_start":"43'10000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2073059,"num_objects":507,"num_object_clones":0,"num_object_copies":1014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":507,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3526}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":88,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223926+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223926+0000","last_peered":"2024-09-17T15:33:11.223926+0000","last_clean":"2024-09-17T15:33:11.223926+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223926+0000","last_undegraded":"2024-09-17T15:33:11.223926+0000","last_fullsized":"2024-09-17T15:33:11.223926+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20208","reported_seq":28623,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:18.306905+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:18.306905+0000","last_peered":"2024-09-17T15:35:18.306905+0000","last_clean":"2024-09-17T15:35:18.306905+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:18.306905+0000","last_undegraded":"2024-09-17T15:35:18.306905+0000","last_fullsized":"2024-09-17T15:35:18.306905+0000","mapping_epoch":46,"log_start":"43'10200","ondisk_log_start":"43'10200","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2162688,"num_objects":528,"num_object_clones":0,"num_object_copies":1056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17970,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20464","reported_seq":20772,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:19.208089+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:19.208089+0000","last_peered":"2024-09-17T15:35:19.208089+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:19.208089+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10400","ondisk_log_start":"43'10400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2056192,"num_objects":502,"num_object_clones":0,"num_object_copies":1004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17012,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223809+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223809+0000","last_peered":"2024-09-17T15:33:11.223809+0000","last_clean":"2024-09-17T15:33:11.223809+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223809+0000","last_undegraded":"2024-09-17T15:33:11.223809+0000","last_fullsized":"2024-09-17T15:33:11.223809+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20270","reported_seq":26555,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:19.236605+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:35:19.236605+0000","last_peered":"2024-09-17T15:35:19.236605+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:35:19.236605+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"43'10200","ondisk_log_start":"43'10200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2056192,"num_objects":502,"num_object_clones":0,"num_object_copies":1004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1094,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16557,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3066,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-592}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20407","reported_seq":20714,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:19.150684+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:19.150684+0000","last_peered":"2024-09-17T15:35:19.150684+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:19.150684+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10400","ondisk_log_start":"43'10400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1961984,"num_objects":479,"num_object_clones":0,"num_object_copies":958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17194,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3501}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223917+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223917+0000","last_peered":"2024-09-17T15:33:11.223917+0000","last_clean":"2024-09-17T15:33:11.223917+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223917+0000","last_undegraded":"2024-09-17T15:33:11.223917+0000","last_fullsized":"2024-09-17T15:33:11.223917+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223714+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223714+0000","last_peered":"2024-09-17T15:33:11.223714+0000","last_clean":"2024-09-17T15:33:11.223714+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223714+0000","last_undegraded":"2024-09-17T15:33:11.223714+0000","last_fullsized":"2024-09-17T15:33:11.223714+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20107","reported_seq":28492,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:18.858503+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:18.858503+0000","last_peered":"2024-09-17T15:35:18.858503+0000","last_clean":"2024-09-17T15:35:18.858503+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:18.858503+0000","last_undegraded":"2024-09-17T15:35:18.858503+0000","last_fullsized":"2024-09-17T15:35:18.858503+0000","mapping_epoch":44,"log_start":"43'10100","ondisk_log_start":"43'10100","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2151366,"num_objects":527,"num_object_clones":0,"num_object_copies":1054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":527,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16549,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":33563382,"num_objects":8204,"num_object_clones":0,"num_object_copies":16408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33505,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8204,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":284783,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15955,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":269869056,"data_stored":269734457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160648,"ondisk_log_size":160648,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051707,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502060,"kb_used_data":149652,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93869780,"statfs":{"total":96636764160,"available":96122654720,"internally_reserved":0,"allocated":153243648,"data_stored":152942050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,1,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724652,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":494204,"kb_used_data":25184,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93877636,"statfs":{"total":96636764160,"available":96130699264,"internally_reserved":0,"allocated":25788416,"data_stored":25464676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480243371},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,3,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247935,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":542128,"kb_used_data":91228,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93829712,"statfs":{"total":96636764160,"available":96081625088,"internally_reserved":0,"allocated":93417472,"data_stored":93115869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":461696680},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,4,3,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":92037120,"data_stored":92000990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":25243648,"data_stored":25185381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":152588288,"data_stored":152548086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:35:26.762 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:35:27.339 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:35:27.340 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:35:27.362 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":371,"stamp":"2024-09-17T15:35:25.807280+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":23384841,"num_objects":5721,"num_object_clones":0,"num_object_copies":11442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33361,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5721,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":287270,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16100,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1536592,"kb_used_data":247644,"kb_used_omap":13,"kb_used_meta":1288882,"kb_avail":281578928,"statfs":{"total":289910292480,"available":288336822272,"internally_reserved":0,"allocated":253587456,"data_stored":252660515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1319816191},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,2,3,8,7,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26963968,"num_objects":-6583,"num_object_clones":0,"num_object_copies":-13166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6583,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006413"},"pg_stats":[{"pgid":"2.f","version":"47'20279","reported_seq":20539,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:24.228423+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:24.228423+0000","last_peered":"2024-09-17T15:35:24.228423+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:24.228423+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10200","ondisk_log_start":"43'10200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511424,"num_objects":369,"num_object_clones":0,"num_object_copies":738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20279,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20529","reported_seq":20641,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:23.425447+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:23.425447+0000","last_peered":"2024-09-17T15:35:23.425447+0000","last_clean":"2024-09-17T15:35:23.425447+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:23.425447+0000","last_undegraded":"2024-09-17T15:35:23.425447+0000","last_fullsized":"2024-09-17T15:35:23.425447+0000","mapping_epoch":33,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1651654,"num_objects":405,"num_object_clones":0,"num_object_copies":810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":405,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18886,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20539","reported_seq":20762,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:23.394082+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:23.394082+0000","last_peered":"2024-09-17T15:35:23.394082+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:23.394082+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613848,"num_objects":395,"num_object_clones":0,"num_object_copies":790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":395,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18696,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3730}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20621","reported_seq":20700,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:23.417040+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:23.417040+0000","last_peered":"2024-09-17T15:35:23.417040+0000","last_clean":"2024-09-17T15:35:23.417040+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:23.417040+0000","last_undegraded":"2024-09-17T15:35:23.417040+0000","last_fullsized":"2024-09-17T15:35:23.417040+0000","mapping_epoch":33,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1429504,"num_objects":349,"num_object_clones":0,"num_object_copies":698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":349,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20621,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20297","reported_seq":20437,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:24.190126+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:24.190126+0000","last_peered":"2024-09-17T15:35:24.190126+0000","last_clean":"2024-09-17T15:35:24.190126+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:24.190126+0000","last_undegraded":"2024-09-17T15:35:24.190126+0000","last_fullsized":"2024-09-17T15:35:24.190126+0000","mapping_epoch":32,"log_start":"43'10200","ondisk_log_start":"43'10200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1458659,"num_objects":357,"num_object_clones":0,"num_object_copies":714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":357,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17279,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'20883","reported_seq":21183,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:24.272965+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:24.272965+0000","last_peered":"2024-09-17T15:35:24.272965+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:24.272965+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507811,"num_objects":369,"num_object_clones":0,"num_object_copies":738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":369,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17382,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3823},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20437","reported_seq":20756,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:24.271453+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:35:24.271453+0000","last_peered":"2024-09-17T15:35:24.271453+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:35:24.271453+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'10400","ondisk_log_start":"43'10400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1356742,"num_objects":333,"num_object_clones":0,"num_object_copies":666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":333,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17206,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3756},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20583","reported_seq":20721,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:24.230348+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:24.230348+0000","last_peered":"2024-09-17T15:35:24.230348+0000","last_clean":"2024-09-17T15:35:24.230348+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:24.230348+0000","last_undegraded":"2024-09-17T15:35:24.230348+0000","last_fullsized":"2024-09-17T15:35:24.230348+0000","mapping_epoch":32,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1363968,"num_objects":333,"num_object_clones":0,"num_object_copies":666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17537,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.224142+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.224142+0000","last_peered":"2024-09-17T15:33:11.224142+0000","last_clean":"2024-09-17T15:33:11.224142+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.224142+0000","last_undegraded":"2024-09-17T15:33:11.224142+0000","last_fullsized":"2024-09-17T15:33:11.224142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20453","reported_seq":28746,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:23.884995+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:23.884995+0000","last_peered":"2024-09-17T15:35:23.884995+0000","last_clean":"2024-09-17T15:35:23.884995+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:23.884995+0000","last_undegraded":"2024-09-17T15:35:23.884995+0000","last_fullsized":"2024-09-17T15:35:23.884995+0000","mapping_epoch":42,"log_start":"43'10400","ondisk_log_start":"43'10400","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1491427,"num_objects":365,"num_object_clones":0,"num_object_copies":730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":365,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18453,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20201","reported_seq":20476,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:24.237080+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:24.237080+0000","last_peered":"2024-09-17T15:35:24.237080+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:24.237080+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10200","ondisk_log_start":"43'10200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1396736,"num_objects":341,"num_object_clones":0,"num_object_copies":682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16797,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3602}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20167","reported_seq":20386,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:23.381142+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:35:23.381142+0000","last_peered":"2024-09-17T15:35:23.381142+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:35:23.381142+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'10100","ondisk_log_start":"43'10100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1442275,"num_objects":353,"num_object_clones":0,"num_object_copies":706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":353,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18062,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3680}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":88,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223926+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223926+0000","last_peered":"2024-09-17T15:33:11.223926+0000","last_clean":"2024-09-17T15:33:11.223926+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223926+0000","last_undegraded":"2024-09-17T15:33:11.223926+0000","last_fullsized":"2024-09-17T15:33:11.223926+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20359","reported_seq":28774,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:23.281743+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:23.281743+0000","last_peered":"2024-09-17T15:35:23.281743+0000","last_clean":"2024-09-17T15:35:23.281743+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:23.281743+0000","last_undegraded":"2024-09-17T15:35:23.281743+0000","last_fullsized":"2024-09-17T15:35:23.281743+0000","mapping_epoch":46,"log_start":"43'10300","ondisk_log_start":"43'10300","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1544192,"num_objects":377,"num_object_clones":0,"num_object_copies":754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18121,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20617","reported_seq":20925,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:24.220269+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:24.220269+0000","last_peered":"2024-09-17T15:35:24.220269+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:24.220269+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1429504,"num_objects":349,"num_object_clones":0,"num_object_copies":698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":349,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17165,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3708}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223809+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223809+0000","last_peered":"2024-09-17T15:33:11.223809+0000","last_clean":"2024-09-17T15:33:11.223809+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223809+0000","last_undegraded":"2024-09-17T15:33:11.223809+0000","last_fullsized":"2024-09-17T15:33:11.223809+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20438","reported_seq":27011,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:24.262461+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:35:24.262461+0000","last_peered":"2024-09-17T15:35:24.262461+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:35:24.262461+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"43'10400","ondisk_log_start":"43'10400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1368064,"num_objects":334,"num_object_clones":0,"num_object_copies":668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":950,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16725,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3210,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-616}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20549","reported_seq":20857,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:24.166372+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:24.166372+0000","last_peered":"2024-09-17T15:35:24.166372+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:24.166372+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1380352,"num_objects":337,"num_object_clones":0,"num_object_copies":674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17336,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3643}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223917+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223917+0000","last_peered":"2024-09-17T15:33:11.223917+0000","last_clean":"2024-09-17T15:33:11.223917+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223917+0000","last_undegraded":"2024-09-17T15:33:11.223917+0000","last_fullsized":"2024-09-17T15:33:11.223917+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223714+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223714+0000","last_peered":"2024-09-17T15:33:11.223714+0000","last_clean":"2024-09-17T15:33:11.223714+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223714+0000","last_undegraded":"2024-09-17T15:33:11.223714+0000","last_fullsized":"2024-09-17T15:33:11.223714+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20281","reported_seq":28666,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:23.873644+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:23.873644+0000","last_peered":"2024-09-17T15:35:23.873644+0000","last_clean":"2024-09-17T15:35:23.873644+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:23.873644+0000","last_undegraded":"2024-09-17T15:35:23.873644+0000","last_fullsized":"2024-09-17T15:35:23.873644+0000","mapping_epoch":44,"log_start":"43'10200","ondisk_log_start":"43'10200","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1438662,"num_objects":353,"num_object_clones":0,"num_object_copies":706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":353,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16723,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":23384822,"num_objects":5719,"num_object_clones":0,"num_object_copies":11438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33361,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5719,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":287268,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16099,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":249552896,"data_stored":249418297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160833,"ondisk_log_size":160833,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051708,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501276,"kb_used_data":144716,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93870564,"statfs":{"total":96636764160,"available":96123457536,"internally_reserved":0,"allocated":148189184,"data_stored":147887586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":365096620},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724653,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":496348,"kb_used_data":19020,"kb_used_omap":4,"kb_used_meta":477307,"kb_avail":93875492,"statfs":{"total":96636764160,"available":96128503808,"internally_reserved":0,"allocated":19476480,"data_stored":19152740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":488763051},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247936,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":538968,"kb_used_data":83908,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93832872,"statfs":{"total":96636764160,"available":96084860928,"internally_reserved":0,"allocated":85921792,"data_stored":85620189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":465956520},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,1,2,3,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":84410368,"data_stored":84374238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":17686528,"data_stored":17628261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":147456000,"data_stored":147415798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:35:27.364 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:35:27.921 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:35:27.922 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:35:27.947 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":372,"stamp":"2024-09-17T15:35:27.807980+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":23384841,"num_objects":5721,"num_object_clones":0,"num_object_copies":11442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33361,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5721,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":287270,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16100,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1536592,"kb_used_data":247644,"kb_used_omap":13,"kb_used_meta":1288882,"kb_avail":281578928,"statfs":{"total":289910292480,"available":288336822272,"internally_reserved":0,"allocated":253587456,"data_stored":252660515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1319816191},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,2,3,8,7,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20086784,"num_objects":-4904,"num_object_clones":0,"num_object_copies":-9808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-294,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4904,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":294,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006439"},"pg_stats":[{"pgid":"2.f","version":"47'20279","reported_seq":20539,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:24.228423+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:24.228423+0000","last_peered":"2024-09-17T15:35:24.228423+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:24.228423+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10200","ondisk_log_start":"43'10200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511424,"num_objects":369,"num_object_clones":0,"num_object_copies":738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20279,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20529","reported_seq":20641,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:23.425447+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:23.425447+0000","last_peered":"2024-09-17T15:35:23.425447+0000","last_clean":"2024-09-17T15:35:23.425447+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:23.425447+0000","last_undegraded":"2024-09-17T15:35:23.425447+0000","last_fullsized":"2024-09-17T15:35:23.425447+0000","mapping_epoch":33,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1651654,"num_objects":405,"num_object_clones":0,"num_object_copies":810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":405,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18886,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20539","reported_seq":20762,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:23.394082+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:23.394082+0000","last_peered":"2024-09-17T15:35:23.394082+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:23.394082+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613848,"num_objects":395,"num_object_clones":0,"num_object_copies":790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":395,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18696,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3730}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20621","reported_seq":20700,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:23.417040+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:23.417040+0000","last_peered":"2024-09-17T15:35:23.417040+0000","last_clean":"2024-09-17T15:35:23.417040+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:23.417040+0000","last_undegraded":"2024-09-17T15:35:23.417040+0000","last_fullsized":"2024-09-17T15:35:23.417040+0000","mapping_epoch":33,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1429504,"num_objects":349,"num_object_clones":0,"num_object_copies":698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":349,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20621,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20297","reported_seq":20437,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:24.190126+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:24.190126+0000","last_peered":"2024-09-17T15:35:24.190126+0000","last_clean":"2024-09-17T15:35:24.190126+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:24.190126+0000","last_undegraded":"2024-09-17T15:35:24.190126+0000","last_fullsized":"2024-09-17T15:35:24.190126+0000","mapping_epoch":32,"log_start":"43'10200","ondisk_log_start":"43'10200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1458659,"num_objects":357,"num_object_clones":0,"num_object_copies":714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":357,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17279,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'20883","reported_seq":21183,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:24.272965+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:24.272965+0000","last_peered":"2024-09-17T15:35:24.272965+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:24.272965+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507811,"num_objects":369,"num_object_clones":0,"num_object_copies":738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":369,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17382,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3823},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20437","reported_seq":20756,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:24.271453+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:35:24.271453+0000","last_peered":"2024-09-17T15:35:24.271453+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:35:24.271453+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'10400","ondisk_log_start":"43'10400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1356742,"num_objects":333,"num_object_clones":0,"num_object_copies":666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":333,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17206,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3756},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20583","reported_seq":20721,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:24.230348+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:24.230348+0000","last_peered":"2024-09-17T15:35:24.230348+0000","last_clean":"2024-09-17T15:35:24.230348+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:24.230348+0000","last_undegraded":"2024-09-17T15:35:24.230348+0000","last_fullsized":"2024-09-17T15:35:24.230348+0000","mapping_epoch":32,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1363968,"num_objects":333,"num_object_clones":0,"num_object_copies":666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17537,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.224142+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.224142+0000","last_peered":"2024-09-17T15:33:11.224142+0000","last_clean":"2024-09-17T15:33:11.224142+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.224142+0000","last_undegraded":"2024-09-17T15:33:11.224142+0000","last_fullsized":"2024-09-17T15:33:11.224142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20453","reported_seq":28746,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:23.884995+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:23.884995+0000","last_peered":"2024-09-17T15:35:23.884995+0000","last_clean":"2024-09-17T15:35:23.884995+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:23.884995+0000","last_undegraded":"2024-09-17T15:35:23.884995+0000","last_fullsized":"2024-09-17T15:35:23.884995+0000","mapping_epoch":42,"log_start":"43'10400","ondisk_log_start":"43'10400","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1491427,"num_objects":365,"num_object_clones":0,"num_object_copies":730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":365,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18453,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20201","reported_seq":20476,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:24.237080+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:24.237080+0000","last_peered":"2024-09-17T15:35:24.237080+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:24.237080+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10200","ondisk_log_start":"43'10200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1396736,"num_objects":341,"num_object_clones":0,"num_object_copies":682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16797,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3602}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20167","reported_seq":20386,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:23.381142+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:35:23.381142+0000","last_peered":"2024-09-17T15:35:23.381142+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:35:23.381142+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'10100","ondisk_log_start":"43'10100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1442275,"num_objects":353,"num_object_clones":0,"num_object_copies":706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":353,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18062,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3680}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":88,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223926+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223926+0000","last_peered":"2024-09-17T15:33:11.223926+0000","last_clean":"2024-09-17T15:33:11.223926+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223926+0000","last_undegraded":"2024-09-17T15:33:11.223926+0000","last_fullsized":"2024-09-17T15:33:11.223926+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20359","reported_seq":28774,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:23.281743+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:23.281743+0000","last_peered":"2024-09-17T15:35:23.281743+0000","last_clean":"2024-09-17T15:35:23.281743+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:23.281743+0000","last_undegraded":"2024-09-17T15:35:23.281743+0000","last_fullsized":"2024-09-17T15:35:23.281743+0000","mapping_epoch":46,"log_start":"43'10300","ondisk_log_start":"43'10300","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1544192,"num_objects":377,"num_object_clones":0,"num_object_copies":754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18121,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20617","reported_seq":20925,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:24.220269+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:24.220269+0000","last_peered":"2024-09-17T15:35:24.220269+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:24.220269+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1429504,"num_objects":349,"num_object_clones":0,"num_object_copies":698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":349,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17165,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3708}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223809+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223809+0000","last_peered":"2024-09-17T15:33:11.223809+0000","last_clean":"2024-09-17T15:33:11.223809+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223809+0000","last_undegraded":"2024-09-17T15:33:11.223809+0000","last_fullsized":"2024-09-17T15:33:11.223809+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20438","reported_seq":27011,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:24.262461+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:35:24.262461+0000","last_peered":"2024-09-17T15:35:24.262461+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:35:24.262461+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"43'10400","ondisk_log_start":"43'10400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1368064,"num_objects":334,"num_object_clones":0,"num_object_copies":668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":950,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16725,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3210,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-616}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20549","reported_seq":20857,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:24.166372+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:24.166372+0000","last_peered":"2024-09-17T15:35:24.166372+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:24.166372+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1380352,"num_objects":337,"num_object_clones":0,"num_object_copies":674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17336,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3643}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223917+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223917+0000","last_peered":"2024-09-17T15:33:11.223917+0000","last_clean":"2024-09-17T15:33:11.223917+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223917+0000","last_undegraded":"2024-09-17T15:33:11.223917+0000","last_fullsized":"2024-09-17T15:33:11.223917+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223714+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223714+0000","last_peered":"2024-09-17T15:33:11.223714+0000","last_clean":"2024-09-17T15:33:11.223714+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223714+0000","last_undegraded":"2024-09-17T15:33:11.223714+0000","last_fullsized":"2024-09-17T15:33:11.223714+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20281","reported_seq":28666,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:23.873644+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:23.873644+0000","last_peered":"2024-09-17T15:35:23.873644+0000","last_clean":"2024-09-17T15:35:23.873644+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:23.873644+0000","last_undegraded":"2024-09-17T15:35:23.873644+0000","last_fullsized":"2024-09-17T15:35:23.873644+0000","mapping_epoch":44,"log_start":"43'10200","ondisk_log_start":"43'10200","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1438662,"num_objects":353,"num_object_clones":0,"num_object_copies":706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":353,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16723,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":23384822,"num_objects":5719,"num_object_clones":0,"num_object_copies":11438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33361,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5719,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":287268,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16099,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":249552896,"data_stored":249418297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160833,"ondisk_log_size":160833,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051708,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501276,"kb_used_data":144716,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93870564,"statfs":{"total":96636764160,"available":96123457536,"internally_reserved":0,"allocated":148189184,"data_stored":147887586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":365096620},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724653,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":496348,"kb_used_data":19020,"kb_used_omap":4,"kb_used_meta":477307,"kb_avail":93875492,"statfs":{"total":96636764160,"available":96128503808,"internally_reserved":0,"allocated":19476480,"data_stored":19152740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":488763051},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247936,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":538968,"kb_used_data":83908,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93832872,"statfs":{"total":96636764160,"available":96084860928,"internally_reserved":0,"allocated":85921792,"data_stored":85620189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":465956520},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,1,2,3,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":84410368,"data_stored":84374238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":17686528,"data_stored":17628261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":147456000,"data_stored":147415798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:35:27.949 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:35:27.950 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'20279", 'reported_seq': 20539, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:24.228423+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:35:24.228423+0000', 'last_peered': '2024-09-17T15:35:24.228423+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:35:24.228423+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'10200", 'ondisk_log_start': "43'10200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1511424, 'num_objects': 369, 'num_object_clones': 0, 'num_object_copies': 738, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 369, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20279, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3615}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:27.950 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:35:27.950 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'20539", 'reported_seq': 20762, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:23.394082+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:35:23.394082+0000', 'last_peered': '2024-09-17T15:35:23.394082+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:35:23.394082+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1613848, 'num_objects': 395, 'num_object_clones': 0, 'num_object_copies': 790, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 395, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18696, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3730}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:35:27.950 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:35:27.951 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'20883", 'reported_seq': 21183, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:24.272965+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:35:24.272965+0000', 'last_peered': '2024-09-17T15:35:24.272965+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:35:24.272965+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "43'10800", 'ondisk_log_start': "43'10800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1507811, 'num_objects': 369, 'num_object_clones': 0, 'num_object_copies': 738, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 369, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17382, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3823}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:27.951 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:35:27.951 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'20437", 'reported_seq': 20756, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:24.271453+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:35:24.271453+0000', 'last_peered': '2024-09-17T15:35:24.271453+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:35:24.271453+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "43'10400", 'ondisk_log_start': "43'10400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1356742, 'num_objects': 333, 'num_object_clones': 0, 'num_object_copies': 666, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4089, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 333, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17206, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 85, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3756}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:27.951 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:35:27.952 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20201", 'reported_seq': 20476, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:24.237080+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:35:24.237080+0000', 'last_peered': '2024-09-17T15:35:24.237080+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:35:24.237080+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10200", 'ondisk_log_start': "43'10200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1396736, 'num_objects': 341, 'num_object_clones': 0, 'num_object_copies': 682, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 341, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16797, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3602}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:27.952 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:35:27.952 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'20167", 'reported_seq': 20386, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:23.381142+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:35:23.381142+0000', 'last_peered': '2024-09-17T15:35:23.381142+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:35:23.381142+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "43'10100", 'ondisk_log_start': "43'10100", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1442275, 'num_objects': 353, 'num_object_clones': 0, 'num_object_copies': 706, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4034, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 353, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18062, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -3680}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:35:27.952 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:35:27.952 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'20617", 'reported_seq': 20925, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:24.220269+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:35:24.220269+0000', 'last_peered': '2024-09-17T15:35:24.220269+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:35:24.220269+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "43'10600", 'ondisk_log_start': "43'10600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1429504, 'num_objects': 349, 'num_object_clones': 0, 'num_object_copies': 698, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 349, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17165, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3708}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:27.953 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:35:27.953 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'20438", 'reported_seq': 27011, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:24.262461+0000', 'last_change': '2024-09-17T15:33:09.763500+0000', 'last_active': '2024-09-17T15:35:24.262461+0000', 'last_peered': '2024-09-17T15:35:24.262461+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:35:24.262461+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "43'10400", 'ondisk_log_start': "43'10400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1368064, 'num_objects': 334, 'num_object_clones': 0, 'num_object_copies': 668, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 950, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 334, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16725, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3210, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -616}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:27.953 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:35:27.953 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'20549", 'reported_seq': 20857, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:24.166372+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:35:24.166372+0000', 'last_peered': '2024-09-17T15:35:24.166372+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:35:24.166372+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1380352, 'num_objects': 337, 'num_object_clones': 0, 'num_object_copies': 674, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 337, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17336, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3643}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:27.954 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:35:28.645 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:35:28.645 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33361/11442 objects degraded (291.566%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":665,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":5721,"data_bytes":23384841,"bytes_used":1573470208,"bytes_avail":288336822272,"bytes_total":289910292480,"degraded_objects":33361,"degraded_total":11442,"degraded_ratio":2.9156615976227931,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":294,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":408},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:35:28.667 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:35:28.668 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:35:29.224 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:35:29.225 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:35:29.255 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":372,"stamp":"2024-09-17T15:35:27.807980+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":23384841,"num_objects":5721,"num_object_clones":0,"num_object_copies":11442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33361,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5721,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":287270,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16100,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1536592,"kb_used_data":247644,"kb_used_omap":13,"kb_used_meta":1288882,"kb_avail":281578928,"statfs":{"total":289910292480,"available":288336822272,"internally_reserved":0,"allocated":253587456,"data_stored":252660515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1319816191},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,2,3,8,7,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20086784,"num_objects":-4904,"num_object_clones":0,"num_object_copies":-9808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-294,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4904,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":294,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006439"},"pg_stats":[{"pgid":"2.f","version":"47'20279","reported_seq":20539,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:24.228423+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:24.228423+0000","last_peered":"2024-09-17T15:35:24.228423+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:24.228423+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10200","ondisk_log_start":"43'10200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511424,"num_objects":369,"num_object_clones":0,"num_object_copies":738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20279,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20529","reported_seq":20641,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:23.425447+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:23.425447+0000","last_peered":"2024-09-17T15:35:23.425447+0000","last_clean":"2024-09-17T15:35:23.425447+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:23.425447+0000","last_undegraded":"2024-09-17T15:35:23.425447+0000","last_fullsized":"2024-09-17T15:35:23.425447+0000","mapping_epoch":33,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1651654,"num_objects":405,"num_object_clones":0,"num_object_copies":810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":405,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18886,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20539","reported_seq":20762,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:23.394082+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:23.394082+0000","last_peered":"2024-09-17T15:35:23.394082+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:23.394082+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613848,"num_objects":395,"num_object_clones":0,"num_object_copies":790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":395,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18696,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3730}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20621","reported_seq":20700,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:23.417040+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:23.417040+0000","last_peered":"2024-09-17T15:35:23.417040+0000","last_clean":"2024-09-17T15:35:23.417040+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:23.417040+0000","last_undegraded":"2024-09-17T15:35:23.417040+0000","last_fullsized":"2024-09-17T15:35:23.417040+0000","mapping_epoch":33,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1429504,"num_objects":349,"num_object_clones":0,"num_object_copies":698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":349,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20621,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20297","reported_seq":20437,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:24.190126+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:24.190126+0000","last_peered":"2024-09-17T15:35:24.190126+0000","last_clean":"2024-09-17T15:35:24.190126+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:24.190126+0000","last_undegraded":"2024-09-17T15:35:24.190126+0000","last_fullsized":"2024-09-17T15:35:24.190126+0000","mapping_epoch":32,"log_start":"43'10200","ondisk_log_start":"43'10200","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1458659,"num_objects":357,"num_object_clones":0,"num_object_copies":714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":357,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17279,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'20883","reported_seq":21183,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:24.272965+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:24.272965+0000","last_peered":"2024-09-17T15:35:24.272965+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:24.272965+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507811,"num_objects":369,"num_object_clones":0,"num_object_copies":738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":369,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17382,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3823},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20437","reported_seq":20756,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:24.271453+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:35:24.271453+0000","last_peered":"2024-09-17T15:35:24.271453+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:35:24.271453+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'10400","ondisk_log_start":"43'10400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1356742,"num_objects":333,"num_object_clones":0,"num_object_copies":666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":333,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17206,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3756},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20583","reported_seq":20721,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:24.230348+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:24.230348+0000","last_peered":"2024-09-17T15:35:24.230348+0000","last_clean":"2024-09-17T15:35:24.230348+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:24.230348+0000","last_undegraded":"2024-09-17T15:35:24.230348+0000","last_fullsized":"2024-09-17T15:35:24.230348+0000","mapping_epoch":32,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1363968,"num_objects":333,"num_object_clones":0,"num_object_copies":666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17537,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.224142+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.224142+0000","last_peered":"2024-09-17T15:33:11.224142+0000","last_clean":"2024-09-17T15:33:11.224142+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.224142+0000","last_undegraded":"2024-09-17T15:33:11.224142+0000","last_fullsized":"2024-09-17T15:33:11.224142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20453","reported_seq":28746,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:23.884995+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:23.884995+0000","last_peered":"2024-09-17T15:35:23.884995+0000","last_clean":"2024-09-17T15:35:23.884995+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:23.884995+0000","last_undegraded":"2024-09-17T15:35:23.884995+0000","last_fullsized":"2024-09-17T15:35:23.884995+0000","mapping_epoch":42,"log_start":"43'10400","ondisk_log_start":"43'10400","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1491427,"num_objects":365,"num_object_clones":0,"num_object_copies":730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":365,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18453,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20201","reported_seq":20476,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:24.237080+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:24.237080+0000","last_peered":"2024-09-17T15:35:24.237080+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:24.237080+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10200","ondisk_log_start":"43'10200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1396736,"num_objects":341,"num_object_clones":0,"num_object_copies":682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16797,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3602}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20167","reported_seq":20386,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:23.381142+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:35:23.381142+0000","last_peered":"2024-09-17T15:35:23.381142+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:35:23.381142+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'10100","ondisk_log_start":"43'10100","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1442275,"num_objects":353,"num_object_clones":0,"num_object_copies":706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":353,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18062,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3680}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":88,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223926+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223926+0000","last_peered":"2024-09-17T15:33:11.223926+0000","last_clean":"2024-09-17T15:33:11.223926+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223926+0000","last_undegraded":"2024-09-17T15:33:11.223926+0000","last_fullsized":"2024-09-17T15:33:11.223926+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20359","reported_seq":28774,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:23.281743+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:23.281743+0000","last_peered":"2024-09-17T15:35:23.281743+0000","last_clean":"2024-09-17T15:35:23.281743+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:23.281743+0000","last_undegraded":"2024-09-17T15:35:23.281743+0000","last_fullsized":"2024-09-17T15:35:23.281743+0000","mapping_epoch":46,"log_start":"43'10300","ondisk_log_start":"43'10300","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1544192,"num_objects":377,"num_object_clones":0,"num_object_copies":754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18121,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20617","reported_seq":20925,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:24.220269+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:24.220269+0000","last_peered":"2024-09-17T15:35:24.220269+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:24.220269+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1429504,"num_objects":349,"num_object_clones":0,"num_object_copies":698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":349,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17165,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3708}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223809+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223809+0000","last_peered":"2024-09-17T15:33:11.223809+0000","last_clean":"2024-09-17T15:33:11.223809+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223809+0000","last_undegraded":"2024-09-17T15:33:11.223809+0000","last_fullsized":"2024-09-17T15:33:11.223809+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20438","reported_seq":27011,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:24.262461+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:35:24.262461+0000","last_peered":"2024-09-17T15:35:24.262461+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:35:24.262461+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"43'10400","ondisk_log_start":"43'10400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1368064,"num_objects":334,"num_object_clones":0,"num_object_copies":668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":950,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16725,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3210,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-616}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20549","reported_seq":20857,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:24.166372+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:24.166372+0000","last_peered":"2024-09-17T15:35:24.166372+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:24.166372+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1380352,"num_objects":337,"num_object_clones":0,"num_object_copies":674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17336,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3643}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223917+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223917+0000","last_peered":"2024-09-17T15:33:11.223917+0000","last_clean":"2024-09-17T15:33:11.223917+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223917+0000","last_undegraded":"2024-09-17T15:33:11.223917+0000","last_fullsized":"2024-09-17T15:33:11.223917+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223714+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223714+0000","last_peered":"2024-09-17T15:33:11.223714+0000","last_clean":"2024-09-17T15:33:11.223714+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223714+0000","last_undegraded":"2024-09-17T15:33:11.223714+0000","last_fullsized":"2024-09-17T15:33:11.223714+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20281","reported_seq":28666,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:23.873644+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:23.873644+0000","last_peered":"2024-09-17T15:35:23.873644+0000","last_clean":"2024-09-17T15:35:23.873644+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:23.873644+0000","last_undegraded":"2024-09-17T15:35:23.873644+0000","last_fullsized":"2024-09-17T15:35:23.873644+0000","mapping_epoch":44,"log_start":"43'10200","ondisk_log_start":"43'10200","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1438662,"num_objects":353,"num_object_clones":0,"num_object_copies":706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":353,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16723,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":23384822,"num_objects":5719,"num_object_clones":0,"num_object_copies":11438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33361,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5719,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":287268,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16099,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":249552896,"data_stored":249418297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160833,"ondisk_log_size":160833,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051708,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501276,"kb_used_data":144716,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93870564,"statfs":{"total":96636764160,"available":96123457536,"internally_reserved":0,"allocated":148189184,"data_stored":147887586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":365096620},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724653,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":496348,"kb_used_data":19020,"kb_used_omap":4,"kb_used_meta":477307,"kb_avail":93875492,"statfs":{"total":96636764160,"available":96128503808,"internally_reserved":0,"allocated":19476480,"data_stored":19152740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":488763051},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247936,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":538968,"kb_used_data":83908,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93832872,"statfs":{"total":96636764160,"available":96084860928,"internally_reserved":0,"allocated":85921792,"data_stored":85620189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":465956520},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,1,2,3,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":84410368,"data_stored":84374238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":17686528,"data_stored":17628261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":147456000,"data_stored":147415798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:35:32.258 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:35:32.835 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:35:32.836 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:35:32.858 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":374,"stamp":"2024-09-17T15:35:31.809467+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":13218569,"num_objects":3239,"num_object_clones":0,"num_object_copies":6478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33201,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3239,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":289752,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16260,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160818,"ondisk_log_size":160818,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1500052,"kb_used_data":227732,"kb_used_omap":13,"kb_used_meta":1272242,"kb_avail":281615468,"statfs":{"total":289910292480,"available":288374239232,"internally_reserved":0,"allocated":233197568,"data_stored":232270627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1302776831},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,7,9,12,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20344832,"num_objects":-4967,"num_object_clones":0,"num_object_copies":-9934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-304,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4967,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004647"},"pg_stats":[{"pgid":"2.f","version":"47'20448","reported_seq":20707,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:29.260154+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:29.260154+0000","last_peered":"2024-09-17T15:35:29.260154+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:29.260154+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10400","ondisk_log_start":"43'10400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":819200,"num_objects":200,"num_object_clones":0,"num_object_copies":400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20448,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3784}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20695","reported_seq":20808,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:28.392990+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:28.392990+0000","last_peered":"2024-09-17T15:35:28.392990+0000","last_clean":"2024-09-17T15:35:28.392990+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:28.392990+0000","last_undegraded":"2024-09-17T15:35:28.392990+0000","last_fullsized":"2024-09-17T15:35:28.392990+0000","mapping_epoch":33,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":971718,"num_objects":239,"num_object_clones":0,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19052,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20697","reported_seq":20920,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:28.269236+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:28.269236+0000","last_peered":"2024-09-17T15:35:28.269236+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:28.269236+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966680,"num_objects":237,"num_object_clones":0,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":237,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18854,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3888}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20757","reported_seq":20837,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:28.272222+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:28.272222+0000","last_peered":"2024-09-17T15:35:28.272222+0000","last_clean":"2024-09-17T15:35:28.272222+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:28.272222+0000","last_undegraded":"2024-09-17T15:35:28.272222+0000","last_fullsized":"2024-09-17T15:35:28.272222+0000","mapping_epoch":33,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":872448,"num_objects":213,"num_object_clones":0,"num_object_copies":426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":213,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20757,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20466","reported_seq":20606,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:29.290966+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:29.290966+0000","last_peered":"2024-09-17T15:35:29.290966+0000","last_clean":"2024-09-17T15:35:29.290966+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:29.290966+0000","last_undegraded":"2024-09-17T15:35:29.290966+0000","last_fullsized":"2024-09-17T15:35:29.290966+0000","mapping_epoch":32,"log_start":"43'10400","ondisk_log_start":"43'10400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":766435,"num_objects":188,"num_object_clones":0,"num_object_copies":376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":188,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17448,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21045","reported_seq":21346,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:29.272830+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:29.272830+0000","last_peered":"2024-09-17T15:35:29.272830+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:29.272830+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11000","ondisk_log_start":"43'11000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":844259,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":207,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17544,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3985},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20584","reported_seq":20903,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:29.324577+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:35:29.324577+0000","last_peered":"2024-09-17T15:35:29.324577+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:35:29.324577+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":754630,"num_objects":186,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":186,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17353,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3903},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20735","reported_seq":20873,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:29.312428+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:29.312428+0000","last_peered":"2024-09-17T15:35:29.312428+0000","last_clean":"2024-09-17T15:35:29.312428+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:29.312428+0000","last_undegraded":"2024-09-17T15:35:29.312428+0000","last_fullsized":"2024-09-17T15:35:29.312428+0000","mapping_epoch":32,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":741376,"num_objects":181,"num_object_clones":0,"num_object_copies":362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17689,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.224142+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.224142+0000","last_peered":"2024-09-17T15:33:11.224142+0000","last_clean":"2024-09-17T15:33:11.224142+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.224142+0000","last_undegraded":"2024-09-17T15:33:11.224142+0000","last_fullsized":"2024-09-17T15:33:11.224142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20602","reported_seq":28895,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:28.904347+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:28.904347+0000","last_peered":"2024-09-17T15:35:28.904347+0000","last_clean":"2024-09-17T15:35:28.904347+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:28.904347+0000","last_undegraded":"2024-09-17T15:35:28.904347+0000","last_fullsized":"2024-09-17T15:35:28.904347+0000","mapping_epoch":42,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":881123,"num_objects":216,"num_object_clones":0,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":216,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18602,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20342","reported_seq":20617,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:29.279835+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:29.279835+0000","last_peered":"2024-09-17T15:35:29.279835+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:29.279835+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10300","ondisk_log_start":"43'10300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":819200,"num_objects":200,"num_object_clones":0,"num_object_copies":400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16938,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3743}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20319","reported_seq":20538,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:28.302647+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:35:28.302647+0000","last_peered":"2024-09-17T15:35:28.302647+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:35:28.302647+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'10300","ondisk_log_start":"43'10300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":819683,"num_objects":201,"num_object_clones":0,"num_object_copies":402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":201,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18214,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3832}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":88,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223926+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223926+0000","last_peered":"2024-09-17T15:33:11.223926+0000","last_clean":"2024-09-17T15:33:11.223926+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223926+0000","last_undegraded":"2024-09-17T15:33:11.223926+0000","last_fullsized":"2024-09-17T15:33:11.223926+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20513","reported_seq":28929,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:28.237586+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:28.237586+0000","last_peered":"2024-09-17T15:35:28.237586+0000","last_clean":"2024-09-17T15:35:28.237586+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:28.237586+0000","last_undegraded":"2024-09-17T15:35:28.237586+0000","last_fullsized":"2024-09-17T15:35:28.237586+0000","mapping_epoch":46,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":913408,"num_objects":223,"num_object_clones":0,"num_object_copies":446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":223,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18275,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20762","reported_seq":21070,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:29.263123+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:29.263123+0000","last_peered":"2024-09-17T15:35:29.263123+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:29.263123+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":835584,"num_objects":204,"num_object_clones":0,"num_object_copies":408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":204,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17310,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3853}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223809+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223809+0000","last_peered":"2024-09-17T15:33:11.223809+0000","last_clean":"2024-09-17T15:33:11.223809+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223809+0000","last_undegraded":"2024-09-17T15:33:11.223809+0000","last_fullsized":"2024-09-17T15:33:11.223809+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20593","reported_seq":27486,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:29.323147+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:35:29.323147+0000","last_peered":"2024-09-17T15:35:29.323147+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:35:29.323147+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":733184,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":790,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16880,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3370,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-611}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20713","reported_seq":21019,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:29.306277+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:29.306277+0000","last_peered":"2024-09-17T15:35:29.306277+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:29.306277+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":708608,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17500,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3807}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223917+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223917+0000","last_peered":"2024-09-17T15:33:11.223917+0000","last_clean":"2024-09-17T15:33:11.223917+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223917+0000","last_undegraded":"2024-09-17T15:33:11.223917+0000","last_fullsized":"2024-09-17T15:33:11.223917+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223714+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223714+0000","last_peered":"2024-09-17T15:33:11.223714+0000","last_clean":"2024-09-17T15:33:11.223714+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223714+0000","last_undegraded":"2024-09-17T15:33:11.223714+0000","last_fullsized":"2024-09-17T15:33:11.223714+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20444","reported_seq":28829,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:28.937167+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:28.937167+0000","last_peered":"2024-09-17T15:35:28.937167+0000","last_clean":"2024-09-17T15:35:28.937167+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:28.937167+0000","last_undegraded":"2024-09-17T15:35:28.937167+0000","last_fullsized":"2024-09-17T15:35:28.937167+0000","mapping_epoch":44,"log_start":"43'10400","ondisk_log_start":"43'10400","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":771014,"num_objects":190,"num_object_clones":0,"num_object_copies":380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16886,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":13218550,"num_objects":3237,"num_object_clones":0,"num_object_copies":6474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33201,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3237,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":289750,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16259,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":229134336,"data_stored":228999737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160815,"ondisk_log_size":160815,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051709,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":500436,"kb_used_data":139704,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93871404,"statfs":{"total":96636764160,"available":96124317696,"internally_reserved":0,"allocated":143056896,"data_stored":142755298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":369356460},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,4,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724654,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476440,"kb_used_data":11588,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93895400,"statfs":{"total":96636764160,"available":96148889600,"internally_reserved":0,"allocated":11866112,"data_stored":11542372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":475983531},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,4,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247937,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523176,"kb_used_data":76440,"kb_used_omap":4,"kb_used_meta":446715,"kb_avail":93848664,"statfs":{"total":96636764160,"available":96101031936,"internally_reserved":0,"allocated":78274560,"data_stored":77972957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":457436840},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,8,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":76738560,"data_stored":76702430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10051584,"data_stored":9993317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":142344192,"data_stored":142303990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:35:32.859 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:35:33.433 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:35:33.436 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:35:33.460 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":374,"stamp":"2024-09-17T15:35:31.809467+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":13218569,"num_objects":3239,"num_object_clones":0,"num_object_copies":6478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33201,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3239,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":289752,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16260,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160818,"ondisk_log_size":160818,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1500052,"kb_used_data":227732,"kb_used_omap":13,"kb_used_meta":1272242,"kb_avail":281615468,"statfs":{"total":289910292480,"available":288374239232,"internally_reserved":0,"allocated":233197568,"data_stored":232270627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1302776831},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,7,9,12,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20344832,"num_objects":-4967,"num_object_clones":0,"num_object_copies":-9934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-304,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4967,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004647"},"pg_stats":[{"pgid":"2.f","version":"47'20448","reported_seq":20707,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:29.260154+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:29.260154+0000","last_peered":"2024-09-17T15:35:29.260154+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:29.260154+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10400","ondisk_log_start":"43'10400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":819200,"num_objects":200,"num_object_clones":0,"num_object_copies":400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20448,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3784}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20695","reported_seq":20808,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:28.392990+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:28.392990+0000","last_peered":"2024-09-17T15:35:28.392990+0000","last_clean":"2024-09-17T15:35:28.392990+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:28.392990+0000","last_undegraded":"2024-09-17T15:35:28.392990+0000","last_fullsized":"2024-09-17T15:35:28.392990+0000","mapping_epoch":33,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":971718,"num_objects":239,"num_object_clones":0,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19052,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20697","reported_seq":20920,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:28.269236+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:28.269236+0000","last_peered":"2024-09-17T15:35:28.269236+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:28.269236+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966680,"num_objects":237,"num_object_clones":0,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":237,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18854,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3888}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20757","reported_seq":20837,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:28.272222+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:28.272222+0000","last_peered":"2024-09-17T15:35:28.272222+0000","last_clean":"2024-09-17T15:35:28.272222+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:28.272222+0000","last_undegraded":"2024-09-17T15:35:28.272222+0000","last_fullsized":"2024-09-17T15:35:28.272222+0000","mapping_epoch":33,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":872448,"num_objects":213,"num_object_clones":0,"num_object_copies":426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":213,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20757,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20466","reported_seq":20606,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:29.290966+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:29.290966+0000","last_peered":"2024-09-17T15:35:29.290966+0000","last_clean":"2024-09-17T15:35:29.290966+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:29.290966+0000","last_undegraded":"2024-09-17T15:35:29.290966+0000","last_fullsized":"2024-09-17T15:35:29.290966+0000","mapping_epoch":32,"log_start":"43'10400","ondisk_log_start":"43'10400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":766435,"num_objects":188,"num_object_clones":0,"num_object_copies":376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":188,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17448,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21045","reported_seq":21346,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:29.272830+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:29.272830+0000","last_peered":"2024-09-17T15:35:29.272830+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:29.272830+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11000","ondisk_log_start":"43'11000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":844259,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":207,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17544,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3985},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20584","reported_seq":20903,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:29.324577+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:35:29.324577+0000","last_peered":"2024-09-17T15:35:29.324577+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:35:29.324577+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":754630,"num_objects":186,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":186,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17353,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3903},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20735","reported_seq":20873,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:29.312428+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:29.312428+0000","last_peered":"2024-09-17T15:35:29.312428+0000","last_clean":"2024-09-17T15:35:29.312428+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:29.312428+0000","last_undegraded":"2024-09-17T15:35:29.312428+0000","last_fullsized":"2024-09-17T15:35:29.312428+0000","mapping_epoch":32,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":741376,"num_objects":181,"num_object_clones":0,"num_object_copies":362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17689,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.224142+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.224142+0000","last_peered":"2024-09-17T15:33:11.224142+0000","last_clean":"2024-09-17T15:33:11.224142+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.224142+0000","last_undegraded":"2024-09-17T15:33:11.224142+0000","last_fullsized":"2024-09-17T15:33:11.224142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20602","reported_seq":28895,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:28.904347+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:28.904347+0000","last_peered":"2024-09-17T15:35:28.904347+0000","last_clean":"2024-09-17T15:35:28.904347+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:28.904347+0000","last_undegraded":"2024-09-17T15:35:28.904347+0000","last_fullsized":"2024-09-17T15:35:28.904347+0000","mapping_epoch":42,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":881123,"num_objects":216,"num_object_clones":0,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":216,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18602,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20342","reported_seq":20617,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:29.279835+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:29.279835+0000","last_peered":"2024-09-17T15:35:29.279835+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:29.279835+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10300","ondisk_log_start":"43'10300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":819200,"num_objects":200,"num_object_clones":0,"num_object_copies":400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16938,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3743}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20319","reported_seq":20538,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:28.302647+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:35:28.302647+0000","last_peered":"2024-09-17T15:35:28.302647+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:35:28.302647+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'10300","ondisk_log_start":"43'10300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":819683,"num_objects":201,"num_object_clones":0,"num_object_copies":402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":201,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18214,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3832}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":88,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223926+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223926+0000","last_peered":"2024-09-17T15:33:11.223926+0000","last_clean":"2024-09-17T15:33:11.223926+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223926+0000","last_undegraded":"2024-09-17T15:33:11.223926+0000","last_fullsized":"2024-09-17T15:33:11.223926+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20513","reported_seq":28929,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:28.237586+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:28.237586+0000","last_peered":"2024-09-17T15:35:28.237586+0000","last_clean":"2024-09-17T15:35:28.237586+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:28.237586+0000","last_undegraded":"2024-09-17T15:35:28.237586+0000","last_fullsized":"2024-09-17T15:35:28.237586+0000","mapping_epoch":46,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":913408,"num_objects":223,"num_object_clones":0,"num_object_copies":446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":223,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18275,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20762","reported_seq":21070,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:29.263123+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:29.263123+0000","last_peered":"2024-09-17T15:35:29.263123+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:29.263123+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":835584,"num_objects":204,"num_object_clones":0,"num_object_copies":408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":204,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17310,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3853}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223809+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223809+0000","last_peered":"2024-09-17T15:33:11.223809+0000","last_clean":"2024-09-17T15:33:11.223809+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223809+0000","last_undegraded":"2024-09-17T15:33:11.223809+0000","last_fullsized":"2024-09-17T15:33:11.223809+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20593","reported_seq":27486,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:29.323147+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:35:29.323147+0000","last_peered":"2024-09-17T15:35:29.323147+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:35:29.323147+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":733184,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":790,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16880,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3370,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-611}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20713","reported_seq":21019,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:29.306277+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:29.306277+0000","last_peered":"2024-09-17T15:35:29.306277+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:29.306277+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":708608,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17500,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3807}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223917+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223917+0000","last_peered":"2024-09-17T15:33:11.223917+0000","last_clean":"2024-09-17T15:33:11.223917+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223917+0000","last_undegraded":"2024-09-17T15:33:11.223917+0000","last_fullsized":"2024-09-17T15:33:11.223917+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223714+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223714+0000","last_peered":"2024-09-17T15:33:11.223714+0000","last_clean":"2024-09-17T15:33:11.223714+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223714+0000","last_undegraded":"2024-09-17T15:33:11.223714+0000","last_fullsized":"2024-09-17T15:33:11.223714+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20444","reported_seq":28829,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:28.937167+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:28.937167+0000","last_peered":"2024-09-17T15:35:28.937167+0000","last_clean":"2024-09-17T15:35:28.937167+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:28.937167+0000","last_undegraded":"2024-09-17T15:35:28.937167+0000","last_fullsized":"2024-09-17T15:35:28.937167+0000","mapping_epoch":44,"log_start":"43'10400","ondisk_log_start":"43'10400","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":771014,"num_objects":190,"num_object_clones":0,"num_object_copies":380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16886,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":13218550,"num_objects":3237,"num_object_clones":0,"num_object_copies":6474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33201,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3237,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":289750,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16259,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":229134336,"data_stored":228999737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160815,"ondisk_log_size":160815,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051709,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":500436,"kb_used_data":139704,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93871404,"statfs":{"total":96636764160,"available":96124317696,"internally_reserved":0,"allocated":143056896,"data_stored":142755298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":369356460},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,4,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724654,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476440,"kb_used_data":11588,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93895400,"statfs":{"total":96636764160,"available":96148889600,"internally_reserved":0,"allocated":11866112,"data_stored":11542372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":475983531},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,4,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247937,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523176,"kb_used_data":76440,"kb_used_omap":4,"kb_used_meta":446715,"kb_avail":93848664,"statfs":{"total":96636764160,"available":96101031936,"internally_reserved":0,"allocated":78274560,"data_stored":77972957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":457436840},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,8,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":76738560,"data_stored":76702430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10051584,"data_stored":9993317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":142344192,"data_stored":142303990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:35:33.462 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:35:33.462 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'20448", 'reported_seq': 20707, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:29.260154+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:35:29.260154+0000', 'last_peered': '2024-09-17T15:35:29.260154+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:35:29.260154+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'10400", 'ondisk_log_start': "43'10400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 819200, 'num_objects': 200, 'num_object_clones': 0, 'num_object_copies': 400, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 200, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20448, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3784}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:33.462 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:35:33.462 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'20697", 'reported_seq': 20920, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:28.269236+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:35:28.269236+0000', 'last_peered': '2024-09-17T15:35:28.269236+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:35:28.269236+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "43'10600", 'ondisk_log_start': "43'10600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966680, 'num_objects': 237, 'num_object_clones': 0, 'num_object_copies': 474, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 237, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18854, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3888}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:35:33.462 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:35:33.462 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'21045", 'reported_seq': 21346, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:29.272830+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:35:29.272830+0000', 'last_peered': '2024-09-17T15:35:29.272830+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:35:29.272830+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "43'11000", 'ondisk_log_start': "43'11000", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 844259, 'num_objects': 207, 'num_object_clones': 0, 'num_object_copies': 414, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 207, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17544, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3985}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:33.462 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:35:33.462 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'20584", 'reported_seq': 20903, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:29.324577+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:35:29.324577+0000', 'last_peered': '2024-09-17T15:35:29.324577+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:35:29.324577+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 754630, 'num_objects': 186, 'num_object_clones': 0, 'num_object_copies': 372, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4089, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 186, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17353, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 85, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3903}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:33.462 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:35:33.462 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20342", 'reported_seq': 20617, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:29.279835+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:35:29.279835+0000', 'last_peered': '2024-09-17T15:35:29.279835+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:35:29.279835+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10300", 'ondisk_log_start': "43'10300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 819200, 'num_objects': 200, 'num_object_clones': 0, 'num_object_copies': 400, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 200, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16938, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3743}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:33.463 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:35:33.463 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'20319", 'reported_seq': 20538, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:28.302647+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:35:28.302647+0000', 'last_peered': '2024-09-17T15:35:28.302647+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:35:28.302647+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "43'10300", 'ondisk_log_start': "43'10300", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 819683, 'num_objects': 201, 'num_object_clones': 0, 'num_object_copies': 402, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4034, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 201, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18214, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -3832}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:35:33.463 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:35:33.463 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'20762", 'reported_seq': 21070, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:29.263123+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:35:29.263123+0000', 'last_peered': '2024-09-17T15:35:29.263123+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:35:29.263123+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "43'10700", 'ondisk_log_start': "43'10700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 835584, 'num_objects': 204, 'num_object_clones': 0, 'num_object_copies': 408, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 204, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17310, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3853}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:33.463 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:35:33.463 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'20593", 'reported_seq': 27486, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:29.323147+0000', 'last_change': '2024-09-17T15:33:09.763500+0000', 'last_active': '2024-09-17T15:35:29.323147+0000', 'last_peered': '2024-09-17T15:35:29.323147+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:35:29.323147+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 733184, 'num_objects': 179, 'num_object_clones': 0, 'num_object_copies': 358, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 790, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 179, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16880, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3370, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -611}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:33.463 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:35:33.463 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'20713", 'reported_seq': 21019, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:29.306277+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:35:29.306277+0000', 'last_peered': '2024-09-17T15:35:29.306277+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:35:29.306277+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'10700", 'ondisk_log_start': "43'10700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 708608, 'num_objects': 173, 'num_object_clones': 0, 'num_object_copies': 346, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 173, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17500, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3807}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:33.464 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:35:34.141 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:35:34.142 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33201/6478 objects degraded (512.519%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":671,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3239,"data_bytes":13218569,"bytes_used":1536053248,"bytes_avail":288374239232,"bytes_total":289910292480,"degraded_objects":33201,"degraded_total":6478,"degraded_ratio":5.1251929607903675,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":304,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":413},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:35:34.165 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:35:34.166 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:35:34.700 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:35:34.702 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:35:34.729 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":375,"stamp":"2024-09-17T15:35:33.810260+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":9986825,"num_objects":2450,"num_object_clones":0,"num_object_copies":4900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33201,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2450,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":290541,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16260,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160807,"ondisk_log_size":160807,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1499256,"kb_used_data":222788,"kb_used_omap":13,"kb_used_meta":1276402,"kb_avail":281616264,"statfs":{"total":289910292480,"available":288375054336,"internally_reserved":0,"allocated":228134912,"data_stored":227207971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1307036671},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,6,5,14,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23576576,"num_objects":-5756,"num_object_clones":0,"num_object_copies":-11512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-304,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5756,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004587"},"pg_stats":[{"pgid":"2.f","version":"47'20448","reported_seq":20707,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:29.260154+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:29.260154+0000","last_peered":"2024-09-17T15:35:29.260154+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:29.260154+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10400","ondisk_log_start":"43'10400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":819200,"num_objects":200,"num_object_clones":0,"num_object_copies":400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20448,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3784}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20868","reported_seq":20981,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:33.403641+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:33.403641+0000","last_peered":"2024-09-17T15:35:33.403641+0000","last_clean":"2024-09-17T15:35:33.403641+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:33.403641+0000","last_undegraded":"2024-09-17T15:35:33.403641+0000","last_fullsized":"2024-09-17T15:35:33.403641+0000","mapping_epoch":33,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":263110,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19225,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20859","reported_seq":21082,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:33.276070+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:33.276070+0000","last_peered":"2024-09-17T15:35:33.276070+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:33.276070+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":303128,"num_objects":75,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19016,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4050}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20906","reported_seq":20986,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:33.378934+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:33.378934+0000","last_peered":"2024-09-17T15:35:33.378934+0000","last_clean":"2024-09-17T15:35:33.378934+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:33.378934+0000","last_undegraded":"2024-09-17T15:35:33.378934+0000","last_fullsized":"2024-09-17T15:35:33.378934+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20906,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20466","reported_seq":20606,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:29.290966+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:29.290966+0000","last_peered":"2024-09-17T15:35:29.290966+0000","last_clean":"2024-09-17T15:35:29.290966+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:29.290966+0000","last_undegraded":"2024-09-17T15:35:29.290966+0000","last_fullsized":"2024-09-17T15:35:29.290966+0000","mapping_epoch":32,"log_start":"43'10400","ondisk_log_start":"43'10400","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":766435,"num_objects":188,"num_object_clones":0,"num_object_copies":376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":188,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17448,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21045","reported_seq":21346,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:29.272830+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:29.272830+0000","last_peered":"2024-09-17T15:35:29.272830+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:29.272830+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11000","ondisk_log_start":"43'11000","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":844259,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":207,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17544,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3985},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20584","reported_seq":20903,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:29.324577+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:35:29.324577+0000","last_peered":"2024-09-17T15:35:29.324577+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:35:29.324577+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":754630,"num_objects":186,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":186,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17353,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3903},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20735","reported_seq":20873,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:29.312428+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:29.312428+0000","last_peered":"2024-09-17T15:35:29.312428+0000","last_clean":"2024-09-17T15:35:29.312428+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:29.312428+0000","last_undegraded":"2024-09-17T15:35:29.312428+0000","last_fullsized":"2024-09-17T15:35:29.312428+0000","mapping_epoch":32,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":741376,"num_objects":181,"num_object_clones":0,"num_object_copies":362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17689,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.224142+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.224142+0000","last_peered":"2024-09-17T15:33:11.224142+0000","last_clean":"2024-09-17T15:33:11.224142+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.224142+0000","last_undegraded":"2024-09-17T15:33:11.224142+0000","last_fullsized":"2024-09-17T15:33:11.224142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20602","reported_seq":28895,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:28.904347+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:28.904347+0000","last_peered":"2024-09-17T15:35:28.904347+0000","last_clean":"2024-09-17T15:35:28.904347+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:28.904347+0000","last_undegraded":"2024-09-17T15:35:28.904347+0000","last_fullsized":"2024-09-17T15:35:28.904347+0000","mapping_epoch":42,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":881123,"num_objects":216,"num_object_clones":0,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":216,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18602,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20342","reported_seq":20617,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:29.279835+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:29.279835+0000","last_peered":"2024-09-17T15:35:29.279835+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:29.279835+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10300","ondisk_log_start":"43'10300","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":819200,"num_objects":200,"num_object_clones":0,"num_object_copies":400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16938,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3743}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.232511+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:11.232511+0000","last_peered":"2024-09-17T15:33:11.232511+0000","last_clean":"2024-09-17T15:33:11.232511+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:11.232511+0000","last_undegraded":"2024-09-17T15:33:11.232511+0000","last_fullsized":"2024-09-17T15:33:11.232511+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20459","reported_seq":20675,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:33.362356+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:35:33.362356+0000","last_peered":"2024-09-17T15:35:33.362356+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:35:33.362356+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'10400","ondisk_log_start":"43'10400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":246243,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18354,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3972}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":88,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223926+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223926+0000","last_peered":"2024-09-17T15:33:11.223926+0000","last_clean":"2024-09-17T15:33:11.223926+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223926+0000","last_undegraded":"2024-09-17T15:33:11.223926+0000","last_fullsized":"2024-09-17T15:33:11.223926+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20678","reported_seq":29094,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:33.315475+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:33.315475+0000","last_peered":"2024-09-17T15:35:33.315475+0000","last_clean":"2024-09-17T15:35:33.315475+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:33.315475+0000","last_undegraded":"2024-09-17T15:35:33.315475+0000","last_fullsized":"2024-09-17T15:35:33.315475+0000","mapping_epoch":46,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":237568,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18440,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20762","reported_seq":21070,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:29.263123+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:29.263123+0000","last_peered":"2024-09-17T15:35:29.263123+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:29.263123+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":835584,"num_objects":204,"num_object_clones":0,"num_object_copies":408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":204,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17310,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3853}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223809+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223809+0000","last_peered":"2024-09-17T15:33:11.223809+0000","last_clean":"2024-09-17T15:33:11.223809+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223809+0000","last_undegraded":"2024-09-17T15:33:11.223809+0000","last_fullsized":"2024-09-17T15:33:11.223809+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20593","reported_seq":27486,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:29.323147+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:35:29.323147+0000","last_peered":"2024-09-17T15:35:29.323147+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:35:29.323147+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":733184,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":790,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16880,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3370,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-611}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20713","reported_seq":21019,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:29.306277+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:29.306277+0000","last_peered":"2024-09-17T15:35:29.306277+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:29.306277+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":708608,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17500,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3807}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223917+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223917+0000","last_peered":"2024-09-17T15:33:11.223917+0000","last_clean":"2024-09-17T15:33:11.223917+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223917+0000","last_undegraded":"2024-09-17T15:33:11.223917+0000","last_fullsized":"2024-09-17T15:33:11.223917+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223714+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223714+0000","last_peered":"2024-09-17T15:33:11.223714+0000","last_clean":"2024-09-17T15:33:11.223714+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223714+0000","last_undegraded":"2024-09-17T15:33:11.223714+0000","last_fullsized":"2024-09-17T15:33:11.223714+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20444","reported_seq":28829,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:28.937167+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:28.937167+0000","last_peered":"2024-09-17T15:35:28.937167+0000","last_clean":"2024-09-17T15:35:28.937167+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:28.937167+0000","last_undegraded":"2024-09-17T15:35:28.937167+0000","last_fullsized":"2024-09-17T15:35:28.937167+0000","mapping_epoch":44,"log_start":"43'10400","ondisk_log_start":"43'10400","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":771014,"num_objects":190,"num_object_clones":0,"num_object_copies":380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16886,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":9986806,"num_objects":2448,"num_object_clones":0,"num_object_copies":4896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33201,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2448,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":290539,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16259,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224043008,"data_stored":223908409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160804,"ondisk_log_size":160804,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051710,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":499640,"kb_used_data":134760,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93872200,"statfs":{"total":96636764160,"available":96125132800,"internally_reserved":0,"allocated":137994240,"data_stored":137692642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,0,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724654,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476440,"kb_used_data":11588,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93895400,"statfs":{"total":96636764160,"available":96148889600,"internally_reserved":0,"allocated":11866112,"data_stored":11542372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":475983531},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,4,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247937,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523176,"kb_used_data":76440,"kb_used_omap":4,"kb_used_meta":446715,"kb_avail":93848664,"statfs":{"total":96636764160,"available":96101031936,"internally_reserved":0,"allocated":78274560,"data_stored":77972957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":457436840},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,8,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":76738560,"data_stored":76702430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10051584,"data_stored":9993317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":137252864,"data_stored":137212662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:35:35.584 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout:Removed 166429 objects 2024-09-17T15:35:35.584 INFO:tasks.lost_unfound.radosbench.client.admin.smithi083.stdout:Clean up completed and total clean up time :356.242 2024-09-17T15:35:37.731 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:35:38.051 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:35:38.051 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:35:38.064 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":378,"stamp":"2024-09-17T15:35:37.863965+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3085065,"num_objects":765,"num_object_clones":0,"num_object_copies":1530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33037,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":765,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292226,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16424,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160692,"ondisk_log_size":160692,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1496760,"kb_used_data":207804,"kb_used_omap":13,"kb_used_meta":1288882,"kb_avail":281618760,"statfs":{"total":289910292480,"available":288377610240,"internally_reserved":0,"allocated":212791296,"data_stored":211864355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1319816191},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,5,1,2,11,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20299776,"num_objects":-4956,"num_object_clones":0,"num_object_copies":-9912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-324,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4956,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4956,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.055985"},"pg_stats":[{"pgid":"2.f","version":"47'20619","reported_seq":20877,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:34.367355+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:34.367355+0000","last_peered":"2024-09-17T15:35:34.367355+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:34.367355+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118784,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20619,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3955}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20868","reported_seq":20981,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:33.403641+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:33.403641+0000","last_peered":"2024-09-17T15:35:33.403641+0000","last_clean":"2024-09-17T15:35:33.403641+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:33.403641+0000","last_undegraded":"2024-09-17T15:35:33.403641+0000","last_fullsized":"2024-09-17T15:35:33.403641+0000","mapping_epoch":33,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":263110,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19225,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20859","reported_seq":21082,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:33.276070+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:33.276070+0000","last_peered":"2024-09-17T15:35:33.276070+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:33.276070+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":303128,"num_objects":75,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19016,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4050}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20906","reported_seq":20986,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:33.378934+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:33.378934+0000","last_peered":"2024-09-17T15:35:33.378934+0000","last_clean":"2024-09-17T15:35:33.378934+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:33.378934+0000","last_undegraded":"2024-09-17T15:35:33.378934+0000","last_fullsized":"2024-09-17T15:35:33.378934+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20906,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20608","reported_seq":20748,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:34.347973+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:34.347973+0000","last_peered":"2024-09-17T15:35:34.347973+0000","last_clean":"2024-09-17T15:35:34.347973+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:34.347973+0000","last_undegraded":"2024-09-17T15:35:34.347973+0000","last_fullsized":"2024-09-17T15:35:34.347973+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":184803,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17590,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21213","reported_seq":21512,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:34.263525+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:34.263525+0000","last_peered":"2024-09-17T15:35:34.263525+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:34.263525+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":156131,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17712,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4153},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20730","reported_seq":21049,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:34.318422+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:35:34.318422+0000","last_peered":"2024-09-17T15:35:34.318422+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:35:34.318422+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":156614,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17499,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4049},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20879","reported_seq":21017,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:34.340781+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:34.340781+0000","last_peered":"2024-09-17T15:35:34.340781+0000","last_clean":"2024-09-17T15:35:34.340781+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:34.340781+0000","last_undegraded":"2024-09-17T15:35:34.340781+0000","last_fullsized":"2024-09-17T15:35:34.340781+0000","mapping_epoch":32,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":151552,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17833,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.224142+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.224142+0000","last_peered":"2024-09-17T15:33:11.224142+0000","last_clean":"2024-09-17T15:33:11.224142+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.224142+0000","last_undegraded":"2024-09-17T15:33:11.224142+0000","last_fullsized":"2024-09-17T15:33:11.224142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20765","reported_seq":29058,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:33.966021+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:33.966021+0000","last_peered":"2024-09-17T15:35:33.966021+0000","last_clean":"2024-09-17T15:35:33.966021+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:33.966021+0000","last_undegraded":"2024-09-17T15:35:33.966021+0000","last_fullsized":"2024-09-17T15:35:33.966021+0000","mapping_epoch":42,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213475,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18765,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20506","reported_seq":20781,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:34.303528+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:34.303528+0000","last_peered":"2024-09-17T15:35:34.303528+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:34.303528+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":147456,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17102,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3907}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":114,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:28.276888+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:28.276888+0000","last_peered":"2024-09-17T15:33:28.276888+0000","last_clean":"2024-09-17T15:33:28.276888+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:28.276888+0000","last_undegraded":"2024-09-17T15:33:28.276888+0000","last_fullsized":"2024-09-17T15:33:28.276888+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20459","reported_seq":20675,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:33.362356+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:35:33.362356+0000","last_peered":"2024-09-17T15:35:33.362356+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:35:33.362356+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'10400","ondisk_log_start":"43'10400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":246243,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18354,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3972}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":88,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223926+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223926+0000","last_peered":"2024-09-17T15:33:11.223926+0000","last_clean":"2024-09-17T15:33:11.223926+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223926+0000","last_undegraded":"2024-09-17T15:33:11.223926+0000","last_fullsized":"2024-09-17T15:33:11.223926+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20678","reported_seq":29094,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:33.315475+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:33.315475+0000","last_peered":"2024-09-17T15:35:33.315475+0000","last_clean":"2024-09-17T15:35:33.315475+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:33.315475+0000","last_undegraded":"2024-09-17T15:35:33.315475+0000","last_fullsized":"2024-09-17T15:35:33.315475+0000","mapping_epoch":46,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":237568,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18440,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20931","reported_seq":21239,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:34.368293+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:34.368293+0000","last_peered":"2024-09-17T15:35:34.368293+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:34.368293+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":143360,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17479,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4022}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223809+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223809+0000","last_peered":"2024-09-17T15:33:11.223809+0000","last_clean":"2024-09-17T15:33:11.223809+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223809+0000","last_undegraded":"2024-09-17T15:33:11.223809+0000","last_fullsized":"2024-09-17T15:33:11.223809+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20730","reported_seq":27950,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:34.364711+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:35:34.364711+0000","last_peered":"2024-09-17T15:35:34.364711+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:35:34.364711+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":172032,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":626,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17017,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3534,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-584}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20855","reported_seq":21162,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:34.329964+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:34.329964+0000","last_peered":"2024-09-17T15:35:34.329964+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:34.329964+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":126976,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17642,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3949}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223917+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223917+0000","last_peered":"2024-09-17T15:33:11.223917+0000","last_clean":"2024-09-17T15:33:11.223917+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223917+0000","last_undegraded":"2024-09-17T15:33:11.223917+0000","last_fullsized":"2024-09-17T15:33:11.223917+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223714+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223714+0000","last_peered":"2024-09-17T15:33:11.223714+0000","last_clean":"2024-09-17T15:33:11.223714+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223714+0000","last_undegraded":"2024-09-17T15:33:11.223714+0000","last_fullsized":"2024-09-17T15:33:11.223714+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20583","reported_seq":28968,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:33.925285+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:33.925285+0000","last_peered":"2024-09-17T15:35:33.925285+0000","last_clean":"2024-09-17T15:35:33.925285+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:33.925285+0000","last_undegraded":"2024-09-17T15:35:33.925285+0000","last_fullsized":"2024-09-17T15:35:33.925285+0000","mapping_epoch":44,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201670,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17025,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":3085046,"num_objects":763,"num_object_clones":0,"num_object_copies":1526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33037,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":763,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292224,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16423,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":208908288,"data_stored":208773689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160689,"ondisk_log_size":160689,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051710,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":499640,"kb_used_data":134760,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93872200,"statfs":{"total":96636764160,"available":96125132800,"internally_reserved":0,"allocated":137994240,"data_stored":137692642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,0,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724655,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":477288,"kb_used_data":4108,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93894552,"statfs":{"total":96636764160,"available":96148021248,"internally_reserved":0,"allocated":4206592,"data_stored":3882852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":484503211},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,1,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247938,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519832,"kb_used_data":68936,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93852008,"statfs":{"total":96636764160,"available":96104456192,"internally_reserved":0,"allocated":70590464,"data_stored":70288861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":461696680},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,5,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69242880,"data_stored":69206750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2412544,"data_stored":2354277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":137252864,"data_stored":137212662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:35:38.066 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:35:38.386 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:35:38.386 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:35:38.399 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":378,"stamp":"2024-09-17T15:35:37.863965+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3085065,"num_objects":765,"num_object_clones":0,"num_object_copies":1530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33037,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":765,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292226,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16424,"num_bytes_recovered":41571289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160692,"ondisk_log_size":160692,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1496760,"kb_used_data":207804,"kb_used_omap":13,"kb_used_meta":1288882,"kb_avail":281618760,"statfs":{"total":289910292480,"available":288377610240,"internally_reserved":0,"allocated":212791296,"data_stored":211864355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1319816191},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,5,1,2,11,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20299776,"num_objects":-4956,"num_object_clones":0,"num_object_copies":-9912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-324,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4956,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4956,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.055985"},"pg_stats":[{"pgid":"2.f","version":"47'20619","reported_seq":20877,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:34.367355+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:34.367355+0000","last_peered":"2024-09-17T15:35:34.367355+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:34.367355+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118784,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20619,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3955}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20868","reported_seq":20981,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:33.403641+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:33.403641+0000","last_peered":"2024-09-17T15:35:33.403641+0000","last_clean":"2024-09-17T15:35:33.403641+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:33.403641+0000","last_undegraded":"2024-09-17T15:35:33.403641+0000","last_fullsized":"2024-09-17T15:35:33.403641+0000","mapping_epoch":33,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":263110,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19225,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20859","reported_seq":21082,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:33.276070+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:33.276070+0000","last_peered":"2024-09-17T15:35:33.276070+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:33.276070+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":303128,"num_objects":75,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19016,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4050}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20906","reported_seq":20986,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:33.378934+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:33.378934+0000","last_peered":"2024-09-17T15:35:33.378934+0000","last_clean":"2024-09-17T15:35:33.378934+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:33.378934+0000","last_undegraded":"2024-09-17T15:35:33.378934+0000","last_fullsized":"2024-09-17T15:35:33.378934+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20906,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20608","reported_seq":20748,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:34.347973+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:34.347973+0000","last_peered":"2024-09-17T15:35:34.347973+0000","last_clean":"2024-09-17T15:35:34.347973+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:34.347973+0000","last_undegraded":"2024-09-17T15:35:34.347973+0000","last_fullsized":"2024-09-17T15:35:34.347973+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":184803,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17590,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21213","reported_seq":21512,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:34.263525+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:34.263525+0000","last_peered":"2024-09-17T15:35:34.263525+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:34.263525+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":156131,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17712,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4153},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20730","reported_seq":21049,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:34.318422+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:35:34.318422+0000","last_peered":"2024-09-17T15:35:34.318422+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:35:34.318422+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":156614,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17499,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4049},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20879","reported_seq":21017,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:34.340781+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:34.340781+0000","last_peered":"2024-09-17T15:35:34.340781+0000","last_clean":"2024-09-17T15:35:34.340781+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:34.340781+0000","last_undegraded":"2024-09-17T15:35:34.340781+0000","last_fullsized":"2024-09-17T15:35:34.340781+0000","mapping_epoch":32,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":151552,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17833,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.224142+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.224142+0000","last_peered":"2024-09-17T15:33:11.224142+0000","last_clean":"2024-09-17T15:33:11.224142+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.224142+0000","last_undegraded":"2024-09-17T15:33:11.224142+0000","last_fullsized":"2024-09-17T15:33:11.224142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20765","reported_seq":29058,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:33.966021+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:33.966021+0000","last_peered":"2024-09-17T15:35:33.966021+0000","last_clean":"2024-09-17T15:35:33.966021+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:33.966021+0000","last_undegraded":"2024-09-17T15:35:33.966021+0000","last_fullsized":"2024-09-17T15:35:33.966021+0000","mapping_epoch":42,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213475,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18765,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225475+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:33:11.225475+0000","last_peered":"2024-09-17T15:33:11.225475+0000","last_clean":"2024-09-17T15:33:11.225475+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:33:11.225475+0000","last_undegraded":"2024-09-17T15:33:11.225475+0000","last_fullsized":"2024-09-17T15:33:11.225475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20506","reported_seq":20781,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:34.303528+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:34.303528+0000","last_peered":"2024-09-17T15:35:34.303528+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:34.303528+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":147456,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17102,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3907}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":114,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:28.276888+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:28.276888+0000","last_peered":"2024-09-17T15:33:28.276888+0000","last_clean":"2024-09-17T15:33:28.276888+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:28.276888+0000","last_undegraded":"2024-09-17T15:33:28.276888+0000","last_fullsized":"2024-09-17T15:33:28.276888+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20459","reported_seq":20675,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:33.362356+0000","last_change":"2024-09-17T15:26:22.597467+0000","last_active":"2024-09-17T15:35:33.362356+0000","last_peered":"2024-09-17T15:35:33.362356+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:35:33.362356+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'10400","ondisk_log_start":"43'10400","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":246243,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18354,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3972}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":88,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223926+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223926+0000","last_peered":"2024-09-17T15:33:11.223926+0000","last_clean":"2024-09-17T15:33:11.223926+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223926+0000","last_undegraded":"2024-09-17T15:33:11.223926+0000","last_fullsized":"2024-09-17T15:33:11.223926+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20678","reported_seq":29094,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:33.315475+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:33.315475+0000","last_peered":"2024-09-17T15:35:33.315475+0000","last_clean":"2024-09-17T15:35:33.315475+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:33.315475+0000","last_undegraded":"2024-09-17T15:35:33.315475+0000","last_fullsized":"2024-09-17T15:35:33.315475+0000","mapping_epoch":46,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":237568,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18440,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20931","reported_seq":21239,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:34.368293+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:34.368293+0000","last_peered":"2024-09-17T15:35:34.368293+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:34.368293+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":143360,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17479,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4022}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223809+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223809+0000","last_peered":"2024-09-17T15:33:11.223809+0000","last_clean":"2024-09-17T15:33:11.223809+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223809+0000","last_undegraded":"2024-09-17T15:33:11.223809+0000","last_fullsized":"2024-09-17T15:33:11.223809+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20730","reported_seq":27950,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:34.364711+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:35:34.364711+0000","last_peered":"2024-09-17T15:35:34.364711+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:35:34.364711+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":172032,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":626,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17017,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3534,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-584}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.225061+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:33:11.225061+0000","last_peered":"2024-09-17T15:33:11.225061+0000","last_clean":"2024-09-17T15:33:11.225061+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:33:11.225061+0000","last_undegraded":"2024-09-17T15:33:11.225061+0000","last_fullsized":"2024-09-17T15:33:11.225061+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20855","reported_seq":21162,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:34.329964+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:34.329964+0000","last_peered":"2024-09-17T15:35:34.329964+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:34.329964+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":126976,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17642,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3949}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223917+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223917+0000","last_peered":"2024-09-17T15:33:11.223917+0000","last_clean":"2024-09-17T15:33:11.223917+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223917+0000","last_undegraded":"2024-09-17T15:33:11.223917+0000","last_fullsized":"2024-09-17T15:33:11.223917+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223714+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223714+0000","last_peered":"2024-09-17T15:33:11.223714+0000","last_clean":"2024-09-17T15:33:11.223714+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223714+0000","last_undegraded":"2024-09-17T15:33:11.223714+0000","last_fullsized":"2024-09-17T15:33:11.223714+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20583","reported_seq":28968,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:33.925285+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:33.925285+0000","last_peered":"2024-09-17T15:35:33.925285+0000","last_clean":"2024-09-17T15:35:33.925285+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:33.925285+0000","last_undegraded":"2024-09-17T15:35:33.925285+0000","last_fullsized":"2024-09-17T15:35:33.925285+0000","mapping_epoch":44,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201670,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17025,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":3085046,"num_objects":763,"num_object_clones":0,"num_object_copies":1526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33037,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":763,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292224,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16423,"num_bytes_recovered":41571270,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":208908288,"data_stored":208773689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160689,"ondisk_log_size":160689,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051710,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":499640,"kb_used_data":134760,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93872200,"statfs":{"total":96636764160,"available":96125132800,"internally_reserved":0,"allocated":137994240,"data_stored":137692642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,0,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724655,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":477288,"kb_used_data":4108,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93894552,"statfs":{"total":96636764160,"available":96148021248,"internally_reserved":0,"allocated":4206592,"data_stored":3882852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":484503211},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,1,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247938,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519832,"kb_used_data":68936,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93852008,"statfs":{"total":96636764160,"available":96104456192,"internally_reserved":0,"allocated":70590464,"data_stored":70288861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":461696680},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,5,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69242880,"data_stored":69206750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2412544,"data_stored":2354277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":137252864,"data_stored":137212662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:35:38.400 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:35:38.400 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'20619", 'reported_seq': 20877, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:34.367355+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:35:34.367355+0000', 'last_peered': '2024-09-17T15:35:34.367355+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:35:34.367355+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'10600", 'ondisk_log_start': "43'10600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 118784, 'num_objects': 29, 'num_object_clones': 0, 'num_object_copies': 58, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 29, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20619, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3955}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:38.401 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:35:38.401 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'20859", 'reported_seq': 21082, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:33.276070+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:35:33.276070+0000', 'last_peered': '2024-09-17T15:35:33.276070+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:35:33.276070+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "43'10800", 'ondisk_log_start': "43'10800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 303128, 'num_objects': 75, 'num_object_clones': 0, 'num_object_copies': 150, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 75, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 19016, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4050}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:35:38.401 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:35:38.401 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'21213", 'reported_seq': 21512, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:34.263525+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:35:34.263525+0000', 'last_peered': '2024-09-17T15:35:34.263525+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:35:34.263525+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "43'11200", 'ondisk_log_start': "43'11200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 156131, 'num_objects': 39, 'num_object_clones': 0, 'num_object_copies': 78, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 39, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17712, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4153}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:38.401 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:35:38.401 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'20730", 'reported_seq': 21049, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:34.318422+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:35:34.318422+0000', 'last_peered': '2024-09-17T15:35:34.318422+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:35:34.318422+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "43'10700", 'ondisk_log_start': "43'10700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 156614, 'num_objects': 40, 'num_object_clones': 0, 'num_object_copies': 80, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4089, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 40, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17499, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 85, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4049}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:38.401 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:35:38.401 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20506", 'reported_seq': 20781, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:34.303528+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:35:34.303528+0000', 'last_peered': '2024-09-17T15:35:34.303528+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:35:34.303528+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 147456, 'num_objects': 36, 'num_object_clones': 0, 'num_object_copies': 72, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 36, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17102, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3907}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:38.402 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:35:38.402 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'20459", 'reported_seq': 20675, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:33.362356+0000', 'last_change': '2024-09-17T15:26:22.597467+0000', 'last_active': '2024-09-17T15:35:33.362356+0000', 'last_peered': '2024-09-17T15:35:33.362356+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:35:33.362356+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "43'10400", 'ondisk_log_start': "43'10400", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 246243, 'num_objects': 61, 'num_object_clones': 0, 'num_object_copies': 122, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4034, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 61, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18354, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -3972}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:35:38.402 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:35:38.402 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'20931", 'reported_seq': 21239, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:34.368293+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:35:34.368293+0000', 'last_peered': '2024-09-17T15:35:34.368293+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:35:34.368293+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "43'10900", 'ondisk_log_start': "43'10900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 143360, 'num_objects': 35, 'num_object_clones': 0, 'num_object_copies': 70, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 35, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17479, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4022}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:38.402 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-17T15:35:38.402 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'20730", 'reported_seq': 27950, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:34.364711+0000', 'last_change': '2024-09-17T15:33:09.763500+0000', 'last_active': '2024-09-17T15:35:34.364711+0000', 'last_peered': '2024-09-17T15:35:34.364711+0000', 'last_clean': '2024-09-17T15:26:20.154604+0000', 'last_became_active': '2024-09-17T15:26:22.671661+0000', 'last_became_peered': '2024-09-17T15:26:22.671661+0000', 'last_unstale': '2024-09-17T15:35:34.364711+0000', 'last_undegraded': '2024-09-17T15:26:22.245946+0000', 'last_fullsized': '2024-09-17T15:26:22.190360+0000', 'mapping_epoch': 40, 'log_start': "43'10700", 'ondisk_log_start': "43'10700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3750", 'last_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:12.900151+0000', 'objects_scrubbed': 3705, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1250, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 172032, 'num_objects': 42, 'num_object_clones': 0, 'num_object_copies': 84, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 626, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 42, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17017, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3534, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -584}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:38.403 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:35:38.403 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'20855", 'reported_seq': 21162, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:34.329964+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:35:34.329964+0000', 'last_peered': '2024-09-17T15:35:34.329964+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:35:34.329964+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'10800", 'ondisk_log_start': "43'10800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 126976, 'num_objects': 31, 'num_object_clones': 0, 'num_object_copies': 62, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 31, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17642, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3949}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:38.403 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:35:38.911 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:35:38.912 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33037/1530 objects degraded (2159.281%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":675,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":765,"data_bytes":3085065,"bytes_used":1532682240,"bytes_avail":288377610240,"bytes_total":289910292480,"degraded_objects":33037,"degraded_total":1530,"degraded_ratio":21.59281045751634,"recovering_objects_per_sec":26,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":324,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":412},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:35:38.927 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:35:38.927 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:35:39.272 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:35:39.272 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:35:39.287 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":379,"stamp":"2024-09-17T15:35:38.865036+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1774321,"num_objects":444,"num_object_clones":0,"num_object_copies":888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32912,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":444,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292547,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16549,"num_bytes_recovered":41571772,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160613,"ondisk_log_size":160613,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1494376,"kb_used_data":205416,"kb_used_omap":13,"kb_used_meta":1288882,"kb_avail":281621144,"statfs":{"total":289910292480,"available":288380051456,"internally_reserved":0,"allocated":210345984,"data_stored":209417850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1319816191},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,1,2,8,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-11444248,"num_objects":-2795,"num_object_clones":0,"num_object_copies":-5590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-289,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2795,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":289,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.056469"},"pg_stats":[{"pgid":"2.f","version":"47'20619","reported_seq":20877,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:34.367355+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:34.367355+0000","last_peered":"2024-09-17T15:35:34.367355+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:34.367355+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118784,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20619,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3955}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21047,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T15:35:37.865744+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:37.865744+0000","last_peered":"2024-09-17T15:35:37.865744+0000","last_clean":"2024-09-17T15:35:37.865744+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:37.865744+0000","last_undegraded":"2024-09-17T15:35:37.865744+0000","last_fullsized":"2024-09-17T15:35:37.865744+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21159,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:37.865786+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:37.865786+0000","last_peered":"2024-09-17T15:35:37.865786+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:37.865786+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4125}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21052,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T15:35:37.865634+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:37.865634+0000","last_peered":"2024-09-17T15:35:37.865634+0000","last_clean":"2024-09-17T15:35:37.865634+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:37.865634+0000","last_undegraded":"2024-09-17T15:35:37.865634+0000","last_fullsized":"2024-09-17T15:35:37.865634+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20608","reported_seq":20748,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:34.347973+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:34.347973+0000","last_peered":"2024-09-17T15:35:34.347973+0000","last_clean":"2024-09-17T15:35:34.347973+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:34.347973+0000","last_undegraded":"2024-09-17T15:35:34.347973+0000","last_fullsized":"2024-09-17T15:35:34.347973+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":184803,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17590,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21213","reported_seq":21512,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:34.263525+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:34.263525+0000","last_peered":"2024-09-17T15:35:34.263525+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:34.263525+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":156131,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17712,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4153},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20730","reported_seq":21049,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:34.318422+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:35:34.318422+0000","last_peered":"2024-09-17T15:35:34.318422+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:35:34.318422+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":156614,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17499,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4049},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20879","reported_seq":21017,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:34.340781+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:34.340781+0000","last_peered":"2024-09-17T15:35:34.340781+0000","last_clean":"2024-09-17T15:35:34.340781+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:34.340781+0000","last_undegraded":"2024-09-17T15:35:34.340781+0000","last_fullsized":"2024-09-17T15:35:34.340781+0000","mapping_epoch":32,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":151552,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17833,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.224142+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:33:11.224142+0000","last_peered":"2024-09-17T15:33:11.224142+0000","last_clean":"2024-09-17T15:33:11.224142+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:33:11.224142+0000","last_undegraded":"2024-09-17T15:33:11.224142+0000","last_fullsized":"2024-09-17T15:33:11.224142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20765","reported_seq":29058,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:33.966021+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:33.966021+0000","last_peered":"2024-09-17T15:35:33.966021+0000","last_clean":"2024-09-17T15:35:33.966021+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:33.966021+0000","last_undegraded":"2024-09-17T15:35:33.966021+0000","last_fullsized":"2024-09-17T15:35:33.966021+0000","mapping_epoch":42,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213475,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18765,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T15:35:37.865651+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:35:37.865651+0000","last_peered":"2024-09-17T15:35:37.865651+0000","last_clean":"2024-09-17T15:35:37.865651+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:35:37.865651+0000","last_undegraded":"2024-09-17T15:35:37.865651+0000","last_fullsized":"2024-09-17T15:35:37.865651+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20506","reported_seq":20781,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:34.303528+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:34.303528+0000","last_peered":"2024-09-17T15:35:34.303528+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:34.303528+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":147456,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17102,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3907}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":114,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:28.276888+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:33:28.276888+0000","last_peered":"2024-09-17T15:33:28.276888+0000","last_clean":"2024-09-17T15:33:28.276888+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:33:28.276888+0000","last_undegraded":"2024-09-17T15:33:28.276888+0000","last_fullsized":"2024-09-17T15:33:28.276888+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":20991,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.431764+0000","last_change":"2024-09-17T15:35:37.572616+0000","last_active":"2024-09-17T15:35:38.431764+0000","last_peered":"2024-09-17T15:35:38.431764+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:35:38.431764+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3909,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3908}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":88,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223926+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:33:11.223926+0000","last_peered":"2024-09-17T15:33:11.223926+0000","last_clean":"2024-09-17T15:33:11.223926+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:33:11.223926+0000","last_undegraded":"2024-09-17T15:33:11.223926+0000","last_fullsized":"2024-09-17T15:33:11.223926+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29154,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T15:35:37.865961+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:37.865961+0000","last_peered":"2024-09-17T15:35:37.865961+0000","last_clean":"2024-09-17T15:35:37.865961+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:37.865961+0000","last_undegraded":"2024-09-17T15:35:37.865961+0000","last_fullsized":"2024-09-17T15:35:37.865961+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20931","reported_seq":21239,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:34.368293+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:34.368293+0000","last_peered":"2024-09-17T15:35:34.368293+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:34.368293+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":143360,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17479,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4022}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223809+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:33:11.223809+0000","last_peered":"2024-09-17T15:33:11.223809+0000","last_clean":"2024-09-17T15:33:11.223809+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:33:11.223809+0000","last_undegraded":"2024-09-17T15:33:11.223809+0000","last_fullsized":"2024-09-17T15:33:11.223809+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20730","reported_seq":27950,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:34.364711+0000","last_change":"2024-09-17T15:33:09.763500+0000","last_active":"2024-09-17T15:35:34.364711+0000","last_peered":"2024-09-17T15:35:34.364711+0000","last_clean":"2024-09-17T15:26:20.154604+0000","last_became_active":"2024-09-17T15:26:22.671661+0000","last_became_peered":"2024-09-17T15:26:22.671661+0000","last_unstale":"2024-09-17T15:35:34.364711+0000","last_undegraded":"2024-09-17T15:26:22.245946+0000","last_fullsized":"2024-09-17T15:26:22.190360+0000","mapping_epoch":40,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":172032,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":626,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17017,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3534,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-584}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T15:35:37.866680+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:35:37.866680+0000","last_peered":"2024-09-17T15:35:37.866680+0000","last_clean":"2024-09-17T15:35:37.866680+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:35:37.866680+0000","last_undegraded":"2024-09-17T15:35:37.866680+0000","last_fullsized":"2024-09-17T15:35:37.866680+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20855","reported_seq":21162,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:34.329964+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:34.329964+0000","last_peered":"2024-09-17T15:35:34.329964+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:34.329964+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":126976,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17642,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3949}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223917+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:33:11.223917+0000","last_peered":"2024-09-17T15:33:11.223917+0000","last_clean":"2024-09-17T15:33:11.223917+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:33:11.223917+0000","last_undegraded":"2024-09-17T15:33:11.223917+0000","last_fullsized":"2024-09-17T15:33:11.223917+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:33:11.223714+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:33:11.223714+0000","last_peered":"2024-09-17T15:33:11.223714+0000","last_clean":"2024-09-17T15:33:11.223714+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:33:11.223714+0000","last_undegraded":"2024-09-17T15:33:11.223714+0000","last_fullsized":"2024-09-17T15:33:11.223714+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20583","reported_seq":28968,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T15:35:33.925285+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:33.925285+0000","last_peered":"2024-09-17T15:35:33.925285+0000","last_clean":"2024-09-17T15:35:33.925285+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:33.925285+0000","last_undegraded":"2024-09-17T15:35:33.925285+0000","last_fullsized":"2024-09-17T15:35:33.925285+0000","mapping_epoch":44,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201670,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17025,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1774302,"num_objects":442,"num_object_clones":0,"num_object_copies":884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32912,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":442,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292545,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16548,"num_bytes_recovered":41571753,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":206802944,"data_stored":206672417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160610,"ondisk_log_size":160610,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051711,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497256,"kb_used_data":132372,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874584,"statfs":{"total":96636764160,"available":96127574016,"internally_reserved":0,"allocated":135548928,"data_stored":135246137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724655,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":477288,"kb_used_data":4108,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93894552,"statfs":{"total":96636764160,"available":96148021248,"internally_reserved":0,"allocated":4206592,"data_stored":3882852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":484503211},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,1,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247938,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519832,"kb_used_data":68936,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93852008,"statfs":{"total":96636764160,"available":96104456192,"internally_reserved":0,"allocated":70590464,"data_stored":70288861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":461696680},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,5,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69242880,"data_stored":69206750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2412544,"data_stored":2354277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135147520,"data_stored":135111390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:35:42.289 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:35:42.624 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:35:42.625 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:35:42.638 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":381,"stamp":"2024-09-17T15:35:41.811954+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32286,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17175,"num_bytes_recovered":41571772,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1501904,"kb_used_data":199064,"kb_used_omap":13,"kb_used_meta":1302770,"kb_avail":281613616,"statfs":{"total":289910292480,"available":288372342784,"internally_reserved":0,"allocated":203841536,"data_stored":202910557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1334037503},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,5,3,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-9981976,"num_objects":-2438,"num_object_clones":0,"num_object_copies":-4876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-915,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2438,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2438,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":915,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001694"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20912,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.872768+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:38.872768+0000","last_peered":"2024-09-17T15:35:38.872768+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:38.872768+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21047,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T15:35:37.865744+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:37.865744+0000","last_peered":"2024-09-17T15:35:37.865744+0000","last_clean":"2024-09-17T15:35:37.865744+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:37.865744+0000","last_undegraded":"2024-09-17T15:35:37.865744+0000","last_fullsized":"2024-09-17T15:35:37.865744+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21159,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:37.865786+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:37.865786+0000","last_peered":"2024-09-17T15:35:37.865786+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:37.865786+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4125}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21052,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T15:35:37.865634+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:37.865634+0000","last_peered":"2024-09-17T15:35:37.865634+0000","last_clean":"2024-09-17T15:35:37.865634+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:37.865634+0000","last_undegraded":"2024-09-17T15:35:37.865634+0000","last_fullsized":"2024-09-17T15:35:37.865634+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20797,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.870725+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:38.870725+0000","last_peered":"2024-09-17T15:35:38.870725+0000","last_clean":"2024-09-17T15:35:38.870725+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:38.870725+0000","last_undegraded":"2024-09-17T15:35:38.870725+0000","last_fullsized":"2024-09-17T15:35:38.870725+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21556,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870736+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:38.870736+0000","last_peered":"2024-09-17T15:35:38.870736+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:38.870736+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4191},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21091,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870726+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:35:38.870726+0000","last_peered":"2024-09-17T15:35:38.870726+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:35:38.870726+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4087},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21058,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.870606+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:38.870606+0000","last_peered":"2024-09-17T15:35:38.870606+0000","last_clean":"2024-09-17T15:35:38.870606+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:38.870606+0000","last_undegraded":"2024-09-17T15:35:38.870606+0000","last_fullsized":"2024-09-17T15:35:38.870606+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869332+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:35:38.869332+0000","last_peered":"2024-09-17T15:35:38.869332+0000","last_clean":"2024-09-17T15:35:38.869332+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:35:38.869332+0000","last_undegraded":"2024-09-17T15:35:38.869332+0000","last_fullsized":"2024-09-17T15:35:38.869332+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29114,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869044+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:38.869044+0000","last_peered":"2024-09-17T15:35:38.869044+0000","last_clean":"2024-09-17T15:35:38.869044+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:38.869044+0000","last_undegraded":"2024-09-17T15:35:38.869044+0000","last_fullsized":"2024-09-17T15:35:38.869044+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T15:35:37.865651+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:35:37.865651+0000","last_peered":"2024-09-17T15:35:37.865651+0000","last_clean":"2024-09-17T15:35:37.865651+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:35:37.865651+0000","last_undegraded":"2024-09-17T15:35:37.865651+0000","last_fullsized":"2024-09-17T15:35:37.865651+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20821,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870877+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:38.870877+0000","last_peered":"2024-09-17T15:35:38.870877+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:38.870877+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":118,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.870615+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:35:38.870615+0000","last_peered":"2024-09-17T15:35:38.870615+0000","last_clean":"2024-09-17T15:35:38.870615+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:35:38.870615+0000","last_undegraded":"2024-09-17T15:35:38.870615+0000","last_fullsized":"2024-09-17T15:35:38.870615+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":20991,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.431764+0000","last_change":"2024-09-17T15:35:37.572616+0000","last_active":"2024-09-17T15:35:38.431764+0000","last_peered":"2024-09-17T15:35:38.431764+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:35:38.431764+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3909,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3908}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":92,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.868946+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:35:38.868946+0000","last_peered":"2024-09-17T15:35:38.868946+0000","last_clean":"2024-09-17T15:35:38.868946+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:35:38.868946+0000","last_undegraded":"2024-09-17T15:35:38.868946+0000","last_fullsized":"2024-09-17T15:35:38.868946+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29154,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T15:35:37.865961+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:37.865961+0000","last_peered":"2024-09-17T15:35:37.865961+0000","last_clean":"2024-09-17T15:35:37.865961+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:37.865961+0000","last_undegraded":"2024-09-17T15:35:37.865961+0000","last_fullsized":"2024-09-17T15:35:37.865961+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21278,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870971+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:38.870971+0000","last_peered":"2024-09-17T15:35:38.870971+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:38.870971+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869077+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:35:38.869077+0000","last_peered":"2024-09-17T15:35:38.869077+0000","last_clean":"2024-09-17T15:35:38.869077+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:35:38.869077+0000","last_undegraded":"2024-09-17T15:35:38.869077+0000","last_fullsized":"2024-09-17T15:35:38.869077+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28070,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.952380+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:35:38.952380+0000","last_peered":"2024-09-17T15:35:38.952380+0000","last_clean":"2024-09-17T15:35:38.952380+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:35:38.952380+0000","last_undegraded":"2024-09-17T15:35:38.952380+0000","last_fullsized":"2024-09-17T15:35:38.952380+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T15:35:37.866680+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:35:37.866680+0000","last_peered":"2024-09-17T15:35:37.866680+0000","last_clean":"2024-09-17T15:35:37.866680+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:35:37.866680+0000","last_undegraded":"2024-09-17T15:35:37.866680+0000","last_fullsized":"2024-09-17T15:35:37.866680+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21198,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.871071+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:38.871071+0000","last_peered":"2024-09-17T15:35:38.871071+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:38.871071+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869142+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:35:38.869142+0000","last_peered":"2024-09-17T15:35:38.869142+0000","last_clean":"2024-09-17T15:35:38.869142+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:35:38.869142+0000","last_undegraded":"2024-09-17T15:35:38.869142+0000","last_fullsized":"2024-09-17T15:35:38.869142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869195+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:35:38.869195+0000","last_peered":"2024-09-17T15:35:38.869195+0000","last_clean":"2024-09-17T15:35:38.869195+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:35:38.869195+0000","last_undegraded":"2024-09-17T15:35:38.869195+0000","last_fullsized":"2024-09-17T15:35:38.869195+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29021,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.868962+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:38.868962+0000","last_peered":"2024-09-17T15:35:38.868962+0000","last_clean":"2024-09-17T15:35:38.868962+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:38.868962+0000","last_undegraded":"2024-09-17T15:35:38.868962+0000","last_fullsized":"2024-09-17T15:35:38.868962+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32286,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17174,"num_bytes_recovered":41571753,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202629120,"data_stored":202506278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051711,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497256,"kb_used_data":132372,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874584,"statfs":{"total":96636764160,"available":96127574016,"internally_reserved":0,"allocated":135548928,"data_stored":135246137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724656,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465300,"kb_used_data":452,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906540,"statfs":{"total":96636764160,"available":96160296960,"internally_reserved":0,"allocated":462848,"data_stored":141528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":475983531},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,5,3,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247939,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539348,"kb_used_data":66240,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832492,"statfs":{"total":96636764160,"available":96084471808,"internally_reserved":0,"allocated":67829760,"data_stored":67522892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135147520,"data_stored":135111390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:35:42.640 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:35:43.034 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:35:43.036 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:35:43.048 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":381,"stamp":"2024-09-17T15:35:41.811954+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32286,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17175,"num_bytes_recovered":41571772,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1501904,"kb_used_data":199064,"kb_used_omap":13,"kb_used_meta":1302770,"kb_avail":281613616,"statfs":{"total":289910292480,"available":288372342784,"internally_reserved":0,"allocated":203841536,"data_stored":202910557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1334037503},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,5,3,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-9981976,"num_objects":-2438,"num_object_clones":0,"num_object_copies":-4876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-915,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2438,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2438,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":915,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001694"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20912,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.872768+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:38.872768+0000","last_peered":"2024-09-17T15:35:38.872768+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:38.872768+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21047,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T15:35:37.865744+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:37.865744+0000","last_peered":"2024-09-17T15:35:37.865744+0000","last_clean":"2024-09-17T15:35:37.865744+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:37.865744+0000","last_undegraded":"2024-09-17T15:35:37.865744+0000","last_fullsized":"2024-09-17T15:35:37.865744+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21159,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:37.865786+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:37.865786+0000","last_peered":"2024-09-17T15:35:37.865786+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:37.865786+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4125}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21052,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T15:35:37.865634+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:37.865634+0000","last_peered":"2024-09-17T15:35:37.865634+0000","last_clean":"2024-09-17T15:35:37.865634+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:37.865634+0000","last_undegraded":"2024-09-17T15:35:37.865634+0000","last_fullsized":"2024-09-17T15:35:37.865634+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20797,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.870725+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:38.870725+0000","last_peered":"2024-09-17T15:35:38.870725+0000","last_clean":"2024-09-17T15:35:38.870725+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:38.870725+0000","last_undegraded":"2024-09-17T15:35:38.870725+0000","last_fullsized":"2024-09-17T15:35:38.870725+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21556,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870736+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:38.870736+0000","last_peered":"2024-09-17T15:35:38.870736+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:38.870736+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4191},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21091,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870726+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:35:38.870726+0000","last_peered":"2024-09-17T15:35:38.870726+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:35:38.870726+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4087},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21058,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.870606+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:38.870606+0000","last_peered":"2024-09-17T15:35:38.870606+0000","last_clean":"2024-09-17T15:35:38.870606+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:38.870606+0000","last_undegraded":"2024-09-17T15:35:38.870606+0000","last_fullsized":"2024-09-17T15:35:38.870606+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869332+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:35:38.869332+0000","last_peered":"2024-09-17T15:35:38.869332+0000","last_clean":"2024-09-17T15:35:38.869332+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:35:38.869332+0000","last_undegraded":"2024-09-17T15:35:38.869332+0000","last_fullsized":"2024-09-17T15:35:38.869332+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29114,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869044+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:38.869044+0000","last_peered":"2024-09-17T15:35:38.869044+0000","last_clean":"2024-09-17T15:35:38.869044+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:38.869044+0000","last_undegraded":"2024-09-17T15:35:38.869044+0000","last_fullsized":"2024-09-17T15:35:38.869044+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T15:35:37.865651+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:35:37.865651+0000","last_peered":"2024-09-17T15:35:37.865651+0000","last_clean":"2024-09-17T15:35:37.865651+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:35:37.865651+0000","last_undegraded":"2024-09-17T15:35:37.865651+0000","last_fullsized":"2024-09-17T15:35:37.865651+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20821,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870877+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:38.870877+0000","last_peered":"2024-09-17T15:35:38.870877+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:38.870877+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":118,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.870615+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:35:38.870615+0000","last_peered":"2024-09-17T15:35:38.870615+0000","last_clean":"2024-09-17T15:35:38.870615+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:35:38.870615+0000","last_undegraded":"2024-09-17T15:35:38.870615+0000","last_fullsized":"2024-09-17T15:35:38.870615+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":20991,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.431764+0000","last_change":"2024-09-17T15:35:37.572616+0000","last_active":"2024-09-17T15:35:38.431764+0000","last_peered":"2024-09-17T15:35:38.431764+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:35:38.431764+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3909,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3908}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":92,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.868946+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:35:38.868946+0000","last_peered":"2024-09-17T15:35:38.868946+0000","last_clean":"2024-09-17T15:35:38.868946+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:35:38.868946+0000","last_undegraded":"2024-09-17T15:35:38.868946+0000","last_fullsized":"2024-09-17T15:35:38.868946+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29154,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T15:35:37.865961+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:37.865961+0000","last_peered":"2024-09-17T15:35:37.865961+0000","last_clean":"2024-09-17T15:35:37.865961+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:37.865961+0000","last_undegraded":"2024-09-17T15:35:37.865961+0000","last_fullsized":"2024-09-17T15:35:37.865961+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21278,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870971+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:38.870971+0000","last_peered":"2024-09-17T15:35:38.870971+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:38.870971+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869077+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:35:38.869077+0000","last_peered":"2024-09-17T15:35:38.869077+0000","last_clean":"2024-09-17T15:35:38.869077+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:35:38.869077+0000","last_undegraded":"2024-09-17T15:35:38.869077+0000","last_fullsized":"2024-09-17T15:35:38.869077+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28070,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.952380+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:35:38.952380+0000","last_peered":"2024-09-17T15:35:38.952380+0000","last_clean":"2024-09-17T15:35:38.952380+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:35:38.952380+0000","last_undegraded":"2024-09-17T15:35:38.952380+0000","last_fullsized":"2024-09-17T15:35:38.952380+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-17T15:35:37.866680+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:35:37.866680+0000","last_peered":"2024-09-17T15:35:37.866680+0000","last_clean":"2024-09-17T15:35:37.866680+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:35:37.866680+0000","last_undegraded":"2024-09-17T15:35:37.866680+0000","last_fullsized":"2024-09-17T15:35:37.866680+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21198,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.871071+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:38.871071+0000","last_peered":"2024-09-17T15:35:38.871071+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:38.871071+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869142+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:35:38.869142+0000","last_peered":"2024-09-17T15:35:38.869142+0000","last_clean":"2024-09-17T15:35:38.869142+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:35:38.869142+0000","last_undegraded":"2024-09-17T15:35:38.869142+0000","last_fullsized":"2024-09-17T15:35:38.869142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869195+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:35:38.869195+0000","last_peered":"2024-09-17T15:35:38.869195+0000","last_clean":"2024-09-17T15:35:38.869195+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:35:38.869195+0000","last_undegraded":"2024-09-17T15:35:38.869195+0000","last_fullsized":"2024-09-17T15:35:38.869195+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29021,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.868962+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:38.868962+0000","last_peered":"2024-09-17T15:35:38.868962+0000","last_clean":"2024-09-17T15:35:38.868962+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:38.868962+0000","last_undegraded":"2024-09-17T15:35:38.868962+0000","last_fullsized":"2024-09-17T15:35:38.868962+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32286,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17174,"num_bytes_recovered":41571753,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202629120,"data_stored":202506278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051711,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497256,"kb_used_data":132372,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874584,"statfs":{"total":96636764160,"available":96127574016,"internally_reserved":0,"allocated":135548928,"data_stored":135246137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724656,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465300,"kb_used_data":452,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906540,"statfs":{"total":96636764160,"available":96160296960,"internally_reserved":0,"allocated":462848,"data_stored":141528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":475983531},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,5,3,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247939,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539348,"kb_used_data":66240,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832492,"statfs":{"total":96636764160,"available":96084471808,"internally_reserved":0,"allocated":67829760,"data_stored":67522892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135147520,"data_stored":135111390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:35:43.049 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:35:43.049 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'20648", 'reported_seq': 20912, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:38.872768+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:35:38.872768+0000', 'last_peered': '2024-09-17T15:35:38.872768+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:35:38.872768+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'10600", 'ondisk_log_start': "43'10600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20648, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:43.049 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:35:43.049 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'20934", 'reported_seq': 21159, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:37.865786+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:35:37.865786+0000', 'last_peered': '2024-09-17T15:35:37.865786+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:35:37.865786+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "43'10900", 'ondisk_log_start': "43'10900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 19091, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4125}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:35:43.050 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:35:43.050 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'21251", 'reported_seq': 21556, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:38.870736+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:35:38.870736+0000', 'last_peered': '2024-09-17T15:35:38.870736+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:35:38.870736+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "43'11200", 'ondisk_log_start': "43'11200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17750, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4191}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:43.050 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:35:43.050 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'20768", 'reported_seq': 21091, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:38.870726+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:35:38.870726+0000', 'last_peered': '2024-09-17T15:35:38.870726+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:35:38.870726+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "43'10700", 'ondisk_log_start': "43'10700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4089, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17537, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 85, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4087}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:43.050 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:35:43.050 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20542", 'reported_seq': 20821, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:38.870877+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:35:38.870877+0000', 'last_peered': '2024-09-17T15:35:38.870877+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:35:38.870877+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17138, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3943}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:43.050 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:35:43.050 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'20519", 'reported_seq': 20991, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:38.431764+0000', 'last_change': '2024-09-17T15:35:37.572616+0000', 'last_active': '2024-09-17T15:35:38.431764+0000', 'last_peered': '2024-09-17T15:35:38.431764+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:35:38.431764+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3909, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18414, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 169, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3908}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:35:43.050 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:35:43.050 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'20966", 'reported_seq': 21278, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:38.870971+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:35:38.870971+0000', 'last_peered': '2024-09-17T15:35:38.870971+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:35:38.870971+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "43'10900", 'ondisk_log_start': "43'10900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17514, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4057}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:43.051 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:35:43.051 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'20886", 'reported_seq': 21198, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:38.871071+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:35:38.871071+0000', 'last_peered': '2024-09-17T15:35:38.871071+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:35:38.871071+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'10800", 'ondisk_log_start': "43'10800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17673, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3980}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:43.051 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:35:43.478 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:35:43.479 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 32286/24 objects degraded (134525.000%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":680,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1537949696,"bytes_avail":288372342784,"bytes_total":289910292480,"degraded_objects":32286,"degraded_total":24,"degraded_ratio":1345.25,"recovering_objects_per_sec":114,"recovering_bytes_per_sec":60,"recovering_keys_per_sec":0,"num_objects_recovered":915,"num_bytes_recovered":483,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":304},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:35:43.495 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:35:43.495 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:35:43.865 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:35:43.866 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:35:43.881 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":382,"stamp":"2024-09-17T15:35:43.812268+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31411,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18050,"num_bytes_recovered":41571772,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1501916,"kb_used_data":199072,"kb_used_omap":13,"kb_used_meta":1302770,"kb_avail":281613604,"statfs":{"total":289910292480,"available":288372330496,"internally_reserved":0,"allocated":203849728,"data_stored":202913463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1334037503},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,5,3,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-3080216,"num_objects":-753,"num_object_clones":0,"num_object_copies":-1506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1626,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":753,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1626,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001378"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20912,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.872768+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:38.872768+0000","last_peered":"2024-09-17T15:35:38.872768+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:38.872768+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21049,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866657+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:38.866657+0000","last_peered":"2024-09-17T15:35:38.866657+0000","last_clean":"2024-09-17T15:35:38.866657+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:38.866657+0000","last_undegraded":"2024-09-17T15:35:38.866657+0000","last_fullsized":"2024-09-17T15:35:38.866657+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21161,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.866979+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:38.866979+0000","last_peered":"2024-09-17T15:35:38.866979+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:38.866979+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4125}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21054,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866816+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:38.866816+0000","last_peered":"2024-09-17T15:35:38.866816+0000","last_clean":"2024-09-17T15:35:38.866816+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:38.866816+0000","last_undegraded":"2024-09-17T15:35:38.866816+0000","last_fullsized":"2024-09-17T15:35:38.866816+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20797,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.870725+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:38.870725+0000","last_peered":"2024-09-17T15:35:38.870725+0000","last_clean":"2024-09-17T15:35:38.870725+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:38.870725+0000","last_undegraded":"2024-09-17T15:35:38.870725+0000","last_fullsized":"2024-09-17T15:35:38.870725+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21556,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870736+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:38.870736+0000","last_peered":"2024-09-17T15:35:38.870736+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:38.870736+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4191},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21091,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870726+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:35:38.870726+0000","last_peered":"2024-09-17T15:35:38.870726+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:35:38.870726+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4087},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21058,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.870606+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:38.870606+0000","last_peered":"2024-09-17T15:35:38.870606+0000","last_clean":"2024-09-17T15:35:38.870606+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:38.870606+0000","last_undegraded":"2024-09-17T15:35:38.870606+0000","last_fullsized":"2024-09-17T15:35:38.870606+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869332+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:35:38.869332+0000","last_peered":"2024-09-17T15:35:38.869332+0000","last_clean":"2024-09-17T15:35:38.869332+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:35:38.869332+0000","last_undegraded":"2024-09-17T15:35:38.869332+0000","last_fullsized":"2024-09-17T15:35:38.869332+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29114,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869044+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:38.869044+0000","last_peered":"2024-09-17T15:35:38.869044+0000","last_clean":"2024-09-17T15:35:38.869044+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:38.869044+0000","last_undegraded":"2024-09-17T15:35:38.869044+0000","last_fullsized":"2024-09-17T15:35:38.869044+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866609+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:35:38.866609+0000","last_peered":"2024-09-17T15:35:38.866609+0000","last_clean":"2024-09-17T15:35:38.866609+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:35:38.866609+0000","last_undegraded":"2024-09-17T15:35:38.866609+0000","last_fullsized":"2024-09-17T15:35:38.866609+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20821,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870877+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:38.870877+0000","last_peered":"2024-09-17T15:35:38.870877+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:38.870877+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":118,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.870615+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:35:38.870615+0000","last_peered":"2024-09-17T15:35:38.870615+0000","last_clean":"2024-09-17T15:35:38.870615+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:35:38.870615+0000","last_undegraded":"2024-09-17T15:35:38.870615+0000","last_fullsized":"2024-09-17T15:35:38.870615+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":22743,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:43.434977+0000","last_change":"2024-09-17T15:35:37.572616+0000","last_active":"2024-09-17T15:35:43.434977+0000","last_peered":"2024-09-17T15:35:43.434977+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:35:43.434977+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1044,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3033}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":92,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.868946+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:35:38.868946+0000","last_peered":"2024-09-17T15:35:38.868946+0000","last_clean":"2024-09-17T15:35:38.868946+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:35:38.868946+0000","last_undegraded":"2024-09-17T15:35:38.868946+0000","last_fullsized":"2024-09-17T15:35:38.868946+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29156,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866426+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:38.866426+0000","last_peered":"2024-09-17T15:35:38.866426+0000","last_clean":"2024-09-17T15:35:38.866426+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:38.866426+0000","last_undegraded":"2024-09-17T15:35:38.866426+0000","last_fullsized":"2024-09-17T15:35:38.866426+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21278,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870971+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:38.870971+0000","last_peered":"2024-09-17T15:35:38.870971+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:38.870971+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869077+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:35:38.869077+0000","last_peered":"2024-09-17T15:35:38.869077+0000","last_clean":"2024-09-17T15:35:38.869077+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:35:38.869077+0000","last_undegraded":"2024-09-17T15:35:38.869077+0000","last_fullsized":"2024-09-17T15:35:38.869077+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28070,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.952380+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:35:38.952380+0000","last_peered":"2024-09-17T15:35:38.952380+0000","last_clean":"2024-09-17T15:35:38.952380+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:35:38.952380+0000","last_undegraded":"2024-09-17T15:35:38.952380+0000","last_fullsized":"2024-09-17T15:35:38.952380+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.867367+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:35:38.867367+0000","last_peered":"2024-09-17T15:35:38.867367+0000","last_clean":"2024-09-17T15:35:38.867367+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:35:38.867367+0000","last_undegraded":"2024-09-17T15:35:38.867367+0000","last_fullsized":"2024-09-17T15:35:38.867367+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21198,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.871071+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:38.871071+0000","last_peered":"2024-09-17T15:35:38.871071+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:38.871071+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869142+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:35:38.869142+0000","last_peered":"2024-09-17T15:35:38.869142+0000","last_clean":"2024-09-17T15:35:38.869142+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:35:38.869142+0000","last_undegraded":"2024-09-17T15:35:38.869142+0000","last_fullsized":"2024-09-17T15:35:38.869142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869195+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:35:38.869195+0000","last_peered":"2024-09-17T15:35:38.869195+0000","last_clean":"2024-09-17T15:35:38.869195+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:35:38.869195+0000","last_undegraded":"2024-09-17T15:35:38.869195+0000","last_fullsized":"2024-09-17T15:35:38.869195+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29021,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.868962+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:38.868962+0000","last_peered":"2024-09-17T15:35:38.868962+0000","last_clean":"2024-09-17T15:35:38.868962+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:38.868962+0000","last_undegraded":"2024-09-17T15:35:38.868962+0000","last_fullsized":"2024-09-17T15:35:38.868962+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31411,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18049,"num_bytes_recovered":41571753,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202629120,"data_stored":202506278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051712,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497268,"kb_used_data":132380,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874572,"statfs":{"total":96636764160,"available":96127561728,"internally_reserved":0,"allocated":135557120,"data_stored":135249043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724656,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465300,"kb_used_data":452,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906540,"statfs":{"total":96636764160,"available":96160296960,"internally_reserved":0,"allocated":462848,"data_stored":141528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":475983531},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,5,3,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247939,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539348,"kb_used_data":66240,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832492,"statfs":{"total":96636764160,"available":96084471808,"internally_reserved":0,"allocated":67829760,"data_stored":67522892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135147520,"data_stored":135111390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:35:46.883 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:35:47.262 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:35:47.263 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:35:47.275 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":383,"stamp":"2024-09-17T15:35:45.812601+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31411,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18050,"num_bytes_recovered":41571772,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1510256,"kb_used_data":199088,"kb_used_omap":13,"kb_used_meta":1311090,"kb_avail":281605264,"statfs":{"total":289910292480,"available":288363790336,"internally_reserved":0,"allocated":203866112,"data_stored":202919275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1342557183},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-3080216,"num_objects":-753,"num_object_clones":0,"num_object_copies":-1506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1626,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":753,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1626,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001346"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20912,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.872768+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:38.872768+0000","last_peered":"2024-09-17T15:35:38.872768+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:38.872768+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21049,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866657+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:38.866657+0000","last_peered":"2024-09-17T15:35:38.866657+0000","last_clean":"2024-09-17T15:35:38.866657+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:38.866657+0000","last_undegraded":"2024-09-17T15:35:38.866657+0000","last_fullsized":"2024-09-17T15:35:38.866657+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21161,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.866979+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:38.866979+0000","last_peered":"2024-09-17T15:35:38.866979+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:38.866979+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4125}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21054,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866816+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:38.866816+0000","last_peered":"2024-09-17T15:35:38.866816+0000","last_clean":"2024-09-17T15:35:38.866816+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:38.866816+0000","last_undegraded":"2024-09-17T15:35:38.866816+0000","last_fullsized":"2024-09-17T15:35:38.866816+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20797,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.870725+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:38.870725+0000","last_peered":"2024-09-17T15:35:38.870725+0000","last_clean":"2024-09-17T15:35:38.870725+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:38.870725+0000","last_undegraded":"2024-09-17T15:35:38.870725+0000","last_fullsized":"2024-09-17T15:35:38.870725+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21556,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870736+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:38.870736+0000","last_peered":"2024-09-17T15:35:38.870736+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:38.870736+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4191},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21091,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870726+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:35:38.870726+0000","last_peered":"2024-09-17T15:35:38.870726+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:35:38.870726+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4087},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21058,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.870606+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:38.870606+0000","last_peered":"2024-09-17T15:35:38.870606+0000","last_clean":"2024-09-17T15:35:38.870606+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:38.870606+0000","last_undegraded":"2024-09-17T15:35:38.870606+0000","last_fullsized":"2024-09-17T15:35:38.870606+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869332+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:35:38.869332+0000","last_peered":"2024-09-17T15:35:38.869332+0000","last_clean":"2024-09-17T15:35:38.869332+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:35:38.869332+0000","last_undegraded":"2024-09-17T15:35:38.869332+0000","last_fullsized":"2024-09-17T15:35:38.869332+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29114,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869044+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:38.869044+0000","last_peered":"2024-09-17T15:35:38.869044+0000","last_clean":"2024-09-17T15:35:38.869044+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:38.869044+0000","last_undegraded":"2024-09-17T15:35:38.869044+0000","last_fullsized":"2024-09-17T15:35:38.869044+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866609+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:35:38.866609+0000","last_peered":"2024-09-17T15:35:38.866609+0000","last_clean":"2024-09-17T15:35:38.866609+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:35:38.866609+0000","last_undegraded":"2024-09-17T15:35:38.866609+0000","last_fullsized":"2024-09-17T15:35:38.866609+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20821,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870877+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:38.870877+0000","last_peered":"2024-09-17T15:35:38.870877+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:38.870877+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":118,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.870615+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:35:38.870615+0000","last_peered":"2024-09-17T15:35:38.870615+0000","last_clean":"2024-09-17T15:35:38.870615+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:35:38.870615+0000","last_undegraded":"2024-09-17T15:35:38.870615+0000","last_fullsized":"2024-09-17T15:35:38.870615+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":22743,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:43.434977+0000","last_change":"2024-09-17T15:35:37.572616+0000","last_active":"2024-09-17T15:35:43.434977+0000","last_peered":"2024-09-17T15:35:43.434977+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:35:43.434977+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1044,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3033}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":92,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.868946+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:35:38.868946+0000","last_peered":"2024-09-17T15:35:38.868946+0000","last_clean":"2024-09-17T15:35:38.868946+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:35:38.868946+0000","last_undegraded":"2024-09-17T15:35:38.868946+0000","last_fullsized":"2024-09-17T15:35:38.868946+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29156,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866426+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:38.866426+0000","last_peered":"2024-09-17T15:35:38.866426+0000","last_clean":"2024-09-17T15:35:38.866426+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:38.866426+0000","last_undegraded":"2024-09-17T15:35:38.866426+0000","last_fullsized":"2024-09-17T15:35:38.866426+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21278,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870971+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:38.870971+0000","last_peered":"2024-09-17T15:35:38.870971+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:38.870971+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869077+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:35:38.869077+0000","last_peered":"2024-09-17T15:35:38.869077+0000","last_clean":"2024-09-17T15:35:38.869077+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:35:38.869077+0000","last_undegraded":"2024-09-17T15:35:38.869077+0000","last_fullsized":"2024-09-17T15:35:38.869077+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28070,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.952380+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:35:38.952380+0000","last_peered":"2024-09-17T15:35:38.952380+0000","last_clean":"2024-09-17T15:35:38.952380+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:35:38.952380+0000","last_undegraded":"2024-09-17T15:35:38.952380+0000","last_fullsized":"2024-09-17T15:35:38.952380+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.867367+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:35:38.867367+0000","last_peered":"2024-09-17T15:35:38.867367+0000","last_clean":"2024-09-17T15:35:38.867367+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:35:38.867367+0000","last_undegraded":"2024-09-17T15:35:38.867367+0000","last_fullsized":"2024-09-17T15:35:38.867367+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21199,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.872159+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:38.872159+0000","last_peered":"2024-09-17T15:35:38.872159+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:38.872159+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869142+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:35:38.869142+0000","last_peered":"2024-09-17T15:35:38.869142+0000","last_clean":"2024-09-17T15:35:38.869142+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:35:38.869142+0000","last_undegraded":"2024-09-17T15:35:38.869142+0000","last_fullsized":"2024-09-17T15:35:38.869142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869195+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:35:38.869195+0000","last_peered":"2024-09-17T15:35:38.869195+0000","last_clean":"2024-09-17T15:35:38.869195+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:35:38.869195+0000","last_undegraded":"2024-09-17T15:35:38.869195+0000","last_fullsized":"2024-09-17T15:35:38.869195+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29021,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.868962+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:38.868962+0000","last_peered":"2024-09-17T15:35:38.868962+0000","last_clean":"2024-09-17T15:35:38.868962+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:38.868962+0000","last_undegraded":"2024-09-17T15:35:38.868962+0000","last_fullsized":"2024-09-17T15:35:38.868962+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31411,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18049,"num_bytes_recovered":41571753,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202629120,"data_stored":202506278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051712,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497268,"kb_used_data":132380,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874572,"statfs":{"total":96636764160,"available":96127561728,"internally_reserved":0,"allocated":135557120,"data_stored":135249043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724657,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473628,"kb_used_data":460,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898212,"statfs":{"total":96636764160,"available":96151769088,"internally_reserved":0,"allocated":471040,"data_stored":144434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":484503211},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247940,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539360,"kb_used_data":66248,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832480,"statfs":{"total":96636764160,"available":96084459520,"internally_reserved":0,"allocated":67837952,"data_stored":67525798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135147520,"data_stored":135111390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:35:47.276 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:35:47.642 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:35:47.642 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:35:47.655 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":383,"stamp":"2024-09-17T15:35:45.812601+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31411,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18050,"num_bytes_recovered":41571772,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1510256,"kb_used_data":199088,"kb_used_omap":13,"kb_used_meta":1311090,"kb_avail":281605264,"statfs":{"total":289910292480,"available":288363790336,"internally_reserved":0,"allocated":203866112,"data_stored":202919275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1342557183},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-3080216,"num_objects":-753,"num_object_clones":0,"num_object_copies":-1506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1626,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":753,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1626,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001346"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20912,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.872768+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:38.872768+0000","last_peered":"2024-09-17T15:35:38.872768+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:38.872768+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21049,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866657+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:38.866657+0000","last_peered":"2024-09-17T15:35:38.866657+0000","last_clean":"2024-09-17T15:35:38.866657+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:38.866657+0000","last_undegraded":"2024-09-17T15:35:38.866657+0000","last_fullsized":"2024-09-17T15:35:38.866657+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21161,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.866979+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:38.866979+0000","last_peered":"2024-09-17T15:35:38.866979+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:38.866979+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4125}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21054,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866816+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:38.866816+0000","last_peered":"2024-09-17T15:35:38.866816+0000","last_clean":"2024-09-17T15:35:38.866816+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:38.866816+0000","last_undegraded":"2024-09-17T15:35:38.866816+0000","last_fullsized":"2024-09-17T15:35:38.866816+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20797,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.870725+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:38.870725+0000","last_peered":"2024-09-17T15:35:38.870725+0000","last_clean":"2024-09-17T15:35:38.870725+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:38.870725+0000","last_undegraded":"2024-09-17T15:35:38.870725+0000","last_fullsized":"2024-09-17T15:35:38.870725+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21556,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870736+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:38.870736+0000","last_peered":"2024-09-17T15:35:38.870736+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:38.870736+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4191},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21091,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870726+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:35:38.870726+0000","last_peered":"2024-09-17T15:35:38.870726+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:35:38.870726+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4087},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21058,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.870606+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:38.870606+0000","last_peered":"2024-09-17T15:35:38.870606+0000","last_clean":"2024-09-17T15:35:38.870606+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:38.870606+0000","last_undegraded":"2024-09-17T15:35:38.870606+0000","last_fullsized":"2024-09-17T15:35:38.870606+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869332+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:35:38.869332+0000","last_peered":"2024-09-17T15:35:38.869332+0000","last_clean":"2024-09-17T15:35:38.869332+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:35:38.869332+0000","last_undegraded":"2024-09-17T15:35:38.869332+0000","last_fullsized":"2024-09-17T15:35:38.869332+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29114,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869044+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:38.869044+0000","last_peered":"2024-09-17T15:35:38.869044+0000","last_clean":"2024-09-17T15:35:38.869044+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:38.869044+0000","last_undegraded":"2024-09-17T15:35:38.869044+0000","last_fullsized":"2024-09-17T15:35:38.869044+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866609+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:35:38.866609+0000","last_peered":"2024-09-17T15:35:38.866609+0000","last_clean":"2024-09-17T15:35:38.866609+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:35:38.866609+0000","last_undegraded":"2024-09-17T15:35:38.866609+0000","last_fullsized":"2024-09-17T15:35:38.866609+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20821,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870877+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:38.870877+0000","last_peered":"2024-09-17T15:35:38.870877+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:38.870877+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":118,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.870615+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:35:38.870615+0000","last_peered":"2024-09-17T15:35:38.870615+0000","last_clean":"2024-09-17T15:35:38.870615+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:35:38.870615+0000","last_undegraded":"2024-09-17T15:35:38.870615+0000","last_fullsized":"2024-09-17T15:35:38.870615+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":22743,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:43.434977+0000","last_change":"2024-09-17T15:35:37.572616+0000","last_active":"2024-09-17T15:35:43.434977+0000","last_peered":"2024-09-17T15:35:43.434977+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:35:43.434977+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1044,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3033}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":92,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.868946+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:35:38.868946+0000","last_peered":"2024-09-17T15:35:38.868946+0000","last_clean":"2024-09-17T15:35:38.868946+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:35:38.868946+0000","last_undegraded":"2024-09-17T15:35:38.868946+0000","last_fullsized":"2024-09-17T15:35:38.868946+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29156,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866426+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:38.866426+0000","last_peered":"2024-09-17T15:35:38.866426+0000","last_clean":"2024-09-17T15:35:38.866426+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:38.866426+0000","last_undegraded":"2024-09-17T15:35:38.866426+0000","last_fullsized":"2024-09-17T15:35:38.866426+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21278,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870971+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:38.870971+0000","last_peered":"2024-09-17T15:35:38.870971+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:38.870971+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869077+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:35:38.869077+0000","last_peered":"2024-09-17T15:35:38.869077+0000","last_clean":"2024-09-17T15:35:38.869077+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:35:38.869077+0000","last_undegraded":"2024-09-17T15:35:38.869077+0000","last_fullsized":"2024-09-17T15:35:38.869077+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28070,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.952380+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:35:38.952380+0000","last_peered":"2024-09-17T15:35:38.952380+0000","last_clean":"2024-09-17T15:35:38.952380+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:35:38.952380+0000","last_undegraded":"2024-09-17T15:35:38.952380+0000","last_fullsized":"2024-09-17T15:35:38.952380+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.867367+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:35:38.867367+0000","last_peered":"2024-09-17T15:35:38.867367+0000","last_clean":"2024-09-17T15:35:38.867367+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:35:38.867367+0000","last_undegraded":"2024-09-17T15:35:38.867367+0000","last_fullsized":"2024-09-17T15:35:38.867367+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21199,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.872159+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:38.872159+0000","last_peered":"2024-09-17T15:35:38.872159+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:38.872159+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869142+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:35:38.869142+0000","last_peered":"2024-09-17T15:35:38.869142+0000","last_clean":"2024-09-17T15:35:38.869142+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:35:38.869142+0000","last_undegraded":"2024-09-17T15:35:38.869142+0000","last_fullsized":"2024-09-17T15:35:38.869142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869195+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:35:38.869195+0000","last_peered":"2024-09-17T15:35:38.869195+0000","last_clean":"2024-09-17T15:35:38.869195+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:35:38.869195+0000","last_undegraded":"2024-09-17T15:35:38.869195+0000","last_fullsized":"2024-09-17T15:35:38.869195+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29021,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.868962+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:38.868962+0000","last_peered":"2024-09-17T15:35:38.868962+0000","last_clean":"2024-09-17T15:35:38.868962+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:38.868962+0000","last_undegraded":"2024-09-17T15:35:38.868962+0000","last_fullsized":"2024-09-17T15:35:38.868962+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31411,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18049,"num_bytes_recovered":41571753,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202629120,"data_stored":202506278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051712,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497268,"kb_used_data":132380,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874572,"statfs":{"total":96636764160,"available":96127561728,"internally_reserved":0,"allocated":135557120,"data_stored":135249043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724657,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473628,"kb_used_data":460,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898212,"statfs":{"total":96636764160,"available":96151769088,"internally_reserved":0,"allocated":471040,"data_stored":144434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":484503211},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247940,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539360,"kb_used_data":66248,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832480,"statfs":{"total":96636764160,"available":96084459520,"internally_reserved":0,"allocated":67837952,"data_stored":67525798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135147520,"data_stored":135111390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:35:47.656 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:35:47.656 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'20648", 'reported_seq': 20912, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:38.872768+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:35:38.872768+0000', 'last_peered': '2024-09-17T15:35:38.872768+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:35:38.872768+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'10600", 'ondisk_log_start': "43'10600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20648, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:47.656 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:35:47.657 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'20934", 'reported_seq': 21161, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:38.866979+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:35:38.866979+0000', 'last_peered': '2024-09-17T15:35:38.866979+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:35:38.866979+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "43'10900", 'ondisk_log_start': "43'10900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 19091, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4125}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:35:47.657 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:35:47.657 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'21251", 'reported_seq': 21556, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:38.870736+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:35:38.870736+0000', 'last_peered': '2024-09-17T15:35:38.870736+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:35:38.870736+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "43'11200", 'ondisk_log_start': "43'11200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17750, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4191}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:47.657 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:35:47.657 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'20768", 'reported_seq': 21091, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:38.870726+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:35:38.870726+0000', 'last_peered': '2024-09-17T15:35:38.870726+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:35:38.870726+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "43'10700", 'ondisk_log_start': "43'10700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4089, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17537, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 85, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4087}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:47.657 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:35:47.657 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20542", 'reported_seq': 20821, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:38.870877+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:35:38.870877+0000', 'last_peered': '2024-09-17T15:35:38.870877+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:35:38.870877+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17138, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3943}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:47.657 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:35:47.657 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'20519", 'reported_seq': 22743, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:43.434977+0000', 'last_change': '2024-09-17T15:35:37.572616+0000', 'last_active': '2024-09-17T15:35:43.434977+0000', 'last_peered': '2024-09-17T15:35:43.434977+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:35:43.434977+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3034, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18414, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1044, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3033}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:35:47.657 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:35:47.658 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'20966", 'reported_seq': 21278, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:38.870971+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:35:38.870971+0000', 'last_peered': '2024-09-17T15:35:38.870971+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:35:38.870971+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "43'10900", 'ondisk_log_start': "43'10900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17514, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4057}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:47.658 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:35:47.658 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'20886", 'reported_seq': 21199, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:38.872159+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:35:38.872159+0000', 'last_peered': '2024-09-17T15:35:38.872159+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:35:38.872159+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'10800", 'ondisk_log_start': "43'10800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17673, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3980}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:47.658 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:35:48.068 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:35:48.068 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 31411/24 objects degraded (130879.167%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":685,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1546502144,"bytes_avail":288363790336,"bytes_total":289910292480,"degraded_objects":31411,"degraded_total":24,"degraded_ratio":1308.7916666666667,"recovering_objects_per_sec":203,"recovering_bytes_per_sec":60,"recovering_keys_per_sec":0,"num_objects_recovered":1626,"num_bytes_recovered":483,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":94},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:35:48.083 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:35:48.083 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:35:48.429 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:35:48.430 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:35:48.443 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":384,"stamp":"2024-09-17T15:35:47.813038+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31411,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18050,"num_bytes_recovered":41571772,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1510256,"kb_used_data":199088,"kb_used_omap":13,"kb_used_meta":1311090,"kb_avail":281605264,"statfs":{"total":289910292480,"available":288363790336,"internally_reserved":0,"allocated":203866112,"data_stored":202919275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1342557183},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-3080216,"num_objects":-753,"num_object_clones":0,"num_object_copies":-1506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1626,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":753,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1626,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.949073"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20912,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.872768+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:38.872768+0000","last_peered":"2024-09-17T15:35:38.872768+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:38.872768+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21049,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866657+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:38.866657+0000","last_peered":"2024-09-17T15:35:38.866657+0000","last_clean":"2024-09-17T15:35:38.866657+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:38.866657+0000","last_undegraded":"2024-09-17T15:35:38.866657+0000","last_fullsized":"2024-09-17T15:35:38.866657+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21161,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.866979+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:38.866979+0000","last_peered":"2024-09-17T15:35:38.866979+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:38.866979+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4125}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21054,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866816+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:38.866816+0000","last_peered":"2024-09-17T15:35:38.866816+0000","last_clean":"2024-09-17T15:35:38.866816+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:38.866816+0000","last_undegraded":"2024-09-17T15:35:38.866816+0000","last_fullsized":"2024-09-17T15:35:38.866816+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20797,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.870725+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:38.870725+0000","last_peered":"2024-09-17T15:35:38.870725+0000","last_clean":"2024-09-17T15:35:38.870725+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:38.870725+0000","last_undegraded":"2024-09-17T15:35:38.870725+0000","last_fullsized":"2024-09-17T15:35:38.870725+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21556,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870736+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:38.870736+0000","last_peered":"2024-09-17T15:35:38.870736+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:38.870736+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4191},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21091,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870726+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:35:38.870726+0000","last_peered":"2024-09-17T15:35:38.870726+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:35:38.870726+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4087},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21058,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.870606+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:38.870606+0000","last_peered":"2024-09-17T15:35:38.870606+0000","last_clean":"2024-09-17T15:35:38.870606+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:38.870606+0000","last_undegraded":"2024-09-17T15:35:38.870606+0000","last_fullsized":"2024-09-17T15:35:38.870606+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869332+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:35:38.869332+0000","last_peered":"2024-09-17T15:35:38.869332+0000","last_clean":"2024-09-17T15:35:38.869332+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:35:38.869332+0000","last_undegraded":"2024-09-17T15:35:38.869332+0000","last_fullsized":"2024-09-17T15:35:38.869332+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29114,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869044+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:38.869044+0000","last_peered":"2024-09-17T15:35:38.869044+0000","last_clean":"2024-09-17T15:35:38.869044+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:38.869044+0000","last_undegraded":"2024-09-17T15:35:38.869044+0000","last_fullsized":"2024-09-17T15:35:38.869044+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866609+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:35:38.866609+0000","last_peered":"2024-09-17T15:35:38.866609+0000","last_clean":"2024-09-17T15:35:38.866609+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:35:38.866609+0000","last_undegraded":"2024-09-17T15:35:38.866609+0000","last_fullsized":"2024-09-17T15:35:38.866609+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20821,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870877+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:38.870877+0000","last_peered":"2024-09-17T15:35:38.870877+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:38.870877+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":118,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.870615+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:35:38.870615+0000","last_peered":"2024-09-17T15:35:38.870615+0000","last_clean":"2024-09-17T15:35:38.870615+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:35:38.870615+0000","last_undegraded":"2024-09-17T15:35:38.870615+0000","last_fullsized":"2024-09-17T15:35:38.870615+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":22743,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:43.434977+0000","last_change":"2024-09-17T15:35:37.572616+0000","last_active":"2024-09-17T15:35:43.434977+0000","last_peered":"2024-09-17T15:35:43.434977+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:35:43.434977+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1044,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3033}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":92,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.868946+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:35:38.868946+0000","last_peered":"2024-09-17T15:35:38.868946+0000","last_clean":"2024-09-17T15:35:38.868946+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:35:38.868946+0000","last_undegraded":"2024-09-17T15:35:38.868946+0000","last_fullsized":"2024-09-17T15:35:38.868946+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29156,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866426+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:38.866426+0000","last_peered":"2024-09-17T15:35:38.866426+0000","last_clean":"2024-09-17T15:35:38.866426+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:38.866426+0000","last_undegraded":"2024-09-17T15:35:38.866426+0000","last_fullsized":"2024-09-17T15:35:38.866426+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21278,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870971+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:38.870971+0000","last_peered":"2024-09-17T15:35:38.870971+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:38.870971+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869077+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:35:38.869077+0000","last_peered":"2024-09-17T15:35:38.869077+0000","last_clean":"2024-09-17T15:35:38.869077+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:35:38.869077+0000","last_undegraded":"2024-09-17T15:35:38.869077+0000","last_fullsized":"2024-09-17T15:35:38.869077+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28070,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.952380+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:35:38.952380+0000","last_peered":"2024-09-17T15:35:38.952380+0000","last_clean":"2024-09-17T15:35:38.952380+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:35:38.952380+0000","last_undegraded":"2024-09-17T15:35:38.952380+0000","last_fullsized":"2024-09-17T15:35:38.952380+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.867367+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:35:38.867367+0000","last_peered":"2024-09-17T15:35:38.867367+0000","last_clean":"2024-09-17T15:35:38.867367+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:35:38.867367+0000","last_undegraded":"2024-09-17T15:35:38.867367+0000","last_fullsized":"2024-09-17T15:35:38.867367+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21199,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.872159+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:38.872159+0000","last_peered":"2024-09-17T15:35:38.872159+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:38.872159+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869142+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:35:38.869142+0000","last_peered":"2024-09-17T15:35:38.869142+0000","last_clean":"2024-09-17T15:35:38.869142+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:35:38.869142+0000","last_undegraded":"2024-09-17T15:35:38.869142+0000","last_fullsized":"2024-09-17T15:35:38.869142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869195+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:35:38.869195+0000","last_peered":"2024-09-17T15:35:38.869195+0000","last_clean":"2024-09-17T15:35:38.869195+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:35:38.869195+0000","last_undegraded":"2024-09-17T15:35:38.869195+0000","last_fullsized":"2024-09-17T15:35:38.869195+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29021,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.868962+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:38.868962+0000","last_peered":"2024-09-17T15:35:38.868962+0000","last_clean":"2024-09-17T15:35:38.868962+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:38.868962+0000","last_undegraded":"2024-09-17T15:35:38.868962+0000","last_fullsized":"2024-09-17T15:35:38.868962+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31411,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18049,"num_bytes_recovered":41571753,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202629120,"data_stored":202506278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051712,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497268,"kb_used_data":132380,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874572,"statfs":{"total":96636764160,"available":96127561728,"internally_reserved":0,"allocated":135557120,"data_stored":135249043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724657,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473628,"kb_used_data":460,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898212,"statfs":{"total":96636764160,"available":96151769088,"internally_reserved":0,"allocated":471040,"data_stored":144434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":484503211},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247940,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539360,"kb_used_data":66248,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832480,"statfs":{"total":96636764160,"available":96084459520,"internally_reserved":0,"allocated":67837952,"data_stored":67525798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135147520,"data_stored":135111390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:35:51.445 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:35:51.760 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:35:51.760 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:35:51.773 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":385,"stamp":"2024-09-17T15:35:49.813364+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30332,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19129,"num_bytes_recovered":41571772,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1501936,"kb_used_data":199088,"kb_used_omap":13,"kb_used_meta":1302770,"kb_avail":281613584,"statfs":{"total":289910292480,"available":288372310016,"internally_reserved":0,"allocated":203866112,"data_stored":202919275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1334037503},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1769472,"num_objects":-432,"num_object_clones":0,"num_object_copies":-864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2580,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":432,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2580,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.948327"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20912,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.872768+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:38.872768+0000","last_peered":"2024-09-17T15:35:38.872768+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:38.872768+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21049,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866657+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:38.866657+0000","last_peered":"2024-09-17T15:35:38.866657+0000","last_clean":"2024-09-17T15:35:38.866657+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:38.866657+0000","last_undegraded":"2024-09-17T15:35:38.866657+0000","last_fullsized":"2024-09-17T15:35:38.866657+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21161,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.866979+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:38.866979+0000","last_peered":"2024-09-17T15:35:38.866979+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:38.866979+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4125}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21054,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866816+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:38.866816+0000","last_peered":"2024-09-17T15:35:38.866816+0000","last_clean":"2024-09-17T15:35:38.866816+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:38.866816+0000","last_undegraded":"2024-09-17T15:35:38.866816+0000","last_fullsized":"2024-09-17T15:35:38.866816+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20797,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.870725+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:38.870725+0000","last_peered":"2024-09-17T15:35:38.870725+0000","last_clean":"2024-09-17T15:35:38.870725+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:38.870725+0000","last_undegraded":"2024-09-17T15:35:38.870725+0000","last_fullsized":"2024-09-17T15:35:38.870725+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21556,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870736+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:38.870736+0000","last_peered":"2024-09-17T15:35:38.870736+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:38.870736+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4191},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21091,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870726+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:35:38.870726+0000","last_peered":"2024-09-17T15:35:38.870726+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:35:38.870726+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4087},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21058,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.870606+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:38.870606+0000","last_peered":"2024-09-17T15:35:38.870606+0000","last_clean":"2024-09-17T15:35:38.870606+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:38.870606+0000","last_undegraded":"2024-09-17T15:35:38.870606+0000","last_fullsized":"2024-09-17T15:35:38.870606+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869332+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:35:38.869332+0000","last_peered":"2024-09-17T15:35:38.869332+0000","last_clean":"2024-09-17T15:35:38.869332+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:35:38.869332+0000","last_undegraded":"2024-09-17T15:35:38.869332+0000","last_fullsized":"2024-09-17T15:35:38.869332+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29114,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869044+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:38.869044+0000","last_peered":"2024-09-17T15:35:38.869044+0000","last_clean":"2024-09-17T15:35:38.869044+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:38.869044+0000","last_undegraded":"2024-09-17T15:35:38.869044+0000","last_fullsized":"2024-09-17T15:35:38.869044+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866649+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:35:38.866649+0000","last_peered":"2024-09-17T15:35:38.866649+0000","last_clean":"2024-09-17T15:35:38.866649+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:35:38.866649+0000","last_undegraded":"2024-09-17T15:35:38.866649+0000","last_fullsized":"2024-09-17T15:35:38.866649+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20821,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870877+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:38.870877+0000","last_peered":"2024-09-17T15:35:38.870877+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:38.870877+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":118,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.870615+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:35:38.870615+0000","last_peered":"2024-09-17T15:35:38.870615+0000","last_clean":"2024-09-17T15:35:38.870615+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:35:38.870615+0000","last_undegraded":"2024-09-17T15:35:38.870615+0000","last_fullsized":"2024-09-17T15:35:38.870615+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":24902,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:48.432407+0000","last_change":"2024-09-17T15:35:37.572616+0000","last_active":"2024-09-17T15:35:48.432407+0000","last_peered":"2024-09-17T15:35:48.432407+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:35:48.432407+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1955,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2123,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1954}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":92,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.868946+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:35:38.868946+0000","last_peered":"2024-09-17T15:35:38.868946+0000","last_clean":"2024-09-17T15:35:38.868946+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:35:38.868946+0000","last_undegraded":"2024-09-17T15:35:38.868946+0000","last_fullsized":"2024-09-17T15:35:38.868946+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29156,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866426+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:38.866426+0000","last_peered":"2024-09-17T15:35:38.866426+0000","last_clean":"2024-09-17T15:35:38.866426+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:38.866426+0000","last_undegraded":"2024-09-17T15:35:38.866426+0000","last_fullsized":"2024-09-17T15:35:38.866426+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21278,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870971+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:38.870971+0000","last_peered":"2024-09-17T15:35:38.870971+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:38.870971+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869077+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:35:38.869077+0000","last_peered":"2024-09-17T15:35:38.869077+0000","last_clean":"2024-09-17T15:35:38.869077+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:35:38.869077+0000","last_undegraded":"2024-09-17T15:35:38.869077+0000","last_fullsized":"2024-09-17T15:35:38.869077+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28070,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.952380+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:35:38.952380+0000","last_peered":"2024-09-17T15:35:38.952380+0000","last_clean":"2024-09-17T15:35:38.952380+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:35:38.952380+0000","last_undegraded":"2024-09-17T15:35:38.952380+0000","last_fullsized":"2024-09-17T15:35:38.952380+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.867463+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:35:38.867463+0000","last_peered":"2024-09-17T15:35:38.867463+0000","last_clean":"2024-09-17T15:35:38.867463+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:35:38.867463+0000","last_undegraded":"2024-09-17T15:35:38.867463+0000","last_fullsized":"2024-09-17T15:35:38.867463+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21199,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.872159+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:38.872159+0000","last_peered":"2024-09-17T15:35:38.872159+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:38.872159+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869142+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:35:38.869142+0000","last_peered":"2024-09-17T15:35:38.869142+0000","last_clean":"2024-09-17T15:35:38.869142+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:35:38.869142+0000","last_undegraded":"2024-09-17T15:35:38.869142+0000","last_fullsized":"2024-09-17T15:35:38.869142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869195+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:35:38.869195+0000","last_peered":"2024-09-17T15:35:38.869195+0000","last_clean":"2024-09-17T15:35:38.869195+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:35:38.869195+0000","last_undegraded":"2024-09-17T15:35:38.869195+0000","last_fullsized":"2024-09-17T15:35:38.869195+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29021,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.868962+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:38.868962+0000","last_peered":"2024-09-17T15:35:38.868962+0000","last_clean":"2024-09-17T15:35:38.868962+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:38.868962+0000","last_undegraded":"2024-09-17T15:35:38.868962+0000","last_fullsized":"2024-09-17T15:35:38.868962+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30332,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19128,"num_bytes_recovered":41571753,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202629120,"data_stored":202506278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051713,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497268,"kb_used_data":132380,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874572,"statfs":{"total":96636764160,"available":96127561728,"internally_reserved":0,"allocated":135557120,"data_stored":135249043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724658,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465308,"kb_used_data":460,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906532,"statfs":{"total":96636764160,"available":96160288768,"internally_reserved":0,"allocated":471040,"data_stored":144434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":475983531},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247941,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539360,"kb_used_data":66248,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832480,"statfs":{"total":96636764160,"available":96084459520,"internally_reserved":0,"allocated":67837952,"data_stored":67525798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135147520,"data_stored":135111390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:35:51.774 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:35:52.093 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:35:52.094 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:35:52.106 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":386,"stamp":"2024-09-17T15:35:51.813772+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30332,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19129,"num_bytes_recovered":41571772,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1501936,"kb_used_data":199088,"kb_used_omap":13,"kb_used_meta":1302770,"kb_avail":281613584,"statfs":{"total":289910292480,"available":288372310016,"internally_reserved":0,"allocated":203866112,"data_stored":202919275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1334037503},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1954,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1954,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002187"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20912,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.872768+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:38.872768+0000","last_peered":"2024-09-17T15:35:38.872768+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:38.872768+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21049,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866657+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:38.866657+0000","last_peered":"2024-09-17T15:35:38.866657+0000","last_clean":"2024-09-17T15:35:38.866657+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:38.866657+0000","last_undegraded":"2024-09-17T15:35:38.866657+0000","last_fullsized":"2024-09-17T15:35:38.866657+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21161,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.866979+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:38.866979+0000","last_peered":"2024-09-17T15:35:38.866979+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:38.866979+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4125}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21054,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866816+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:38.866816+0000","last_peered":"2024-09-17T15:35:38.866816+0000","last_clean":"2024-09-17T15:35:38.866816+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:38.866816+0000","last_undegraded":"2024-09-17T15:35:38.866816+0000","last_fullsized":"2024-09-17T15:35:38.866816+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20797,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.870725+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:38.870725+0000","last_peered":"2024-09-17T15:35:38.870725+0000","last_clean":"2024-09-17T15:35:38.870725+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:38.870725+0000","last_undegraded":"2024-09-17T15:35:38.870725+0000","last_fullsized":"2024-09-17T15:35:38.870725+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21556,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870736+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:38.870736+0000","last_peered":"2024-09-17T15:35:38.870736+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:38.870736+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4191},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21091,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870726+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:35:38.870726+0000","last_peered":"2024-09-17T15:35:38.870726+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:35:38.870726+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4087},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21058,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.870606+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:38.870606+0000","last_peered":"2024-09-17T15:35:38.870606+0000","last_clean":"2024-09-17T15:35:38.870606+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:38.870606+0000","last_undegraded":"2024-09-17T15:35:38.870606+0000","last_fullsized":"2024-09-17T15:35:38.870606+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869332+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:35:38.869332+0000","last_peered":"2024-09-17T15:35:38.869332+0000","last_clean":"2024-09-17T15:35:38.869332+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:35:38.869332+0000","last_undegraded":"2024-09-17T15:35:38.869332+0000","last_fullsized":"2024-09-17T15:35:38.869332+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29114,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869044+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:38.869044+0000","last_peered":"2024-09-17T15:35:38.869044+0000","last_clean":"2024-09-17T15:35:38.869044+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:38.869044+0000","last_undegraded":"2024-09-17T15:35:38.869044+0000","last_fullsized":"2024-09-17T15:35:38.869044+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866649+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:35:38.866649+0000","last_peered":"2024-09-17T15:35:38.866649+0000","last_clean":"2024-09-17T15:35:38.866649+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:35:38.866649+0000","last_undegraded":"2024-09-17T15:35:38.866649+0000","last_fullsized":"2024-09-17T15:35:38.866649+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20821,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870877+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:38.870877+0000","last_peered":"2024-09-17T15:35:38.870877+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:38.870877+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":118,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.870615+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:35:38.870615+0000","last_peered":"2024-09-17T15:35:38.870615+0000","last_clean":"2024-09-17T15:35:38.870615+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:35:38.870615+0000","last_undegraded":"2024-09-17T15:35:38.870615+0000","last_fullsized":"2024-09-17T15:35:38.870615+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":24902,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:48.432407+0000","last_change":"2024-09-17T15:35:37.572616+0000","last_active":"2024-09-17T15:35:48.432407+0000","last_peered":"2024-09-17T15:35:48.432407+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:35:48.432407+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1955,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2123,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1954}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":92,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.868946+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:35:38.868946+0000","last_peered":"2024-09-17T15:35:38.868946+0000","last_clean":"2024-09-17T15:35:38.868946+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:35:38.868946+0000","last_undegraded":"2024-09-17T15:35:38.868946+0000","last_fullsized":"2024-09-17T15:35:38.868946+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29156,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866426+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:38.866426+0000","last_peered":"2024-09-17T15:35:38.866426+0000","last_clean":"2024-09-17T15:35:38.866426+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:38.866426+0000","last_undegraded":"2024-09-17T15:35:38.866426+0000","last_fullsized":"2024-09-17T15:35:38.866426+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21278,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870971+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:38.870971+0000","last_peered":"2024-09-17T15:35:38.870971+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:38.870971+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869077+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:35:38.869077+0000","last_peered":"2024-09-17T15:35:38.869077+0000","last_clean":"2024-09-17T15:35:38.869077+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:35:38.869077+0000","last_undegraded":"2024-09-17T15:35:38.869077+0000","last_fullsized":"2024-09-17T15:35:38.869077+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28070,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.952380+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:35:38.952380+0000","last_peered":"2024-09-17T15:35:38.952380+0000","last_clean":"2024-09-17T15:35:38.952380+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:35:38.952380+0000","last_undegraded":"2024-09-17T15:35:38.952380+0000","last_fullsized":"2024-09-17T15:35:38.952380+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.867463+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:35:38.867463+0000","last_peered":"2024-09-17T15:35:38.867463+0000","last_clean":"2024-09-17T15:35:38.867463+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:35:38.867463+0000","last_undegraded":"2024-09-17T15:35:38.867463+0000","last_fullsized":"2024-09-17T15:35:38.867463+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21199,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.872159+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:38.872159+0000","last_peered":"2024-09-17T15:35:38.872159+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:38.872159+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869142+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:35:38.869142+0000","last_peered":"2024-09-17T15:35:38.869142+0000","last_clean":"2024-09-17T15:35:38.869142+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:35:38.869142+0000","last_undegraded":"2024-09-17T15:35:38.869142+0000","last_fullsized":"2024-09-17T15:35:38.869142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869195+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:35:38.869195+0000","last_peered":"2024-09-17T15:35:38.869195+0000","last_clean":"2024-09-17T15:35:38.869195+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:35:38.869195+0000","last_undegraded":"2024-09-17T15:35:38.869195+0000","last_fullsized":"2024-09-17T15:35:38.869195+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29021,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.868962+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:38.868962+0000","last_peered":"2024-09-17T15:35:38.868962+0000","last_clean":"2024-09-17T15:35:38.868962+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:38.868962+0000","last_undegraded":"2024-09-17T15:35:38.868962+0000","last_fullsized":"2024-09-17T15:35:38.868962+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30332,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19128,"num_bytes_recovered":41571753,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202629120,"data_stored":202506278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051713,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497268,"kb_used_data":132380,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874572,"statfs":{"total":96636764160,"available":96127561728,"internally_reserved":0,"allocated":135557120,"data_stored":135249043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724658,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465308,"kb_used_data":460,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906532,"statfs":{"total":96636764160,"available":96160288768,"internally_reserved":0,"allocated":471040,"data_stored":144434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":475983531},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247941,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539360,"kb_used_data":66248,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832480,"statfs":{"total":96636764160,"available":96084459520,"internally_reserved":0,"allocated":67837952,"data_stored":67525798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135147520,"data_stored":135111390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:35:52.108 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:35:52.108 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'20648", 'reported_seq': 20912, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:38.872768+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:35:38.872768+0000', 'last_peered': '2024-09-17T15:35:38.872768+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:35:38.872768+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'10600", 'ondisk_log_start': "43'10600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20648, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:52.108 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:35:52.108 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'20934", 'reported_seq': 21161, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:38.866979+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:35:38.866979+0000', 'last_peered': '2024-09-17T15:35:38.866979+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:35:38.866979+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "43'10900", 'ondisk_log_start': "43'10900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 19091, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4125}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:35:52.108 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:35:52.108 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'21251", 'reported_seq': 21556, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:38.870736+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:35:38.870736+0000', 'last_peered': '2024-09-17T15:35:38.870736+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:35:38.870736+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "43'11200", 'ondisk_log_start': "43'11200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17750, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4191}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:52.108 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:35:52.108 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'20768", 'reported_seq': 21091, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:38.870726+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:35:38.870726+0000', 'last_peered': '2024-09-17T15:35:38.870726+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:35:38.870726+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "43'10700", 'ondisk_log_start': "43'10700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4089, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17537, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 85, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4087}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:52.109 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:35:52.109 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20542", 'reported_seq': 20821, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:38.870877+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:35:38.870877+0000', 'last_peered': '2024-09-17T15:35:38.870877+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:35:38.870877+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17138, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3943}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:52.109 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:35:52.109 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'20519", 'reported_seq': 24902, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:48.432407+0000', 'last_change': '2024-09-17T15:35:37.572616+0000', 'last_active': '2024-09-17T15:35:48.432407+0000', 'last_peered': '2024-09-17T15:35:48.432407+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:35:48.432407+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1955, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18414, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2123, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1954}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:35:52.109 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:35:52.109 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'20966", 'reported_seq': 21278, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:38.870971+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:35:38.870971+0000', 'last_peered': '2024-09-17T15:35:38.870971+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:35:38.870971+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "43'10900", 'ondisk_log_start': "43'10900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17514, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4057}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:52.109 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:35:52.109 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'20886", 'reported_seq': 21199, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:38.872159+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:35:38.872159+0000', 'last_peered': '2024-09-17T15:35:38.872159+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:35:38.872159+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'10800", 'ondisk_log_start': "43'10800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17673, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3980}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:52.110 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:35:52.496 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:35:52.496 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 30332/24 objects degraded (126383.333%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":689,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1537982464,"bytes_avail":288372310016,"bytes_total":289910292480,"degraded_objects":30332,"degraded_total":24,"degraded_ratio":1263.8333333333333,"recovering_objects_per_sec":162,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1954,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:35:52.510 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:35:52.510 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:35:52.812 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:35:52.813 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:35:52.825 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":386,"stamp":"2024-09-17T15:35:51.813772+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30332,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19129,"num_bytes_recovered":41571772,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1501936,"kb_used_data":199088,"kb_used_omap":13,"kb_used_meta":1302770,"kb_avail":281613584,"statfs":{"total":289910292480,"available":288372310016,"internally_reserved":0,"allocated":203866112,"data_stored":202919275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1334037503},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1954,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1954,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002187"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20912,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.872768+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:38.872768+0000","last_peered":"2024-09-17T15:35:38.872768+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:38.872768+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21049,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866657+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:38.866657+0000","last_peered":"2024-09-17T15:35:38.866657+0000","last_clean":"2024-09-17T15:35:38.866657+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:38.866657+0000","last_undegraded":"2024-09-17T15:35:38.866657+0000","last_fullsized":"2024-09-17T15:35:38.866657+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21161,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.866979+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:38.866979+0000","last_peered":"2024-09-17T15:35:38.866979+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:38.866979+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4125}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21054,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866816+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:38.866816+0000","last_peered":"2024-09-17T15:35:38.866816+0000","last_clean":"2024-09-17T15:35:38.866816+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:38.866816+0000","last_undegraded":"2024-09-17T15:35:38.866816+0000","last_fullsized":"2024-09-17T15:35:38.866816+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20797,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.870725+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:38.870725+0000","last_peered":"2024-09-17T15:35:38.870725+0000","last_clean":"2024-09-17T15:35:38.870725+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:38.870725+0000","last_undegraded":"2024-09-17T15:35:38.870725+0000","last_fullsized":"2024-09-17T15:35:38.870725+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21556,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870736+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:38.870736+0000","last_peered":"2024-09-17T15:35:38.870736+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:38.870736+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4191},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21091,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870726+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:35:38.870726+0000","last_peered":"2024-09-17T15:35:38.870726+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:35:38.870726+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4087},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21058,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.870606+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:38.870606+0000","last_peered":"2024-09-17T15:35:38.870606+0000","last_clean":"2024-09-17T15:35:38.870606+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:38.870606+0000","last_undegraded":"2024-09-17T15:35:38.870606+0000","last_fullsized":"2024-09-17T15:35:38.870606+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869332+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:35:38.869332+0000","last_peered":"2024-09-17T15:35:38.869332+0000","last_clean":"2024-09-17T15:35:38.869332+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:35:38.869332+0000","last_undegraded":"2024-09-17T15:35:38.869332+0000","last_fullsized":"2024-09-17T15:35:38.869332+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29114,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869044+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:38.869044+0000","last_peered":"2024-09-17T15:35:38.869044+0000","last_clean":"2024-09-17T15:35:38.869044+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:38.869044+0000","last_undegraded":"2024-09-17T15:35:38.869044+0000","last_fullsized":"2024-09-17T15:35:38.869044+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866649+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:35:38.866649+0000","last_peered":"2024-09-17T15:35:38.866649+0000","last_clean":"2024-09-17T15:35:38.866649+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:35:38.866649+0000","last_undegraded":"2024-09-17T15:35:38.866649+0000","last_fullsized":"2024-09-17T15:35:38.866649+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20821,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870877+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:38.870877+0000","last_peered":"2024-09-17T15:35:38.870877+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:38.870877+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":118,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.870615+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:35:38.870615+0000","last_peered":"2024-09-17T15:35:38.870615+0000","last_clean":"2024-09-17T15:35:38.870615+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:35:38.870615+0000","last_undegraded":"2024-09-17T15:35:38.870615+0000","last_fullsized":"2024-09-17T15:35:38.870615+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":24902,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:48.432407+0000","last_change":"2024-09-17T15:35:37.572616+0000","last_active":"2024-09-17T15:35:48.432407+0000","last_peered":"2024-09-17T15:35:48.432407+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:35:48.432407+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1955,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2123,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1954}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":92,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.868946+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:35:38.868946+0000","last_peered":"2024-09-17T15:35:38.868946+0000","last_clean":"2024-09-17T15:35:38.868946+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:35:38.868946+0000","last_undegraded":"2024-09-17T15:35:38.868946+0000","last_fullsized":"2024-09-17T15:35:38.868946+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29156,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866426+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:38.866426+0000","last_peered":"2024-09-17T15:35:38.866426+0000","last_clean":"2024-09-17T15:35:38.866426+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:38.866426+0000","last_undegraded":"2024-09-17T15:35:38.866426+0000","last_fullsized":"2024-09-17T15:35:38.866426+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21278,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870971+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:38.870971+0000","last_peered":"2024-09-17T15:35:38.870971+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:38.870971+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869077+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:35:38.869077+0000","last_peered":"2024-09-17T15:35:38.869077+0000","last_clean":"2024-09-17T15:35:38.869077+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:35:38.869077+0000","last_undegraded":"2024-09-17T15:35:38.869077+0000","last_fullsized":"2024-09-17T15:35:38.869077+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28070,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.952380+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:35:38.952380+0000","last_peered":"2024-09-17T15:35:38.952380+0000","last_clean":"2024-09-17T15:35:38.952380+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:35:38.952380+0000","last_undegraded":"2024-09-17T15:35:38.952380+0000","last_fullsized":"2024-09-17T15:35:38.952380+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.867463+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:35:38.867463+0000","last_peered":"2024-09-17T15:35:38.867463+0000","last_clean":"2024-09-17T15:35:38.867463+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:35:38.867463+0000","last_undegraded":"2024-09-17T15:35:38.867463+0000","last_fullsized":"2024-09-17T15:35:38.867463+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21199,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.872159+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:38.872159+0000","last_peered":"2024-09-17T15:35:38.872159+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:38.872159+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869142+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:35:38.869142+0000","last_peered":"2024-09-17T15:35:38.869142+0000","last_clean":"2024-09-17T15:35:38.869142+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:35:38.869142+0000","last_undegraded":"2024-09-17T15:35:38.869142+0000","last_fullsized":"2024-09-17T15:35:38.869142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869195+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:35:38.869195+0000","last_peered":"2024-09-17T15:35:38.869195+0000","last_clean":"2024-09-17T15:35:38.869195+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:35:38.869195+0000","last_undegraded":"2024-09-17T15:35:38.869195+0000","last_fullsized":"2024-09-17T15:35:38.869195+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29021,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.868962+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:38.868962+0000","last_peered":"2024-09-17T15:35:38.868962+0000","last_clean":"2024-09-17T15:35:38.868962+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:38.868962+0000","last_undegraded":"2024-09-17T15:35:38.868962+0000","last_fullsized":"2024-09-17T15:35:38.868962+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30332,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19128,"num_bytes_recovered":41571753,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202629120,"data_stored":202506278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051713,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497268,"kb_used_data":132380,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874572,"statfs":{"total":96636764160,"available":96127561728,"internally_reserved":0,"allocated":135557120,"data_stored":135249043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724658,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465308,"kb_used_data":460,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906532,"statfs":{"total":96636764160,"available":96160288768,"internally_reserved":0,"allocated":471040,"data_stored":144434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":475983531},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247941,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539360,"kb_used_data":66248,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832480,"statfs":{"total":96636764160,"available":96084459520,"internally_reserved":0,"allocated":67837952,"data_stored":67525798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135147520,"data_stored":135111390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:35:55.827 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:35:56.152 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:35:56.152 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:35:56.164 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":388,"stamp":"2024-09-17T15:35:55.814409+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20005,"num_bytes_recovered":41571772,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1524144,"kb_used_data":199088,"kb_used_omap":13,"kb_used_meta":1324978,"kb_avail":281591376,"statfs":{"total":289910292480,"available":288349569024,"internally_reserved":0,"allocated":203866112,"data_stored":202919275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1356778495},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1955,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1955,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002141"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20912,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.872768+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:38.872768+0000","last_peered":"2024-09-17T15:35:38.872768+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:38.872768+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21049,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866657+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:38.866657+0000","last_peered":"2024-09-17T15:35:38.866657+0000","last_clean":"2024-09-17T15:35:38.866657+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:38.866657+0000","last_undegraded":"2024-09-17T15:35:38.866657+0000","last_fullsized":"2024-09-17T15:35:38.866657+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21161,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.866979+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:38.866979+0000","last_peered":"2024-09-17T15:35:38.866979+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:38.866979+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4125}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21054,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866816+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:38.866816+0000","last_peered":"2024-09-17T15:35:38.866816+0000","last_clean":"2024-09-17T15:35:38.866816+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:38.866816+0000","last_undegraded":"2024-09-17T15:35:38.866816+0000","last_fullsized":"2024-09-17T15:35:38.866816+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20797,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.870725+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:38.870725+0000","last_peered":"2024-09-17T15:35:38.870725+0000","last_clean":"2024-09-17T15:35:38.870725+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:38.870725+0000","last_undegraded":"2024-09-17T15:35:38.870725+0000","last_fullsized":"2024-09-17T15:35:38.870725+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21556,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870736+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:38.870736+0000","last_peered":"2024-09-17T15:35:38.870736+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:38.870736+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4191},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21091,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870726+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:35:38.870726+0000","last_peered":"2024-09-17T15:35:38.870726+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:35:38.870726+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4087},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21058,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.870606+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:38.870606+0000","last_peered":"2024-09-17T15:35:38.870606+0000","last_clean":"2024-09-17T15:35:38.870606+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:38.870606+0000","last_undegraded":"2024-09-17T15:35:38.870606+0000","last_fullsized":"2024-09-17T15:35:38.870606+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869332+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:35:38.869332+0000","last_peered":"2024-09-17T15:35:38.869332+0000","last_clean":"2024-09-17T15:35:38.869332+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:35:38.869332+0000","last_undegraded":"2024-09-17T15:35:38.869332+0000","last_fullsized":"2024-09-17T15:35:38.869332+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29114,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869044+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:38.869044+0000","last_peered":"2024-09-17T15:35:38.869044+0000","last_clean":"2024-09-17T15:35:38.869044+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:38.869044+0000","last_undegraded":"2024-09-17T15:35:38.869044+0000","last_fullsized":"2024-09-17T15:35:38.869044+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866649+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:35:38.866649+0000","last_peered":"2024-09-17T15:35:38.866649+0000","last_clean":"2024-09-17T15:35:38.866649+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:35:38.866649+0000","last_undegraded":"2024-09-17T15:35:38.866649+0000","last_fullsized":"2024-09-17T15:35:38.866649+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20821,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870877+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:38.870877+0000","last_peered":"2024-09-17T15:35:38.870877+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:38.870877+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":118,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.870615+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:35:38.870615+0000","last_peered":"2024-09-17T15:35:38.870615+0000","last_clean":"2024-09-17T15:35:38.870615+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:35:38.870615+0000","last_undegraded":"2024-09-17T15:35:38.870615+0000","last_fullsized":"2024-09-17T15:35:38.870615+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":26654,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:53.439616+0000","last_change":"2024-09-17T15:35:37.572616+0000","last_active":"2024-09-17T15:35:53.439616+0000","last_peered":"2024-09-17T15:35:53.439616+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:35:53.439616+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1079,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2999,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1078}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":92,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.868946+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:35:38.868946+0000","last_peered":"2024-09-17T15:35:38.868946+0000","last_clean":"2024-09-17T15:35:38.868946+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:35:38.868946+0000","last_undegraded":"2024-09-17T15:35:38.868946+0000","last_fullsized":"2024-09-17T15:35:38.868946+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29156,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866426+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:38.866426+0000","last_peered":"2024-09-17T15:35:38.866426+0000","last_clean":"2024-09-17T15:35:38.866426+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:38.866426+0000","last_undegraded":"2024-09-17T15:35:38.866426+0000","last_fullsized":"2024-09-17T15:35:38.866426+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21278,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870971+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:38.870971+0000","last_peered":"2024-09-17T15:35:38.870971+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:38.870971+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869077+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:35:38.869077+0000","last_peered":"2024-09-17T15:35:38.869077+0000","last_clean":"2024-09-17T15:35:38.869077+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:35:38.869077+0000","last_undegraded":"2024-09-17T15:35:38.869077+0000","last_fullsized":"2024-09-17T15:35:38.869077+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28070,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.952380+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:35:38.952380+0000","last_peered":"2024-09-17T15:35:38.952380+0000","last_clean":"2024-09-17T15:35:38.952380+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:35:38.952380+0000","last_undegraded":"2024-09-17T15:35:38.952380+0000","last_fullsized":"2024-09-17T15:35:38.952380+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.867463+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:35:38.867463+0000","last_peered":"2024-09-17T15:35:38.867463+0000","last_clean":"2024-09-17T15:35:38.867463+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:35:38.867463+0000","last_undegraded":"2024-09-17T15:35:38.867463+0000","last_fullsized":"2024-09-17T15:35:38.867463+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21199,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.872159+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:38.872159+0000","last_peered":"2024-09-17T15:35:38.872159+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:38.872159+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869142+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:35:38.869142+0000","last_peered":"2024-09-17T15:35:38.869142+0000","last_clean":"2024-09-17T15:35:38.869142+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:35:38.869142+0000","last_undegraded":"2024-09-17T15:35:38.869142+0000","last_fullsized":"2024-09-17T15:35:38.869142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869195+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:35:38.869195+0000","last_peered":"2024-09-17T15:35:38.869195+0000","last_clean":"2024-09-17T15:35:38.869195+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:35:38.869195+0000","last_undegraded":"2024-09-17T15:35:38.869195+0000","last_fullsized":"2024-09-17T15:35:38.869195+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29021,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.868962+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:38.868962+0000","last_peered":"2024-09-17T15:35:38.868962+0000","last_clean":"2024-09-17T15:35:38.868962+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:38.868962+0000","last_undegraded":"2024-09-17T15:35:38.868962+0000","last_fullsized":"2024-09-17T15:35:38.868962+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20004,"num_bytes_recovered":41571753,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202629120,"data_stored":202506278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051714,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497268,"kb_used_data":132380,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874572,"statfs":{"total":96636764160,"available":96127561728,"internally_reserved":0,"allocated":135557120,"data_stored":135249043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724659,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487516,"kb_used_data":460,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884324,"statfs":{"total":96636764160,"available":96137547776,"internally_reserved":0,"allocated":471040,"data_stored":144434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":498724523},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247942,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539360,"kb_used_data":66248,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832480,"statfs":{"total":96636764160,"available":96084459520,"internally_reserved":0,"allocated":67837952,"data_stored":67525798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135147520,"data_stored":135111390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:35:56.165 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:35:56.469 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:35:56.469 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:35:56.481 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":388,"stamp":"2024-09-17T15:35:55.814409+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20005,"num_bytes_recovered":41571772,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1524144,"kb_used_data":199088,"kb_used_omap":13,"kb_used_meta":1324978,"kb_avail":281591376,"statfs":{"total":289910292480,"available":288349569024,"internally_reserved":0,"allocated":203866112,"data_stored":202919275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1356778495},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1955,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1955,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002141"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20912,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.872768+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:38.872768+0000","last_peered":"2024-09-17T15:35:38.872768+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:38.872768+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21049,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866657+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:38.866657+0000","last_peered":"2024-09-17T15:35:38.866657+0000","last_clean":"2024-09-17T15:35:38.866657+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:38.866657+0000","last_undegraded":"2024-09-17T15:35:38.866657+0000","last_fullsized":"2024-09-17T15:35:38.866657+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21161,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.866979+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:38.866979+0000","last_peered":"2024-09-17T15:35:38.866979+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:38.866979+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4125}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21054,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866816+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:38.866816+0000","last_peered":"2024-09-17T15:35:38.866816+0000","last_clean":"2024-09-17T15:35:38.866816+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:38.866816+0000","last_undegraded":"2024-09-17T15:35:38.866816+0000","last_fullsized":"2024-09-17T15:35:38.866816+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20797,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.870725+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:38.870725+0000","last_peered":"2024-09-17T15:35:38.870725+0000","last_clean":"2024-09-17T15:35:38.870725+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:38.870725+0000","last_undegraded":"2024-09-17T15:35:38.870725+0000","last_fullsized":"2024-09-17T15:35:38.870725+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21556,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870736+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:38.870736+0000","last_peered":"2024-09-17T15:35:38.870736+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:38.870736+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4191},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21091,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870726+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:35:38.870726+0000","last_peered":"2024-09-17T15:35:38.870726+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:35:38.870726+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4087},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21058,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.870606+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:38.870606+0000","last_peered":"2024-09-17T15:35:38.870606+0000","last_clean":"2024-09-17T15:35:38.870606+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:38.870606+0000","last_undegraded":"2024-09-17T15:35:38.870606+0000","last_fullsized":"2024-09-17T15:35:38.870606+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869332+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:35:38.869332+0000","last_peered":"2024-09-17T15:35:38.869332+0000","last_clean":"2024-09-17T15:35:38.869332+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:35:38.869332+0000","last_undegraded":"2024-09-17T15:35:38.869332+0000","last_fullsized":"2024-09-17T15:35:38.869332+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29114,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869044+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:38.869044+0000","last_peered":"2024-09-17T15:35:38.869044+0000","last_clean":"2024-09-17T15:35:38.869044+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:38.869044+0000","last_undegraded":"2024-09-17T15:35:38.869044+0000","last_fullsized":"2024-09-17T15:35:38.869044+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866649+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:35:38.866649+0000","last_peered":"2024-09-17T15:35:38.866649+0000","last_clean":"2024-09-17T15:35:38.866649+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:35:38.866649+0000","last_undegraded":"2024-09-17T15:35:38.866649+0000","last_fullsized":"2024-09-17T15:35:38.866649+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20821,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870877+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:38.870877+0000","last_peered":"2024-09-17T15:35:38.870877+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:38.870877+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":118,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.870615+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:35:38.870615+0000","last_peered":"2024-09-17T15:35:38.870615+0000","last_clean":"2024-09-17T15:35:38.870615+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:35:38.870615+0000","last_undegraded":"2024-09-17T15:35:38.870615+0000","last_fullsized":"2024-09-17T15:35:38.870615+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":26654,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:53.439616+0000","last_change":"2024-09-17T15:35:37.572616+0000","last_active":"2024-09-17T15:35:53.439616+0000","last_peered":"2024-09-17T15:35:53.439616+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:35:53.439616+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1079,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2999,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1078}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":92,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.868946+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:35:38.868946+0000","last_peered":"2024-09-17T15:35:38.868946+0000","last_clean":"2024-09-17T15:35:38.868946+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:35:38.868946+0000","last_undegraded":"2024-09-17T15:35:38.868946+0000","last_fullsized":"2024-09-17T15:35:38.868946+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29156,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866426+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:38.866426+0000","last_peered":"2024-09-17T15:35:38.866426+0000","last_clean":"2024-09-17T15:35:38.866426+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:38.866426+0000","last_undegraded":"2024-09-17T15:35:38.866426+0000","last_fullsized":"2024-09-17T15:35:38.866426+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21278,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870971+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:38.870971+0000","last_peered":"2024-09-17T15:35:38.870971+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:38.870971+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869077+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:35:38.869077+0000","last_peered":"2024-09-17T15:35:38.869077+0000","last_clean":"2024-09-17T15:35:38.869077+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:35:38.869077+0000","last_undegraded":"2024-09-17T15:35:38.869077+0000","last_fullsized":"2024-09-17T15:35:38.869077+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28070,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.952380+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:35:38.952380+0000","last_peered":"2024-09-17T15:35:38.952380+0000","last_clean":"2024-09-17T15:35:38.952380+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:35:38.952380+0000","last_undegraded":"2024-09-17T15:35:38.952380+0000","last_fullsized":"2024-09-17T15:35:38.952380+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.867463+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:35:38.867463+0000","last_peered":"2024-09-17T15:35:38.867463+0000","last_clean":"2024-09-17T15:35:38.867463+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:35:38.867463+0000","last_undegraded":"2024-09-17T15:35:38.867463+0000","last_fullsized":"2024-09-17T15:35:38.867463+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21199,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.872159+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:38.872159+0000","last_peered":"2024-09-17T15:35:38.872159+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:38.872159+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869142+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:35:38.869142+0000","last_peered":"2024-09-17T15:35:38.869142+0000","last_clean":"2024-09-17T15:35:38.869142+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:35:38.869142+0000","last_undegraded":"2024-09-17T15:35:38.869142+0000","last_fullsized":"2024-09-17T15:35:38.869142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869195+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:35:38.869195+0000","last_peered":"2024-09-17T15:35:38.869195+0000","last_clean":"2024-09-17T15:35:38.869195+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:35:38.869195+0000","last_undegraded":"2024-09-17T15:35:38.869195+0000","last_fullsized":"2024-09-17T15:35:38.869195+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29021,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.868962+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:38.868962+0000","last_peered":"2024-09-17T15:35:38.868962+0000","last_clean":"2024-09-17T15:35:38.868962+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:38.868962+0000","last_undegraded":"2024-09-17T15:35:38.868962+0000","last_fullsized":"2024-09-17T15:35:38.868962+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20004,"num_bytes_recovered":41571753,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202629120,"data_stored":202506278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051714,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497268,"kb_used_data":132380,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874572,"statfs":{"total":96636764160,"available":96127561728,"internally_reserved":0,"allocated":135557120,"data_stored":135249043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724659,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487516,"kb_used_data":460,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884324,"statfs":{"total":96636764160,"available":96137547776,"internally_reserved":0,"allocated":471040,"data_stored":144434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":498724523},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247942,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539360,"kb_used_data":66248,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832480,"statfs":{"total":96636764160,"available":96084459520,"internally_reserved":0,"allocated":67837952,"data_stored":67525798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135147520,"data_stored":135111390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:35:56.482 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:35:56.482 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'20648", 'reported_seq': 20912, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:38.872768+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:35:38.872768+0000', 'last_peered': '2024-09-17T15:35:38.872768+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:35:38.872768+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'10600", 'ondisk_log_start': "43'10600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20648, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:56.482 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:35:56.482 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'20934", 'reported_seq': 21161, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:38.866979+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:35:38.866979+0000', 'last_peered': '2024-09-17T15:35:38.866979+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:35:38.866979+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "43'10900", 'ondisk_log_start': "43'10900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 19091, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4125}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:35:56.482 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:35:56.483 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'21251", 'reported_seq': 21556, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:38.870736+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:35:38.870736+0000', 'last_peered': '2024-09-17T15:35:38.870736+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:35:38.870736+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "43'11200", 'ondisk_log_start': "43'11200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17750, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4191}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:56.483 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:35:56.483 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'20768", 'reported_seq': 21091, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:38.870726+0000', 'last_change': '2024-09-17T15:26:22.690248+0000', 'last_active': '2024-09-17T15:35:38.870726+0000', 'last_peered': '2024-09-17T15:35:38.870726+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:35:38.870726+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "43'10700", 'ondisk_log_start': "43'10700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4089, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17537, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 85, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4087}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:56.483 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:35:56.483 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20542", 'reported_seq': 20821, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:38.870877+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:35:38.870877+0000', 'last_peered': '2024-09-17T15:35:38.870877+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:35:38.870877+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17138, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3943}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:56.483 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:35:56.483 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'20519", 'reported_seq': 26654, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:53.439616+0000', 'last_change': '2024-09-17T15:35:37.572616+0000', 'last_active': '2024-09-17T15:35:53.439616+0000', 'last_peered': '2024-09-17T15:35:53.439616+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:35:53.439616+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1079, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18414, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2999, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1078}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:35:56.483 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:35:56.483 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'20966", 'reported_seq': 21278, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:38.870971+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:35:38.870971+0000', 'last_peered': '2024-09-17T15:35:38.870971+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:35:38.870971+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "43'10900", 'ondisk_log_start': "43'10900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17514, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4057}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:56.484 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:35:56.484 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'20886", 'reported_seq': 21199, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:38.872159+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:35:38.872159+0000', 'last_peered': '2024-09-17T15:35:38.872159+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:35:38.872159+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'10800", 'ondisk_log_start': "43'10800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17673, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3980}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:35:56.484 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:35:56.895 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:35:56.895 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 29456/24 objects degraded (122733.333%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":693,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1560723456,"bytes_avail":288349569024,"bytes_total":289910292480,"degraded_objects":29456,"degraded_total":24,"degraded_ratio":1227.3333333333333,"recovering_objects_per_sec":162,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1955,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:35:56.908 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:35:56.909 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:35:57.219 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:35:57.219 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:35:57.231 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":388,"stamp":"2024-09-17T15:35:55.814409+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20005,"num_bytes_recovered":41571772,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1524144,"kb_used_data":199088,"kb_used_omap":13,"kb_used_meta":1324978,"kb_avail":281591376,"statfs":{"total":289910292480,"available":288349569024,"internally_reserved":0,"allocated":203866112,"data_stored":202919275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1356778495},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1955,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1955,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002141"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20912,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.872768+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:38.872768+0000","last_peered":"2024-09-17T15:35:38.872768+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:38.872768+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21049,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866657+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:38.866657+0000","last_peered":"2024-09-17T15:35:38.866657+0000","last_clean":"2024-09-17T15:35:38.866657+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:38.866657+0000","last_undegraded":"2024-09-17T15:35:38.866657+0000","last_fullsized":"2024-09-17T15:35:38.866657+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21161,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.866979+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:38.866979+0000","last_peered":"2024-09-17T15:35:38.866979+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:38.866979+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4125}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21054,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866816+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:38.866816+0000","last_peered":"2024-09-17T15:35:38.866816+0000","last_clean":"2024-09-17T15:35:38.866816+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:38.866816+0000","last_undegraded":"2024-09-17T15:35:38.866816+0000","last_fullsized":"2024-09-17T15:35:38.866816+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20797,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.870725+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:38.870725+0000","last_peered":"2024-09-17T15:35:38.870725+0000","last_clean":"2024-09-17T15:35:38.870725+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:38.870725+0000","last_undegraded":"2024-09-17T15:35:38.870725+0000","last_fullsized":"2024-09-17T15:35:38.870725+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21556,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870736+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:38.870736+0000","last_peered":"2024-09-17T15:35:38.870736+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:38.870736+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4191},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21091,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870726+0000","last_change":"2024-09-17T15:26:22.690248+0000","last_active":"2024-09-17T15:35:38.870726+0000","last_peered":"2024-09-17T15:35:38.870726+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:35:38.870726+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4087},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21058,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.870606+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:38.870606+0000","last_peered":"2024-09-17T15:35:38.870606+0000","last_clean":"2024-09-17T15:35:38.870606+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:38.870606+0000","last_undegraded":"2024-09-17T15:35:38.870606+0000","last_fullsized":"2024-09-17T15:35:38.870606+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869332+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:35:38.869332+0000","last_peered":"2024-09-17T15:35:38.869332+0000","last_clean":"2024-09-17T15:35:38.869332+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:35:38.869332+0000","last_undegraded":"2024-09-17T15:35:38.869332+0000","last_fullsized":"2024-09-17T15:35:38.869332+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29114,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869044+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:38.869044+0000","last_peered":"2024-09-17T15:35:38.869044+0000","last_clean":"2024-09-17T15:35:38.869044+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:38.869044+0000","last_undegraded":"2024-09-17T15:35:38.869044+0000","last_fullsized":"2024-09-17T15:35:38.869044+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866649+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:35:38.866649+0000","last_peered":"2024-09-17T15:35:38.866649+0000","last_clean":"2024-09-17T15:35:38.866649+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:35:38.866649+0000","last_undegraded":"2024-09-17T15:35:38.866649+0000","last_fullsized":"2024-09-17T15:35:38.866649+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20821,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870877+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:38.870877+0000","last_peered":"2024-09-17T15:35:38.870877+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:38.870877+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":118,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.870615+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:35:38.870615+0000","last_peered":"2024-09-17T15:35:38.870615+0000","last_clean":"2024-09-17T15:35:38.870615+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:35:38.870615+0000","last_undegraded":"2024-09-17T15:35:38.870615+0000","last_fullsized":"2024-09-17T15:35:38.870615+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":26654,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:53.439616+0000","last_change":"2024-09-17T15:35:37.572616+0000","last_active":"2024-09-17T15:35:53.439616+0000","last_peered":"2024-09-17T15:35:53.439616+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:35:53.439616+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1079,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2999,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1078}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":92,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.868946+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:35:38.868946+0000","last_peered":"2024-09-17T15:35:38.868946+0000","last_clean":"2024-09-17T15:35:38.868946+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:35:38.868946+0000","last_undegraded":"2024-09-17T15:35:38.868946+0000","last_fullsized":"2024-09-17T15:35:38.868946+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29156,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866426+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:38.866426+0000","last_peered":"2024-09-17T15:35:38.866426+0000","last_clean":"2024-09-17T15:35:38.866426+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:38.866426+0000","last_undegraded":"2024-09-17T15:35:38.866426+0000","last_fullsized":"2024-09-17T15:35:38.866426+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21278,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.870971+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:38.870971+0000","last_peered":"2024-09-17T15:35:38.870971+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:38.870971+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869077+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:35:38.869077+0000","last_peered":"2024-09-17T15:35:38.869077+0000","last_clean":"2024-09-17T15:35:38.869077+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:35:38.869077+0000","last_undegraded":"2024-09-17T15:35:38.869077+0000","last_fullsized":"2024-09-17T15:35:38.869077+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28070,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.952380+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:35:38.952380+0000","last_peered":"2024-09-17T15:35:38.952380+0000","last_clean":"2024-09-17T15:35:38.952380+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:35:38.952380+0000","last_undegraded":"2024-09-17T15:35:38.952380+0000","last_fullsized":"2024-09-17T15:35:38.952380+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.867463+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:35:38.867463+0000","last_peered":"2024-09-17T15:35:38.867463+0000","last_clean":"2024-09-17T15:35:38.867463+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:35:38.867463+0000","last_undegraded":"2024-09-17T15:35:38.867463+0000","last_fullsized":"2024-09-17T15:35:38.867463+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21199,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.872159+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:38.872159+0000","last_peered":"2024-09-17T15:35:38.872159+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:38.872159+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869142+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:35:38.869142+0000","last_peered":"2024-09-17T15:35:38.869142+0000","last_clean":"2024-09-17T15:35:38.869142+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:35:38.869142+0000","last_undegraded":"2024-09-17T15:35:38.869142+0000","last_fullsized":"2024-09-17T15:35:38.869142+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.869195+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:35:38.869195+0000","last_peered":"2024-09-17T15:35:38.869195+0000","last_clean":"2024-09-17T15:35:38.869195+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:35:38.869195+0000","last_undegraded":"2024-09-17T15:35:38.869195+0000","last_fullsized":"2024-09-17T15:35:38.869195+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29021,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.868962+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:38.868962+0000","last_peered":"2024-09-17T15:35:38.868962+0000","last_clean":"2024-09-17T15:35:38.868962+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:38.868962+0000","last_undegraded":"2024-09-17T15:35:38.868962+0000","last_fullsized":"2024-09-17T15:35:38.868962+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20004,"num_bytes_recovered":41571753,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202629120,"data_stored":202506278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051714,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497268,"kb_used_data":132380,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874572,"statfs":{"total":96636764160,"available":96127561728,"internally_reserved":0,"allocated":135557120,"data_stored":135249043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724659,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487516,"kb_used_data":460,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884324,"statfs":{"total":96636764160,"available":96137547776,"internally_reserved":0,"allocated":471040,"data_stored":144434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":498724523},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247942,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539360,"kb_used_data":66248,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832480,"statfs":{"total":96636764160,"available":96084459520,"internally_reserved":0,"allocated":67837952,"data_stored":67525798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135147520,"data_stored":135111390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:36:00.236 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:36:00.602 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:00.603 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:36:00.616 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":392,"stamp":"2024-09-17T15:35:59.896268+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28212,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21249,"num_bytes_recovered":41572738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1519996,"kb_used_data":199096,"kb_used_omap":13,"kb_used_meta":1320818,"kb_avail":281595524,"statfs":{"total":289910292480,"available":288353816576,"internally_reserved":0,"allocated":203874304,"data_stored":202922177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1352518655},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2120,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.082496"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20914,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:58.896975+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:58.896975+0000","last_peered":"2024-09-17T15:35:58.896975+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:58.896975+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21049,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866657+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:38.866657+0000","last_peered":"2024-09-17T15:35:38.866657+0000","last_clean":"2024-09-17T15:35:38.866657+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:38.866657+0000","last_undegraded":"2024-09-17T15:35:38.866657+0000","last_fullsized":"2024-09-17T15:35:38.866657+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21161,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.866979+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:38.866979+0000","last_peered":"2024-09-17T15:35:38.866979+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:38.866979+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4125}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21054,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866816+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:38.866816+0000","last_peered":"2024-09-17T15:35:38.866816+0000","last_clean":"2024-09-17T15:35:38.866816+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:38.866816+0000","last_undegraded":"2024-09-17T15:35:38.866816+0000","last_fullsized":"2024-09-17T15:35:38.866816+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20799,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.895883+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:58.895883+0000","last_peered":"2024-09-17T15:35:58.895883+0000","last_clean":"2024-09-17T15:35:58.895883+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:58.895883+0000","last_undegraded":"2024-09-17T15:35:58.895883+0000","last_fullsized":"2024-09-17T15:35:58.895883+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21558,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:58.895307+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:58.895307+0000","last_peered":"2024-09-17T15:35:58.895307+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:58.895307+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4191},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21424,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.389955+0000","last_change":"2024-09-17T15:35:58.120749+0000","last_active":"2024-09-17T15:35:59.389955+0000","last_peered":"2024-09-17T15:35:59.389955+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:35:59.389955+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3924,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":250,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3922}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21061,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.895312+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:58.895312+0000","last_peered":"2024-09-17T15:35:58.895312+0000","last_clean":"2024-09-17T15:35:58.895312+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:58.895312+0000","last_undegraded":"2024-09-17T15:35:58.895312+0000","last_fullsized":"2024-09-17T15:35:58.895312+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.891661+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:35:58.891661+0000","last_peered":"2024-09-17T15:35:58.891661+0000","last_clean":"2024-09-17T15:35:58.891661+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:35:58.891661+0000","last_undegraded":"2024-09-17T15:35:58.891661+0000","last_fullsized":"2024-09-17T15:35:58.891661+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29116,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.892028+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:58.892028+0000","last_peered":"2024-09-17T15:35:58.892028+0000","last_clean":"2024-09-17T15:35:58.892028+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:58.892028+0000","last_undegraded":"2024-09-17T15:35:58.892028+0000","last_fullsized":"2024-09-17T15:35:58.892028+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866649+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:35:38.866649+0000","last_peered":"2024-09-17T15:35:38.866649+0000","last_clean":"2024-09-17T15:35:38.866649+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:35:38.866649+0000","last_undegraded":"2024-09-17T15:35:38.866649+0000","last_fullsized":"2024-09-17T15:35:38.866649+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20823,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:58.896110+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:58.896110+0000","last_peered":"2024-09-17T15:35:58.896110+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:58.896110+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":120,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.895305+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:35:58.895305+0000","last_peered":"2024-09-17T15:35:58.895305+0000","last_clean":"2024-09-17T15:35:58.895305+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:35:58.895305+0000","last_undegraded":"2024-09-17T15:35:58.895305+0000","last_fullsized":"2024-09-17T15:35:58.895305+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28813,"reported_epoch":49,"state":"active+undersized+remapped","last_fresh":"2024-09-17T15:35:58.120315+0000","last_change":"2024-09-17T15:35:58.120473+0000","last_active":"2024-09-17T15:35:58.120315+0000","last_peered":"2024-09-17T15:35:58.120315+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:35:58.120315+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0","1"],"object_location_counts":[{"shards":"2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":94,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.891932+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:35:58.891932+0000","last_peered":"2024-09-17T15:35:58.891932+0000","last_clean":"2024-09-17T15:35:58.891932+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:35:58.891932+0000","last_undegraded":"2024-09-17T15:35:58.891932+0000","last_fullsized":"2024-09-17T15:35:58.891932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29156,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866426+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:38.866426+0000","last_peered":"2024-09-17T15:35:38.866426+0000","last_clean":"2024-09-17T15:35:38.866426+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:38.866426+0000","last_undegraded":"2024-09-17T15:35:38.866426+0000","last_fullsized":"2024-09-17T15:35:38.866426+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21280,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:58.896971+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:58.896971+0000","last_peered":"2024-09-17T15:35:58.896971+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:58.896971+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.892023+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:35:58.892023+0000","last_peered":"2024-09-17T15:35:58.892023+0000","last_clean":"2024-09-17T15:35:58.892023+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:35:58.892023+0000","last_undegraded":"2024-09-17T15:35:58.892023+0000","last_fullsized":"2024-09-17T15:35:58.892023+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28072,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.892209+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:35:58.892209+0000","last_peered":"2024-09-17T15:35:58.892209+0000","last_clean":"2024-09-17T15:35:58.892209+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:35:58.892209+0000","last_undegraded":"2024-09-17T15:35:58.892209+0000","last_fullsized":"2024-09-17T15:35:58.892209+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.867463+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:35:38.867463+0000","last_peered":"2024-09-17T15:35:38.867463+0000","last_clean":"2024-09-17T15:35:38.867463+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:35:38.867463+0000","last_undegraded":"2024-09-17T15:35:38.867463+0000","last_fullsized":"2024-09-17T15:35:38.867463+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21201,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:58.896015+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:58.896015+0000","last_peered":"2024-09-17T15:35:58.896015+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:58.896015+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.892126+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:35:58.892126+0000","last_peered":"2024-09-17T15:35:58.892126+0000","last_clean":"2024-09-17T15:35:58.892126+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:35:58.892126+0000","last_undegraded":"2024-09-17T15:35:58.892126+0000","last_fullsized":"2024-09-17T15:35:58.892126+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.892366+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:35:58.892366+0000","last_peered":"2024-09-17T15:35:58.892366+0000","last_clean":"2024-09-17T15:35:58.892366+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:35:58.892366+0000","last_undegraded":"2024-09-17T15:35:58.892366+0000","last_fullsized":"2024-09-17T15:35:58.892366+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29023,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.891923+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:58.891923+0000","last_peered":"2024-09-17T15:35:58.891923+0000","last_clean":"2024-09-17T15:35:58.891923+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:58.891923+0000","last_undegraded":"2024-09-17T15:35:58.891923+0000","last_fullsized":"2024-09-17T15:35:58.891923+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28212,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21248,"num_bytes_recovered":41572719,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202629120,"data_stored":202506278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051715,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497268,"kb_used_data":132380,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874572,"statfs":{"total":96636764160,"available":96127561728,"internally_reserved":0,"allocated":135557120,"data_stored":135249043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724660,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483368,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888472,"statfs":{"total":96636764160,"available":96141795328,"internally_reserved":0,"allocated":479232,"data_stored":147336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":494464683},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247943,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539360,"kb_used_data":66248,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832480,"statfs":{"total":96636764160,"available":96084459520,"internally_reserved":0,"allocated":67837952,"data_stored":67525798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135147520,"data_stored":135111390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:36:00.617 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:36:01.009 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:01.010 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:36:01.023 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":392,"stamp":"2024-09-17T15:35:59.896268+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28212,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21249,"num_bytes_recovered":41572738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1519996,"kb_used_data":199096,"kb_used_omap":13,"kb_used_meta":1320818,"kb_avail":281595524,"statfs":{"total":289910292480,"available":288353816576,"internally_reserved":0,"allocated":203874304,"data_stored":202922177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1352518655},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2120,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.082496"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20914,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:58.896975+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:58.896975+0000","last_peered":"2024-09-17T15:35:58.896975+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:58.896975+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21049,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866657+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:38.866657+0000","last_peered":"2024-09-17T15:35:38.866657+0000","last_clean":"2024-09-17T15:35:38.866657+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:38.866657+0000","last_undegraded":"2024-09-17T15:35:38.866657+0000","last_fullsized":"2024-09-17T15:35:38.866657+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21161,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.866979+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:38.866979+0000","last_peered":"2024-09-17T15:35:38.866979+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:38.866979+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4125}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21054,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866816+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:38.866816+0000","last_peered":"2024-09-17T15:35:38.866816+0000","last_clean":"2024-09-17T15:35:38.866816+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:38.866816+0000","last_undegraded":"2024-09-17T15:35:38.866816+0000","last_fullsized":"2024-09-17T15:35:38.866816+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20799,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.895883+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:58.895883+0000","last_peered":"2024-09-17T15:35:58.895883+0000","last_clean":"2024-09-17T15:35:58.895883+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:58.895883+0000","last_undegraded":"2024-09-17T15:35:58.895883+0000","last_fullsized":"2024-09-17T15:35:58.895883+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21558,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:58.895307+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:58.895307+0000","last_peered":"2024-09-17T15:35:58.895307+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:58.895307+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4191},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21424,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.389955+0000","last_change":"2024-09-17T15:35:58.120749+0000","last_active":"2024-09-17T15:35:59.389955+0000","last_peered":"2024-09-17T15:35:59.389955+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:35:59.389955+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3924,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":250,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3922}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21061,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.895312+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:58.895312+0000","last_peered":"2024-09-17T15:35:58.895312+0000","last_clean":"2024-09-17T15:35:58.895312+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:58.895312+0000","last_undegraded":"2024-09-17T15:35:58.895312+0000","last_fullsized":"2024-09-17T15:35:58.895312+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.891661+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:35:58.891661+0000","last_peered":"2024-09-17T15:35:58.891661+0000","last_clean":"2024-09-17T15:35:58.891661+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:35:58.891661+0000","last_undegraded":"2024-09-17T15:35:58.891661+0000","last_fullsized":"2024-09-17T15:35:58.891661+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29116,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.892028+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:58.892028+0000","last_peered":"2024-09-17T15:35:58.892028+0000","last_clean":"2024-09-17T15:35:58.892028+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:58.892028+0000","last_undegraded":"2024-09-17T15:35:58.892028+0000","last_fullsized":"2024-09-17T15:35:58.892028+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866649+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:35:38.866649+0000","last_peered":"2024-09-17T15:35:38.866649+0000","last_clean":"2024-09-17T15:35:38.866649+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:35:38.866649+0000","last_undegraded":"2024-09-17T15:35:38.866649+0000","last_fullsized":"2024-09-17T15:35:38.866649+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20823,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:58.896110+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:58.896110+0000","last_peered":"2024-09-17T15:35:58.896110+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:58.896110+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":120,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.895305+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:35:58.895305+0000","last_peered":"2024-09-17T15:35:58.895305+0000","last_clean":"2024-09-17T15:35:58.895305+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:35:58.895305+0000","last_undegraded":"2024-09-17T15:35:58.895305+0000","last_fullsized":"2024-09-17T15:35:58.895305+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28813,"reported_epoch":49,"state":"active+undersized+remapped","last_fresh":"2024-09-17T15:35:58.120315+0000","last_change":"2024-09-17T15:35:58.120473+0000","last_active":"2024-09-17T15:35:58.120315+0000","last_peered":"2024-09-17T15:35:58.120315+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:35:58.120315+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0","1"],"object_location_counts":[{"shards":"2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":94,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.891932+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:35:58.891932+0000","last_peered":"2024-09-17T15:35:58.891932+0000","last_clean":"2024-09-17T15:35:58.891932+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:35:58.891932+0000","last_undegraded":"2024-09-17T15:35:58.891932+0000","last_fullsized":"2024-09-17T15:35:58.891932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29156,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866426+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:38.866426+0000","last_peered":"2024-09-17T15:35:38.866426+0000","last_clean":"2024-09-17T15:35:38.866426+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:38.866426+0000","last_undegraded":"2024-09-17T15:35:38.866426+0000","last_fullsized":"2024-09-17T15:35:38.866426+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21280,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:58.896971+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:58.896971+0000","last_peered":"2024-09-17T15:35:58.896971+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:58.896971+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.892023+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:35:58.892023+0000","last_peered":"2024-09-17T15:35:58.892023+0000","last_clean":"2024-09-17T15:35:58.892023+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:35:58.892023+0000","last_undegraded":"2024-09-17T15:35:58.892023+0000","last_fullsized":"2024-09-17T15:35:58.892023+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28072,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.892209+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:35:58.892209+0000","last_peered":"2024-09-17T15:35:58.892209+0000","last_clean":"2024-09-17T15:35:58.892209+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:35:58.892209+0000","last_undegraded":"2024-09-17T15:35:58.892209+0000","last_fullsized":"2024-09-17T15:35:58.892209+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.867463+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:35:38.867463+0000","last_peered":"2024-09-17T15:35:38.867463+0000","last_clean":"2024-09-17T15:35:38.867463+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:35:38.867463+0000","last_undegraded":"2024-09-17T15:35:38.867463+0000","last_fullsized":"2024-09-17T15:35:38.867463+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21201,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:58.896015+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:58.896015+0000","last_peered":"2024-09-17T15:35:58.896015+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:58.896015+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.892126+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:35:58.892126+0000","last_peered":"2024-09-17T15:35:58.892126+0000","last_clean":"2024-09-17T15:35:58.892126+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:35:58.892126+0000","last_undegraded":"2024-09-17T15:35:58.892126+0000","last_fullsized":"2024-09-17T15:35:58.892126+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.892366+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:35:58.892366+0000","last_peered":"2024-09-17T15:35:58.892366+0000","last_clean":"2024-09-17T15:35:58.892366+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:35:58.892366+0000","last_undegraded":"2024-09-17T15:35:58.892366+0000","last_fullsized":"2024-09-17T15:35:58.892366+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29023,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.891923+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:58.891923+0000","last_peered":"2024-09-17T15:35:58.891923+0000","last_clean":"2024-09-17T15:35:58.891923+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:58.891923+0000","last_undegraded":"2024-09-17T15:35:58.891923+0000","last_fullsized":"2024-09-17T15:35:58.891923+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28212,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21248,"num_bytes_recovered":41572719,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202629120,"data_stored":202506278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051715,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497268,"kb_used_data":132380,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874572,"statfs":{"total":96636764160,"available":96127561728,"internally_reserved":0,"allocated":135557120,"data_stored":135249043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724660,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483368,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888472,"statfs":{"total":96636764160,"available":96141795328,"internally_reserved":0,"allocated":479232,"data_stored":147336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":494464683},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247943,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539360,"kb_used_data":66248,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832480,"statfs":{"total":96636764160,"available":96084459520,"internally_reserved":0,"allocated":67837952,"data_stored":67525798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135147520,"data_stored":135111390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:36:01.024 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:36:01.024 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'20648", 'reported_seq': 20914, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:58.896975+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:35:58.896975+0000', 'last_peered': '2024-09-17T15:35:58.896975+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:35:58.896975+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'10600", 'ondisk_log_start': "43'10600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20648, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:01.024 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:36:01.024 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'20934", 'reported_seq': 21161, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:38.866979+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:35:38.866979+0000', 'last_peered': '2024-09-17T15:35:38.866979+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:35:38.866979+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "43'10900", 'ondisk_log_start': "43'10900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 19091, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4125}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:36:01.025 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:36:01.025 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'21251", 'reported_seq': 21558, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:58.895307+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:35:58.895307+0000', 'last_peered': '2024-09-17T15:35:58.895307+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:35:58.895307+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "43'11200", 'ondisk_log_start': "43'11200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17750, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4191}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:01.025 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:36:01.025 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'20768", 'reported_seq': 21424, 'reported_epoch': 50, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:59.389955+0000', 'last_change': '2024-09-17T15:35:58.120749+0000', 'last_active': '2024-09-17T15:35:59.389955+0000', 'last_peered': '2024-09-17T15:35:59.389955+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:35:59.389955+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "43'10700", 'ondisk_log_start': "43'10700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3924, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17537, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 250, 'num_bytes_recovered': 966, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3922}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:01.025 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:36:01.025 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20542", 'reported_seq': 20823, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:58.896110+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:35:58.896110+0000', 'last_peered': '2024-09-17T15:35:58.896110+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:35:58.896110+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17138, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3943}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:01.025 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-17T15:36:01.025 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'20519", 'reported_seq': 28813, 'reported_epoch': 49, 'state': 'active+undersized+remapped', 'last_fresh': '2024-09-17T15:35:58.120315+0000', 'last_change': '2024-09-17T15:35:58.120473+0000', 'last_active': '2024-09-17T15:35:58.120315+0000', 'last_peered': '2024-09-17T15:35:58.120315+0000', 'last_clean': '2024-09-17T15:26:20.152702+0000', 'last_became_active': '2024-09-17T15:26:22.589565+0000', 'last_became_peered': '2024-09-17T15:26:22.589565+0000', 'last_unstale': '2024-09-17T15:35:58.120315+0000', 'last_undegraded': '2024-09-17T15:26:22.230922+0000', 'last_fullsized': '2024-09-17T15:26:22.189912+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'2142", 'last_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:55.123199+0000', 'objects_scrubbed': 2101, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 715, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18414, 'num_write_kb': 32628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4078, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0', '1'], 'object_location_counts': [{'shards': '2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:36:01.026 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:36:01.026 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'20966", 'reported_seq': 21280, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:58.896971+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:35:58.896971+0000', 'last_peered': '2024-09-17T15:35:58.896971+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:35:58.896971+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "43'10900", 'ondisk_log_start': "43'10900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17514, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4057}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:01.026 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:36:01.026 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'20886", 'reported_seq': 21201, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:58.896015+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:35:58.896015+0000', 'last_peered': '2024-09-17T15:35:58.896015+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:35:58.896015+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'10800", 'ondisk_log_start': "43'10800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17673, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3980}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:01.026 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:36:01.483 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:01.483 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 28212/24 objects degraded (117550.000%), 7 pgs degraded, 8 pgs undersized","count":15},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":698,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+undersized+remapped","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1556475904,"bytes_avail":288353816576,"bytes_total":289910292480,"degraded_objects":28212,"degraded_total":24,"degraded_ratio":1175.5,"recovering_objects_per_sec":211,"recovering_bytes_per_sec":96,"recovering_keys_per_sec":0,"num_objects_recovered":2120,"num_bytes_recovered":966,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:36:01.501 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:36:01.501 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:36:01.858 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:01.859 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:36:01.871 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":393,"stamp":"2024-09-17T15:36:01.815551+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28212,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21249,"num_bytes_recovered":41572738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1519996,"kb_used_data":199096,"kb_used_omap":13,"kb_used_meta":1320818,"kb_avail":281595524,"statfs":{"total":289910292480,"available":288353816576,"internally_reserved":0,"allocated":203874304,"data_stored":202922177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1352518655},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1244,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1244,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001473"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20914,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:58.896975+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:58.896975+0000","last_peered":"2024-09-17T15:35:58.896975+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:58.896975+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21049,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866657+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:38.866657+0000","last_peered":"2024-09-17T15:35:38.866657+0000","last_clean":"2024-09-17T15:35:38.866657+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:38.866657+0000","last_undegraded":"2024-09-17T15:35:38.866657+0000","last_fullsized":"2024-09-17T15:35:38.866657+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21161,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:38.866979+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:38.866979+0000","last_peered":"2024-09-17T15:35:38.866979+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:38.866979+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4125}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21054,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866816+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:38.866816+0000","last_peered":"2024-09-17T15:35:38.866816+0000","last_clean":"2024-09-17T15:35:38.866816+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:38.866816+0000","last_undegraded":"2024-09-17T15:35:38.866816+0000","last_fullsized":"2024-09-17T15:35:38.866816+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20799,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.895883+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:58.895883+0000","last_peered":"2024-09-17T15:35:58.895883+0000","last_clean":"2024-09-17T15:35:58.895883+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:58.895883+0000","last_undegraded":"2024-09-17T15:35:58.895883+0000","last_fullsized":"2024-09-17T15:35:58.895883+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21558,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:58.895307+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:58.895307+0000","last_peered":"2024-09-17T15:35:58.895307+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:58.895307+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4191},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21424,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.389955+0000","last_change":"2024-09-17T15:35:58.120749+0000","last_active":"2024-09-17T15:35:59.389955+0000","last_peered":"2024-09-17T15:35:59.389955+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:35:59.389955+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3924,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":250,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3922}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21061,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.895312+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:58.895312+0000","last_peered":"2024-09-17T15:35:58.895312+0000","last_clean":"2024-09-17T15:35:58.895312+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:58.895312+0000","last_undegraded":"2024-09-17T15:35:58.895312+0000","last_fullsized":"2024-09-17T15:35:58.895312+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.891661+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:35:58.891661+0000","last_peered":"2024-09-17T15:35:58.891661+0000","last_clean":"2024-09-17T15:35:58.891661+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:35:58.891661+0000","last_undegraded":"2024-09-17T15:35:58.891661+0000","last_fullsized":"2024-09-17T15:35:58.891661+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29116,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.892028+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:58.892028+0000","last_peered":"2024-09-17T15:35:58.892028+0000","last_clean":"2024-09-17T15:35:58.892028+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:58.892028+0000","last_undegraded":"2024-09-17T15:35:58.892028+0000","last_fullsized":"2024-09-17T15:35:58.892028+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866649+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:35:38.866649+0000","last_peered":"2024-09-17T15:35:38.866649+0000","last_clean":"2024-09-17T15:35:38.866649+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:35:38.866649+0000","last_undegraded":"2024-09-17T15:35:38.866649+0000","last_fullsized":"2024-09-17T15:35:38.866649+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20823,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:58.896110+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:58.896110+0000","last_peered":"2024-09-17T15:35:58.896110+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:58.896110+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":120,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.895305+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:35:58.895305+0000","last_peered":"2024-09-17T15:35:58.895305+0000","last_clean":"2024-09-17T15:35:58.895305+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:35:58.895305+0000","last_undegraded":"2024-09-17T15:35:58.895305+0000","last_fullsized":"2024-09-17T15:35:58.895305+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28813,"reported_epoch":49,"state":"active+undersized+remapped","last_fresh":"2024-09-17T15:35:58.120315+0000","last_change":"2024-09-17T15:35:58.120473+0000","last_active":"2024-09-17T15:35:58.120315+0000","last_peered":"2024-09-17T15:35:58.120315+0000","last_clean":"2024-09-17T15:26:20.152702+0000","last_became_active":"2024-09-17T15:26:22.589565+0000","last_became_peered":"2024-09-17T15:26:22.589565+0000","last_unstale":"2024-09-17T15:35:58.120315+0000","last_undegraded":"2024-09-17T15:26:22.230922+0000","last_fullsized":"2024-09-17T15:26:22.189912+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0","1"],"object_location_counts":[{"shards":"2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":94,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.891932+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:35:58.891932+0000","last_peered":"2024-09-17T15:35:58.891932+0000","last_clean":"2024-09-17T15:35:58.891932+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:35:58.891932+0000","last_undegraded":"2024-09-17T15:35:58.891932+0000","last_fullsized":"2024-09-17T15:35:58.891932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29156,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.866426+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:38.866426+0000","last_peered":"2024-09-17T15:35:38.866426+0000","last_clean":"2024-09-17T15:35:38.866426+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:38.866426+0000","last_undegraded":"2024-09-17T15:35:38.866426+0000","last_fullsized":"2024-09-17T15:35:38.866426+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21280,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:58.896971+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:58.896971+0000","last_peered":"2024-09-17T15:35:58.896971+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:58.896971+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.892023+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:35:58.892023+0000","last_peered":"2024-09-17T15:35:58.892023+0000","last_clean":"2024-09-17T15:35:58.892023+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:35:58.892023+0000","last_undegraded":"2024-09-17T15:35:58.892023+0000","last_fullsized":"2024-09-17T15:35:58.892023+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28072,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.892209+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:35:58.892209+0000","last_peered":"2024-09-17T15:35:58.892209+0000","last_clean":"2024-09-17T15:35:58.892209+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:35:58.892209+0000","last_undegraded":"2024-09-17T15:35:58.892209+0000","last_fullsized":"2024-09-17T15:35:58.892209+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T15:35:38.867463+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:35:38.867463+0000","last_peered":"2024-09-17T15:35:38.867463+0000","last_clean":"2024-09-17T15:35:38.867463+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:35:38.867463+0000","last_undegraded":"2024-09-17T15:35:38.867463+0000","last_fullsized":"2024-09-17T15:35:38.867463+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21201,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:58.896015+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:58.896015+0000","last_peered":"2024-09-17T15:35:58.896015+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:58.896015+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.892126+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:35:58.892126+0000","last_peered":"2024-09-17T15:35:58.892126+0000","last_clean":"2024-09-17T15:35:58.892126+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:35:58.892126+0000","last_undegraded":"2024-09-17T15:35:58.892126+0000","last_fullsized":"2024-09-17T15:35:58.892126+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.892366+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:35:58.892366+0000","last_peered":"2024-09-17T15:35:58.892366+0000","last_clean":"2024-09-17T15:35:58.892366+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:35:58.892366+0000","last_undegraded":"2024-09-17T15:35:58.892366+0000","last_fullsized":"2024-09-17T15:35:58.892366+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29023,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.891923+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:58.891923+0000","last_peered":"2024-09-17T15:35:58.891923+0000","last_clean":"2024-09-17T15:35:58.891923+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:58.891923+0000","last_undegraded":"2024-09-17T15:35:58.891923+0000","last_fullsized":"2024-09-17T15:35:58.891923+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28212,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21248,"num_bytes_recovered":41572719,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202629120,"data_stored":202506278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051715,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497268,"kb_used_data":132380,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874572,"statfs":{"total":96636764160,"available":96127561728,"internally_reserved":0,"allocated":135557120,"data_stored":135249043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724660,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483368,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888472,"statfs":{"total":96636764160,"available":96141795328,"internally_reserved":0,"allocated":479232,"data_stored":147336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":494464683},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247943,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539360,"kb_used_data":66248,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832480,"statfs":{"total":96636764160,"available":96084459520,"internally_reserved":0,"allocated":67837952,"data_stored":67525798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135147520,"data_stored":135111390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:36:04.873 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:36:05.296 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:05.296 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:36:05.310 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":394,"stamp":"2024-09-17T15:36:03.815946+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28212,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21249,"num_bytes_recovered":41572738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1520012,"kb_used_data":199112,"kb_used_omap":13,"kb_used_meta":1320818,"kb_avail":281595508,"statfs":{"total":289910292480,"available":288353800192,"internally_reserved":0,"allocated":203890688,"data_stored":202927960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1352518655},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1244,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1244,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001536"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20914,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:58.896975+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:58.896975+0000","last_peered":"2024-09-17T15:35:58.896975+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:58.896975+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21053,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899472+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:59.899472+0000","last_peered":"2024-09-17T15:35:59.899472+0000","last_clean":"2024-09-17T15:35:59.899472+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:59.899472+0000","last_undegraded":"2024-09-17T15:35:59.899472+0000","last_fullsized":"2024-09-17T15:35:59.899472+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21165,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.899948+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:59.899948+0000","last_peered":"2024-09-17T15:35:59.899948+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:59.899948+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4125}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21058,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899239+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:59.899239+0000","last_peered":"2024-09-17T15:35:59.899239+0000","last_clean":"2024-09-17T15:35:59.899239+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:59.899239+0000","last_undegraded":"2024-09-17T15:35:59.899239+0000","last_fullsized":"2024-09-17T15:35:59.899239+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20799,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.895883+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:58.895883+0000","last_peered":"2024-09-17T15:35:58.895883+0000","last_clean":"2024-09-17T15:35:58.895883+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:58.895883+0000","last_undegraded":"2024-09-17T15:35:58.895883+0000","last_fullsized":"2024-09-17T15:35:58.895883+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21558,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:58.895307+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:58.895307+0000","last_peered":"2024-09-17T15:35:58.895307+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:58.895307+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4191},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21424,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.389955+0000","last_change":"2024-09-17T15:35:58.120749+0000","last_active":"2024-09-17T15:35:59.389955+0000","last_peered":"2024-09-17T15:35:59.389955+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:35:59.389955+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3924,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":250,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3922}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21061,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.895312+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:58.895312+0000","last_peered":"2024-09-17T15:35:58.895312+0000","last_clean":"2024-09-17T15:35:58.895312+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:58.895312+0000","last_undegraded":"2024-09-17T15:35:58.895312+0000","last_fullsized":"2024-09-17T15:35:58.895312+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.891661+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:35:58.891661+0000","last_peered":"2024-09-17T15:35:58.891661+0000","last_clean":"2024-09-17T15:35:58.891661+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:35:58.891661+0000","last_undegraded":"2024-09-17T15:35:58.891661+0000","last_fullsized":"2024-09-17T15:35:58.891661+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29116,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.892028+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:58.892028+0000","last_peered":"2024-09-17T15:35:58.892028+0000","last_clean":"2024-09-17T15:35:58.892028+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:58.892028+0000","last_undegraded":"2024-09-17T15:35:58.892028+0000","last_fullsized":"2024-09-17T15:35:58.892028+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899621+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:35:59.899621+0000","last_peered":"2024-09-17T15:35:59.899621+0000","last_clean":"2024-09-17T15:35:59.899621+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:35:59.899621+0000","last_undegraded":"2024-09-17T15:35:59.899621+0000","last_fullsized":"2024-09-17T15:35:59.899621+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20823,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:58.896110+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:58.896110+0000","last_peered":"2024-09-17T15:35:58.896110+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:58.896110+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":120,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.895305+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:35:58.895305+0000","last_peered":"2024-09-17T15:35:58.895305+0000","last_clean":"2024-09-17T15:35:58.895305+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:35:58.895305+0000","last_undegraded":"2024-09-17T15:35:58.895305+0000","last_fullsized":"2024-09-17T15:35:58.895305+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28823,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.967113+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:35:59.967113+0000","last_peered":"2024-09-17T15:35:59.967113+0000","last_clean":"2024-09-17T15:35:59.967113+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:35:59.967113+0000","last_undegraded":"2024-09-17T15:35:59.967113+0000","last_fullsized":"2024-09-17T15:35:59.967113+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":94,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.891932+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:35:58.891932+0000","last_peered":"2024-09-17T15:35:58.891932+0000","last_clean":"2024-09-17T15:35:58.891932+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:35:58.891932+0000","last_undegraded":"2024-09-17T15:35:58.891932+0000","last_fullsized":"2024-09-17T15:35:58.891932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29160,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.898948+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:59.898948+0000","last_peered":"2024-09-17T15:35:59.898948+0000","last_clean":"2024-09-17T15:35:59.898948+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:59.898948+0000","last_undegraded":"2024-09-17T15:35:59.898948+0000","last_fullsized":"2024-09-17T15:35:59.898948+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21280,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:58.896971+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:58.896971+0000","last_peered":"2024-09-17T15:35:58.896971+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:58.896971+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.892023+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:35:58.892023+0000","last_peered":"2024-09-17T15:35:58.892023+0000","last_clean":"2024-09-17T15:35:58.892023+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:35:58.892023+0000","last_undegraded":"2024-09-17T15:35:58.892023+0000","last_fullsized":"2024-09-17T15:35:58.892023+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28072,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.892209+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:35:58.892209+0000","last_peered":"2024-09-17T15:35:58.892209+0000","last_clean":"2024-09-17T15:35:58.892209+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:35:58.892209+0000","last_undegraded":"2024-09-17T15:35:58.892209+0000","last_fullsized":"2024-09-17T15:35:58.892209+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.900949+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:35:59.900949+0000","last_peered":"2024-09-17T15:35:59.900949+0000","last_clean":"2024-09-17T15:35:59.900949+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:35:59.900949+0000","last_undegraded":"2024-09-17T15:35:59.900949+0000","last_fullsized":"2024-09-17T15:35:59.900949+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21201,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:58.896015+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:58.896015+0000","last_peered":"2024-09-17T15:35:58.896015+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:58.896015+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.892126+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:35:58.892126+0000","last_peered":"2024-09-17T15:35:58.892126+0000","last_clean":"2024-09-17T15:35:58.892126+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:35:58.892126+0000","last_undegraded":"2024-09-17T15:35:58.892126+0000","last_fullsized":"2024-09-17T15:35:58.892126+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.892366+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:35:58.892366+0000","last_peered":"2024-09-17T15:35:58.892366+0000","last_clean":"2024-09-17T15:35:58.892366+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:35:58.892366+0000","last_undegraded":"2024-09-17T15:35:58.892366+0000","last_fullsized":"2024-09-17T15:35:58.892366+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29023,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.891923+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:58.891923+0000","last_peered":"2024-09-17T15:35:58.891923+0000","last_clean":"2024-09-17T15:35:58.891923+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:58.891923+0000","last_undegraded":"2024-09-17T15:35:58.891923+0000","last_fullsized":"2024-09-17T15:35:58.891923+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28212,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21248,"num_bytes_recovered":41572719,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202629120,"data_stored":202506278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051716,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497284,"kb_used_data":132396,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874556,"statfs":{"total":96636764160,"available":96127545344,"internally_reserved":0,"allocated":135573504,"data_stored":135254826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724660,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483368,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888472,"statfs":{"total":96636764160,"available":96141795328,"internally_reserved":0,"allocated":479232,"data_stored":147336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":494464683},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247943,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539360,"kb_used_data":66248,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832480,"statfs":{"total":96636764160,"available":96084459520,"internally_reserved":0,"allocated":67837952,"data_stored":67525798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135147520,"data_stored":135111390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:36:05.312 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:36:05.687 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:05.688 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:36:05.700 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":394,"stamp":"2024-09-17T15:36:03.815946+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28212,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21249,"num_bytes_recovered":41572738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1520012,"kb_used_data":199112,"kb_used_omap":13,"kb_used_meta":1320818,"kb_avail":281595508,"statfs":{"total":289910292480,"available":288353800192,"internally_reserved":0,"allocated":203890688,"data_stored":202927960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1352518655},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1244,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1244,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001536"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20914,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:58.896975+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:58.896975+0000","last_peered":"2024-09-17T15:35:58.896975+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:58.896975+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21053,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899472+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:59.899472+0000","last_peered":"2024-09-17T15:35:59.899472+0000","last_clean":"2024-09-17T15:35:59.899472+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:59.899472+0000","last_undegraded":"2024-09-17T15:35:59.899472+0000","last_fullsized":"2024-09-17T15:35:59.899472+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21165,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.899948+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:59.899948+0000","last_peered":"2024-09-17T15:35:59.899948+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:59.899948+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4125}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21058,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899239+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:59.899239+0000","last_peered":"2024-09-17T15:35:59.899239+0000","last_clean":"2024-09-17T15:35:59.899239+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:59.899239+0000","last_undegraded":"2024-09-17T15:35:59.899239+0000","last_fullsized":"2024-09-17T15:35:59.899239+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20799,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.895883+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:58.895883+0000","last_peered":"2024-09-17T15:35:58.895883+0000","last_clean":"2024-09-17T15:35:58.895883+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:58.895883+0000","last_undegraded":"2024-09-17T15:35:58.895883+0000","last_fullsized":"2024-09-17T15:35:58.895883+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21558,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:58.895307+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:58.895307+0000","last_peered":"2024-09-17T15:35:58.895307+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:58.895307+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4191},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21424,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.389955+0000","last_change":"2024-09-17T15:35:58.120749+0000","last_active":"2024-09-17T15:35:59.389955+0000","last_peered":"2024-09-17T15:35:59.389955+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:35:59.389955+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3924,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":250,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3922}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21061,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.895312+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:58.895312+0000","last_peered":"2024-09-17T15:35:58.895312+0000","last_clean":"2024-09-17T15:35:58.895312+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:58.895312+0000","last_undegraded":"2024-09-17T15:35:58.895312+0000","last_fullsized":"2024-09-17T15:35:58.895312+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.891661+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:35:58.891661+0000","last_peered":"2024-09-17T15:35:58.891661+0000","last_clean":"2024-09-17T15:35:58.891661+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:35:58.891661+0000","last_undegraded":"2024-09-17T15:35:58.891661+0000","last_fullsized":"2024-09-17T15:35:58.891661+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29116,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.892028+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:58.892028+0000","last_peered":"2024-09-17T15:35:58.892028+0000","last_clean":"2024-09-17T15:35:58.892028+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:58.892028+0000","last_undegraded":"2024-09-17T15:35:58.892028+0000","last_fullsized":"2024-09-17T15:35:58.892028+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899621+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:35:59.899621+0000","last_peered":"2024-09-17T15:35:59.899621+0000","last_clean":"2024-09-17T15:35:59.899621+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:35:59.899621+0000","last_undegraded":"2024-09-17T15:35:59.899621+0000","last_fullsized":"2024-09-17T15:35:59.899621+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20823,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:58.896110+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:58.896110+0000","last_peered":"2024-09-17T15:35:58.896110+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:58.896110+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":120,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.895305+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:35:58.895305+0000","last_peered":"2024-09-17T15:35:58.895305+0000","last_clean":"2024-09-17T15:35:58.895305+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:35:58.895305+0000","last_undegraded":"2024-09-17T15:35:58.895305+0000","last_fullsized":"2024-09-17T15:35:58.895305+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28823,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.967113+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:35:59.967113+0000","last_peered":"2024-09-17T15:35:59.967113+0000","last_clean":"2024-09-17T15:35:59.967113+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:35:59.967113+0000","last_undegraded":"2024-09-17T15:35:59.967113+0000","last_fullsized":"2024-09-17T15:35:59.967113+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":94,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.891932+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:35:58.891932+0000","last_peered":"2024-09-17T15:35:58.891932+0000","last_clean":"2024-09-17T15:35:58.891932+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:35:58.891932+0000","last_undegraded":"2024-09-17T15:35:58.891932+0000","last_fullsized":"2024-09-17T15:35:58.891932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29160,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.898948+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:59.898948+0000","last_peered":"2024-09-17T15:35:59.898948+0000","last_clean":"2024-09-17T15:35:59.898948+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:59.898948+0000","last_undegraded":"2024-09-17T15:35:59.898948+0000","last_fullsized":"2024-09-17T15:35:59.898948+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21280,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:58.896971+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:58.896971+0000","last_peered":"2024-09-17T15:35:58.896971+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:58.896971+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.892023+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:35:58.892023+0000","last_peered":"2024-09-17T15:35:58.892023+0000","last_clean":"2024-09-17T15:35:58.892023+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:35:58.892023+0000","last_undegraded":"2024-09-17T15:35:58.892023+0000","last_fullsized":"2024-09-17T15:35:58.892023+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28072,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.892209+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:35:58.892209+0000","last_peered":"2024-09-17T15:35:58.892209+0000","last_clean":"2024-09-17T15:35:58.892209+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:35:58.892209+0000","last_undegraded":"2024-09-17T15:35:58.892209+0000","last_fullsized":"2024-09-17T15:35:58.892209+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.900949+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:35:59.900949+0000","last_peered":"2024-09-17T15:35:59.900949+0000","last_clean":"2024-09-17T15:35:59.900949+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:35:59.900949+0000","last_undegraded":"2024-09-17T15:35:59.900949+0000","last_fullsized":"2024-09-17T15:35:59.900949+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21201,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:58.896015+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:58.896015+0000","last_peered":"2024-09-17T15:35:58.896015+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:58.896015+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.892126+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:35:58.892126+0000","last_peered":"2024-09-17T15:35:58.892126+0000","last_clean":"2024-09-17T15:35:58.892126+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:35:58.892126+0000","last_undegraded":"2024-09-17T15:35:58.892126+0000","last_fullsized":"2024-09-17T15:35:58.892126+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.892366+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:35:58.892366+0000","last_peered":"2024-09-17T15:35:58.892366+0000","last_clean":"2024-09-17T15:35:58.892366+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:35:58.892366+0000","last_undegraded":"2024-09-17T15:35:58.892366+0000","last_fullsized":"2024-09-17T15:35:58.892366+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29023,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T15:35:58.891923+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:58.891923+0000","last_peered":"2024-09-17T15:35:58.891923+0000","last_clean":"2024-09-17T15:35:58.891923+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:58.891923+0000","last_undegraded":"2024-09-17T15:35:58.891923+0000","last_fullsized":"2024-09-17T15:35:58.891923+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28212,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21248,"num_bytes_recovered":41572719,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202629120,"data_stored":202506278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051716,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497284,"kb_used_data":132396,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874556,"statfs":{"total":96636764160,"available":96127545344,"internally_reserved":0,"allocated":135573504,"data_stored":135254826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724660,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483368,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888472,"statfs":{"total":96636764160,"available":96141795328,"internally_reserved":0,"allocated":479232,"data_stored":147336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":494464683},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247943,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539360,"kb_used_data":66248,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832480,"statfs":{"total":96636764160,"available":96084459520,"internally_reserved":0,"allocated":67837952,"data_stored":67525798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135147520,"data_stored":135111390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:36:05.701 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:36:05.702 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'20648", 'reported_seq': 20914, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:58.896975+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:35:58.896975+0000', 'last_peered': '2024-09-17T15:35:58.896975+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:35:58.896975+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'10600", 'ondisk_log_start': "43'10600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20648, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:05.702 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:36:05.702 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'20934", 'reported_seq': 21165, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:59.899948+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:35:59.899948+0000', 'last_peered': '2024-09-17T15:35:59.899948+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:35:59.899948+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "43'10900", 'ondisk_log_start': "43'10900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 19091, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4125}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:36:05.702 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:36:05.702 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'21251", 'reported_seq': 21558, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:58.895307+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:35:58.895307+0000', 'last_peered': '2024-09-17T15:35:58.895307+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:35:58.895307+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "43'11200", 'ondisk_log_start': "43'11200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17750, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4191}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:05.702 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:36:05.702 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'20768", 'reported_seq': 21424, 'reported_epoch': 50, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:59.389955+0000', 'last_change': '2024-09-17T15:35:58.120749+0000', 'last_active': '2024-09-17T15:35:59.389955+0000', 'last_peered': '2024-09-17T15:35:59.389955+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:35:59.389955+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "43'10700", 'ondisk_log_start': "43'10700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3924, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17537, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 250, 'num_bytes_recovered': 966, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3922}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:05.702 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:36:05.702 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20542", 'reported_seq': 20823, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:58.896110+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:35:58.896110+0000', 'last_peered': '2024-09-17T15:35:58.896110+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:35:58.896110+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17138, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3943}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:05.703 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:36:05.703 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'20966", 'reported_seq': 21280, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:58.896971+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:35:58.896971+0000', 'last_peered': '2024-09-17T15:35:58.896971+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:35:58.896971+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "43'10900", 'ondisk_log_start': "43'10900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17514, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4057}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:05.703 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:36:05.703 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'20886", 'reported_seq': 21201, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:58.896015+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:35:58.896015+0000', 'last_peered': '2024-09-17T15:35:58.896015+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:35:58.896015+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'10800", 'ondisk_log_start': "43'10800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17673, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3980}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:05.703 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:36:06.128 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:06.128 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 27307/24 objects degraded (113779.167%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":703,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1565032448,"bytes_avail":288345260032,"bytes_total":289910292480,"degraded_objects":27307,"degraded_total":24,"degraded_ratio":1137.7916666666667,"recovering_objects_per_sec":268,"recovering_bytes_per_sec":120,"recovering_keys_per_sec":0,"num_objects_recovered":2149,"num_bytes_recovered":966,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:36:06.143 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:36:06.143 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:36:06.514 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:06.514 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:36:06.529 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":395,"stamp":"2024-09-17T15:36:05.816299+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27307,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22154,"num_bytes_recovered":41572738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1528352,"kb_used_data":199136,"kb_used_omap":13,"kb_used_meta":1329138,"kb_avail":281587168,"statfs":{"total":289910292480,"available":288345260032,"internally_reserved":0,"allocated":203915264,"data_stored":202936624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1361038335},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2149,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001423"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20916,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900219+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:59.900219+0000","last_peered":"2024-09-17T15:35:59.900219+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:59.900219+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21053,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899472+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:59.899472+0000","last_peered":"2024-09-17T15:35:59.899472+0000","last_clean":"2024-09-17T15:35:59.899472+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:59.899472+0000","last_undegraded":"2024-09-17T15:35:59.899472+0000","last_fullsized":"2024-09-17T15:35:59.899472+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21165,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.899948+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:59.899948+0000","last_peered":"2024-09-17T15:35:59.899948+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:59.899948+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4125}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21058,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899239+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:59.899239+0000","last_peered":"2024-09-17T15:35:59.899239+0000","last_clean":"2024-09-17T15:35:59.899239+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:59.899239+0000","last_undegraded":"2024-09-17T15:35:59.899239+0000","last_fullsized":"2024-09-17T15:35:59.899239+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20801,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899356+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:59.899356+0000","last_peered":"2024-09-17T15:35:59.899356+0000","last_clean":"2024-09-17T15:35:59.899356+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:59.899356+0000","last_undegraded":"2024-09-17T15:35:59.899356+0000","last_fullsized":"2024-09-17T15:35:59.899356+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21560,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.899441+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:59.899441+0000","last_peered":"2024-09-17T15:35:59.899441+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:59.899441+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4191},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":23236,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:04.399924+0000","last_change":"2024-09-17T15:35:58.120749+0000","last_active":"2024-09-17T15:36:04.399924+0000","last_peered":"2024-09-17T15:36:04.399924+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:36:04.399924+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3019,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1155,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3017}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21063,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899226+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:59.899226+0000","last_peered":"2024-09-17T15:35:59.899226+0000","last_clean":"2024-09-17T15:35:59.899226+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:59.899226+0000","last_undegraded":"2024-09-17T15:35:59.899226+0000","last_fullsized":"2024-09-17T15:35:59.899226+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":86,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896693+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:35:59.896693+0000","last_peered":"2024-09-17T15:35:59.896693+0000","last_clean":"2024-09-17T15:35:59.896693+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:35:59.896693+0000","last_undegraded":"2024-09-17T15:35:59.896693+0000","last_fullsized":"2024-09-17T15:35:59.896693+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29118,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896980+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:59.896980+0000","last_peered":"2024-09-17T15:35:59.896980+0000","last_clean":"2024-09-17T15:35:59.896980+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:59.896980+0000","last_undegraded":"2024-09-17T15:35:59.896980+0000","last_fullsized":"2024-09-17T15:35:59.896980+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899621+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:35:59.899621+0000","last_peered":"2024-09-17T15:35:59.899621+0000","last_clean":"2024-09-17T15:35:59.899621+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:35:59.899621+0000","last_undegraded":"2024-09-17T15:35:59.899621+0000","last_fullsized":"2024-09-17T15:35:59.899621+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20825,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.899685+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:59.899685+0000","last_peered":"2024-09-17T15:35:59.899685+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:59.899685+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":122,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899340+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:35:59.899340+0000","last_peered":"2024-09-17T15:35:59.899340+0000","last_clean":"2024-09-17T15:35:59.899340+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:35:59.899340+0000","last_undegraded":"2024-09-17T15:35:59.899340+0000","last_fullsized":"2024-09-17T15:35:59.899340+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28823,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.967113+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:35:59.967113+0000","last_peered":"2024-09-17T15:35:59.967113+0000","last_clean":"2024-09-17T15:35:59.967113+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:35:59.967113+0000","last_undegraded":"2024-09-17T15:35:59.967113+0000","last_fullsized":"2024-09-17T15:35:59.967113+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":96,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896940+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:35:59.896940+0000","last_peered":"2024-09-17T15:35:59.896940+0000","last_clean":"2024-09-17T15:35:59.896940+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:35:59.896940+0000","last_undegraded":"2024-09-17T15:35:59.896940+0000","last_fullsized":"2024-09-17T15:35:59.896940+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29160,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.898948+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:59.898948+0000","last_peered":"2024-09-17T15:35:59.898948+0000","last_clean":"2024-09-17T15:35:59.898948+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:59.898948+0000","last_undegraded":"2024-09-17T15:35:59.898948+0000","last_fullsized":"2024-09-17T15:35:59.898948+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21282,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900110+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:59.900110+0000","last_peered":"2024-09-17T15:35:59.900110+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:59.900110+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":86,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897175+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:35:59.897175+0000","last_peered":"2024-09-17T15:35:59.897175+0000","last_clean":"2024-09-17T15:35:59.897175+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:35:59.897175+0000","last_undegraded":"2024-09-17T15:35:59.897175+0000","last_fullsized":"2024-09-17T15:35:59.897175+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28074,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897466+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:35:59.897466+0000","last_peered":"2024-09-17T15:35:59.897466+0000","last_clean":"2024-09-17T15:35:59.897466+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:35:59.897466+0000","last_undegraded":"2024-09-17T15:35:59.897466+0000","last_fullsized":"2024-09-17T15:35:59.897466+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.900949+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:35:59.900949+0000","last_peered":"2024-09-17T15:35:59.900949+0000","last_clean":"2024-09-17T15:35:59.900949+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:35:59.900949+0000","last_undegraded":"2024-09-17T15:35:59.900949+0000","last_fullsized":"2024-09-17T15:35:59.900949+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21203,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900017+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:59.900017+0000","last_peered":"2024-09-17T15:35:59.900017+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:59.900017+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":86,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897393+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:35:59.897393+0000","last_peered":"2024-09-17T15:35:59.897393+0000","last_clean":"2024-09-17T15:35:59.897393+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:35:59.897393+0000","last_undegraded":"2024-09-17T15:35:59.897393+0000","last_fullsized":"2024-09-17T15:35:59.897393+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":91,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896940+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:35:59.896940+0000","last_peered":"2024-09-17T15:35:59.896940+0000","last_clean":"2024-09-17T15:35:59.896940+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:35:59.896940+0000","last_undegraded":"2024-09-17T15:35:59.896940+0000","last_fullsized":"2024-09-17T15:35:59.896940+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29025,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897092+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:59.897092+0000","last_peered":"2024-09-17T15:35:59.897092+0000","last_clean":"2024-09-17T15:35:59.897092+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:59.897092+0000","last_undegraded":"2024-09-17T15:35:59.897092+0000","last_fullsized":"2024-09-17T15:35:59.897092+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27307,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22153,"num_bytes_recovered":41572719,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202629120,"data_stored":202506278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051716,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497284,"kb_used_data":132396,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874556,"statfs":{"total":96636764160,"available":96127545344,"internally_reserved":0,"allocated":135573504,"data_stored":135254826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724661,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491692,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880148,"statfs":{"total":96636764160,"available":96133271552,"internally_reserved":0,"allocated":487424,"data_stored":150217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":502984363},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247944,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539376,"kb_used_data":66264,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832464,"statfs":{"total":96636764160,"available":96084443136,"internally_reserved":0,"allocated":67854336,"data_stored":67531581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135147520,"data_stored":135111390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:36:09.532 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:36:09.855 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:09.856 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:36:09.869 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":397,"stamp":"2024-09-17T15:36:09.817053+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26423,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23038,"num_bytes_recovered":41572738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1515872,"kb_used_data":199136,"kb_used_omap":13,"kb_used_meta":1316658,"kb_avail":281599648,"statfs":{"total":289910292480,"available":288358039552,"internally_reserved":0,"allocated":203915264,"data_stored":202936624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1348258815},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1789,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1789,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001863"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20916,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900219+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:59.900219+0000","last_peered":"2024-09-17T15:35:59.900219+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:59.900219+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21053,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899472+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:59.899472+0000","last_peered":"2024-09-17T15:35:59.899472+0000","last_clean":"2024-09-17T15:35:59.899472+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:59.899472+0000","last_undegraded":"2024-09-17T15:35:59.899472+0000","last_fullsized":"2024-09-17T15:35:59.899472+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21165,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.899948+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:59.899948+0000","last_peered":"2024-09-17T15:35:59.899948+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:59.899948+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4125}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21058,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899239+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:59.899239+0000","last_peered":"2024-09-17T15:35:59.899239+0000","last_clean":"2024-09-17T15:35:59.899239+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:59.899239+0000","last_undegraded":"2024-09-17T15:35:59.899239+0000","last_fullsized":"2024-09-17T15:35:59.899239+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20801,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899356+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:59.899356+0000","last_peered":"2024-09-17T15:35:59.899356+0000","last_clean":"2024-09-17T15:35:59.899356+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:59.899356+0000","last_undegraded":"2024-09-17T15:35:59.899356+0000","last_fullsized":"2024-09-17T15:35:59.899356+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21561,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900373+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:59.900373+0000","last_peered":"2024-09-17T15:35:59.900373+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:59.900373+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4191},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":25005,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:09.396562+0000","last_change":"2024-09-17T15:35:58.120749+0000","last_active":"2024-09-17T15:36:09.396562+0000","last_peered":"2024-09-17T15:36:09.396562+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:36:09.396562+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2135,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2039,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2133}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21063,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899226+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:59.899226+0000","last_peered":"2024-09-17T15:35:59.899226+0000","last_clean":"2024-09-17T15:35:59.899226+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:59.899226+0000","last_undegraded":"2024-09-17T15:35:59.899226+0000","last_fullsized":"2024-09-17T15:35:59.899226+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":86,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896693+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:35:59.896693+0000","last_peered":"2024-09-17T15:35:59.896693+0000","last_clean":"2024-09-17T15:35:59.896693+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:35:59.896693+0000","last_undegraded":"2024-09-17T15:35:59.896693+0000","last_fullsized":"2024-09-17T15:35:59.896693+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29118,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896980+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:59.896980+0000","last_peered":"2024-09-17T15:35:59.896980+0000","last_clean":"2024-09-17T15:35:59.896980+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:59.896980+0000","last_undegraded":"2024-09-17T15:35:59.896980+0000","last_fullsized":"2024-09-17T15:35:59.896980+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899621+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:35:59.899621+0000","last_peered":"2024-09-17T15:35:59.899621+0000","last_clean":"2024-09-17T15:35:59.899621+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:35:59.899621+0000","last_undegraded":"2024-09-17T15:35:59.899621+0000","last_fullsized":"2024-09-17T15:35:59.899621+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20825,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.899685+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:59.899685+0000","last_peered":"2024-09-17T15:35:59.899685+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:59.899685+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":122,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899340+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:35:59.899340+0000","last_peered":"2024-09-17T15:35:59.899340+0000","last_clean":"2024-09-17T15:35:59.899340+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:35:59.899340+0000","last_undegraded":"2024-09-17T15:35:59.899340+0000","last_fullsized":"2024-09-17T15:35:59.899340+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28823,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.967113+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:35:59.967113+0000","last_peered":"2024-09-17T15:35:59.967113+0000","last_clean":"2024-09-17T15:35:59.967113+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:35:59.967113+0000","last_undegraded":"2024-09-17T15:35:59.967113+0000","last_fullsized":"2024-09-17T15:35:59.967113+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":96,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896940+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:35:59.896940+0000","last_peered":"2024-09-17T15:35:59.896940+0000","last_clean":"2024-09-17T15:35:59.896940+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:35:59.896940+0000","last_undegraded":"2024-09-17T15:35:59.896940+0000","last_fullsized":"2024-09-17T15:35:59.896940+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29160,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.898948+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:59.898948+0000","last_peered":"2024-09-17T15:35:59.898948+0000","last_clean":"2024-09-17T15:35:59.898948+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:59.898948+0000","last_undegraded":"2024-09-17T15:35:59.898948+0000","last_fullsized":"2024-09-17T15:35:59.898948+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21282,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900110+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:59.900110+0000","last_peered":"2024-09-17T15:35:59.900110+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:59.900110+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":86,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897175+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:35:59.897175+0000","last_peered":"2024-09-17T15:35:59.897175+0000","last_clean":"2024-09-17T15:35:59.897175+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:35:59.897175+0000","last_undegraded":"2024-09-17T15:35:59.897175+0000","last_fullsized":"2024-09-17T15:35:59.897175+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28074,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897466+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:35:59.897466+0000","last_peered":"2024-09-17T15:35:59.897466+0000","last_clean":"2024-09-17T15:35:59.897466+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:35:59.897466+0000","last_undegraded":"2024-09-17T15:35:59.897466+0000","last_fullsized":"2024-09-17T15:35:59.897466+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.900949+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:35:59.900949+0000","last_peered":"2024-09-17T15:35:59.900949+0000","last_clean":"2024-09-17T15:35:59.900949+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:35:59.900949+0000","last_undegraded":"2024-09-17T15:35:59.900949+0000","last_fullsized":"2024-09-17T15:35:59.900949+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21203,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900017+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:59.900017+0000","last_peered":"2024-09-17T15:35:59.900017+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:59.900017+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":86,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897393+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:35:59.897393+0000","last_peered":"2024-09-17T15:35:59.897393+0000","last_clean":"2024-09-17T15:35:59.897393+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:35:59.897393+0000","last_undegraded":"2024-09-17T15:35:59.897393+0000","last_fullsized":"2024-09-17T15:35:59.897393+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":91,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896940+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:35:59.896940+0000","last_peered":"2024-09-17T15:35:59.896940+0000","last_clean":"2024-09-17T15:35:59.896940+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:35:59.896940+0000","last_undegraded":"2024-09-17T15:35:59.896940+0000","last_fullsized":"2024-09-17T15:35:59.896940+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29025,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897092+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:59.897092+0000","last_peered":"2024-09-17T15:35:59.897092+0000","last_clean":"2024-09-17T15:35:59.897092+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:59.897092+0000","last_undegraded":"2024-09-17T15:35:59.897092+0000","last_fullsized":"2024-09-17T15:35:59.897092+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26423,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23037,"num_bytes_recovered":41572719,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202629120,"data_stored":202506278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051717,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497284,"kb_used_data":132396,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874556,"statfs":{"total":96636764160,"available":96127545344,"internally_reserved":0,"allocated":135573504,"data_stored":135254826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724662,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479212,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892628,"statfs":{"total":96636764160,"available":96146051072,"internally_reserved":0,"allocated":487424,"data_stored":150217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":490204843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247945,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539376,"kb_used_data":66264,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832464,"statfs":{"total":96636764160,"available":96084443136,"internally_reserved":0,"allocated":67854336,"data_stored":67531581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135147520,"data_stored":135111390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:36:09.870 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:36:10.216 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:10.216 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:36:10.231 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":397,"stamp":"2024-09-17T15:36:09.817053+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26423,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23038,"num_bytes_recovered":41572738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1515872,"kb_used_data":199136,"kb_used_omap":13,"kb_used_meta":1316658,"kb_avail":281599648,"statfs":{"total":289910292480,"available":288358039552,"internally_reserved":0,"allocated":203915264,"data_stored":202936624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1348258815},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1789,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1789,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001863"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20916,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900219+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:59.900219+0000","last_peered":"2024-09-17T15:35:59.900219+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:59.900219+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21053,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899472+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:59.899472+0000","last_peered":"2024-09-17T15:35:59.899472+0000","last_clean":"2024-09-17T15:35:59.899472+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:59.899472+0000","last_undegraded":"2024-09-17T15:35:59.899472+0000","last_fullsized":"2024-09-17T15:35:59.899472+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21165,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.899948+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:59.899948+0000","last_peered":"2024-09-17T15:35:59.899948+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:59.899948+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4125}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21058,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899239+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:59.899239+0000","last_peered":"2024-09-17T15:35:59.899239+0000","last_clean":"2024-09-17T15:35:59.899239+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:59.899239+0000","last_undegraded":"2024-09-17T15:35:59.899239+0000","last_fullsized":"2024-09-17T15:35:59.899239+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20801,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899356+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:59.899356+0000","last_peered":"2024-09-17T15:35:59.899356+0000","last_clean":"2024-09-17T15:35:59.899356+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:59.899356+0000","last_undegraded":"2024-09-17T15:35:59.899356+0000","last_fullsized":"2024-09-17T15:35:59.899356+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21561,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900373+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:59.900373+0000","last_peered":"2024-09-17T15:35:59.900373+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:59.900373+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4191},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":25005,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:09.396562+0000","last_change":"2024-09-17T15:35:58.120749+0000","last_active":"2024-09-17T15:36:09.396562+0000","last_peered":"2024-09-17T15:36:09.396562+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:36:09.396562+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2135,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2039,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2133}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21063,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899226+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:59.899226+0000","last_peered":"2024-09-17T15:35:59.899226+0000","last_clean":"2024-09-17T15:35:59.899226+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:59.899226+0000","last_undegraded":"2024-09-17T15:35:59.899226+0000","last_fullsized":"2024-09-17T15:35:59.899226+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":86,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896693+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:35:59.896693+0000","last_peered":"2024-09-17T15:35:59.896693+0000","last_clean":"2024-09-17T15:35:59.896693+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:35:59.896693+0000","last_undegraded":"2024-09-17T15:35:59.896693+0000","last_fullsized":"2024-09-17T15:35:59.896693+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29118,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896980+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:59.896980+0000","last_peered":"2024-09-17T15:35:59.896980+0000","last_clean":"2024-09-17T15:35:59.896980+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:59.896980+0000","last_undegraded":"2024-09-17T15:35:59.896980+0000","last_fullsized":"2024-09-17T15:35:59.896980+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899621+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:35:59.899621+0000","last_peered":"2024-09-17T15:35:59.899621+0000","last_clean":"2024-09-17T15:35:59.899621+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:35:59.899621+0000","last_undegraded":"2024-09-17T15:35:59.899621+0000","last_fullsized":"2024-09-17T15:35:59.899621+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20825,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.899685+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:59.899685+0000","last_peered":"2024-09-17T15:35:59.899685+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:59.899685+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":122,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899340+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:35:59.899340+0000","last_peered":"2024-09-17T15:35:59.899340+0000","last_clean":"2024-09-17T15:35:59.899340+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:35:59.899340+0000","last_undegraded":"2024-09-17T15:35:59.899340+0000","last_fullsized":"2024-09-17T15:35:59.899340+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28823,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.967113+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:35:59.967113+0000","last_peered":"2024-09-17T15:35:59.967113+0000","last_clean":"2024-09-17T15:35:59.967113+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:35:59.967113+0000","last_undegraded":"2024-09-17T15:35:59.967113+0000","last_fullsized":"2024-09-17T15:35:59.967113+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":96,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896940+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:35:59.896940+0000","last_peered":"2024-09-17T15:35:59.896940+0000","last_clean":"2024-09-17T15:35:59.896940+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:35:59.896940+0000","last_undegraded":"2024-09-17T15:35:59.896940+0000","last_fullsized":"2024-09-17T15:35:59.896940+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29160,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.898948+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:59.898948+0000","last_peered":"2024-09-17T15:35:59.898948+0000","last_clean":"2024-09-17T15:35:59.898948+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:59.898948+0000","last_undegraded":"2024-09-17T15:35:59.898948+0000","last_fullsized":"2024-09-17T15:35:59.898948+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21282,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900110+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:59.900110+0000","last_peered":"2024-09-17T15:35:59.900110+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:59.900110+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":86,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897175+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:35:59.897175+0000","last_peered":"2024-09-17T15:35:59.897175+0000","last_clean":"2024-09-17T15:35:59.897175+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:35:59.897175+0000","last_undegraded":"2024-09-17T15:35:59.897175+0000","last_fullsized":"2024-09-17T15:35:59.897175+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28074,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897466+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:35:59.897466+0000","last_peered":"2024-09-17T15:35:59.897466+0000","last_clean":"2024-09-17T15:35:59.897466+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:35:59.897466+0000","last_undegraded":"2024-09-17T15:35:59.897466+0000","last_fullsized":"2024-09-17T15:35:59.897466+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.900949+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:35:59.900949+0000","last_peered":"2024-09-17T15:35:59.900949+0000","last_clean":"2024-09-17T15:35:59.900949+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:35:59.900949+0000","last_undegraded":"2024-09-17T15:35:59.900949+0000","last_fullsized":"2024-09-17T15:35:59.900949+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21203,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900017+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:59.900017+0000","last_peered":"2024-09-17T15:35:59.900017+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:59.900017+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":86,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897393+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:35:59.897393+0000","last_peered":"2024-09-17T15:35:59.897393+0000","last_clean":"2024-09-17T15:35:59.897393+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:35:59.897393+0000","last_undegraded":"2024-09-17T15:35:59.897393+0000","last_fullsized":"2024-09-17T15:35:59.897393+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":91,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896940+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:35:59.896940+0000","last_peered":"2024-09-17T15:35:59.896940+0000","last_clean":"2024-09-17T15:35:59.896940+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:35:59.896940+0000","last_undegraded":"2024-09-17T15:35:59.896940+0000","last_fullsized":"2024-09-17T15:35:59.896940+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29025,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897092+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:59.897092+0000","last_peered":"2024-09-17T15:35:59.897092+0000","last_clean":"2024-09-17T15:35:59.897092+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:59.897092+0000","last_undegraded":"2024-09-17T15:35:59.897092+0000","last_fullsized":"2024-09-17T15:35:59.897092+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26423,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23037,"num_bytes_recovered":41572719,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202629120,"data_stored":202506278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051717,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497284,"kb_used_data":132396,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874556,"statfs":{"total":96636764160,"available":96127545344,"internally_reserved":0,"allocated":135573504,"data_stored":135254826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724662,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479212,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892628,"statfs":{"total":96636764160,"available":96146051072,"internally_reserved":0,"allocated":487424,"data_stored":150217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":490204843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247945,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539376,"kb_used_data":66264,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832464,"statfs":{"total":96636764160,"available":96084443136,"internally_reserved":0,"allocated":67854336,"data_stored":67531581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135147520,"data_stored":135111390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:36:10.233 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:36:10.233 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'20648", 'reported_seq': 20916, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:59.900219+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:35:59.900219+0000', 'last_peered': '2024-09-17T15:35:59.900219+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:35:59.900219+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'10600", 'ondisk_log_start': "43'10600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20648, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:10.233 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:36:10.233 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'20934", 'reported_seq': 21165, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:59.899948+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:35:59.899948+0000', 'last_peered': '2024-09-17T15:35:59.899948+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:35:59.899948+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "43'10900", 'ondisk_log_start': "43'10900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 19091, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4125}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:36:10.233 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:36:10.233 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'21251", 'reported_seq': 21561, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:59.900373+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:35:59.900373+0000', 'last_peered': '2024-09-17T15:35:59.900373+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:35:59.900373+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "43'11200", 'ondisk_log_start': "43'11200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17750, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4191}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:10.233 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:36:10.233 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'20768", 'reported_seq': 25005, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:09.396562+0000', 'last_change': '2024-09-17T15:35:58.120749+0000', 'last_active': '2024-09-17T15:36:09.396562+0000', 'last_peered': '2024-09-17T15:36:09.396562+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:36:09.396562+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "43'10700", 'ondisk_log_start': "43'10700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2135, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17537, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2039, 'num_bytes_recovered': 966, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2133}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:10.234 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:36:10.234 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20542", 'reported_seq': 20825, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:59.899685+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:35:59.899685+0000', 'last_peered': '2024-09-17T15:35:59.899685+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:35:59.899685+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17138, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3943}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:10.234 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:36:10.234 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'20966", 'reported_seq': 21282, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:59.900110+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:35:59.900110+0000', 'last_peered': '2024-09-17T15:35:59.900110+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:35:59.900110+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "43'10900", 'ondisk_log_start': "43'10900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17514, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4057}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:10.234 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:36:10.234 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'20886", 'reported_seq': 21203, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:59.900017+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:35:59.900017+0000', 'last_peered': '2024-09-17T15:35:59.900017+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:35:59.900017+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'10800", 'ondisk_log_start': "43'10800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17673, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3980}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:10.234 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:36:10.607 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:10.607 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 26423/24 objects degraded (110095.833%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":707,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1552252928,"bytes_avail":288358039552,"bytes_total":289910292480,"degraded_objects":26423,"degraded_total":24,"degraded_ratio":1100.9583333333333,"recovering_objects_per_sec":178,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1789,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:36:10.619 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:36:10.620 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:36:10.917 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:10.917 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:36:10.931 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":397,"stamp":"2024-09-17T15:36:09.817053+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26423,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23038,"num_bytes_recovered":41572738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1515872,"kb_used_data":199136,"kb_used_omap":13,"kb_used_meta":1316658,"kb_avail":281599648,"statfs":{"total":289910292480,"available":288358039552,"internally_reserved":0,"allocated":203915264,"data_stored":202936624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1348258815},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1789,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1789,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001863"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20916,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900219+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:59.900219+0000","last_peered":"2024-09-17T15:35:59.900219+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:59.900219+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21053,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899472+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:59.899472+0000","last_peered":"2024-09-17T15:35:59.899472+0000","last_clean":"2024-09-17T15:35:59.899472+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:59.899472+0000","last_undegraded":"2024-09-17T15:35:59.899472+0000","last_fullsized":"2024-09-17T15:35:59.899472+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21165,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.899948+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:59.899948+0000","last_peered":"2024-09-17T15:35:59.899948+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:59.899948+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4125}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21058,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899239+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:59.899239+0000","last_peered":"2024-09-17T15:35:59.899239+0000","last_clean":"2024-09-17T15:35:59.899239+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:59.899239+0000","last_undegraded":"2024-09-17T15:35:59.899239+0000","last_fullsized":"2024-09-17T15:35:59.899239+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20801,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899356+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:59.899356+0000","last_peered":"2024-09-17T15:35:59.899356+0000","last_clean":"2024-09-17T15:35:59.899356+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:59.899356+0000","last_undegraded":"2024-09-17T15:35:59.899356+0000","last_fullsized":"2024-09-17T15:35:59.899356+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21561,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900373+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:59.900373+0000","last_peered":"2024-09-17T15:35:59.900373+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:59.900373+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4191},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":25005,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:09.396562+0000","last_change":"2024-09-17T15:35:58.120749+0000","last_active":"2024-09-17T15:36:09.396562+0000","last_peered":"2024-09-17T15:36:09.396562+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:36:09.396562+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2135,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2039,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2133}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21063,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899226+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:59.899226+0000","last_peered":"2024-09-17T15:35:59.899226+0000","last_clean":"2024-09-17T15:35:59.899226+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:59.899226+0000","last_undegraded":"2024-09-17T15:35:59.899226+0000","last_fullsized":"2024-09-17T15:35:59.899226+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":86,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896693+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:35:59.896693+0000","last_peered":"2024-09-17T15:35:59.896693+0000","last_clean":"2024-09-17T15:35:59.896693+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:35:59.896693+0000","last_undegraded":"2024-09-17T15:35:59.896693+0000","last_fullsized":"2024-09-17T15:35:59.896693+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29118,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896980+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:59.896980+0000","last_peered":"2024-09-17T15:35:59.896980+0000","last_clean":"2024-09-17T15:35:59.896980+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:59.896980+0000","last_undegraded":"2024-09-17T15:35:59.896980+0000","last_fullsized":"2024-09-17T15:35:59.896980+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899621+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:35:59.899621+0000","last_peered":"2024-09-17T15:35:59.899621+0000","last_clean":"2024-09-17T15:35:59.899621+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:35:59.899621+0000","last_undegraded":"2024-09-17T15:35:59.899621+0000","last_fullsized":"2024-09-17T15:35:59.899621+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20825,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.899685+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:59.899685+0000","last_peered":"2024-09-17T15:35:59.899685+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:59.899685+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":122,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899340+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:35:59.899340+0000","last_peered":"2024-09-17T15:35:59.899340+0000","last_clean":"2024-09-17T15:35:59.899340+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:35:59.899340+0000","last_undegraded":"2024-09-17T15:35:59.899340+0000","last_fullsized":"2024-09-17T15:35:59.899340+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28823,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.967113+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:35:59.967113+0000","last_peered":"2024-09-17T15:35:59.967113+0000","last_clean":"2024-09-17T15:35:59.967113+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:35:59.967113+0000","last_undegraded":"2024-09-17T15:35:59.967113+0000","last_fullsized":"2024-09-17T15:35:59.967113+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":96,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896940+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:35:59.896940+0000","last_peered":"2024-09-17T15:35:59.896940+0000","last_clean":"2024-09-17T15:35:59.896940+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:35:59.896940+0000","last_undegraded":"2024-09-17T15:35:59.896940+0000","last_fullsized":"2024-09-17T15:35:59.896940+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29160,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.898948+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:59.898948+0000","last_peered":"2024-09-17T15:35:59.898948+0000","last_clean":"2024-09-17T15:35:59.898948+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:59.898948+0000","last_undegraded":"2024-09-17T15:35:59.898948+0000","last_fullsized":"2024-09-17T15:35:59.898948+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21282,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900110+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:59.900110+0000","last_peered":"2024-09-17T15:35:59.900110+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:59.900110+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":86,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897175+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:35:59.897175+0000","last_peered":"2024-09-17T15:35:59.897175+0000","last_clean":"2024-09-17T15:35:59.897175+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:35:59.897175+0000","last_undegraded":"2024-09-17T15:35:59.897175+0000","last_fullsized":"2024-09-17T15:35:59.897175+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28074,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897466+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:35:59.897466+0000","last_peered":"2024-09-17T15:35:59.897466+0000","last_clean":"2024-09-17T15:35:59.897466+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:35:59.897466+0000","last_undegraded":"2024-09-17T15:35:59.897466+0000","last_fullsized":"2024-09-17T15:35:59.897466+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.900949+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:35:59.900949+0000","last_peered":"2024-09-17T15:35:59.900949+0000","last_clean":"2024-09-17T15:35:59.900949+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:35:59.900949+0000","last_undegraded":"2024-09-17T15:35:59.900949+0000","last_fullsized":"2024-09-17T15:35:59.900949+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21203,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900017+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:59.900017+0000","last_peered":"2024-09-17T15:35:59.900017+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:59.900017+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":86,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897393+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:35:59.897393+0000","last_peered":"2024-09-17T15:35:59.897393+0000","last_clean":"2024-09-17T15:35:59.897393+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:35:59.897393+0000","last_undegraded":"2024-09-17T15:35:59.897393+0000","last_fullsized":"2024-09-17T15:35:59.897393+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":91,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896940+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:35:59.896940+0000","last_peered":"2024-09-17T15:35:59.896940+0000","last_clean":"2024-09-17T15:35:59.896940+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:35:59.896940+0000","last_undegraded":"2024-09-17T15:35:59.896940+0000","last_fullsized":"2024-09-17T15:35:59.896940+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29025,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897092+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:59.897092+0000","last_peered":"2024-09-17T15:35:59.897092+0000","last_clean":"2024-09-17T15:35:59.897092+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:59.897092+0000","last_undegraded":"2024-09-17T15:35:59.897092+0000","last_fullsized":"2024-09-17T15:35:59.897092+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26423,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23037,"num_bytes_recovered":41572719,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202629120,"data_stored":202506278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051717,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497284,"kb_used_data":132396,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874556,"statfs":{"total":96636764160,"available":96127545344,"internally_reserved":0,"allocated":135573504,"data_stored":135254826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724662,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479212,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892628,"statfs":{"total":96636764160,"available":96146051072,"internally_reserved":0,"allocated":487424,"data_stored":150217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":490204843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247945,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539376,"kb_used_data":66264,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832464,"statfs":{"total":96636764160,"available":96084443136,"internally_reserved":0,"allocated":67854336,"data_stored":67531581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135147520,"data_stored":135111390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:36:13.933 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:36:14.267 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:14.267 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:36:14.281 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":399,"stamp":"2024-09-17T15:36:13.817821+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26423,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23038,"num_bytes_recovered":41572738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1515872,"kb_used_data":199136,"kb_used_omap":13,"kb_used_meta":1316658,"kb_avail":281599648,"statfs":{"total":289910292480,"available":288358039552,"internally_reserved":0,"allocated":203915264,"data_stored":202936624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1348258815},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1789,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1789,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002270"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20916,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900219+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:59.900219+0000","last_peered":"2024-09-17T15:35:59.900219+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:59.900219+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21053,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899472+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:59.899472+0000","last_peered":"2024-09-17T15:35:59.899472+0000","last_clean":"2024-09-17T15:35:59.899472+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:59.899472+0000","last_undegraded":"2024-09-17T15:35:59.899472+0000","last_fullsized":"2024-09-17T15:35:59.899472+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21165,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.899948+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:59.899948+0000","last_peered":"2024-09-17T15:35:59.899948+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:59.899948+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4125}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21058,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899239+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:59.899239+0000","last_peered":"2024-09-17T15:35:59.899239+0000","last_clean":"2024-09-17T15:35:59.899239+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:59.899239+0000","last_undegraded":"2024-09-17T15:35:59.899239+0000","last_fullsized":"2024-09-17T15:35:59.899239+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20801,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899356+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:59.899356+0000","last_peered":"2024-09-17T15:35:59.899356+0000","last_clean":"2024-09-17T15:35:59.899356+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:59.899356+0000","last_undegraded":"2024-09-17T15:35:59.899356+0000","last_fullsized":"2024-09-17T15:35:59.899356+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21561,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900373+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:59.900373+0000","last_peered":"2024-09-17T15:35:59.900373+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:59.900373+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4191},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":25005,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:09.396562+0000","last_change":"2024-09-17T15:35:58.120749+0000","last_active":"2024-09-17T15:36:09.396562+0000","last_peered":"2024-09-17T15:36:09.396562+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:36:09.396562+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2135,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2039,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2133}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21063,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899226+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:59.899226+0000","last_peered":"2024-09-17T15:35:59.899226+0000","last_clean":"2024-09-17T15:35:59.899226+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:59.899226+0000","last_undegraded":"2024-09-17T15:35:59.899226+0000","last_fullsized":"2024-09-17T15:35:59.899226+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":86,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896693+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:35:59.896693+0000","last_peered":"2024-09-17T15:35:59.896693+0000","last_clean":"2024-09-17T15:35:59.896693+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:35:59.896693+0000","last_undegraded":"2024-09-17T15:35:59.896693+0000","last_fullsized":"2024-09-17T15:35:59.896693+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29118,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896980+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:59.896980+0000","last_peered":"2024-09-17T15:35:59.896980+0000","last_clean":"2024-09-17T15:35:59.896980+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:59.896980+0000","last_undegraded":"2024-09-17T15:35:59.896980+0000","last_fullsized":"2024-09-17T15:35:59.896980+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899621+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:35:59.899621+0000","last_peered":"2024-09-17T15:35:59.899621+0000","last_clean":"2024-09-17T15:35:59.899621+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:35:59.899621+0000","last_undegraded":"2024-09-17T15:35:59.899621+0000","last_fullsized":"2024-09-17T15:35:59.899621+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20825,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.899685+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:59.899685+0000","last_peered":"2024-09-17T15:35:59.899685+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:59.899685+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":122,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899340+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:35:59.899340+0000","last_peered":"2024-09-17T15:35:59.899340+0000","last_clean":"2024-09-17T15:35:59.899340+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:35:59.899340+0000","last_undegraded":"2024-09-17T15:35:59.899340+0000","last_fullsized":"2024-09-17T15:35:59.899340+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28823,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.967113+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:35:59.967113+0000","last_peered":"2024-09-17T15:35:59.967113+0000","last_clean":"2024-09-17T15:35:59.967113+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:35:59.967113+0000","last_undegraded":"2024-09-17T15:35:59.967113+0000","last_fullsized":"2024-09-17T15:35:59.967113+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":96,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896940+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:35:59.896940+0000","last_peered":"2024-09-17T15:35:59.896940+0000","last_clean":"2024-09-17T15:35:59.896940+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:35:59.896940+0000","last_undegraded":"2024-09-17T15:35:59.896940+0000","last_fullsized":"2024-09-17T15:35:59.896940+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29160,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.898948+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:59.898948+0000","last_peered":"2024-09-17T15:35:59.898948+0000","last_clean":"2024-09-17T15:35:59.898948+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:59.898948+0000","last_undegraded":"2024-09-17T15:35:59.898948+0000","last_fullsized":"2024-09-17T15:35:59.898948+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21282,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900110+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:59.900110+0000","last_peered":"2024-09-17T15:35:59.900110+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:59.900110+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":86,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897175+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:35:59.897175+0000","last_peered":"2024-09-17T15:35:59.897175+0000","last_clean":"2024-09-17T15:35:59.897175+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:35:59.897175+0000","last_undegraded":"2024-09-17T15:35:59.897175+0000","last_fullsized":"2024-09-17T15:35:59.897175+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28074,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897466+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:35:59.897466+0000","last_peered":"2024-09-17T15:35:59.897466+0000","last_clean":"2024-09-17T15:35:59.897466+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:35:59.897466+0000","last_undegraded":"2024-09-17T15:35:59.897466+0000","last_fullsized":"2024-09-17T15:35:59.897466+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.900949+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:35:59.900949+0000","last_peered":"2024-09-17T15:35:59.900949+0000","last_clean":"2024-09-17T15:35:59.900949+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:35:59.900949+0000","last_undegraded":"2024-09-17T15:35:59.900949+0000","last_fullsized":"2024-09-17T15:35:59.900949+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21203,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900017+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:59.900017+0000","last_peered":"2024-09-17T15:35:59.900017+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:59.900017+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":86,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897393+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:35:59.897393+0000","last_peered":"2024-09-17T15:35:59.897393+0000","last_clean":"2024-09-17T15:35:59.897393+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:35:59.897393+0000","last_undegraded":"2024-09-17T15:35:59.897393+0000","last_fullsized":"2024-09-17T15:35:59.897393+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":91,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896940+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:35:59.896940+0000","last_peered":"2024-09-17T15:35:59.896940+0000","last_clean":"2024-09-17T15:35:59.896940+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:35:59.896940+0000","last_undegraded":"2024-09-17T15:35:59.896940+0000","last_fullsized":"2024-09-17T15:35:59.896940+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29025,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897092+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:59.897092+0000","last_peered":"2024-09-17T15:35:59.897092+0000","last_clean":"2024-09-17T15:35:59.897092+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:59.897092+0000","last_undegraded":"2024-09-17T15:35:59.897092+0000","last_fullsized":"2024-09-17T15:35:59.897092+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26423,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23037,"num_bytes_recovered":41572719,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202629120,"data_stored":202506278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051718,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497284,"kb_used_data":132396,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874556,"statfs":{"total":96636764160,"available":96127545344,"internally_reserved":0,"allocated":135573504,"data_stored":135254826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724662,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479212,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892628,"statfs":{"total":96636764160,"available":96146051072,"internally_reserved":0,"allocated":487424,"data_stored":150217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":490204843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247945,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539376,"kb_used_data":66264,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832464,"statfs":{"total":96636764160,"available":96084443136,"internally_reserved":0,"allocated":67854336,"data_stored":67531581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135147520,"data_stored":135111390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:36:14.282 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:36:14.632 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:14.633 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:36:14.645 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":399,"stamp":"2024-09-17T15:36:13.817821+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26423,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23038,"num_bytes_recovered":41572738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1515872,"kb_used_data":199136,"kb_used_omap":13,"kb_used_meta":1316658,"kb_avail":281599648,"statfs":{"total":289910292480,"available":288358039552,"internally_reserved":0,"allocated":203915264,"data_stored":202936624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1348258815},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1789,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1789,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002270"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20916,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900219+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:59.900219+0000","last_peered":"2024-09-17T15:35:59.900219+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:59.900219+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21053,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899472+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:59.899472+0000","last_peered":"2024-09-17T15:35:59.899472+0000","last_clean":"2024-09-17T15:35:59.899472+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:59.899472+0000","last_undegraded":"2024-09-17T15:35:59.899472+0000","last_fullsized":"2024-09-17T15:35:59.899472+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21165,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.899948+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:59.899948+0000","last_peered":"2024-09-17T15:35:59.899948+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:59.899948+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4125}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21058,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899239+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:59.899239+0000","last_peered":"2024-09-17T15:35:59.899239+0000","last_clean":"2024-09-17T15:35:59.899239+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:59.899239+0000","last_undegraded":"2024-09-17T15:35:59.899239+0000","last_fullsized":"2024-09-17T15:35:59.899239+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20801,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899356+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:59.899356+0000","last_peered":"2024-09-17T15:35:59.899356+0000","last_clean":"2024-09-17T15:35:59.899356+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:59.899356+0000","last_undegraded":"2024-09-17T15:35:59.899356+0000","last_fullsized":"2024-09-17T15:35:59.899356+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21561,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900373+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:59.900373+0000","last_peered":"2024-09-17T15:35:59.900373+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:59.900373+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4191},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":25005,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:09.396562+0000","last_change":"2024-09-17T15:35:58.120749+0000","last_active":"2024-09-17T15:36:09.396562+0000","last_peered":"2024-09-17T15:36:09.396562+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:36:09.396562+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2135,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2039,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2133}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21063,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899226+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:59.899226+0000","last_peered":"2024-09-17T15:35:59.899226+0000","last_clean":"2024-09-17T15:35:59.899226+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:59.899226+0000","last_undegraded":"2024-09-17T15:35:59.899226+0000","last_fullsized":"2024-09-17T15:35:59.899226+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":86,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896693+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:35:59.896693+0000","last_peered":"2024-09-17T15:35:59.896693+0000","last_clean":"2024-09-17T15:35:59.896693+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:35:59.896693+0000","last_undegraded":"2024-09-17T15:35:59.896693+0000","last_fullsized":"2024-09-17T15:35:59.896693+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29118,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896980+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:59.896980+0000","last_peered":"2024-09-17T15:35:59.896980+0000","last_clean":"2024-09-17T15:35:59.896980+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:59.896980+0000","last_undegraded":"2024-09-17T15:35:59.896980+0000","last_fullsized":"2024-09-17T15:35:59.896980+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899621+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:35:59.899621+0000","last_peered":"2024-09-17T15:35:59.899621+0000","last_clean":"2024-09-17T15:35:59.899621+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:35:59.899621+0000","last_undegraded":"2024-09-17T15:35:59.899621+0000","last_fullsized":"2024-09-17T15:35:59.899621+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20825,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.899685+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:59.899685+0000","last_peered":"2024-09-17T15:35:59.899685+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:59.899685+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":122,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899340+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:35:59.899340+0000","last_peered":"2024-09-17T15:35:59.899340+0000","last_clean":"2024-09-17T15:35:59.899340+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:35:59.899340+0000","last_undegraded":"2024-09-17T15:35:59.899340+0000","last_fullsized":"2024-09-17T15:35:59.899340+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28823,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.967113+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:35:59.967113+0000","last_peered":"2024-09-17T15:35:59.967113+0000","last_clean":"2024-09-17T15:35:59.967113+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:35:59.967113+0000","last_undegraded":"2024-09-17T15:35:59.967113+0000","last_fullsized":"2024-09-17T15:35:59.967113+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":96,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896940+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:35:59.896940+0000","last_peered":"2024-09-17T15:35:59.896940+0000","last_clean":"2024-09-17T15:35:59.896940+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:35:59.896940+0000","last_undegraded":"2024-09-17T15:35:59.896940+0000","last_fullsized":"2024-09-17T15:35:59.896940+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29160,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.898948+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:59.898948+0000","last_peered":"2024-09-17T15:35:59.898948+0000","last_clean":"2024-09-17T15:35:59.898948+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:59.898948+0000","last_undegraded":"2024-09-17T15:35:59.898948+0000","last_fullsized":"2024-09-17T15:35:59.898948+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21282,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900110+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:59.900110+0000","last_peered":"2024-09-17T15:35:59.900110+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:59.900110+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":86,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897175+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:35:59.897175+0000","last_peered":"2024-09-17T15:35:59.897175+0000","last_clean":"2024-09-17T15:35:59.897175+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:35:59.897175+0000","last_undegraded":"2024-09-17T15:35:59.897175+0000","last_fullsized":"2024-09-17T15:35:59.897175+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28074,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897466+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:35:59.897466+0000","last_peered":"2024-09-17T15:35:59.897466+0000","last_clean":"2024-09-17T15:35:59.897466+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:35:59.897466+0000","last_undegraded":"2024-09-17T15:35:59.897466+0000","last_fullsized":"2024-09-17T15:35:59.897466+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.900949+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:35:59.900949+0000","last_peered":"2024-09-17T15:35:59.900949+0000","last_clean":"2024-09-17T15:35:59.900949+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:35:59.900949+0000","last_undegraded":"2024-09-17T15:35:59.900949+0000","last_fullsized":"2024-09-17T15:35:59.900949+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21203,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900017+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:59.900017+0000","last_peered":"2024-09-17T15:35:59.900017+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:59.900017+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":86,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897393+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:35:59.897393+0000","last_peered":"2024-09-17T15:35:59.897393+0000","last_clean":"2024-09-17T15:35:59.897393+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:35:59.897393+0000","last_undegraded":"2024-09-17T15:35:59.897393+0000","last_fullsized":"2024-09-17T15:35:59.897393+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":91,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896940+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:35:59.896940+0000","last_peered":"2024-09-17T15:35:59.896940+0000","last_clean":"2024-09-17T15:35:59.896940+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:35:59.896940+0000","last_undegraded":"2024-09-17T15:35:59.896940+0000","last_fullsized":"2024-09-17T15:35:59.896940+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29025,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897092+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:59.897092+0000","last_peered":"2024-09-17T15:35:59.897092+0000","last_clean":"2024-09-17T15:35:59.897092+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:59.897092+0000","last_undegraded":"2024-09-17T15:35:59.897092+0000","last_fullsized":"2024-09-17T15:35:59.897092+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26423,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23037,"num_bytes_recovered":41572719,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202629120,"data_stored":202506278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051718,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497284,"kb_used_data":132396,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874556,"statfs":{"total":96636764160,"available":96127545344,"internally_reserved":0,"allocated":135573504,"data_stored":135254826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724662,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479212,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892628,"statfs":{"total":96636764160,"available":96146051072,"internally_reserved":0,"allocated":487424,"data_stored":150217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":490204843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247945,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539376,"kb_used_data":66264,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832464,"statfs":{"total":96636764160,"available":96084443136,"internally_reserved":0,"allocated":67854336,"data_stored":67531581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135147520,"data_stored":135111390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:36:14.646 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:36:14.646 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'20648", 'reported_seq': 20916, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:59.900219+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:35:59.900219+0000', 'last_peered': '2024-09-17T15:35:59.900219+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:35:59.900219+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'10600", 'ondisk_log_start': "43'10600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20648, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:14.646 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:36:14.646 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'20934", 'reported_seq': 21165, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:59.899948+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:35:59.899948+0000', 'last_peered': '2024-09-17T15:35:59.899948+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:35:59.899948+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "43'10900", 'ondisk_log_start': "43'10900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 19091, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4125}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:36:14.646 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:36:14.646 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'21251", 'reported_seq': 21561, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:59.900373+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:35:59.900373+0000', 'last_peered': '2024-09-17T15:35:59.900373+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:35:59.900373+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "43'11200", 'ondisk_log_start': "43'11200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17750, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4191}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:14.647 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:36:14.647 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'20768", 'reported_seq': 25005, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:09.396562+0000', 'last_change': '2024-09-17T15:35:58.120749+0000', 'last_active': '2024-09-17T15:36:09.396562+0000', 'last_peered': '2024-09-17T15:36:09.396562+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:36:09.396562+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "43'10700", 'ondisk_log_start': "43'10700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2135, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17537, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2039, 'num_bytes_recovered': 966, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2133}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:14.647 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:36:14.647 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20542", 'reported_seq': 20825, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:59.899685+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:35:59.899685+0000', 'last_peered': '2024-09-17T15:35:59.899685+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:35:59.899685+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17138, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3943}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:14.647 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:36:14.647 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'20966", 'reported_seq': 21282, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:59.900110+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:35:59.900110+0000', 'last_peered': '2024-09-17T15:35:59.900110+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:35:59.900110+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "43'10900", 'ondisk_log_start': "43'10900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17514, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4057}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:14.647 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:36:14.647 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'20886", 'reported_seq': 21203, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:59.900017+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:35:59.900017+0000', 'last_peered': '2024-09-17T15:35:59.900017+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:35:59.900017+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'10800", 'ondisk_log_start': "43'10800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17673, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3980}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:14.648 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:36:15.028 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:15.028 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 26423/24 objects degraded (110095.833%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":712,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1552252928,"bytes_avail":288358039552,"bytes_total":289910292480,"degraded_objects":26423,"degraded_total":24,"degraded_ratio":1100.9583333333333,"recovering_objects_per_sec":149,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1789,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:36:15.041 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:36:15.041 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:36:15.337 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:15.337 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:36:15.350 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":399,"stamp":"2024-09-17T15:36:13.817821+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26423,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23038,"num_bytes_recovered":41572738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1515872,"kb_used_data":199136,"kb_used_omap":13,"kb_used_meta":1316658,"kb_avail":281599648,"statfs":{"total":289910292480,"available":288358039552,"internally_reserved":0,"allocated":203915264,"data_stored":202936624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1348258815},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1789,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1789,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002270"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20916,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900219+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:59.900219+0000","last_peered":"2024-09-17T15:35:59.900219+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:59.900219+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21053,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899472+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:59.899472+0000","last_peered":"2024-09-17T15:35:59.899472+0000","last_clean":"2024-09-17T15:35:59.899472+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:59.899472+0000","last_undegraded":"2024-09-17T15:35:59.899472+0000","last_fullsized":"2024-09-17T15:35:59.899472+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21165,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.899948+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:59.899948+0000","last_peered":"2024-09-17T15:35:59.899948+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:59.899948+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4125}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21058,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899239+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:59.899239+0000","last_peered":"2024-09-17T15:35:59.899239+0000","last_clean":"2024-09-17T15:35:59.899239+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:59.899239+0000","last_undegraded":"2024-09-17T15:35:59.899239+0000","last_fullsized":"2024-09-17T15:35:59.899239+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20801,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899356+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:59.899356+0000","last_peered":"2024-09-17T15:35:59.899356+0000","last_clean":"2024-09-17T15:35:59.899356+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:59.899356+0000","last_undegraded":"2024-09-17T15:35:59.899356+0000","last_fullsized":"2024-09-17T15:35:59.899356+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21561,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900373+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:59.900373+0000","last_peered":"2024-09-17T15:35:59.900373+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:59.900373+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4191},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":25005,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:09.396562+0000","last_change":"2024-09-17T15:35:58.120749+0000","last_active":"2024-09-17T15:36:09.396562+0000","last_peered":"2024-09-17T15:36:09.396562+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:36:09.396562+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2135,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2039,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2133}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21063,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899226+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:59.899226+0000","last_peered":"2024-09-17T15:35:59.899226+0000","last_clean":"2024-09-17T15:35:59.899226+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:59.899226+0000","last_undegraded":"2024-09-17T15:35:59.899226+0000","last_fullsized":"2024-09-17T15:35:59.899226+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":86,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896693+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:35:59.896693+0000","last_peered":"2024-09-17T15:35:59.896693+0000","last_clean":"2024-09-17T15:35:59.896693+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:35:59.896693+0000","last_undegraded":"2024-09-17T15:35:59.896693+0000","last_fullsized":"2024-09-17T15:35:59.896693+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29118,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896980+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:59.896980+0000","last_peered":"2024-09-17T15:35:59.896980+0000","last_clean":"2024-09-17T15:35:59.896980+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:59.896980+0000","last_undegraded":"2024-09-17T15:35:59.896980+0000","last_fullsized":"2024-09-17T15:35:59.896980+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899621+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:35:59.899621+0000","last_peered":"2024-09-17T15:35:59.899621+0000","last_clean":"2024-09-17T15:35:59.899621+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:35:59.899621+0000","last_undegraded":"2024-09-17T15:35:59.899621+0000","last_fullsized":"2024-09-17T15:35:59.899621+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20825,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.899685+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:59.899685+0000","last_peered":"2024-09-17T15:35:59.899685+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:59.899685+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":122,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899340+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:35:59.899340+0000","last_peered":"2024-09-17T15:35:59.899340+0000","last_clean":"2024-09-17T15:35:59.899340+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:35:59.899340+0000","last_undegraded":"2024-09-17T15:35:59.899340+0000","last_fullsized":"2024-09-17T15:35:59.899340+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28823,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.967113+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:35:59.967113+0000","last_peered":"2024-09-17T15:35:59.967113+0000","last_clean":"2024-09-17T15:35:59.967113+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:35:59.967113+0000","last_undegraded":"2024-09-17T15:35:59.967113+0000","last_fullsized":"2024-09-17T15:35:59.967113+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":96,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896940+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:35:59.896940+0000","last_peered":"2024-09-17T15:35:59.896940+0000","last_clean":"2024-09-17T15:35:59.896940+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:35:59.896940+0000","last_undegraded":"2024-09-17T15:35:59.896940+0000","last_fullsized":"2024-09-17T15:35:59.896940+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29160,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.898948+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:59.898948+0000","last_peered":"2024-09-17T15:35:59.898948+0000","last_clean":"2024-09-17T15:35:59.898948+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:59.898948+0000","last_undegraded":"2024-09-17T15:35:59.898948+0000","last_fullsized":"2024-09-17T15:35:59.898948+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21282,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900110+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:59.900110+0000","last_peered":"2024-09-17T15:35:59.900110+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:59.900110+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":86,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897175+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:35:59.897175+0000","last_peered":"2024-09-17T15:35:59.897175+0000","last_clean":"2024-09-17T15:35:59.897175+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:35:59.897175+0000","last_undegraded":"2024-09-17T15:35:59.897175+0000","last_fullsized":"2024-09-17T15:35:59.897175+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28074,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897466+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:35:59.897466+0000","last_peered":"2024-09-17T15:35:59.897466+0000","last_clean":"2024-09-17T15:35:59.897466+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:35:59.897466+0000","last_undegraded":"2024-09-17T15:35:59.897466+0000","last_fullsized":"2024-09-17T15:35:59.897466+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.900949+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:35:59.900949+0000","last_peered":"2024-09-17T15:35:59.900949+0000","last_clean":"2024-09-17T15:35:59.900949+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:35:59.900949+0000","last_undegraded":"2024-09-17T15:35:59.900949+0000","last_fullsized":"2024-09-17T15:35:59.900949+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21203,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900017+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:59.900017+0000","last_peered":"2024-09-17T15:35:59.900017+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:59.900017+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":86,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897393+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:35:59.897393+0000","last_peered":"2024-09-17T15:35:59.897393+0000","last_clean":"2024-09-17T15:35:59.897393+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:35:59.897393+0000","last_undegraded":"2024-09-17T15:35:59.897393+0000","last_fullsized":"2024-09-17T15:35:59.897393+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":91,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896940+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:35:59.896940+0000","last_peered":"2024-09-17T15:35:59.896940+0000","last_clean":"2024-09-17T15:35:59.896940+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:35:59.896940+0000","last_undegraded":"2024-09-17T15:35:59.896940+0000","last_fullsized":"2024-09-17T15:35:59.896940+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29025,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897092+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:59.897092+0000","last_peered":"2024-09-17T15:35:59.897092+0000","last_clean":"2024-09-17T15:35:59.897092+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:59.897092+0000","last_undegraded":"2024-09-17T15:35:59.897092+0000","last_fullsized":"2024-09-17T15:35:59.897092+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26423,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23037,"num_bytes_recovered":41572719,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202629120,"data_stored":202506278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051718,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497284,"kb_used_data":132396,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874556,"statfs":{"total":96636764160,"available":96127545344,"internally_reserved":0,"allocated":135573504,"data_stored":135254826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724662,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479212,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892628,"statfs":{"total":96636764160,"available":96146051072,"internally_reserved":0,"allocated":487424,"data_stored":150217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":490204843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247945,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539376,"kb_used_data":66264,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832464,"statfs":{"total":96636764160,"available":96084443136,"internally_reserved":0,"allocated":67854336,"data_stored":67531581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135147520,"data_stored":135111390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:36:18.352 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:36:18.655 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:18.655 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:36:18.668 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":401,"stamp":"2024-09-17T15:36:17.818635+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25642,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23819,"num_bytes_recovered":41572738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1524192,"kb_used_data":199136,"kb_used_omap":13,"kb_used_meta":1324978,"kb_avail":281591328,"statfs":{"total":289910292480,"available":288349519872,"internally_reserved":0,"allocated":203915264,"data_stored":202936624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1356778495},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1665,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1665,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002335"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20916,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900219+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:59.900219+0000","last_peered":"2024-09-17T15:35:59.900219+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:59.900219+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21053,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899472+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:59.899472+0000","last_peered":"2024-09-17T15:35:59.899472+0000","last_clean":"2024-09-17T15:35:59.899472+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:59.899472+0000","last_undegraded":"2024-09-17T15:35:59.899472+0000","last_fullsized":"2024-09-17T15:35:59.899472+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21165,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.899948+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:59.899948+0000","last_peered":"2024-09-17T15:35:59.899948+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:59.899948+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4125}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21058,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899239+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:59.899239+0000","last_peered":"2024-09-17T15:35:59.899239+0000","last_clean":"2024-09-17T15:35:59.899239+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:59.899239+0000","last_undegraded":"2024-09-17T15:35:59.899239+0000","last_fullsized":"2024-09-17T15:35:59.899239+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20801,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899356+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:59.899356+0000","last_peered":"2024-09-17T15:35:59.899356+0000","last_clean":"2024-09-17T15:35:59.899356+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:59.899356+0000","last_undegraded":"2024-09-17T15:35:59.899356+0000","last_fullsized":"2024-09-17T15:35:59.899356+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21561,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900373+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:59.900373+0000","last_peered":"2024-09-17T15:35:59.900373+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:59.900373+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4191},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":26567,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:14.397560+0000","last_change":"2024-09-17T15:35:58.120749+0000","last_active":"2024-09-17T15:36:14.397560+0000","last_peered":"2024-09-17T15:36:14.397560+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:36:14.397560+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2820,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1352}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21063,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899226+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:59.899226+0000","last_peered":"2024-09-17T15:35:59.899226+0000","last_clean":"2024-09-17T15:35:59.899226+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:59.899226+0000","last_undegraded":"2024-09-17T15:35:59.899226+0000","last_fullsized":"2024-09-17T15:35:59.899226+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":86,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896693+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:35:59.896693+0000","last_peered":"2024-09-17T15:35:59.896693+0000","last_clean":"2024-09-17T15:35:59.896693+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:35:59.896693+0000","last_undegraded":"2024-09-17T15:35:59.896693+0000","last_fullsized":"2024-09-17T15:35:59.896693+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29118,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896980+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:59.896980+0000","last_peered":"2024-09-17T15:35:59.896980+0000","last_clean":"2024-09-17T15:35:59.896980+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:59.896980+0000","last_undegraded":"2024-09-17T15:35:59.896980+0000","last_fullsized":"2024-09-17T15:35:59.896980+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899621+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:35:59.899621+0000","last_peered":"2024-09-17T15:35:59.899621+0000","last_clean":"2024-09-17T15:35:59.899621+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:35:59.899621+0000","last_undegraded":"2024-09-17T15:35:59.899621+0000","last_fullsized":"2024-09-17T15:35:59.899621+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20825,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.899685+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:59.899685+0000","last_peered":"2024-09-17T15:35:59.899685+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:59.899685+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":122,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899340+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:35:59.899340+0000","last_peered":"2024-09-17T15:35:59.899340+0000","last_clean":"2024-09-17T15:35:59.899340+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:35:59.899340+0000","last_undegraded":"2024-09-17T15:35:59.899340+0000","last_fullsized":"2024-09-17T15:35:59.899340+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28823,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.967113+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:35:59.967113+0000","last_peered":"2024-09-17T15:35:59.967113+0000","last_clean":"2024-09-17T15:35:59.967113+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:35:59.967113+0000","last_undegraded":"2024-09-17T15:35:59.967113+0000","last_fullsized":"2024-09-17T15:35:59.967113+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":96,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896940+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:35:59.896940+0000","last_peered":"2024-09-17T15:35:59.896940+0000","last_clean":"2024-09-17T15:35:59.896940+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:35:59.896940+0000","last_undegraded":"2024-09-17T15:35:59.896940+0000","last_fullsized":"2024-09-17T15:35:59.896940+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29160,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.898948+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:59.898948+0000","last_peered":"2024-09-17T15:35:59.898948+0000","last_clean":"2024-09-17T15:35:59.898948+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:59.898948+0000","last_undegraded":"2024-09-17T15:35:59.898948+0000","last_fullsized":"2024-09-17T15:35:59.898948+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21282,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900110+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:59.900110+0000","last_peered":"2024-09-17T15:35:59.900110+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:59.900110+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":86,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897175+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:35:59.897175+0000","last_peered":"2024-09-17T15:35:59.897175+0000","last_clean":"2024-09-17T15:35:59.897175+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:35:59.897175+0000","last_undegraded":"2024-09-17T15:35:59.897175+0000","last_fullsized":"2024-09-17T15:35:59.897175+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28074,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897466+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:35:59.897466+0000","last_peered":"2024-09-17T15:35:59.897466+0000","last_clean":"2024-09-17T15:35:59.897466+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:35:59.897466+0000","last_undegraded":"2024-09-17T15:35:59.897466+0000","last_fullsized":"2024-09-17T15:35:59.897466+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.900949+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:35:59.900949+0000","last_peered":"2024-09-17T15:35:59.900949+0000","last_clean":"2024-09-17T15:35:59.900949+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:35:59.900949+0000","last_undegraded":"2024-09-17T15:35:59.900949+0000","last_fullsized":"2024-09-17T15:35:59.900949+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21203,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900017+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:59.900017+0000","last_peered":"2024-09-17T15:35:59.900017+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:59.900017+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":86,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897393+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:35:59.897393+0000","last_peered":"2024-09-17T15:35:59.897393+0000","last_clean":"2024-09-17T15:35:59.897393+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:35:59.897393+0000","last_undegraded":"2024-09-17T15:35:59.897393+0000","last_fullsized":"2024-09-17T15:35:59.897393+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":91,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896940+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:35:59.896940+0000","last_peered":"2024-09-17T15:35:59.896940+0000","last_clean":"2024-09-17T15:35:59.896940+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:35:59.896940+0000","last_undegraded":"2024-09-17T15:35:59.896940+0000","last_fullsized":"2024-09-17T15:35:59.896940+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29025,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897092+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:59.897092+0000","last_peered":"2024-09-17T15:35:59.897092+0000","last_clean":"2024-09-17T15:35:59.897092+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:59.897092+0000","last_undegraded":"2024-09-17T15:35:59.897092+0000","last_fullsized":"2024-09-17T15:35:59.897092+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25642,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23818,"num_bytes_recovered":41572719,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202629120,"data_stored":202506278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051718,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497284,"kb_used_data":132396,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874556,"statfs":{"total":96636764160,"available":96127545344,"internally_reserved":0,"allocated":135573504,"data_stored":135254826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724663,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487532,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884308,"statfs":{"total":96636764160,"available":96137531392,"internally_reserved":0,"allocated":487424,"data_stored":150217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":498724523},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247946,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539376,"kb_used_data":66264,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832464,"statfs":{"total":96636764160,"available":96084443136,"internally_reserved":0,"allocated":67854336,"data_stored":67531581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135147520,"data_stored":135111390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:36:18.669 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:36:18.983 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:18.983 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:36:18.997 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":401,"stamp":"2024-09-17T15:36:17.818635+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25642,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23819,"num_bytes_recovered":41572738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1524192,"kb_used_data":199136,"kb_used_omap":13,"kb_used_meta":1324978,"kb_avail":281591328,"statfs":{"total":289910292480,"available":288349519872,"internally_reserved":0,"allocated":203915264,"data_stored":202936624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1356778495},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1665,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1665,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002335"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20916,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900219+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:59.900219+0000","last_peered":"2024-09-17T15:35:59.900219+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:59.900219+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21053,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899472+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:59.899472+0000","last_peered":"2024-09-17T15:35:59.899472+0000","last_clean":"2024-09-17T15:35:59.899472+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:59.899472+0000","last_undegraded":"2024-09-17T15:35:59.899472+0000","last_fullsized":"2024-09-17T15:35:59.899472+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21165,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.899948+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:59.899948+0000","last_peered":"2024-09-17T15:35:59.899948+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:59.899948+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4125}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21058,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899239+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:59.899239+0000","last_peered":"2024-09-17T15:35:59.899239+0000","last_clean":"2024-09-17T15:35:59.899239+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:59.899239+0000","last_undegraded":"2024-09-17T15:35:59.899239+0000","last_fullsized":"2024-09-17T15:35:59.899239+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20801,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899356+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:59.899356+0000","last_peered":"2024-09-17T15:35:59.899356+0000","last_clean":"2024-09-17T15:35:59.899356+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:59.899356+0000","last_undegraded":"2024-09-17T15:35:59.899356+0000","last_fullsized":"2024-09-17T15:35:59.899356+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21561,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900373+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:59.900373+0000","last_peered":"2024-09-17T15:35:59.900373+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:59.900373+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4191},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":26567,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:14.397560+0000","last_change":"2024-09-17T15:35:58.120749+0000","last_active":"2024-09-17T15:36:14.397560+0000","last_peered":"2024-09-17T15:36:14.397560+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:36:14.397560+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2820,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1352}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21063,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899226+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:59.899226+0000","last_peered":"2024-09-17T15:35:59.899226+0000","last_clean":"2024-09-17T15:35:59.899226+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:59.899226+0000","last_undegraded":"2024-09-17T15:35:59.899226+0000","last_fullsized":"2024-09-17T15:35:59.899226+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":86,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896693+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:35:59.896693+0000","last_peered":"2024-09-17T15:35:59.896693+0000","last_clean":"2024-09-17T15:35:59.896693+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:35:59.896693+0000","last_undegraded":"2024-09-17T15:35:59.896693+0000","last_fullsized":"2024-09-17T15:35:59.896693+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29118,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896980+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:59.896980+0000","last_peered":"2024-09-17T15:35:59.896980+0000","last_clean":"2024-09-17T15:35:59.896980+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:59.896980+0000","last_undegraded":"2024-09-17T15:35:59.896980+0000","last_fullsized":"2024-09-17T15:35:59.896980+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899621+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:35:59.899621+0000","last_peered":"2024-09-17T15:35:59.899621+0000","last_clean":"2024-09-17T15:35:59.899621+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:35:59.899621+0000","last_undegraded":"2024-09-17T15:35:59.899621+0000","last_fullsized":"2024-09-17T15:35:59.899621+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20825,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.899685+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:59.899685+0000","last_peered":"2024-09-17T15:35:59.899685+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:59.899685+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":122,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899340+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:35:59.899340+0000","last_peered":"2024-09-17T15:35:59.899340+0000","last_clean":"2024-09-17T15:35:59.899340+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:35:59.899340+0000","last_undegraded":"2024-09-17T15:35:59.899340+0000","last_fullsized":"2024-09-17T15:35:59.899340+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28823,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.967113+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:35:59.967113+0000","last_peered":"2024-09-17T15:35:59.967113+0000","last_clean":"2024-09-17T15:35:59.967113+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:35:59.967113+0000","last_undegraded":"2024-09-17T15:35:59.967113+0000","last_fullsized":"2024-09-17T15:35:59.967113+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":96,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896940+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:35:59.896940+0000","last_peered":"2024-09-17T15:35:59.896940+0000","last_clean":"2024-09-17T15:35:59.896940+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:35:59.896940+0000","last_undegraded":"2024-09-17T15:35:59.896940+0000","last_fullsized":"2024-09-17T15:35:59.896940+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29160,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.898948+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:59.898948+0000","last_peered":"2024-09-17T15:35:59.898948+0000","last_clean":"2024-09-17T15:35:59.898948+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:59.898948+0000","last_undegraded":"2024-09-17T15:35:59.898948+0000","last_fullsized":"2024-09-17T15:35:59.898948+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21282,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900110+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:59.900110+0000","last_peered":"2024-09-17T15:35:59.900110+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:59.900110+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":86,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897175+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:35:59.897175+0000","last_peered":"2024-09-17T15:35:59.897175+0000","last_clean":"2024-09-17T15:35:59.897175+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:35:59.897175+0000","last_undegraded":"2024-09-17T15:35:59.897175+0000","last_fullsized":"2024-09-17T15:35:59.897175+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28074,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897466+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:35:59.897466+0000","last_peered":"2024-09-17T15:35:59.897466+0000","last_clean":"2024-09-17T15:35:59.897466+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:35:59.897466+0000","last_undegraded":"2024-09-17T15:35:59.897466+0000","last_fullsized":"2024-09-17T15:35:59.897466+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.900949+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:35:59.900949+0000","last_peered":"2024-09-17T15:35:59.900949+0000","last_clean":"2024-09-17T15:35:59.900949+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:35:59.900949+0000","last_undegraded":"2024-09-17T15:35:59.900949+0000","last_fullsized":"2024-09-17T15:35:59.900949+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21203,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900017+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:59.900017+0000","last_peered":"2024-09-17T15:35:59.900017+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:59.900017+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":86,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897393+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:35:59.897393+0000","last_peered":"2024-09-17T15:35:59.897393+0000","last_clean":"2024-09-17T15:35:59.897393+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:35:59.897393+0000","last_undegraded":"2024-09-17T15:35:59.897393+0000","last_fullsized":"2024-09-17T15:35:59.897393+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":91,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896940+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:35:59.896940+0000","last_peered":"2024-09-17T15:35:59.896940+0000","last_clean":"2024-09-17T15:35:59.896940+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:35:59.896940+0000","last_undegraded":"2024-09-17T15:35:59.896940+0000","last_fullsized":"2024-09-17T15:35:59.896940+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29025,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897092+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:59.897092+0000","last_peered":"2024-09-17T15:35:59.897092+0000","last_clean":"2024-09-17T15:35:59.897092+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:59.897092+0000","last_undegraded":"2024-09-17T15:35:59.897092+0000","last_fullsized":"2024-09-17T15:35:59.897092+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25642,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23818,"num_bytes_recovered":41572719,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202629120,"data_stored":202506278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051718,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497284,"kb_used_data":132396,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874556,"statfs":{"total":96636764160,"available":96127545344,"internally_reserved":0,"allocated":135573504,"data_stored":135254826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724663,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487532,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884308,"statfs":{"total":96636764160,"available":96137531392,"internally_reserved":0,"allocated":487424,"data_stored":150217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":498724523},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247946,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539376,"kb_used_data":66264,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832464,"statfs":{"total":96636764160,"available":96084443136,"internally_reserved":0,"allocated":67854336,"data_stored":67531581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135147520,"data_stored":135111390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:36:18.999 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:36:18.999 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'20648", 'reported_seq': 20916, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:59.900219+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:35:59.900219+0000', 'last_peered': '2024-09-17T15:35:59.900219+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:35:59.900219+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'10600", 'ondisk_log_start': "43'10600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20648, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:18.999 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:36:18.999 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'20934", 'reported_seq': 21165, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:59.899948+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:35:59.899948+0000', 'last_peered': '2024-09-17T15:35:59.899948+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:35:59.899948+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "43'10900", 'ondisk_log_start': "43'10900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 19091, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4125}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:36:18.999 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:36:18.999 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'21251", 'reported_seq': 21561, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:59.900373+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:35:59.900373+0000', 'last_peered': '2024-09-17T15:35:59.900373+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:35:59.900373+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "43'11200", 'ondisk_log_start': "43'11200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17750, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4191}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:18.999 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:36:19.000 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'20768", 'reported_seq': 26567, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:14.397560+0000', 'last_change': '2024-09-17T15:35:58.120749+0000', 'last_active': '2024-09-17T15:36:14.397560+0000', 'last_peered': '2024-09-17T15:36:14.397560+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:36:14.397560+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "43'10700", 'ondisk_log_start': "43'10700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17537, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2820, 'num_bytes_recovered': 966, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1352}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:19.000 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:36:19.000 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20542", 'reported_seq': 20825, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:59.899685+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:35:59.899685+0000', 'last_peered': '2024-09-17T15:35:59.899685+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:35:59.899685+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17138, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3943}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:19.000 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:36:19.000 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'20966", 'reported_seq': 21282, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:59.900110+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:35:59.900110+0000', 'last_peered': '2024-09-17T15:35:59.900110+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:35:59.900110+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "43'10900", 'ondisk_log_start': "43'10900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17514, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4057}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:19.000 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:36:19.000 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'20886", 'reported_seq': 21203, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:59.900017+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:35:59.900017+0000', 'last_peered': '2024-09-17T15:35:59.900017+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:35:59.900017+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'10800", 'ondisk_log_start': "43'10800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17673, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3980}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:19.000 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:36:19.394 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:19.395 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 25642/24 objects degraded (106841.667%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":716,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1560772608,"bytes_avail":288349519872,"bytes_total":289910292480,"degraded_objects":25642,"degraded_total":24,"degraded_ratio":1068.4166666666667,"recovering_objects_per_sec":138,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1665,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:36:19.408 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:36:19.409 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:36:19.743 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:19.744 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:36:19.758 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":401,"stamp":"2024-09-17T15:36:17.818635+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25642,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23819,"num_bytes_recovered":41572738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1524192,"kb_used_data":199136,"kb_used_omap":13,"kb_used_meta":1324978,"kb_avail":281591328,"statfs":{"total":289910292480,"available":288349519872,"internally_reserved":0,"allocated":203915264,"data_stored":202936624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1356778495},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1665,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1665,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002335"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20916,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900219+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:59.900219+0000","last_peered":"2024-09-17T15:35:59.900219+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:59.900219+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21053,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899472+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:59.899472+0000","last_peered":"2024-09-17T15:35:59.899472+0000","last_clean":"2024-09-17T15:35:59.899472+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:59.899472+0000","last_undegraded":"2024-09-17T15:35:59.899472+0000","last_fullsized":"2024-09-17T15:35:59.899472+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21165,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.899948+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:59.899948+0000","last_peered":"2024-09-17T15:35:59.899948+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:59.899948+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4125}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21058,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899239+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:59.899239+0000","last_peered":"2024-09-17T15:35:59.899239+0000","last_clean":"2024-09-17T15:35:59.899239+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:59.899239+0000","last_undegraded":"2024-09-17T15:35:59.899239+0000","last_fullsized":"2024-09-17T15:35:59.899239+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20801,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899356+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:59.899356+0000","last_peered":"2024-09-17T15:35:59.899356+0000","last_clean":"2024-09-17T15:35:59.899356+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:59.899356+0000","last_undegraded":"2024-09-17T15:35:59.899356+0000","last_fullsized":"2024-09-17T15:35:59.899356+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21561,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900373+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:59.900373+0000","last_peered":"2024-09-17T15:35:59.900373+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:59.900373+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4191},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":26567,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:14.397560+0000","last_change":"2024-09-17T15:35:58.120749+0000","last_active":"2024-09-17T15:36:14.397560+0000","last_peered":"2024-09-17T15:36:14.397560+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:36:14.397560+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2820,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1352}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21063,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899226+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:59.899226+0000","last_peered":"2024-09-17T15:35:59.899226+0000","last_clean":"2024-09-17T15:35:59.899226+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:59.899226+0000","last_undegraded":"2024-09-17T15:35:59.899226+0000","last_fullsized":"2024-09-17T15:35:59.899226+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":86,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896693+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:35:59.896693+0000","last_peered":"2024-09-17T15:35:59.896693+0000","last_clean":"2024-09-17T15:35:59.896693+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:35:59.896693+0000","last_undegraded":"2024-09-17T15:35:59.896693+0000","last_fullsized":"2024-09-17T15:35:59.896693+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29118,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896980+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:59.896980+0000","last_peered":"2024-09-17T15:35:59.896980+0000","last_clean":"2024-09-17T15:35:59.896980+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:59.896980+0000","last_undegraded":"2024-09-17T15:35:59.896980+0000","last_fullsized":"2024-09-17T15:35:59.896980+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899621+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:35:59.899621+0000","last_peered":"2024-09-17T15:35:59.899621+0000","last_clean":"2024-09-17T15:35:59.899621+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:35:59.899621+0000","last_undegraded":"2024-09-17T15:35:59.899621+0000","last_fullsized":"2024-09-17T15:35:59.899621+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20825,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.899685+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:59.899685+0000","last_peered":"2024-09-17T15:35:59.899685+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:59.899685+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":122,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899340+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:35:59.899340+0000","last_peered":"2024-09-17T15:35:59.899340+0000","last_clean":"2024-09-17T15:35:59.899340+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:35:59.899340+0000","last_undegraded":"2024-09-17T15:35:59.899340+0000","last_fullsized":"2024-09-17T15:35:59.899340+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28823,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.967113+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:35:59.967113+0000","last_peered":"2024-09-17T15:35:59.967113+0000","last_clean":"2024-09-17T15:35:59.967113+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:35:59.967113+0000","last_undegraded":"2024-09-17T15:35:59.967113+0000","last_fullsized":"2024-09-17T15:35:59.967113+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":96,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896940+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:35:59.896940+0000","last_peered":"2024-09-17T15:35:59.896940+0000","last_clean":"2024-09-17T15:35:59.896940+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:35:59.896940+0000","last_undegraded":"2024-09-17T15:35:59.896940+0000","last_fullsized":"2024-09-17T15:35:59.896940+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29160,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.898948+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:59.898948+0000","last_peered":"2024-09-17T15:35:59.898948+0000","last_clean":"2024-09-17T15:35:59.898948+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:59.898948+0000","last_undegraded":"2024-09-17T15:35:59.898948+0000","last_fullsized":"2024-09-17T15:35:59.898948+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21282,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900110+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:59.900110+0000","last_peered":"2024-09-17T15:35:59.900110+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:59.900110+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":86,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897175+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:35:59.897175+0000","last_peered":"2024-09-17T15:35:59.897175+0000","last_clean":"2024-09-17T15:35:59.897175+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:35:59.897175+0000","last_undegraded":"2024-09-17T15:35:59.897175+0000","last_fullsized":"2024-09-17T15:35:59.897175+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28074,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897466+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:35:59.897466+0000","last_peered":"2024-09-17T15:35:59.897466+0000","last_clean":"2024-09-17T15:35:59.897466+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:35:59.897466+0000","last_undegraded":"2024-09-17T15:35:59.897466+0000","last_fullsized":"2024-09-17T15:35:59.897466+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.900949+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:35:59.900949+0000","last_peered":"2024-09-17T15:35:59.900949+0000","last_clean":"2024-09-17T15:35:59.900949+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:35:59.900949+0000","last_undegraded":"2024-09-17T15:35:59.900949+0000","last_fullsized":"2024-09-17T15:35:59.900949+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21203,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900017+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:59.900017+0000","last_peered":"2024-09-17T15:35:59.900017+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:59.900017+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":86,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897393+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:35:59.897393+0000","last_peered":"2024-09-17T15:35:59.897393+0000","last_clean":"2024-09-17T15:35:59.897393+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:35:59.897393+0000","last_undegraded":"2024-09-17T15:35:59.897393+0000","last_fullsized":"2024-09-17T15:35:59.897393+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":91,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896940+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:35:59.896940+0000","last_peered":"2024-09-17T15:35:59.896940+0000","last_clean":"2024-09-17T15:35:59.896940+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:35:59.896940+0000","last_undegraded":"2024-09-17T15:35:59.896940+0000","last_fullsized":"2024-09-17T15:35:59.896940+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29025,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897092+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:59.897092+0000","last_peered":"2024-09-17T15:35:59.897092+0000","last_clean":"2024-09-17T15:35:59.897092+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:59.897092+0000","last_undegraded":"2024-09-17T15:35:59.897092+0000","last_fullsized":"2024-09-17T15:35:59.897092+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25642,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23818,"num_bytes_recovered":41572719,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202629120,"data_stored":202506278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051718,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497284,"kb_used_data":132396,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874556,"statfs":{"total":96636764160,"available":96127545344,"internally_reserved":0,"allocated":135573504,"data_stored":135254826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724663,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487532,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884308,"statfs":{"total":96636764160,"available":96137531392,"internally_reserved":0,"allocated":487424,"data_stored":150217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":498724523},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247946,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539376,"kb_used_data":66264,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832464,"statfs":{"total":96636764160,"available":96084443136,"internally_reserved":0,"allocated":67854336,"data_stored":67531581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135147520,"data_stored":135111390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:36:22.760 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:36:23.140 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:23.151 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:36:23.157 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":405,"stamp":"2024-09-17T15:36:21.953620+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24623,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24838,"num_bytes_recovered":41572738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1515872,"kb_used_data":199136,"kb_used_omap":13,"kb_used_meta":1316658,"kb_avail":281599648,"statfs":{"total":289910292480,"available":288358039552,"internally_reserved":0,"allocated":203915264,"data_stored":202936624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1348258815},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1800,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1800,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.135798"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20916,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900219+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:59.900219+0000","last_peered":"2024-09-17T15:35:59.900219+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:59.900219+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21053,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899472+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:59.899472+0000","last_peered":"2024-09-17T15:35:59.899472+0000","last_clean":"2024-09-17T15:35:59.899472+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:59.899472+0000","last_undegraded":"2024-09-17T15:35:59.899472+0000","last_fullsized":"2024-09-17T15:35:59.899472+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21165,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.899948+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:59.899948+0000","last_peered":"2024-09-17T15:35:59.899948+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:59.899948+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4125}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21058,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899239+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:59.899239+0000","last_peered":"2024-09-17T15:35:59.899239+0000","last_clean":"2024-09-17T15:35:59.899239+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:59.899239+0000","last_undegraded":"2024-09-17T15:35:59.899239+0000","last_fullsized":"2024-09-17T15:35:59.899239+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20801,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899356+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:59.899356+0000","last_peered":"2024-09-17T15:35:59.899356+0000","last_clean":"2024-09-17T15:35:59.899356+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:59.899356+0000","last_undegraded":"2024-09-17T15:35:59.899356+0000","last_fullsized":"2024-09-17T15:35:59.899356+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21561,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900373+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:59.900373+0000","last_peered":"2024-09-17T15:35:59.900373+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:59.900373+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4191},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":28605,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:19.398388+0000","last_change":"2024-09-17T15:35:58.120749+0000","last_active":"2024-09-17T15:36:19.398388+0000","last_peered":"2024-09-17T15:36:19.398388+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:36:19.398388+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":335,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3839,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-333}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21063,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899226+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:59.899226+0000","last_peered":"2024-09-17T15:35:59.899226+0000","last_clean":"2024-09-17T15:35:59.899226+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:59.899226+0000","last_undegraded":"2024-09-17T15:35:59.899226+0000","last_fullsized":"2024-09-17T15:35:59.899226+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":86,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896693+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:35:59.896693+0000","last_peered":"2024-09-17T15:35:59.896693+0000","last_clean":"2024-09-17T15:35:59.896693+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:35:59.896693+0000","last_undegraded":"2024-09-17T15:35:59.896693+0000","last_fullsized":"2024-09-17T15:35:59.896693+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29118,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896980+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:59.896980+0000","last_peered":"2024-09-17T15:35:59.896980+0000","last_clean":"2024-09-17T15:35:59.896980+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:59.896980+0000","last_undegraded":"2024-09-17T15:35:59.896980+0000","last_fullsized":"2024-09-17T15:35:59.896980+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899621+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:35:59.899621+0000","last_peered":"2024-09-17T15:35:59.899621+0000","last_clean":"2024-09-17T15:35:59.899621+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:35:59.899621+0000","last_undegraded":"2024-09-17T15:35:59.899621+0000","last_fullsized":"2024-09-17T15:35:59.899621+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20825,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.899685+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:59.899685+0000","last_peered":"2024-09-17T15:35:59.899685+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:59.899685+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":122,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899340+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:35:59.899340+0000","last_peered":"2024-09-17T15:35:59.899340+0000","last_clean":"2024-09-17T15:35:59.899340+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:35:59.899340+0000","last_undegraded":"2024-09-17T15:35:59.899340+0000","last_fullsized":"2024-09-17T15:35:59.899340+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28823,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.967113+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:35:59.967113+0000","last_peered":"2024-09-17T15:35:59.967113+0000","last_clean":"2024-09-17T15:35:59.967113+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:35:59.967113+0000","last_undegraded":"2024-09-17T15:35:59.967113+0000","last_fullsized":"2024-09-17T15:35:59.967113+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":96,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896940+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:35:59.896940+0000","last_peered":"2024-09-17T15:35:59.896940+0000","last_clean":"2024-09-17T15:35:59.896940+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:35:59.896940+0000","last_undegraded":"2024-09-17T15:35:59.896940+0000","last_fullsized":"2024-09-17T15:35:59.896940+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29160,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.898948+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:59.898948+0000","last_peered":"2024-09-17T15:35:59.898948+0000","last_clean":"2024-09-17T15:35:59.898948+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:59.898948+0000","last_undegraded":"2024-09-17T15:35:59.898948+0000","last_fullsized":"2024-09-17T15:35:59.898948+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21282,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900110+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:59.900110+0000","last_peered":"2024-09-17T15:35:59.900110+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:59.900110+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":86,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897175+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:35:59.897175+0000","last_peered":"2024-09-17T15:35:59.897175+0000","last_clean":"2024-09-17T15:35:59.897175+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:35:59.897175+0000","last_undegraded":"2024-09-17T15:35:59.897175+0000","last_fullsized":"2024-09-17T15:35:59.897175+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28074,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897466+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:35:59.897466+0000","last_peered":"2024-09-17T15:35:59.897466+0000","last_clean":"2024-09-17T15:35:59.897466+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:35:59.897466+0000","last_undegraded":"2024-09-17T15:35:59.897466+0000","last_fullsized":"2024-09-17T15:35:59.897466+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.900949+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:35:59.900949+0000","last_peered":"2024-09-17T15:35:59.900949+0000","last_clean":"2024-09-17T15:35:59.900949+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:35:59.900949+0000","last_undegraded":"2024-09-17T15:35:59.900949+0000","last_fullsized":"2024-09-17T15:35:59.900949+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21203,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900017+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:59.900017+0000","last_peered":"2024-09-17T15:35:59.900017+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:59.900017+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":86,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897393+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:35:59.897393+0000","last_peered":"2024-09-17T15:35:59.897393+0000","last_clean":"2024-09-17T15:35:59.897393+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:35:59.897393+0000","last_undegraded":"2024-09-17T15:35:59.897393+0000","last_fullsized":"2024-09-17T15:35:59.897393+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":91,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896940+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:35:59.896940+0000","last_peered":"2024-09-17T15:35:59.896940+0000","last_clean":"2024-09-17T15:35:59.896940+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:35:59.896940+0000","last_undegraded":"2024-09-17T15:35:59.896940+0000","last_fullsized":"2024-09-17T15:35:59.896940+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29025,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897092+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:59.897092+0000","last_peered":"2024-09-17T15:35:59.897092+0000","last_clean":"2024-09-17T15:35:59.897092+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:59.897092+0000","last_undegraded":"2024-09-17T15:35:59.897092+0000","last_fullsized":"2024-09-17T15:35:59.897092+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24623,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24837,"num_bytes_recovered":41572719,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202629120,"data_stored":202506278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051719,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497284,"kb_used_data":132396,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874556,"statfs":{"total":96636764160,"available":96127545344,"internally_reserved":0,"allocated":135573504,"data_stored":135254826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724664,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479212,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892628,"statfs":{"total":96636764160,"available":96146051072,"internally_reserved":0,"allocated":487424,"data_stored":150217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":490204843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247947,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539376,"kb_used_data":66264,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832464,"statfs":{"total":96636764160,"available":96084443136,"internally_reserved":0,"allocated":67854336,"data_stored":67531581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135147520,"data_stored":135111390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:36:23.158 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:36:23.489 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:23.490 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:36:23.502 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":405,"stamp":"2024-09-17T15:36:21.953620+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24623,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24838,"num_bytes_recovered":41572738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1515872,"kb_used_data":199136,"kb_used_omap":13,"kb_used_meta":1316658,"kb_avail":281599648,"statfs":{"total":289910292480,"available":288358039552,"internally_reserved":0,"allocated":203915264,"data_stored":202936624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1348258815},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1800,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1800,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.135798"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20916,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900219+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:59.900219+0000","last_peered":"2024-09-17T15:35:59.900219+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:59.900219+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21053,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899472+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:35:59.899472+0000","last_peered":"2024-09-17T15:35:59.899472+0000","last_clean":"2024-09-17T15:35:59.899472+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:35:59.899472+0000","last_undegraded":"2024-09-17T15:35:59.899472+0000","last_fullsized":"2024-09-17T15:35:59.899472+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21165,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.899948+0000","last_change":"2024-09-17T15:26:22.693154+0000","last_active":"2024-09-17T15:35:59.899948+0000","last_peered":"2024-09-17T15:35:59.899948+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:35:59.899948+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4125}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21058,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899239+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:35:59.899239+0000","last_peered":"2024-09-17T15:35:59.899239+0000","last_clean":"2024-09-17T15:35:59.899239+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:35:59.899239+0000","last_undegraded":"2024-09-17T15:35:59.899239+0000","last_fullsized":"2024-09-17T15:35:59.899239+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20801,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899356+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:59.899356+0000","last_peered":"2024-09-17T15:35:59.899356+0000","last_clean":"2024-09-17T15:35:59.899356+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:59.899356+0000","last_undegraded":"2024-09-17T15:35:59.899356+0000","last_fullsized":"2024-09-17T15:35:59.899356+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21561,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900373+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:59.900373+0000","last_peered":"2024-09-17T15:35:59.900373+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:59.900373+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4191},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":28605,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:19.398388+0000","last_change":"2024-09-17T15:35:58.120749+0000","last_active":"2024-09-17T15:36:19.398388+0000","last_peered":"2024-09-17T15:36:19.398388+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:36:19.398388+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":335,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3839,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-333}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21063,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899226+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:59.899226+0000","last_peered":"2024-09-17T15:35:59.899226+0000","last_clean":"2024-09-17T15:35:59.899226+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:59.899226+0000","last_undegraded":"2024-09-17T15:35:59.899226+0000","last_fullsized":"2024-09-17T15:35:59.899226+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":86,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896693+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:35:59.896693+0000","last_peered":"2024-09-17T15:35:59.896693+0000","last_clean":"2024-09-17T15:35:59.896693+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:35:59.896693+0000","last_undegraded":"2024-09-17T15:35:59.896693+0000","last_fullsized":"2024-09-17T15:35:59.896693+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29118,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896980+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:59.896980+0000","last_peered":"2024-09-17T15:35:59.896980+0000","last_clean":"2024-09-17T15:35:59.896980+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:59.896980+0000","last_undegraded":"2024-09-17T15:35:59.896980+0000","last_fullsized":"2024-09-17T15:35:59.896980+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899621+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:35:59.899621+0000","last_peered":"2024-09-17T15:35:59.899621+0000","last_clean":"2024-09-17T15:35:59.899621+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:35:59.899621+0000","last_undegraded":"2024-09-17T15:35:59.899621+0000","last_fullsized":"2024-09-17T15:35:59.899621+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20825,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.899685+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:59.899685+0000","last_peered":"2024-09-17T15:35:59.899685+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:59.899685+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":122,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899340+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:35:59.899340+0000","last_peered":"2024-09-17T15:35:59.899340+0000","last_clean":"2024-09-17T15:35:59.899340+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:35:59.899340+0000","last_undegraded":"2024-09-17T15:35:59.899340+0000","last_fullsized":"2024-09-17T15:35:59.899340+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28823,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.967113+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:35:59.967113+0000","last_peered":"2024-09-17T15:35:59.967113+0000","last_clean":"2024-09-17T15:35:59.967113+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:35:59.967113+0000","last_undegraded":"2024-09-17T15:35:59.967113+0000","last_fullsized":"2024-09-17T15:35:59.967113+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":96,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896940+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:35:59.896940+0000","last_peered":"2024-09-17T15:35:59.896940+0000","last_clean":"2024-09-17T15:35:59.896940+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:35:59.896940+0000","last_undegraded":"2024-09-17T15:35:59.896940+0000","last_fullsized":"2024-09-17T15:35:59.896940+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29160,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.898948+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:35:59.898948+0000","last_peered":"2024-09-17T15:35:59.898948+0000","last_clean":"2024-09-17T15:35:59.898948+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:35:59.898948+0000","last_undegraded":"2024-09-17T15:35:59.898948+0000","last_fullsized":"2024-09-17T15:35:59.898948+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21282,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900110+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:59.900110+0000","last_peered":"2024-09-17T15:35:59.900110+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:59.900110+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":86,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897175+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:35:59.897175+0000","last_peered":"2024-09-17T15:35:59.897175+0000","last_clean":"2024-09-17T15:35:59.897175+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:35:59.897175+0000","last_undegraded":"2024-09-17T15:35:59.897175+0000","last_fullsized":"2024-09-17T15:35:59.897175+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28074,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897466+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:35:59.897466+0000","last_peered":"2024-09-17T15:35:59.897466+0000","last_clean":"2024-09-17T15:35:59.897466+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:35:59.897466+0000","last_undegraded":"2024-09-17T15:35:59.897466+0000","last_fullsized":"2024-09-17T15:35:59.897466+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.900949+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:35:59.900949+0000","last_peered":"2024-09-17T15:35:59.900949+0000","last_clean":"2024-09-17T15:35:59.900949+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:35:59.900949+0000","last_undegraded":"2024-09-17T15:35:59.900949+0000","last_fullsized":"2024-09-17T15:35:59.900949+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21203,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900017+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:59.900017+0000","last_peered":"2024-09-17T15:35:59.900017+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:59.900017+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":86,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897393+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:35:59.897393+0000","last_peered":"2024-09-17T15:35:59.897393+0000","last_clean":"2024-09-17T15:35:59.897393+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:35:59.897393+0000","last_undegraded":"2024-09-17T15:35:59.897393+0000","last_fullsized":"2024-09-17T15:35:59.897393+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":91,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896940+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:35:59.896940+0000","last_peered":"2024-09-17T15:35:59.896940+0000","last_clean":"2024-09-17T15:35:59.896940+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:35:59.896940+0000","last_undegraded":"2024-09-17T15:35:59.896940+0000","last_fullsized":"2024-09-17T15:35:59.896940+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29025,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897092+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:59.897092+0000","last_peered":"2024-09-17T15:35:59.897092+0000","last_clean":"2024-09-17T15:35:59.897092+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:59.897092+0000","last_undegraded":"2024-09-17T15:35:59.897092+0000","last_fullsized":"2024-09-17T15:35:59.897092+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24623,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24837,"num_bytes_recovered":41572719,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202629120,"data_stored":202506278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051719,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497284,"kb_used_data":132396,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874556,"statfs":{"total":96636764160,"available":96127545344,"internally_reserved":0,"allocated":135573504,"data_stored":135254826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724664,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479212,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892628,"statfs":{"total":96636764160,"available":96146051072,"internally_reserved":0,"allocated":487424,"data_stored":150217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":490204843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247947,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539376,"kb_used_data":66264,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832464,"statfs":{"total":96636764160,"available":96084443136,"internally_reserved":0,"allocated":67854336,"data_stored":67531581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135147520,"data_stored":135111390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:36:23.504 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:36:23.504 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'20648", 'reported_seq': 20916, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:59.900219+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:35:59.900219+0000', 'last_peered': '2024-09-17T15:35:59.900219+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:35:59.900219+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'10600", 'ondisk_log_start': "43'10600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20648, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:23.504 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:36:23.504 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'20934", 'reported_seq': 21165, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:59.899948+0000', 'last_change': '2024-09-17T15:26:22.693154+0000', 'last_active': '2024-09-17T15:35:59.899948+0000', 'last_peered': '2024-09-17T15:35:59.899948+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:35:59.899948+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "43'10900", 'ondisk_log_start': "43'10900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 19091, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4125}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:36:23.504 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:36:23.504 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'21251", 'reported_seq': 21561, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:59.900373+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:35:59.900373+0000', 'last_peered': '2024-09-17T15:35:59.900373+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:35:59.900373+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "43'11200", 'ondisk_log_start': "43'11200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17750, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4191}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:23.505 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-17T15:36:23.505 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'20768", 'reported_seq': 28605, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:19.398388+0000', 'last_change': '2024-09-17T15:35:58.120749+0000', 'last_active': '2024-09-17T15:36:19.398388+0000', 'last_peered': '2024-09-17T15:36:19.398388+0000', 'last_clean': '2024-09-17T15:26:20.157764+0000', 'last_became_active': '2024-09-17T15:26:22.679891+0000', 'last_became_peered': '2024-09-17T15:26:22.679891+0000', 'last_unstale': '2024-09-17T15:36:19.398388+0000', 'last_undegraded': '2024-09-17T15:26:22.283848+0000', 'last_fullsized': '2024-09-17T15:26:22.190222+0000', 'mapping_epoch': 40, 'log_start': "43'10700", 'ondisk_log_start': "43'10700", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3290", 'last_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:06.806681+0000', 'objects_scrubbed': 3227, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1180, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 335, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17537, 'num_write_kb': 28624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3839, 'num_bytes_recovered': 966, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -333}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:23.505 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:36:23.505 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20542", 'reported_seq': 20825, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:59.899685+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:35:59.899685+0000', 'last_peered': '2024-09-17T15:35:59.899685+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:35:59.899685+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17138, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3943}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:23.505 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:36:23.505 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'20966", 'reported_seq': 21282, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:59.900110+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:35:59.900110+0000', 'last_peered': '2024-09-17T15:35:59.900110+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:35:59.900110+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "43'10900", 'ondisk_log_start': "43'10900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17514, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4057}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:23.505 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:36:23.505 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'20886", 'reported_seq': 21203, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:35:59.900017+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:35:59.900017+0000', 'last_peered': '2024-09-17T15:35:59.900017+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:35:59.900017+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'10800", 'ondisk_log_start': "43'10800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17673, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3980}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:23.506 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:36:23.943 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:23.944 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 24623/24 objects degraded (102595.833%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":720,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1552252928,"bytes_avail":288358039552,"bytes_total":289910292480,"degraded_objects":24623,"degraded_total":24,"degraded_ratio":1025.9583333333333,"recovering_objects_per_sec":179,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1800,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:36:23.958 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:36:23.958 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:36:24.311 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:24.312 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:36:24.324 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":406,"stamp":"2024-09-17T15:36:23.819664+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24229,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25232,"num_bytes_recovered":41572738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1515888,"kb_used_data":199152,"kb_used_omap":13,"kb_used_meta":1316658,"kb_avail":281599632,"statfs":{"total":289910292480,"available":288358023168,"internally_reserved":0,"allocated":203931648,"data_stored":202942357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1348258815},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1413,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1413,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001430"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20916,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900219+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:35:59.900219+0000","last_peered":"2024-09-17T15:35:59.900219+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:35:59.900219+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21057,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.956046+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:36:21.956046+0000","last_peered":"2024-09-17T15:36:21.956046+0000","last_clean":"2024-09-17T15:36:21.956046+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:36:21.956046+0000","last_undegraded":"2024-09-17T15:36:21.956046+0000","last_fullsized":"2024-09-17T15:36:21.956046+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21958,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:23.444457+0000","last_change":"2024-09-17T15:36:20.543951+0000","last_active":"2024-09-17T15:36:23.444457+0000","last_peered":"2024-09-17T15:36:23.444457+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:36:23.444457+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3732,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":437,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3732}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21062,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955609+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:36:21.955609+0000","last_peered":"2024-09-17T15:36:21.955609+0000","last_clean":"2024-09-17T15:36:21.955609+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:36:21.955609+0000","last_undegraded":"2024-09-17T15:36:21.955609+0000","last_fullsized":"2024-09-17T15:36:21.955609+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20801,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899356+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:35:59.899356+0000","last_peered":"2024-09-17T15:35:59.899356+0000","last_clean":"2024-09-17T15:35:59.899356+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:35:59.899356+0000","last_undegraded":"2024-09-17T15:35:59.899356+0000","last_fullsized":"2024-09-17T15:35:59.899356+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21561,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900373+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:35:59.900373+0000","last_peered":"2024-09-17T15:35:59.900373+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:35:59.900373+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4191},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":28605,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:19.398388+0000","last_change":"2024-09-17T15:35:58.120749+0000","last_active":"2024-09-17T15:36:19.398388+0000","last_peered":"2024-09-17T15:36:19.398388+0000","last_clean":"2024-09-17T15:26:20.157764+0000","last_became_active":"2024-09-17T15:26:22.679891+0000","last_became_peered":"2024-09-17T15:26:22.679891+0000","last_unstale":"2024-09-17T15:36:19.398388+0000","last_undegraded":"2024-09-17T15:26:22.283848+0000","last_fullsized":"2024-09-17T15:26:22.190222+0000","mapping_epoch":40,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":335,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3839,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-333}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21063,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899226+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:35:59.899226+0000","last_peered":"2024-09-17T15:35:59.899226+0000","last_clean":"2024-09-17T15:35:59.899226+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:35:59.899226+0000","last_undegraded":"2024-09-17T15:35:59.899226+0000","last_fullsized":"2024-09-17T15:35:59.899226+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":86,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896693+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:35:59.896693+0000","last_peered":"2024-09-17T15:35:59.896693+0000","last_clean":"2024-09-17T15:35:59.896693+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:35:59.896693+0000","last_undegraded":"2024-09-17T15:35:59.896693+0000","last_fullsized":"2024-09-17T15:35:59.896693+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29118,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896980+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:35:59.896980+0000","last_peered":"2024-09-17T15:35:59.896980+0000","last_clean":"2024-09-17T15:35:59.896980+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:35:59.896980+0000","last_undegraded":"2024-09-17T15:35:59.896980+0000","last_fullsized":"2024-09-17T15:35:59.896980+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955084+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:36:21.955084+0000","last_peered":"2024-09-17T15:36:21.955084+0000","last_clean":"2024-09-17T15:36:21.955084+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:36:21.955084+0000","last_undegraded":"2024-09-17T15:36:21.955084+0000","last_fullsized":"2024-09-17T15:36:21.955084+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20825,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.899685+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:35:59.899685+0000","last_peered":"2024-09-17T15:35:59.899685+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:35:59.899685+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":122,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.899340+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:35:59.899340+0000","last_peered":"2024-09-17T15:35:59.899340+0000","last_clean":"2024-09-17T15:35:59.899340+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:35:59.899340+0000","last_undegraded":"2024-09-17T15:35:59.899340+0000","last_fullsized":"2024-09-17T15:35:59.899340+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28827,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955167+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:36:21.955167+0000","last_peered":"2024-09-17T15:36:21.955167+0000","last_clean":"2024-09-17T15:36:21.955167+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:36:21.955167+0000","last_undegraded":"2024-09-17T15:36:21.955167+0000","last_fullsized":"2024-09-17T15:36:21.955167+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":96,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896940+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:35:59.896940+0000","last_peered":"2024-09-17T15:35:59.896940+0000","last_clean":"2024-09-17T15:35:59.896940+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:35:59.896940+0000","last_undegraded":"2024-09-17T15:35:59.896940+0000","last_fullsized":"2024-09-17T15:35:59.896940+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29164,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954760+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:36:21.954760+0000","last_peered":"2024-09-17T15:36:21.954760+0000","last_clean":"2024-09-17T15:36:21.954760+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:36:21.954760+0000","last_undegraded":"2024-09-17T15:36:21.954760+0000","last_fullsized":"2024-09-17T15:36:21.954760+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21282,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900110+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:35:59.900110+0000","last_peered":"2024-09-17T15:35:59.900110+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:35:59.900110+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":86,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897175+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:35:59.897175+0000","last_peered":"2024-09-17T15:35:59.897175+0000","last_clean":"2024-09-17T15:35:59.897175+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:35:59.897175+0000","last_undegraded":"2024-09-17T15:35:59.897175+0000","last_fullsized":"2024-09-17T15:35:59.897175+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28074,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897466+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:35:59.897466+0000","last_peered":"2024-09-17T15:35:59.897466+0000","last_clean":"2024-09-17T15:35:59.897466+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:35:59.897466+0000","last_undegraded":"2024-09-17T15:35:59.897466+0000","last_fullsized":"2024-09-17T15:35:59.897466+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955852+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:36:21.955852+0000","last_peered":"2024-09-17T15:36:21.955852+0000","last_clean":"2024-09-17T15:36:21.955852+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:36:21.955852+0000","last_undegraded":"2024-09-17T15:36:21.955852+0000","last_fullsized":"2024-09-17T15:36:21.955852+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21203,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:35:59.900017+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:35:59.900017+0000","last_peered":"2024-09-17T15:35:59.900017+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:35:59.900017+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":86,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897393+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:35:59.897393+0000","last_peered":"2024-09-17T15:35:59.897393+0000","last_clean":"2024-09-17T15:35:59.897393+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:35:59.897393+0000","last_undegraded":"2024-09-17T15:35:59.897393+0000","last_fullsized":"2024-09-17T15:35:59.897393+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":91,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.896940+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:35:59.896940+0000","last_peered":"2024-09-17T15:35:59.896940+0000","last_clean":"2024-09-17T15:35:59.896940+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:35:59.896940+0000","last_undegraded":"2024-09-17T15:35:59.896940+0000","last_fullsized":"2024-09-17T15:35:59.896940+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29025,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T15:35:59.897092+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:35:59.897092+0000","last_peered":"2024-09-17T15:35:59.897092+0000","last_clean":"2024-09-17T15:35:59.897092+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:35:59.897092+0000","last_undegraded":"2024-09-17T15:35:59.897092+0000","last_fullsized":"2024-09-17T15:35:59.897092+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24229,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25231,"num_bytes_recovered":41572719,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202629120,"data_stored":202506278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051720,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497300,"kb_used_data":132412,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874540,"statfs":{"total":96636764160,"available":96127528960,"internally_reserved":0,"allocated":135589888,"data_stored":135260559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724664,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479212,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892628,"statfs":{"total":96636764160,"available":96146051072,"internally_reserved":0,"allocated":487424,"data_stored":150217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":490204843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247947,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539376,"kb_used_data":66264,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832464,"statfs":{"total":96636764160,"available":96084443136,"internally_reserved":0,"allocated":67854336,"data_stored":67531581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135147520,"data_stored":135111390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:36:27.326 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:36:27.632 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:27.632 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:36:27.645 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":407,"stamp":"2024-09-17T15:36:25.820110+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23894,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25567,"num_bytes_recovered":41572738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1524232,"kb_used_data":199180,"kb_used_omap":13,"kb_used_meta":1324978,"kb_avail":281591288,"statfs":{"total":289910292480,"available":288349478912,"internally_reserved":0,"allocated":203960320,"data_stored":202953340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1356778495},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1748,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1748,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001475"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20921,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.961364+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:36:21.961364+0000","last_peered":"2024-09-17T15:36:21.961364+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:36:21.961364+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21057,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.956046+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:36:21.956046+0000","last_peered":"2024-09-17T15:36:21.956046+0000","last_clean":"2024-09-17T15:36:21.956046+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:36:21.956046+0000","last_undegraded":"2024-09-17T15:36:21.956046+0000","last_fullsized":"2024-09-17T15:36:21.956046+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21958,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:23.444457+0000","last_change":"2024-09-17T15:36:20.543951+0000","last_active":"2024-09-17T15:36:23.444457+0000","last_peered":"2024-09-17T15:36:23.444457+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:36:23.444457+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3732,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":437,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3732}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21062,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955609+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:36:21.955609+0000","last_peered":"2024-09-17T15:36:21.955609+0000","last_clean":"2024-09-17T15:36:21.955609+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:36:21.955609+0000","last_undegraded":"2024-09-17T15:36:21.955609+0000","last_fullsized":"2024-09-17T15:36:21.955609+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20805,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959779+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:36:21.959779+0000","last_peered":"2024-09-17T15:36:21.959779+0000","last_clean":"2024-09-17T15:36:21.959779+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:36:21.959779+0000","last_undegraded":"2024-09-17T15:36:21.959779+0000","last_fullsized":"2024-09-17T15:36:21.959779+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21565,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.959782+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:36:21.959782+0000","last_peered":"2024-09-17T15:36:21.959782+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:36:21.959782+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4191},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:22.048419+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:36:22.048419+0000","last_peered":"2024-09-17T15:36:22.048419+0000","last_clean":"2024-09-17T15:36:22.048419+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:36:22.048419+0000","last_undegraded":"2024-09-17T15:36:22.048419+0000","last_fullsized":"2024-09-17T15:36:22.048419+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21067,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959637+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:36:21.959637+0000","last_peered":"2024-09-17T15:36:21.959637+0000","last_clean":"2024-09-17T15:36:21.959637+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:36:21.959637+0000","last_undegraded":"2024-09-17T15:36:21.959637+0000","last_fullsized":"2024-09-17T15:36:21.959637+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954544+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:36:21.954544+0000","last_peered":"2024-09-17T15:36:21.954544+0000","last_clean":"2024-09-17T15:36:21.954544+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:36:21.954544+0000","last_undegraded":"2024-09-17T15:36:21.954544+0000","last_fullsized":"2024-09-17T15:36:21.954544+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955888+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:36:21.955888+0000","last_peered":"2024-09-17T15:36:21.955888+0000","last_clean":"2024-09-17T15:36:21.955888+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:36:21.955888+0000","last_undegraded":"2024-09-17T15:36:21.955888+0000","last_fullsized":"2024-09-17T15:36:21.955888+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955084+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:36:21.955084+0000","last_peered":"2024-09-17T15:36:21.955084+0000","last_clean":"2024-09-17T15:36:21.955084+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:36:21.955084+0000","last_undegraded":"2024-09-17T15:36:21.955084+0000","last_fullsized":"2024-09-17T15:36:21.955084+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20829,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.960010+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:36:21.960010+0000","last_peered":"2024-09-17T15:36:21.960010+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:36:21.960010+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959684+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:36:21.959684+0000","last_peered":"2024-09-17T15:36:21.959684+0000","last_clean":"2024-09-17T15:36:21.959684+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:36:21.959684+0000","last_undegraded":"2024-09-17T15:36:21.959684+0000","last_fullsized":"2024-09-17T15:36:21.959684+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28827,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955167+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:36:21.955167+0000","last_peered":"2024-09-17T15:36:21.955167+0000","last_clean":"2024-09-17T15:36:21.955167+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:36:21.955167+0000","last_undegraded":"2024-09-17T15:36:21.955167+0000","last_fullsized":"2024-09-17T15:36:21.955167+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955801+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:36:21.955801+0000","last_peered":"2024-09-17T15:36:21.955801+0000","last_clean":"2024-09-17T15:36:21.955801+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:36:21.955801+0000","last_undegraded":"2024-09-17T15:36:21.955801+0000","last_fullsized":"2024-09-17T15:36:21.955801+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29164,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954760+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:36:21.954760+0000","last_peered":"2024-09-17T15:36:21.954760+0000","last_clean":"2024-09-17T15:36:21.954760+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:36:21.954760+0000","last_undegraded":"2024-09-17T15:36:21.954760+0000","last_fullsized":"2024-09-17T15:36:21.954760+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21286,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.960250+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:36:21.960250+0000","last_peered":"2024-09-17T15:36:21.960250+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:36:21.960250+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":90,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954762+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:36:21.954762+0000","last_peered":"2024-09-17T15:36:21.954762+0000","last_clean":"2024-09-17T15:36:21.954762+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:36:21.954762+0000","last_undegraded":"2024-09-17T15:36:21.954762+0000","last_fullsized":"2024-09-17T15:36:21.954762+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28078,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955344+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:36:21.955344+0000","last_peered":"2024-09-17T15:36:21.955344+0000","last_clean":"2024-09-17T15:36:21.955344+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:36:21.955344+0000","last_undegraded":"2024-09-17T15:36:21.955344+0000","last_fullsized":"2024-09-17T15:36:21.955344+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955852+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:36:21.955852+0000","last_peered":"2024-09-17T15:36:21.955852+0000","last_clean":"2024-09-17T15:36:21.955852+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:36:21.955852+0000","last_undegraded":"2024-09-17T15:36:21.955852+0000","last_fullsized":"2024-09-17T15:36:21.955852+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21207,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.959971+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:36:21.959971+0000","last_peered":"2024-09-17T15:36:21.959971+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:36:21.959971+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":90,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955263+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:36:21.955263+0000","last_peered":"2024-09-17T15:36:21.955263+0000","last_clean":"2024-09-17T15:36:21.955263+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:36:21.955263+0000","last_undegraded":"2024-09-17T15:36:21.955263+0000","last_fullsized":"2024-09-17T15:36:21.955263+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":95,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.956005+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:36:21.956005+0000","last_peered":"2024-09-17T15:36:21.956005+0000","last_clean":"2024-09-17T15:36:21.956005+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:36:21.956005+0000","last_undegraded":"2024-09-17T15:36:21.956005+0000","last_fullsized":"2024-09-17T15:36:21.956005+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29029,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954672+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:36:21.954672+0000","last_peered":"2024-09-17T15:36:21.954672+0000","last_clean":"2024-09-17T15:36:21.954672+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:36:21.954672+0000","last_undegraded":"2024-09-17T15:36:21.954672+0000","last_fullsized":"2024-09-17T15:36:21.954672+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23894,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25566,"num_bytes_recovered":41572719,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202625024,"data_stored":202505795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051720,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497300,"kb_used_data":132412,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874540,"statfs":{"total":96636764160,"available":96127528960,"internally_reserved":0,"allocated":135589888,"data_stored":135260559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724665,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487544,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884296,"statfs":{"total":96636764160,"available":96137519104,"internally_reserved":0,"allocated":499712,"data_stored":155467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":498724523},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247948,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539388,"kb_used_data":66280,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832452,"statfs":{"total":96636764160,"available":96084430848,"internally_reserved":0,"allocated":67870720,"data_stored":67537314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135147520,"data_stored":135111390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:36:27.647 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:36:27.950 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:27.950 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:36:27.963 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":408,"stamp":"2024-09-17T15:36:27.820431+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23894,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25567,"num_bytes_recovered":41572738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1524232,"kb_used_data":199180,"kb_used_omap":13,"kb_used_meta":1324978,"kb_avail":281591288,"statfs":{"total":289910292480,"available":288349478912,"internally_reserved":0,"allocated":203960320,"data_stored":202953340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1356778495},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-729,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":729,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001471"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20921,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.961364+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:36:21.961364+0000","last_peered":"2024-09-17T15:36:21.961364+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:36:21.961364+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21057,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.956046+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:36:21.956046+0000","last_peered":"2024-09-17T15:36:21.956046+0000","last_clean":"2024-09-17T15:36:21.956046+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:36:21.956046+0000","last_undegraded":"2024-09-17T15:36:21.956046+0000","last_fullsized":"2024-09-17T15:36:21.956046+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21958,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:23.444457+0000","last_change":"2024-09-17T15:36:20.543951+0000","last_active":"2024-09-17T15:36:23.444457+0000","last_peered":"2024-09-17T15:36:23.444457+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:36:23.444457+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3732,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":437,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3732}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21062,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955609+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:36:21.955609+0000","last_peered":"2024-09-17T15:36:21.955609+0000","last_clean":"2024-09-17T15:36:21.955609+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:36:21.955609+0000","last_undegraded":"2024-09-17T15:36:21.955609+0000","last_fullsized":"2024-09-17T15:36:21.955609+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20805,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959779+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:36:21.959779+0000","last_peered":"2024-09-17T15:36:21.959779+0000","last_clean":"2024-09-17T15:36:21.959779+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:36:21.959779+0000","last_undegraded":"2024-09-17T15:36:21.959779+0000","last_fullsized":"2024-09-17T15:36:21.959779+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21565,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.959782+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:36:21.959782+0000","last_peered":"2024-09-17T15:36:21.959782+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:36:21.959782+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4191},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:22.048419+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:36:22.048419+0000","last_peered":"2024-09-17T15:36:22.048419+0000","last_clean":"2024-09-17T15:36:22.048419+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:36:22.048419+0000","last_undegraded":"2024-09-17T15:36:22.048419+0000","last_fullsized":"2024-09-17T15:36:22.048419+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21067,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959637+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:36:21.959637+0000","last_peered":"2024-09-17T15:36:21.959637+0000","last_clean":"2024-09-17T15:36:21.959637+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:36:21.959637+0000","last_undegraded":"2024-09-17T15:36:21.959637+0000","last_fullsized":"2024-09-17T15:36:21.959637+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954544+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:36:21.954544+0000","last_peered":"2024-09-17T15:36:21.954544+0000","last_clean":"2024-09-17T15:36:21.954544+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:36:21.954544+0000","last_undegraded":"2024-09-17T15:36:21.954544+0000","last_fullsized":"2024-09-17T15:36:21.954544+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955888+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:36:21.955888+0000","last_peered":"2024-09-17T15:36:21.955888+0000","last_clean":"2024-09-17T15:36:21.955888+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:36:21.955888+0000","last_undegraded":"2024-09-17T15:36:21.955888+0000","last_fullsized":"2024-09-17T15:36:21.955888+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955084+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:36:21.955084+0000","last_peered":"2024-09-17T15:36:21.955084+0000","last_clean":"2024-09-17T15:36:21.955084+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:36:21.955084+0000","last_undegraded":"2024-09-17T15:36:21.955084+0000","last_fullsized":"2024-09-17T15:36:21.955084+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20829,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.960010+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:36:21.960010+0000","last_peered":"2024-09-17T15:36:21.960010+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:36:21.960010+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959684+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:36:21.959684+0000","last_peered":"2024-09-17T15:36:21.959684+0000","last_clean":"2024-09-17T15:36:21.959684+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:36:21.959684+0000","last_undegraded":"2024-09-17T15:36:21.959684+0000","last_fullsized":"2024-09-17T15:36:21.959684+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28827,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955167+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:36:21.955167+0000","last_peered":"2024-09-17T15:36:21.955167+0000","last_clean":"2024-09-17T15:36:21.955167+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:36:21.955167+0000","last_undegraded":"2024-09-17T15:36:21.955167+0000","last_fullsized":"2024-09-17T15:36:21.955167+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955801+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:36:21.955801+0000","last_peered":"2024-09-17T15:36:21.955801+0000","last_clean":"2024-09-17T15:36:21.955801+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:36:21.955801+0000","last_undegraded":"2024-09-17T15:36:21.955801+0000","last_fullsized":"2024-09-17T15:36:21.955801+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29164,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954760+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:36:21.954760+0000","last_peered":"2024-09-17T15:36:21.954760+0000","last_clean":"2024-09-17T15:36:21.954760+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:36:21.954760+0000","last_undegraded":"2024-09-17T15:36:21.954760+0000","last_fullsized":"2024-09-17T15:36:21.954760+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21286,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.960250+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:36:21.960250+0000","last_peered":"2024-09-17T15:36:21.960250+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:36:21.960250+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":90,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954762+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:36:21.954762+0000","last_peered":"2024-09-17T15:36:21.954762+0000","last_clean":"2024-09-17T15:36:21.954762+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:36:21.954762+0000","last_undegraded":"2024-09-17T15:36:21.954762+0000","last_fullsized":"2024-09-17T15:36:21.954762+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28078,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955344+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:36:21.955344+0000","last_peered":"2024-09-17T15:36:21.955344+0000","last_clean":"2024-09-17T15:36:21.955344+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:36:21.955344+0000","last_undegraded":"2024-09-17T15:36:21.955344+0000","last_fullsized":"2024-09-17T15:36:21.955344+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955852+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:36:21.955852+0000","last_peered":"2024-09-17T15:36:21.955852+0000","last_clean":"2024-09-17T15:36:21.955852+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:36:21.955852+0000","last_undegraded":"2024-09-17T15:36:21.955852+0000","last_fullsized":"2024-09-17T15:36:21.955852+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21207,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.959971+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:36:21.959971+0000","last_peered":"2024-09-17T15:36:21.959971+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:36:21.959971+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":90,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955263+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:36:21.955263+0000","last_peered":"2024-09-17T15:36:21.955263+0000","last_clean":"2024-09-17T15:36:21.955263+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:36:21.955263+0000","last_undegraded":"2024-09-17T15:36:21.955263+0000","last_fullsized":"2024-09-17T15:36:21.955263+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":95,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.956005+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:36:21.956005+0000","last_peered":"2024-09-17T15:36:21.956005+0000","last_clean":"2024-09-17T15:36:21.956005+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:36:21.956005+0000","last_undegraded":"2024-09-17T15:36:21.956005+0000","last_fullsized":"2024-09-17T15:36:21.956005+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29029,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954672+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:36:21.954672+0000","last_peered":"2024-09-17T15:36:21.954672+0000","last_clean":"2024-09-17T15:36:21.954672+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:36:21.954672+0000","last_undegraded":"2024-09-17T15:36:21.954672+0000","last_fullsized":"2024-09-17T15:36:21.954672+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23894,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25566,"num_bytes_recovered":41572719,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202625024,"data_stored":202505795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051720,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497300,"kb_used_data":132412,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874540,"statfs":{"total":96636764160,"available":96127528960,"internally_reserved":0,"allocated":135589888,"data_stored":135260559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724665,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487544,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884296,"statfs":{"total":96636764160,"available":96137519104,"internally_reserved":0,"allocated":499712,"data_stored":155467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":498724523},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247948,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539388,"kb_used_data":66280,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832452,"statfs":{"total":96636764160,"available":96084430848,"internally_reserved":0,"allocated":67870720,"data_stored":67537314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135147520,"data_stored":135111390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:36:27.964 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:36:27.964 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'20648", 'reported_seq': 20921, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:21.961364+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:36:21.961364+0000', 'last_peered': '2024-09-17T15:36:21.961364+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:36:21.961364+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'10600", 'ondisk_log_start': "43'10600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20648, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:27.965 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:36:27.965 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'20934", 'reported_seq': 21958, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:23.444457+0000', 'last_change': '2024-09-17T15:36:20.543951+0000', 'last_active': '2024-09-17T15:36:23.444457+0000', 'last_peered': '2024-09-17T15:36:23.444457+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:36:23.444457+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "43'10900", 'ondisk_log_start': "43'10900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3732, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 19091, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 437, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3732}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:36:27.965 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:36:27.965 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'21251", 'reported_seq': 21565, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:21.959782+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:36:21.959782+0000', 'last_peered': '2024-09-17T15:36:21.959782+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:36:21.959782+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "43'11200", 'ondisk_log_start': "43'11200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17750, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4191}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:27.965 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:36:27.965 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20542", 'reported_seq': 20829, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:21.960010+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:36:21.960010+0000', 'last_peered': '2024-09-17T15:36:21.960010+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:36:21.960010+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17138, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3943}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:27.965 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:36:27.965 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'20966", 'reported_seq': 21286, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:21.960250+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:36:21.960250+0000', 'last_peered': '2024-09-17T15:36:21.960250+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:36:21.960250+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "43'10900", 'ondisk_log_start': "43'10900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17514, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4057}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:27.965 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:36:27.965 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'20886", 'reported_seq': 21207, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:21.959971+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:36:21.959971+0000', 'last_peered': '2024-09-17T15:36:21.959971+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:36:21.959971+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'10800", 'ondisk_log_start': "43'10800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17673, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3980}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:27.966 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:36:28.369 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:28.370 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 23894/24 objects degraded (99558.333%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":725,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1560813568,"bytes_avail":288349478912,"bytes_total":289910292480,"degraded_objects":23894,"degraded_total":24,"degraded_ratio":995.58333333333348,"recovering_objects_per_sec":91,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":729,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:36:28.383 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:36:28.383 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:36:28.698 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:28.699 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:36:28.712 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":408,"stamp":"2024-09-17T15:36:27.820431+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23894,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25567,"num_bytes_recovered":41572738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1524232,"kb_used_data":199180,"kb_used_omap":13,"kb_used_meta":1324978,"kb_avail":281591288,"statfs":{"total":289910292480,"available":288349478912,"internally_reserved":0,"allocated":203960320,"data_stored":202953340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1356778495},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-729,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":729,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001471"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20921,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.961364+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:36:21.961364+0000","last_peered":"2024-09-17T15:36:21.961364+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:36:21.961364+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21057,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.956046+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:36:21.956046+0000","last_peered":"2024-09-17T15:36:21.956046+0000","last_clean":"2024-09-17T15:36:21.956046+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:36:21.956046+0000","last_undegraded":"2024-09-17T15:36:21.956046+0000","last_fullsized":"2024-09-17T15:36:21.956046+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21958,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:23.444457+0000","last_change":"2024-09-17T15:36:20.543951+0000","last_active":"2024-09-17T15:36:23.444457+0000","last_peered":"2024-09-17T15:36:23.444457+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:36:23.444457+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3732,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":437,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3732}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21062,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955609+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:36:21.955609+0000","last_peered":"2024-09-17T15:36:21.955609+0000","last_clean":"2024-09-17T15:36:21.955609+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:36:21.955609+0000","last_undegraded":"2024-09-17T15:36:21.955609+0000","last_fullsized":"2024-09-17T15:36:21.955609+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20805,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959779+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:36:21.959779+0000","last_peered":"2024-09-17T15:36:21.959779+0000","last_clean":"2024-09-17T15:36:21.959779+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:36:21.959779+0000","last_undegraded":"2024-09-17T15:36:21.959779+0000","last_fullsized":"2024-09-17T15:36:21.959779+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21565,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.959782+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:36:21.959782+0000","last_peered":"2024-09-17T15:36:21.959782+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:36:21.959782+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4191},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:22.048419+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:36:22.048419+0000","last_peered":"2024-09-17T15:36:22.048419+0000","last_clean":"2024-09-17T15:36:22.048419+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:36:22.048419+0000","last_undegraded":"2024-09-17T15:36:22.048419+0000","last_fullsized":"2024-09-17T15:36:22.048419+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21067,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959637+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:36:21.959637+0000","last_peered":"2024-09-17T15:36:21.959637+0000","last_clean":"2024-09-17T15:36:21.959637+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:36:21.959637+0000","last_undegraded":"2024-09-17T15:36:21.959637+0000","last_fullsized":"2024-09-17T15:36:21.959637+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954544+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:36:21.954544+0000","last_peered":"2024-09-17T15:36:21.954544+0000","last_clean":"2024-09-17T15:36:21.954544+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:36:21.954544+0000","last_undegraded":"2024-09-17T15:36:21.954544+0000","last_fullsized":"2024-09-17T15:36:21.954544+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955888+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:36:21.955888+0000","last_peered":"2024-09-17T15:36:21.955888+0000","last_clean":"2024-09-17T15:36:21.955888+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:36:21.955888+0000","last_undegraded":"2024-09-17T15:36:21.955888+0000","last_fullsized":"2024-09-17T15:36:21.955888+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955084+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:36:21.955084+0000","last_peered":"2024-09-17T15:36:21.955084+0000","last_clean":"2024-09-17T15:36:21.955084+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:36:21.955084+0000","last_undegraded":"2024-09-17T15:36:21.955084+0000","last_fullsized":"2024-09-17T15:36:21.955084+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20829,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.960010+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:36:21.960010+0000","last_peered":"2024-09-17T15:36:21.960010+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:36:21.960010+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959684+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:36:21.959684+0000","last_peered":"2024-09-17T15:36:21.959684+0000","last_clean":"2024-09-17T15:36:21.959684+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:36:21.959684+0000","last_undegraded":"2024-09-17T15:36:21.959684+0000","last_fullsized":"2024-09-17T15:36:21.959684+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28827,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955167+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:36:21.955167+0000","last_peered":"2024-09-17T15:36:21.955167+0000","last_clean":"2024-09-17T15:36:21.955167+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:36:21.955167+0000","last_undegraded":"2024-09-17T15:36:21.955167+0000","last_fullsized":"2024-09-17T15:36:21.955167+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955801+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:36:21.955801+0000","last_peered":"2024-09-17T15:36:21.955801+0000","last_clean":"2024-09-17T15:36:21.955801+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:36:21.955801+0000","last_undegraded":"2024-09-17T15:36:21.955801+0000","last_fullsized":"2024-09-17T15:36:21.955801+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29164,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954760+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:36:21.954760+0000","last_peered":"2024-09-17T15:36:21.954760+0000","last_clean":"2024-09-17T15:36:21.954760+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:36:21.954760+0000","last_undegraded":"2024-09-17T15:36:21.954760+0000","last_fullsized":"2024-09-17T15:36:21.954760+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21286,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.960250+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:36:21.960250+0000","last_peered":"2024-09-17T15:36:21.960250+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:36:21.960250+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":90,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954762+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:36:21.954762+0000","last_peered":"2024-09-17T15:36:21.954762+0000","last_clean":"2024-09-17T15:36:21.954762+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:36:21.954762+0000","last_undegraded":"2024-09-17T15:36:21.954762+0000","last_fullsized":"2024-09-17T15:36:21.954762+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28078,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955344+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:36:21.955344+0000","last_peered":"2024-09-17T15:36:21.955344+0000","last_clean":"2024-09-17T15:36:21.955344+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:36:21.955344+0000","last_undegraded":"2024-09-17T15:36:21.955344+0000","last_fullsized":"2024-09-17T15:36:21.955344+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955852+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:36:21.955852+0000","last_peered":"2024-09-17T15:36:21.955852+0000","last_clean":"2024-09-17T15:36:21.955852+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:36:21.955852+0000","last_undegraded":"2024-09-17T15:36:21.955852+0000","last_fullsized":"2024-09-17T15:36:21.955852+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21207,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.959971+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:36:21.959971+0000","last_peered":"2024-09-17T15:36:21.959971+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:36:21.959971+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":90,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955263+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:36:21.955263+0000","last_peered":"2024-09-17T15:36:21.955263+0000","last_clean":"2024-09-17T15:36:21.955263+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:36:21.955263+0000","last_undegraded":"2024-09-17T15:36:21.955263+0000","last_fullsized":"2024-09-17T15:36:21.955263+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":95,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.956005+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:36:21.956005+0000","last_peered":"2024-09-17T15:36:21.956005+0000","last_clean":"2024-09-17T15:36:21.956005+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:36:21.956005+0000","last_undegraded":"2024-09-17T15:36:21.956005+0000","last_fullsized":"2024-09-17T15:36:21.956005+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29029,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954672+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:36:21.954672+0000","last_peered":"2024-09-17T15:36:21.954672+0000","last_clean":"2024-09-17T15:36:21.954672+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:36:21.954672+0000","last_undegraded":"2024-09-17T15:36:21.954672+0000","last_fullsized":"2024-09-17T15:36:21.954672+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23894,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25566,"num_bytes_recovered":41572719,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202625024,"data_stored":202505795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051720,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497300,"kb_used_data":132412,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874540,"statfs":{"total":96636764160,"available":96127528960,"internally_reserved":0,"allocated":135589888,"data_stored":135260559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724665,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487544,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884296,"statfs":{"total":96636764160,"available":96137519104,"internally_reserved":0,"allocated":499712,"data_stored":155467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":498724523},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247948,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539388,"kb_used_data":66280,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832452,"statfs":{"total":96636764160,"available":96084430848,"internally_reserved":0,"allocated":67870720,"data_stored":67537314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135147520,"data_stored":135111390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:36:31.714 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:36:32.062 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:32.062 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:36:32.076 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":410,"stamp":"2024-09-17T15:36:31.821096+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23261,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26200,"num_bytes_recovered":41572738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1528392,"kb_used_data":199180,"kb_used_omap":13,"kb_used_meta":1329138,"kb_avail":281587128,"statfs":{"total":289910292480,"available":288345219072,"internally_reserved":0,"allocated":203960320,"data_stored":202953340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1361038335},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1362,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1362,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001739"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20921,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.961364+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:36:21.961364+0000","last_peered":"2024-09-17T15:36:21.961364+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:36:21.961364+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21057,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.956046+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:36:21.956046+0000","last_peered":"2024-09-17T15:36:21.956046+0000","last_clean":"2024-09-17T15:36:21.956046+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:36:21.956046+0000","last_undegraded":"2024-09-17T15:36:21.956046+0000","last_fullsized":"2024-09-17T15:36:21.956046+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":23224,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:28.442089+0000","last_change":"2024-09-17T15:36:20.543951+0000","last_active":"2024-09-17T15:36:28.442089+0000","last_peered":"2024-09-17T15:36:28.442089+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:36:28.442089+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3099,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1070,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3099}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21062,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955609+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:36:21.955609+0000","last_peered":"2024-09-17T15:36:21.955609+0000","last_clean":"2024-09-17T15:36:21.955609+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:36:21.955609+0000","last_undegraded":"2024-09-17T15:36:21.955609+0000","last_fullsized":"2024-09-17T15:36:21.955609+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20805,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959779+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:36:21.959779+0000","last_peered":"2024-09-17T15:36:21.959779+0000","last_clean":"2024-09-17T15:36:21.959779+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:36:21.959779+0000","last_undegraded":"2024-09-17T15:36:21.959779+0000","last_fullsized":"2024-09-17T15:36:21.959779+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21565,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.959782+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:36:21.959782+0000","last_peered":"2024-09-17T15:36:21.959782+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:36:21.959782+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4191},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:22.048419+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:36:22.048419+0000","last_peered":"2024-09-17T15:36:22.048419+0000","last_clean":"2024-09-17T15:36:22.048419+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:36:22.048419+0000","last_undegraded":"2024-09-17T15:36:22.048419+0000","last_fullsized":"2024-09-17T15:36:22.048419+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21067,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959637+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:36:21.959637+0000","last_peered":"2024-09-17T15:36:21.959637+0000","last_clean":"2024-09-17T15:36:21.959637+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:36:21.959637+0000","last_undegraded":"2024-09-17T15:36:21.959637+0000","last_fullsized":"2024-09-17T15:36:21.959637+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954544+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:36:21.954544+0000","last_peered":"2024-09-17T15:36:21.954544+0000","last_clean":"2024-09-17T15:36:21.954544+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:36:21.954544+0000","last_undegraded":"2024-09-17T15:36:21.954544+0000","last_fullsized":"2024-09-17T15:36:21.954544+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955888+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:36:21.955888+0000","last_peered":"2024-09-17T15:36:21.955888+0000","last_clean":"2024-09-17T15:36:21.955888+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:36:21.955888+0000","last_undegraded":"2024-09-17T15:36:21.955888+0000","last_fullsized":"2024-09-17T15:36:21.955888+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955084+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:36:21.955084+0000","last_peered":"2024-09-17T15:36:21.955084+0000","last_clean":"2024-09-17T15:36:21.955084+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:36:21.955084+0000","last_undegraded":"2024-09-17T15:36:21.955084+0000","last_fullsized":"2024-09-17T15:36:21.955084+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20829,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.960010+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:36:21.960010+0000","last_peered":"2024-09-17T15:36:21.960010+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:36:21.960010+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959684+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:36:21.959684+0000","last_peered":"2024-09-17T15:36:21.959684+0000","last_clean":"2024-09-17T15:36:21.959684+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:36:21.959684+0000","last_undegraded":"2024-09-17T15:36:21.959684+0000","last_fullsized":"2024-09-17T15:36:21.959684+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28827,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955167+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:36:21.955167+0000","last_peered":"2024-09-17T15:36:21.955167+0000","last_clean":"2024-09-17T15:36:21.955167+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:36:21.955167+0000","last_undegraded":"2024-09-17T15:36:21.955167+0000","last_fullsized":"2024-09-17T15:36:21.955167+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955801+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:36:21.955801+0000","last_peered":"2024-09-17T15:36:21.955801+0000","last_clean":"2024-09-17T15:36:21.955801+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:36:21.955801+0000","last_undegraded":"2024-09-17T15:36:21.955801+0000","last_fullsized":"2024-09-17T15:36:21.955801+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29164,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954760+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:36:21.954760+0000","last_peered":"2024-09-17T15:36:21.954760+0000","last_clean":"2024-09-17T15:36:21.954760+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:36:21.954760+0000","last_undegraded":"2024-09-17T15:36:21.954760+0000","last_fullsized":"2024-09-17T15:36:21.954760+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21286,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.960250+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:36:21.960250+0000","last_peered":"2024-09-17T15:36:21.960250+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:36:21.960250+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":90,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954762+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:36:21.954762+0000","last_peered":"2024-09-17T15:36:21.954762+0000","last_clean":"2024-09-17T15:36:21.954762+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:36:21.954762+0000","last_undegraded":"2024-09-17T15:36:21.954762+0000","last_fullsized":"2024-09-17T15:36:21.954762+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28078,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955344+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:36:21.955344+0000","last_peered":"2024-09-17T15:36:21.955344+0000","last_clean":"2024-09-17T15:36:21.955344+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:36:21.955344+0000","last_undegraded":"2024-09-17T15:36:21.955344+0000","last_fullsized":"2024-09-17T15:36:21.955344+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955852+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:36:21.955852+0000","last_peered":"2024-09-17T15:36:21.955852+0000","last_clean":"2024-09-17T15:36:21.955852+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:36:21.955852+0000","last_undegraded":"2024-09-17T15:36:21.955852+0000","last_fullsized":"2024-09-17T15:36:21.955852+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21207,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.959971+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:36:21.959971+0000","last_peered":"2024-09-17T15:36:21.959971+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:36:21.959971+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":90,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955263+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:36:21.955263+0000","last_peered":"2024-09-17T15:36:21.955263+0000","last_clean":"2024-09-17T15:36:21.955263+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:36:21.955263+0000","last_undegraded":"2024-09-17T15:36:21.955263+0000","last_fullsized":"2024-09-17T15:36:21.955263+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":95,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.956005+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:36:21.956005+0000","last_peered":"2024-09-17T15:36:21.956005+0000","last_clean":"2024-09-17T15:36:21.956005+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:36:21.956005+0000","last_undegraded":"2024-09-17T15:36:21.956005+0000","last_fullsized":"2024-09-17T15:36:21.956005+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29029,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954672+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:36:21.954672+0000","last_peered":"2024-09-17T15:36:21.954672+0000","last_clean":"2024-09-17T15:36:21.954672+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:36:21.954672+0000","last_undegraded":"2024-09-17T15:36:21.954672+0000","last_fullsized":"2024-09-17T15:36:21.954672+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23261,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26199,"num_bytes_recovered":41572719,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202625024,"data_stored":202505795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051721,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497300,"kb_used_data":132412,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874540,"statfs":{"total":96636764160,"available":96127528960,"internally_reserved":0,"allocated":135589888,"data_stored":135260559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724666,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491704,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880136,"statfs":{"total":96636764160,"available":96133259264,"internally_reserved":0,"allocated":499712,"data_stored":155467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":502984363},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247949,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539388,"kb_used_data":66280,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832452,"statfs":{"total":96636764160,"available":96084430848,"internally_reserved":0,"allocated":67870720,"data_stored":67537314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135147520,"data_stored":135111390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:36:32.077 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:36:32.405 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:32.406 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:36:32.419 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":410,"stamp":"2024-09-17T15:36:31.821096+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23261,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26200,"num_bytes_recovered":41572738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1528392,"kb_used_data":199180,"kb_used_omap":13,"kb_used_meta":1329138,"kb_avail":281587128,"statfs":{"total":289910292480,"available":288345219072,"internally_reserved":0,"allocated":203960320,"data_stored":202953340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1361038335},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1362,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1362,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001739"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20921,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.961364+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:36:21.961364+0000","last_peered":"2024-09-17T15:36:21.961364+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:36:21.961364+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21057,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.956046+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:36:21.956046+0000","last_peered":"2024-09-17T15:36:21.956046+0000","last_clean":"2024-09-17T15:36:21.956046+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:36:21.956046+0000","last_undegraded":"2024-09-17T15:36:21.956046+0000","last_fullsized":"2024-09-17T15:36:21.956046+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":23224,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:28.442089+0000","last_change":"2024-09-17T15:36:20.543951+0000","last_active":"2024-09-17T15:36:28.442089+0000","last_peered":"2024-09-17T15:36:28.442089+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:36:28.442089+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3099,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1070,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3099}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21062,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955609+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:36:21.955609+0000","last_peered":"2024-09-17T15:36:21.955609+0000","last_clean":"2024-09-17T15:36:21.955609+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:36:21.955609+0000","last_undegraded":"2024-09-17T15:36:21.955609+0000","last_fullsized":"2024-09-17T15:36:21.955609+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20805,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959779+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:36:21.959779+0000","last_peered":"2024-09-17T15:36:21.959779+0000","last_clean":"2024-09-17T15:36:21.959779+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:36:21.959779+0000","last_undegraded":"2024-09-17T15:36:21.959779+0000","last_fullsized":"2024-09-17T15:36:21.959779+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21565,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.959782+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:36:21.959782+0000","last_peered":"2024-09-17T15:36:21.959782+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:36:21.959782+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4191},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:22.048419+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:36:22.048419+0000","last_peered":"2024-09-17T15:36:22.048419+0000","last_clean":"2024-09-17T15:36:22.048419+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:36:22.048419+0000","last_undegraded":"2024-09-17T15:36:22.048419+0000","last_fullsized":"2024-09-17T15:36:22.048419+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21067,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959637+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:36:21.959637+0000","last_peered":"2024-09-17T15:36:21.959637+0000","last_clean":"2024-09-17T15:36:21.959637+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:36:21.959637+0000","last_undegraded":"2024-09-17T15:36:21.959637+0000","last_fullsized":"2024-09-17T15:36:21.959637+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954544+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:36:21.954544+0000","last_peered":"2024-09-17T15:36:21.954544+0000","last_clean":"2024-09-17T15:36:21.954544+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:36:21.954544+0000","last_undegraded":"2024-09-17T15:36:21.954544+0000","last_fullsized":"2024-09-17T15:36:21.954544+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955888+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:36:21.955888+0000","last_peered":"2024-09-17T15:36:21.955888+0000","last_clean":"2024-09-17T15:36:21.955888+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:36:21.955888+0000","last_undegraded":"2024-09-17T15:36:21.955888+0000","last_fullsized":"2024-09-17T15:36:21.955888+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955084+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:36:21.955084+0000","last_peered":"2024-09-17T15:36:21.955084+0000","last_clean":"2024-09-17T15:36:21.955084+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:36:21.955084+0000","last_undegraded":"2024-09-17T15:36:21.955084+0000","last_fullsized":"2024-09-17T15:36:21.955084+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20829,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.960010+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:36:21.960010+0000","last_peered":"2024-09-17T15:36:21.960010+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:36:21.960010+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959684+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:36:21.959684+0000","last_peered":"2024-09-17T15:36:21.959684+0000","last_clean":"2024-09-17T15:36:21.959684+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:36:21.959684+0000","last_undegraded":"2024-09-17T15:36:21.959684+0000","last_fullsized":"2024-09-17T15:36:21.959684+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28827,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955167+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:36:21.955167+0000","last_peered":"2024-09-17T15:36:21.955167+0000","last_clean":"2024-09-17T15:36:21.955167+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:36:21.955167+0000","last_undegraded":"2024-09-17T15:36:21.955167+0000","last_fullsized":"2024-09-17T15:36:21.955167+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955801+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:36:21.955801+0000","last_peered":"2024-09-17T15:36:21.955801+0000","last_clean":"2024-09-17T15:36:21.955801+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:36:21.955801+0000","last_undegraded":"2024-09-17T15:36:21.955801+0000","last_fullsized":"2024-09-17T15:36:21.955801+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29164,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954760+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:36:21.954760+0000","last_peered":"2024-09-17T15:36:21.954760+0000","last_clean":"2024-09-17T15:36:21.954760+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:36:21.954760+0000","last_undegraded":"2024-09-17T15:36:21.954760+0000","last_fullsized":"2024-09-17T15:36:21.954760+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21286,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.960250+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:36:21.960250+0000","last_peered":"2024-09-17T15:36:21.960250+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:36:21.960250+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":90,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954762+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:36:21.954762+0000","last_peered":"2024-09-17T15:36:21.954762+0000","last_clean":"2024-09-17T15:36:21.954762+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:36:21.954762+0000","last_undegraded":"2024-09-17T15:36:21.954762+0000","last_fullsized":"2024-09-17T15:36:21.954762+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28078,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955344+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:36:21.955344+0000","last_peered":"2024-09-17T15:36:21.955344+0000","last_clean":"2024-09-17T15:36:21.955344+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:36:21.955344+0000","last_undegraded":"2024-09-17T15:36:21.955344+0000","last_fullsized":"2024-09-17T15:36:21.955344+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955852+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:36:21.955852+0000","last_peered":"2024-09-17T15:36:21.955852+0000","last_clean":"2024-09-17T15:36:21.955852+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:36:21.955852+0000","last_undegraded":"2024-09-17T15:36:21.955852+0000","last_fullsized":"2024-09-17T15:36:21.955852+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21207,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.959971+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:36:21.959971+0000","last_peered":"2024-09-17T15:36:21.959971+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:36:21.959971+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":90,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955263+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:36:21.955263+0000","last_peered":"2024-09-17T15:36:21.955263+0000","last_clean":"2024-09-17T15:36:21.955263+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:36:21.955263+0000","last_undegraded":"2024-09-17T15:36:21.955263+0000","last_fullsized":"2024-09-17T15:36:21.955263+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":95,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.956005+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:36:21.956005+0000","last_peered":"2024-09-17T15:36:21.956005+0000","last_clean":"2024-09-17T15:36:21.956005+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:36:21.956005+0000","last_undegraded":"2024-09-17T15:36:21.956005+0000","last_fullsized":"2024-09-17T15:36:21.956005+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29029,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954672+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:36:21.954672+0000","last_peered":"2024-09-17T15:36:21.954672+0000","last_clean":"2024-09-17T15:36:21.954672+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:36:21.954672+0000","last_undegraded":"2024-09-17T15:36:21.954672+0000","last_fullsized":"2024-09-17T15:36:21.954672+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23261,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26199,"num_bytes_recovered":41572719,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202625024,"data_stored":202505795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051721,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497300,"kb_used_data":132412,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874540,"statfs":{"total":96636764160,"available":96127528960,"internally_reserved":0,"allocated":135589888,"data_stored":135260559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724666,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491704,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880136,"statfs":{"total":96636764160,"available":96133259264,"internally_reserved":0,"allocated":499712,"data_stored":155467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":502984363},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247949,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539388,"kb_used_data":66280,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832452,"statfs":{"total":96636764160,"available":96084430848,"internally_reserved":0,"allocated":67870720,"data_stored":67537314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135147520,"data_stored":135111390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:36:32.421 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:36:32.421 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'20648", 'reported_seq': 20921, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:21.961364+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:36:21.961364+0000', 'last_peered': '2024-09-17T15:36:21.961364+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:36:21.961364+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'10600", 'ondisk_log_start': "43'10600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20648, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:32.421 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:36:32.421 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'20934", 'reported_seq': 23224, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:28.442089+0000', 'last_change': '2024-09-17T15:36:20.543951+0000', 'last_active': '2024-09-17T15:36:28.442089+0000', 'last_peered': '2024-09-17T15:36:28.442089+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:36:28.442089+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "43'10900", 'ondisk_log_start': "43'10900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3099, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 19091, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1070, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3099}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:36:32.421 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:36:32.421 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'21251", 'reported_seq': 21565, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:21.959782+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:36:21.959782+0000', 'last_peered': '2024-09-17T15:36:21.959782+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:36:21.959782+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "43'11200", 'ondisk_log_start': "43'11200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17750, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4191}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:32.421 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:36:32.421 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20542", 'reported_seq': 20829, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:21.960010+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:36:21.960010+0000', 'last_peered': '2024-09-17T15:36:21.960010+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:36:21.960010+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17138, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3943}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:32.422 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:36:32.422 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'20966", 'reported_seq': 21286, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:21.960250+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:36:21.960250+0000', 'last_peered': '2024-09-17T15:36:21.960250+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:36:21.960250+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "43'10900", 'ondisk_log_start': "43'10900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17514, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4057}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:32.422 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:36:32.422 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'20886", 'reported_seq': 21207, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:21.959971+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:36:21.959971+0000', 'last_peered': '2024-09-17T15:36:21.959971+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:36:21.959971+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'10800", 'ondisk_log_start': "43'10800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17673, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3980}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:32.422 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:36:32.829 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:32.829 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 23261/24 objects degraded (96920.833%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":729,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1565073408,"bytes_avail":288345219072,"bytes_total":289910292480,"degraded_objects":23261,"degraded_total":24,"degraded_ratio":969.20833333333348,"recovering_objects_per_sec":136,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1362,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:36:32.842 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:36:32.843 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:36:33.214 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:33.215 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:36:33.229 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":410,"stamp":"2024-09-17T15:36:31.821096+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23261,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26200,"num_bytes_recovered":41572738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1528392,"kb_used_data":199180,"kb_used_omap":13,"kb_used_meta":1329138,"kb_avail":281587128,"statfs":{"total":289910292480,"available":288345219072,"internally_reserved":0,"allocated":203960320,"data_stored":202953340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1361038335},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1362,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1362,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001739"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20921,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.961364+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:36:21.961364+0000","last_peered":"2024-09-17T15:36:21.961364+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:36:21.961364+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21057,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.956046+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:36:21.956046+0000","last_peered":"2024-09-17T15:36:21.956046+0000","last_clean":"2024-09-17T15:36:21.956046+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:36:21.956046+0000","last_undegraded":"2024-09-17T15:36:21.956046+0000","last_fullsized":"2024-09-17T15:36:21.956046+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":23224,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:28.442089+0000","last_change":"2024-09-17T15:36:20.543951+0000","last_active":"2024-09-17T15:36:28.442089+0000","last_peered":"2024-09-17T15:36:28.442089+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:36:28.442089+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3099,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1070,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3099}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21062,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955609+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:36:21.955609+0000","last_peered":"2024-09-17T15:36:21.955609+0000","last_clean":"2024-09-17T15:36:21.955609+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:36:21.955609+0000","last_undegraded":"2024-09-17T15:36:21.955609+0000","last_fullsized":"2024-09-17T15:36:21.955609+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20805,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959779+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:36:21.959779+0000","last_peered":"2024-09-17T15:36:21.959779+0000","last_clean":"2024-09-17T15:36:21.959779+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:36:21.959779+0000","last_undegraded":"2024-09-17T15:36:21.959779+0000","last_fullsized":"2024-09-17T15:36:21.959779+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21565,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.959782+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:36:21.959782+0000","last_peered":"2024-09-17T15:36:21.959782+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:36:21.959782+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4191},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:22.048419+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:36:22.048419+0000","last_peered":"2024-09-17T15:36:22.048419+0000","last_clean":"2024-09-17T15:36:22.048419+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:36:22.048419+0000","last_undegraded":"2024-09-17T15:36:22.048419+0000","last_fullsized":"2024-09-17T15:36:22.048419+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21067,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959637+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:36:21.959637+0000","last_peered":"2024-09-17T15:36:21.959637+0000","last_clean":"2024-09-17T15:36:21.959637+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:36:21.959637+0000","last_undegraded":"2024-09-17T15:36:21.959637+0000","last_fullsized":"2024-09-17T15:36:21.959637+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954544+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:36:21.954544+0000","last_peered":"2024-09-17T15:36:21.954544+0000","last_clean":"2024-09-17T15:36:21.954544+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:36:21.954544+0000","last_undegraded":"2024-09-17T15:36:21.954544+0000","last_fullsized":"2024-09-17T15:36:21.954544+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955888+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:36:21.955888+0000","last_peered":"2024-09-17T15:36:21.955888+0000","last_clean":"2024-09-17T15:36:21.955888+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:36:21.955888+0000","last_undegraded":"2024-09-17T15:36:21.955888+0000","last_fullsized":"2024-09-17T15:36:21.955888+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955084+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:36:21.955084+0000","last_peered":"2024-09-17T15:36:21.955084+0000","last_clean":"2024-09-17T15:36:21.955084+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:36:21.955084+0000","last_undegraded":"2024-09-17T15:36:21.955084+0000","last_fullsized":"2024-09-17T15:36:21.955084+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20829,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.960010+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:36:21.960010+0000","last_peered":"2024-09-17T15:36:21.960010+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:36:21.960010+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959684+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:36:21.959684+0000","last_peered":"2024-09-17T15:36:21.959684+0000","last_clean":"2024-09-17T15:36:21.959684+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:36:21.959684+0000","last_undegraded":"2024-09-17T15:36:21.959684+0000","last_fullsized":"2024-09-17T15:36:21.959684+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28827,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955167+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:36:21.955167+0000","last_peered":"2024-09-17T15:36:21.955167+0000","last_clean":"2024-09-17T15:36:21.955167+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:36:21.955167+0000","last_undegraded":"2024-09-17T15:36:21.955167+0000","last_fullsized":"2024-09-17T15:36:21.955167+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955801+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:36:21.955801+0000","last_peered":"2024-09-17T15:36:21.955801+0000","last_clean":"2024-09-17T15:36:21.955801+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:36:21.955801+0000","last_undegraded":"2024-09-17T15:36:21.955801+0000","last_fullsized":"2024-09-17T15:36:21.955801+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29164,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954760+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:36:21.954760+0000","last_peered":"2024-09-17T15:36:21.954760+0000","last_clean":"2024-09-17T15:36:21.954760+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:36:21.954760+0000","last_undegraded":"2024-09-17T15:36:21.954760+0000","last_fullsized":"2024-09-17T15:36:21.954760+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21286,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.960250+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:36:21.960250+0000","last_peered":"2024-09-17T15:36:21.960250+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:36:21.960250+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":90,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954762+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:36:21.954762+0000","last_peered":"2024-09-17T15:36:21.954762+0000","last_clean":"2024-09-17T15:36:21.954762+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:36:21.954762+0000","last_undegraded":"2024-09-17T15:36:21.954762+0000","last_fullsized":"2024-09-17T15:36:21.954762+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28078,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955344+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:36:21.955344+0000","last_peered":"2024-09-17T15:36:21.955344+0000","last_clean":"2024-09-17T15:36:21.955344+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:36:21.955344+0000","last_undegraded":"2024-09-17T15:36:21.955344+0000","last_fullsized":"2024-09-17T15:36:21.955344+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955852+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:36:21.955852+0000","last_peered":"2024-09-17T15:36:21.955852+0000","last_clean":"2024-09-17T15:36:21.955852+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:36:21.955852+0000","last_undegraded":"2024-09-17T15:36:21.955852+0000","last_fullsized":"2024-09-17T15:36:21.955852+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21207,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.959971+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:36:21.959971+0000","last_peered":"2024-09-17T15:36:21.959971+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:36:21.959971+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":90,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955263+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:36:21.955263+0000","last_peered":"2024-09-17T15:36:21.955263+0000","last_clean":"2024-09-17T15:36:21.955263+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:36:21.955263+0000","last_undegraded":"2024-09-17T15:36:21.955263+0000","last_fullsized":"2024-09-17T15:36:21.955263+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":95,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.956005+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:36:21.956005+0000","last_peered":"2024-09-17T15:36:21.956005+0000","last_clean":"2024-09-17T15:36:21.956005+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:36:21.956005+0000","last_undegraded":"2024-09-17T15:36:21.956005+0000","last_fullsized":"2024-09-17T15:36:21.956005+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29029,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954672+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:36:21.954672+0000","last_peered":"2024-09-17T15:36:21.954672+0000","last_clean":"2024-09-17T15:36:21.954672+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:36:21.954672+0000","last_undegraded":"2024-09-17T15:36:21.954672+0000","last_fullsized":"2024-09-17T15:36:21.954672+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23261,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26199,"num_bytes_recovered":41572719,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202625024,"data_stored":202505795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051721,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497300,"kb_used_data":132412,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874540,"statfs":{"total":96636764160,"available":96127528960,"internally_reserved":0,"allocated":135589888,"data_stored":135260559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724666,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491704,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880136,"statfs":{"total":96636764160,"available":96133259264,"internally_reserved":0,"allocated":499712,"data_stored":155467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":502984363},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247949,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539388,"kb_used_data":66280,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832452,"statfs":{"total":96636764160,"available":96084430848,"internally_reserved":0,"allocated":67870720,"data_stored":67537314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135147520,"data_stored":135111390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:36:36.231 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:36:36.552 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:36.552 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:36:36.565 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":412,"stamp":"2024-09-17T15:36:35.821758+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26812,"num_bytes_recovered":41572738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1515912,"kb_used_data":199180,"kb_used_omap":13,"kb_used_meta":1316658,"kb_avail":281599608,"statfs":{"total":289910292480,"available":288357998592,"internally_reserved":0,"allocated":203960320,"data_stored":202953340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1348258815},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1580,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1580,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002094"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20921,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.961364+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:36:21.961364+0000","last_peered":"2024-09-17T15:36:21.961364+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:36:21.961364+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21057,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.956046+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:36:21.956046+0000","last_peered":"2024-09-17T15:36:21.956046+0000","last_clean":"2024-09-17T15:36:21.956046+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:36:21.956046+0000","last_undegraded":"2024-09-17T15:36:21.956046+0000","last_fullsized":"2024-09-17T15:36:21.956046+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":24448,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:33.445527+0000","last_change":"2024-09-17T15:36:20.543951+0000","last_active":"2024-09-17T15:36:33.445527+0000","last_peered":"2024-09-17T15:36:33.445527+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:36:33.445527+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1682,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2487}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21062,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955609+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:36:21.955609+0000","last_peered":"2024-09-17T15:36:21.955609+0000","last_clean":"2024-09-17T15:36:21.955609+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:36:21.955609+0000","last_undegraded":"2024-09-17T15:36:21.955609+0000","last_fullsized":"2024-09-17T15:36:21.955609+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20805,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959779+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:36:21.959779+0000","last_peered":"2024-09-17T15:36:21.959779+0000","last_clean":"2024-09-17T15:36:21.959779+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:36:21.959779+0000","last_undegraded":"2024-09-17T15:36:21.959779+0000","last_fullsized":"2024-09-17T15:36:21.959779+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21565,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.959782+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:36:21.959782+0000","last_peered":"2024-09-17T15:36:21.959782+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:36:21.959782+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4191},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:22.048419+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:36:22.048419+0000","last_peered":"2024-09-17T15:36:22.048419+0000","last_clean":"2024-09-17T15:36:22.048419+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:36:22.048419+0000","last_undegraded":"2024-09-17T15:36:22.048419+0000","last_fullsized":"2024-09-17T15:36:22.048419+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21067,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959637+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:36:21.959637+0000","last_peered":"2024-09-17T15:36:21.959637+0000","last_clean":"2024-09-17T15:36:21.959637+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:36:21.959637+0000","last_undegraded":"2024-09-17T15:36:21.959637+0000","last_fullsized":"2024-09-17T15:36:21.959637+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954544+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:36:21.954544+0000","last_peered":"2024-09-17T15:36:21.954544+0000","last_clean":"2024-09-17T15:36:21.954544+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:36:21.954544+0000","last_undegraded":"2024-09-17T15:36:21.954544+0000","last_fullsized":"2024-09-17T15:36:21.954544+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955888+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:36:21.955888+0000","last_peered":"2024-09-17T15:36:21.955888+0000","last_clean":"2024-09-17T15:36:21.955888+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:36:21.955888+0000","last_undegraded":"2024-09-17T15:36:21.955888+0000","last_fullsized":"2024-09-17T15:36:21.955888+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955084+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:36:21.955084+0000","last_peered":"2024-09-17T15:36:21.955084+0000","last_clean":"2024-09-17T15:36:21.955084+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:36:21.955084+0000","last_undegraded":"2024-09-17T15:36:21.955084+0000","last_fullsized":"2024-09-17T15:36:21.955084+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20829,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.960010+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:36:21.960010+0000","last_peered":"2024-09-17T15:36:21.960010+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:36:21.960010+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959684+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:36:21.959684+0000","last_peered":"2024-09-17T15:36:21.959684+0000","last_clean":"2024-09-17T15:36:21.959684+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:36:21.959684+0000","last_undegraded":"2024-09-17T15:36:21.959684+0000","last_fullsized":"2024-09-17T15:36:21.959684+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28827,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955167+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:36:21.955167+0000","last_peered":"2024-09-17T15:36:21.955167+0000","last_clean":"2024-09-17T15:36:21.955167+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:36:21.955167+0000","last_undegraded":"2024-09-17T15:36:21.955167+0000","last_fullsized":"2024-09-17T15:36:21.955167+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955801+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:36:21.955801+0000","last_peered":"2024-09-17T15:36:21.955801+0000","last_clean":"2024-09-17T15:36:21.955801+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:36:21.955801+0000","last_undegraded":"2024-09-17T15:36:21.955801+0000","last_fullsized":"2024-09-17T15:36:21.955801+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29164,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954760+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:36:21.954760+0000","last_peered":"2024-09-17T15:36:21.954760+0000","last_clean":"2024-09-17T15:36:21.954760+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:36:21.954760+0000","last_undegraded":"2024-09-17T15:36:21.954760+0000","last_fullsized":"2024-09-17T15:36:21.954760+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21286,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.960250+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:36:21.960250+0000","last_peered":"2024-09-17T15:36:21.960250+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:36:21.960250+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":90,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954762+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:36:21.954762+0000","last_peered":"2024-09-17T15:36:21.954762+0000","last_clean":"2024-09-17T15:36:21.954762+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:36:21.954762+0000","last_undegraded":"2024-09-17T15:36:21.954762+0000","last_fullsized":"2024-09-17T15:36:21.954762+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28078,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955344+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:36:21.955344+0000","last_peered":"2024-09-17T15:36:21.955344+0000","last_clean":"2024-09-17T15:36:21.955344+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:36:21.955344+0000","last_undegraded":"2024-09-17T15:36:21.955344+0000","last_fullsized":"2024-09-17T15:36:21.955344+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955852+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:36:21.955852+0000","last_peered":"2024-09-17T15:36:21.955852+0000","last_clean":"2024-09-17T15:36:21.955852+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:36:21.955852+0000","last_undegraded":"2024-09-17T15:36:21.955852+0000","last_fullsized":"2024-09-17T15:36:21.955852+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21207,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.959971+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:36:21.959971+0000","last_peered":"2024-09-17T15:36:21.959971+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:36:21.959971+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":90,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955263+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:36:21.955263+0000","last_peered":"2024-09-17T15:36:21.955263+0000","last_clean":"2024-09-17T15:36:21.955263+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:36:21.955263+0000","last_undegraded":"2024-09-17T15:36:21.955263+0000","last_fullsized":"2024-09-17T15:36:21.955263+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":95,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.956005+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:36:21.956005+0000","last_peered":"2024-09-17T15:36:21.956005+0000","last_clean":"2024-09-17T15:36:21.956005+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:36:21.956005+0000","last_undegraded":"2024-09-17T15:36:21.956005+0000","last_fullsized":"2024-09-17T15:36:21.956005+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29029,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954672+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:36:21.954672+0000","last_peered":"2024-09-17T15:36:21.954672+0000","last_clean":"2024-09-17T15:36:21.954672+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:36:21.954672+0000","last_undegraded":"2024-09-17T15:36:21.954672+0000","last_fullsized":"2024-09-17T15:36:21.954672+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26811,"num_bytes_recovered":41572719,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202625024,"data_stored":202505795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051722,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497300,"kb_used_data":132412,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874540,"statfs":{"total":96636764160,"available":96127528960,"internally_reserved":0,"allocated":135589888,"data_stored":135260559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724667,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479224,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892616,"statfs":{"total":96636764160,"available":96146038784,"internally_reserved":0,"allocated":499712,"data_stored":155467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":490204843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247950,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539388,"kb_used_data":66280,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832452,"statfs":{"total":96636764160,"available":96084430848,"internally_reserved":0,"allocated":67870720,"data_stored":67537314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135147520,"data_stored":135111390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:36:36.567 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:36:36.864 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:36.864 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:36:36.877 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":412,"stamp":"2024-09-17T15:36:35.821758+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26812,"num_bytes_recovered":41572738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1515912,"kb_used_data":199180,"kb_used_omap":13,"kb_used_meta":1316658,"kb_avail":281599608,"statfs":{"total":289910292480,"available":288357998592,"internally_reserved":0,"allocated":203960320,"data_stored":202953340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1348258815},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1580,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1580,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002094"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20921,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.961364+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:36:21.961364+0000","last_peered":"2024-09-17T15:36:21.961364+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:36:21.961364+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21057,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.956046+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:36:21.956046+0000","last_peered":"2024-09-17T15:36:21.956046+0000","last_clean":"2024-09-17T15:36:21.956046+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:36:21.956046+0000","last_undegraded":"2024-09-17T15:36:21.956046+0000","last_fullsized":"2024-09-17T15:36:21.956046+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":24448,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:33.445527+0000","last_change":"2024-09-17T15:36:20.543951+0000","last_active":"2024-09-17T15:36:33.445527+0000","last_peered":"2024-09-17T15:36:33.445527+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:36:33.445527+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1682,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2487}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21062,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955609+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:36:21.955609+0000","last_peered":"2024-09-17T15:36:21.955609+0000","last_clean":"2024-09-17T15:36:21.955609+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:36:21.955609+0000","last_undegraded":"2024-09-17T15:36:21.955609+0000","last_fullsized":"2024-09-17T15:36:21.955609+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20805,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959779+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:36:21.959779+0000","last_peered":"2024-09-17T15:36:21.959779+0000","last_clean":"2024-09-17T15:36:21.959779+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:36:21.959779+0000","last_undegraded":"2024-09-17T15:36:21.959779+0000","last_fullsized":"2024-09-17T15:36:21.959779+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21565,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.959782+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:36:21.959782+0000","last_peered":"2024-09-17T15:36:21.959782+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:36:21.959782+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4191},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:22.048419+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:36:22.048419+0000","last_peered":"2024-09-17T15:36:22.048419+0000","last_clean":"2024-09-17T15:36:22.048419+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:36:22.048419+0000","last_undegraded":"2024-09-17T15:36:22.048419+0000","last_fullsized":"2024-09-17T15:36:22.048419+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21067,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959637+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:36:21.959637+0000","last_peered":"2024-09-17T15:36:21.959637+0000","last_clean":"2024-09-17T15:36:21.959637+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:36:21.959637+0000","last_undegraded":"2024-09-17T15:36:21.959637+0000","last_fullsized":"2024-09-17T15:36:21.959637+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954544+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:36:21.954544+0000","last_peered":"2024-09-17T15:36:21.954544+0000","last_clean":"2024-09-17T15:36:21.954544+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:36:21.954544+0000","last_undegraded":"2024-09-17T15:36:21.954544+0000","last_fullsized":"2024-09-17T15:36:21.954544+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955888+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:36:21.955888+0000","last_peered":"2024-09-17T15:36:21.955888+0000","last_clean":"2024-09-17T15:36:21.955888+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:36:21.955888+0000","last_undegraded":"2024-09-17T15:36:21.955888+0000","last_fullsized":"2024-09-17T15:36:21.955888+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955084+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:36:21.955084+0000","last_peered":"2024-09-17T15:36:21.955084+0000","last_clean":"2024-09-17T15:36:21.955084+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:36:21.955084+0000","last_undegraded":"2024-09-17T15:36:21.955084+0000","last_fullsized":"2024-09-17T15:36:21.955084+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20829,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.960010+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:36:21.960010+0000","last_peered":"2024-09-17T15:36:21.960010+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:36:21.960010+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959684+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:36:21.959684+0000","last_peered":"2024-09-17T15:36:21.959684+0000","last_clean":"2024-09-17T15:36:21.959684+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:36:21.959684+0000","last_undegraded":"2024-09-17T15:36:21.959684+0000","last_fullsized":"2024-09-17T15:36:21.959684+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28827,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955167+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:36:21.955167+0000","last_peered":"2024-09-17T15:36:21.955167+0000","last_clean":"2024-09-17T15:36:21.955167+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:36:21.955167+0000","last_undegraded":"2024-09-17T15:36:21.955167+0000","last_fullsized":"2024-09-17T15:36:21.955167+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955801+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:36:21.955801+0000","last_peered":"2024-09-17T15:36:21.955801+0000","last_clean":"2024-09-17T15:36:21.955801+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:36:21.955801+0000","last_undegraded":"2024-09-17T15:36:21.955801+0000","last_fullsized":"2024-09-17T15:36:21.955801+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29164,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954760+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:36:21.954760+0000","last_peered":"2024-09-17T15:36:21.954760+0000","last_clean":"2024-09-17T15:36:21.954760+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:36:21.954760+0000","last_undegraded":"2024-09-17T15:36:21.954760+0000","last_fullsized":"2024-09-17T15:36:21.954760+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21286,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.960250+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:36:21.960250+0000","last_peered":"2024-09-17T15:36:21.960250+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:36:21.960250+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":90,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954762+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:36:21.954762+0000","last_peered":"2024-09-17T15:36:21.954762+0000","last_clean":"2024-09-17T15:36:21.954762+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:36:21.954762+0000","last_undegraded":"2024-09-17T15:36:21.954762+0000","last_fullsized":"2024-09-17T15:36:21.954762+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28078,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955344+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:36:21.955344+0000","last_peered":"2024-09-17T15:36:21.955344+0000","last_clean":"2024-09-17T15:36:21.955344+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:36:21.955344+0000","last_undegraded":"2024-09-17T15:36:21.955344+0000","last_fullsized":"2024-09-17T15:36:21.955344+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955852+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:36:21.955852+0000","last_peered":"2024-09-17T15:36:21.955852+0000","last_clean":"2024-09-17T15:36:21.955852+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:36:21.955852+0000","last_undegraded":"2024-09-17T15:36:21.955852+0000","last_fullsized":"2024-09-17T15:36:21.955852+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21207,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.959971+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:36:21.959971+0000","last_peered":"2024-09-17T15:36:21.959971+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:36:21.959971+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":90,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955263+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:36:21.955263+0000","last_peered":"2024-09-17T15:36:21.955263+0000","last_clean":"2024-09-17T15:36:21.955263+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:36:21.955263+0000","last_undegraded":"2024-09-17T15:36:21.955263+0000","last_fullsized":"2024-09-17T15:36:21.955263+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":95,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.956005+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:36:21.956005+0000","last_peered":"2024-09-17T15:36:21.956005+0000","last_clean":"2024-09-17T15:36:21.956005+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:36:21.956005+0000","last_undegraded":"2024-09-17T15:36:21.956005+0000","last_fullsized":"2024-09-17T15:36:21.956005+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29029,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954672+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:36:21.954672+0000","last_peered":"2024-09-17T15:36:21.954672+0000","last_clean":"2024-09-17T15:36:21.954672+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:36:21.954672+0000","last_undegraded":"2024-09-17T15:36:21.954672+0000","last_fullsized":"2024-09-17T15:36:21.954672+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26811,"num_bytes_recovered":41572719,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202625024,"data_stored":202505795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051722,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497300,"kb_used_data":132412,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874540,"statfs":{"total":96636764160,"available":96127528960,"internally_reserved":0,"allocated":135589888,"data_stored":135260559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724667,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479224,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892616,"statfs":{"total":96636764160,"available":96146038784,"internally_reserved":0,"allocated":499712,"data_stored":155467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":490204843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247950,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539388,"kb_used_data":66280,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832452,"statfs":{"total":96636764160,"available":96084430848,"internally_reserved":0,"allocated":67870720,"data_stored":67537314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135147520,"data_stored":135111390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:36:36.878 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:36:36.878 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'20648", 'reported_seq': 20921, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:21.961364+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:36:21.961364+0000', 'last_peered': '2024-09-17T15:36:21.961364+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:36:21.961364+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'10600", 'ondisk_log_start': "43'10600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20648, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:36.878 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:36:36.878 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'20934", 'reported_seq': 24448, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:33.445527+0000', 'last_change': '2024-09-17T15:36:20.543951+0000', 'last_active': '2024-09-17T15:36:33.445527+0000', 'last_peered': '2024-09-17T15:36:33.445527+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:36:33.445527+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "43'10900", 'ondisk_log_start': "43'10900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2487, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 19091, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1682, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2487}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:36:36.878 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:36:36.879 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'21251", 'reported_seq': 21565, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:21.959782+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:36:21.959782+0000', 'last_peered': '2024-09-17T15:36:21.959782+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:36:21.959782+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "43'11200", 'ondisk_log_start': "43'11200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17750, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4191}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:36.879 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:36:36.879 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20542", 'reported_seq': 20829, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:21.960010+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:36:21.960010+0000', 'last_peered': '2024-09-17T15:36:21.960010+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:36:21.960010+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17138, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3943}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:36.879 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:36:36.879 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'20966", 'reported_seq': 21286, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:21.960250+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:36:21.960250+0000', 'last_peered': '2024-09-17T15:36:21.960250+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:36:21.960250+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "43'10900", 'ondisk_log_start': "43'10900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17514, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4057}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:36.879 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:36:36.879 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'20886", 'reported_seq': 21207, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:21.959971+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:36:21.959971+0000', 'last_peered': '2024-09-17T15:36:21.959971+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:36:21.959971+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'10800", 'ondisk_log_start': "43'10800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17673, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3980}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:36.879 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:36:37.253 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:37.253 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 22649/24 objects degraded (94370.833%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":734,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1552293888,"bytes_avail":288357998592,"bytes_total":289910292480,"degraded_objects":22649,"degraded_total":24,"degraded_ratio":943.70833333333348,"recovering_objects_per_sec":131,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1580,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:36:37.266 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:36:37.266 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:36:37.611 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:37.611 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:36:37.624 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":412,"stamp":"2024-09-17T15:36:35.821758+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26812,"num_bytes_recovered":41572738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1515912,"kb_used_data":199180,"kb_used_omap":13,"kb_used_meta":1316658,"kb_avail":281599608,"statfs":{"total":289910292480,"available":288357998592,"internally_reserved":0,"allocated":203960320,"data_stored":202953340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1348258815},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1580,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1580,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002094"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20921,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.961364+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:36:21.961364+0000","last_peered":"2024-09-17T15:36:21.961364+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:36:21.961364+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21057,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.956046+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:36:21.956046+0000","last_peered":"2024-09-17T15:36:21.956046+0000","last_clean":"2024-09-17T15:36:21.956046+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:36:21.956046+0000","last_undegraded":"2024-09-17T15:36:21.956046+0000","last_fullsized":"2024-09-17T15:36:21.956046+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":24448,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:33.445527+0000","last_change":"2024-09-17T15:36:20.543951+0000","last_active":"2024-09-17T15:36:33.445527+0000","last_peered":"2024-09-17T15:36:33.445527+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:36:33.445527+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1682,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2487}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21062,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955609+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:36:21.955609+0000","last_peered":"2024-09-17T15:36:21.955609+0000","last_clean":"2024-09-17T15:36:21.955609+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:36:21.955609+0000","last_undegraded":"2024-09-17T15:36:21.955609+0000","last_fullsized":"2024-09-17T15:36:21.955609+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20805,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959779+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:36:21.959779+0000","last_peered":"2024-09-17T15:36:21.959779+0000","last_clean":"2024-09-17T15:36:21.959779+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:36:21.959779+0000","last_undegraded":"2024-09-17T15:36:21.959779+0000","last_fullsized":"2024-09-17T15:36:21.959779+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21565,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.959782+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:36:21.959782+0000","last_peered":"2024-09-17T15:36:21.959782+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:36:21.959782+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4191},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:22.048419+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:36:22.048419+0000","last_peered":"2024-09-17T15:36:22.048419+0000","last_clean":"2024-09-17T15:36:22.048419+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:36:22.048419+0000","last_undegraded":"2024-09-17T15:36:22.048419+0000","last_fullsized":"2024-09-17T15:36:22.048419+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21067,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959637+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:36:21.959637+0000","last_peered":"2024-09-17T15:36:21.959637+0000","last_clean":"2024-09-17T15:36:21.959637+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:36:21.959637+0000","last_undegraded":"2024-09-17T15:36:21.959637+0000","last_fullsized":"2024-09-17T15:36:21.959637+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954544+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:36:21.954544+0000","last_peered":"2024-09-17T15:36:21.954544+0000","last_clean":"2024-09-17T15:36:21.954544+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:36:21.954544+0000","last_undegraded":"2024-09-17T15:36:21.954544+0000","last_fullsized":"2024-09-17T15:36:21.954544+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955888+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:36:21.955888+0000","last_peered":"2024-09-17T15:36:21.955888+0000","last_clean":"2024-09-17T15:36:21.955888+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:36:21.955888+0000","last_undegraded":"2024-09-17T15:36:21.955888+0000","last_fullsized":"2024-09-17T15:36:21.955888+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955084+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:36:21.955084+0000","last_peered":"2024-09-17T15:36:21.955084+0000","last_clean":"2024-09-17T15:36:21.955084+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:36:21.955084+0000","last_undegraded":"2024-09-17T15:36:21.955084+0000","last_fullsized":"2024-09-17T15:36:21.955084+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20829,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.960010+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:36:21.960010+0000","last_peered":"2024-09-17T15:36:21.960010+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:36:21.960010+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959684+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:36:21.959684+0000","last_peered":"2024-09-17T15:36:21.959684+0000","last_clean":"2024-09-17T15:36:21.959684+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:36:21.959684+0000","last_undegraded":"2024-09-17T15:36:21.959684+0000","last_fullsized":"2024-09-17T15:36:21.959684+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28827,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955167+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:36:21.955167+0000","last_peered":"2024-09-17T15:36:21.955167+0000","last_clean":"2024-09-17T15:36:21.955167+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:36:21.955167+0000","last_undegraded":"2024-09-17T15:36:21.955167+0000","last_fullsized":"2024-09-17T15:36:21.955167+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955801+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:36:21.955801+0000","last_peered":"2024-09-17T15:36:21.955801+0000","last_clean":"2024-09-17T15:36:21.955801+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:36:21.955801+0000","last_undegraded":"2024-09-17T15:36:21.955801+0000","last_fullsized":"2024-09-17T15:36:21.955801+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29164,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954760+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:36:21.954760+0000","last_peered":"2024-09-17T15:36:21.954760+0000","last_clean":"2024-09-17T15:36:21.954760+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:36:21.954760+0000","last_undegraded":"2024-09-17T15:36:21.954760+0000","last_fullsized":"2024-09-17T15:36:21.954760+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21286,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.960250+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:36:21.960250+0000","last_peered":"2024-09-17T15:36:21.960250+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:36:21.960250+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":90,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954762+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:36:21.954762+0000","last_peered":"2024-09-17T15:36:21.954762+0000","last_clean":"2024-09-17T15:36:21.954762+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:36:21.954762+0000","last_undegraded":"2024-09-17T15:36:21.954762+0000","last_fullsized":"2024-09-17T15:36:21.954762+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28078,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955344+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:36:21.955344+0000","last_peered":"2024-09-17T15:36:21.955344+0000","last_clean":"2024-09-17T15:36:21.955344+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:36:21.955344+0000","last_undegraded":"2024-09-17T15:36:21.955344+0000","last_fullsized":"2024-09-17T15:36:21.955344+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955852+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:36:21.955852+0000","last_peered":"2024-09-17T15:36:21.955852+0000","last_clean":"2024-09-17T15:36:21.955852+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:36:21.955852+0000","last_undegraded":"2024-09-17T15:36:21.955852+0000","last_fullsized":"2024-09-17T15:36:21.955852+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21207,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.959971+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:36:21.959971+0000","last_peered":"2024-09-17T15:36:21.959971+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:36:21.959971+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":90,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955263+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:36:21.955263+0000","last_peered":"2024-09-17T15:36:21.955263+0000","last_clean":"2024-09-17T15:36:21.955263+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:36:21.955263+0000","last_undegraded":"2024-09-17T15:36:21.955263+0000","last_fullsized":"2024-09-17T15:36:21.955263+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":95,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.956005+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:36:21.956005+0000","last_peered":"2024-09-17T15:36:21.956005+0000","last_clean":"2024-09-17T15:36:21.956005+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:36:21.956005+0000","last_undegraded":"2024-09-17T15:36:21.956005+0000","last_fullsized":"2024-09-17T15:36:21.956005+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29029,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954672+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:36:21.954672+0000","last_peered":"2024-09-17T15:36:21.954672+0000","last_clean":"2024-09-17T15:36:21.954672+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:36:21.954672+0000","last_undegraded":"2024-09-17T15:36:21.954672+0000","last_fullsized":"2024-09-17T15:36:21.954672+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26811,"num_bytes_recovered":41572719,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202625024,"data_stored":202505795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051722,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497300,"kb_used_data":132412,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874540,"statfs":{"total":96636764160,"available":96127528960,"internally_reserved":0,"allocated":135589888,"data_stored":135260559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724667,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479224,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892616,"statfs":{"total":96636764160,"available":96146038784,"internally_reserved":0,"allocated":499712,"data_stored":155467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":490204843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247950,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539388,"kb_used_data":66280,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832452,"statfs":{"total":96636764160,"available":96084430848,"internally_reserved":0,"allocated":67870720,"data_stored":67537314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135147520,"data_stored":135111390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:36:40.627 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:36:40.963 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:40.963 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:36:40.975 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":414,"stamp":"2024-09-17T15:36:39.822416+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21914,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27547,"num_bytes_recovered":41572738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1524232,"kb_used_data":199180,"kb_used_omap":13,"kb_used_meta":1324978,"kb_avail":281591288,"statfs":{"total":289910292480,"available":288349478912,"internally_reserved":0,"allocated":203960320,"data_stored":202953340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1356778495},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1980,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1980,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001985"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20921,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.961364+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:36:21.961364+0000","last_peered":"2024-09-17T15:36:21.961364+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:36:21.961364+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21058,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.956094+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:36:21.956094+0000","last_peered":"2024-09-17T15:36:21.956094+0000","last_clean":"2024-09-17T15:36:21.956094+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:36:21.956094+0000","last_undegraded":"2024-09-17T15:36:21.956094+0000","last_fullsized":"2024-09-17T15:36:21.956094+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":25918,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:38.447960+0000","last_change":"2024-09-17T15:36:20.543951+0000","last_active":"2024-09-17T15:36:38.447960+0000","last_peered":"2024-09-17T15:36:38.447960+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:36:38.447960+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1752,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2417,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1752}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21063,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955651+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:36:21.955651+0000","last_peered":"2024-09-17T15:36:21.955651+0000","last_clean":"2024-09-17T15:36:21.955651+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:36:21.955651+0000","last_undegraded":"2024-09-17T15:36:21.955651+0000","last_fullsized":"2024-09-17T15:36:21.955651+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20805,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959779+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:36:21.959779+0000","last_peered":"2024-09-17T15:36:21.959779+0000","last_clean":"2024-09-17T15:36:21.959779+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:36:21.959779+0000","last_undegraded":"2024-09-17T15:36:21.959779+0000","last_fullsized":"2024-09-17T15:36:21.959779+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21565,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.959782+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:36:21.959782+0000","last_peered":"2024-09-17T15:36:21.959782+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:36:21.959782+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4191},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:22.048419+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:36:22.048419+0000","last_peered":"2024-09-17T15:36:22.048419+0000","last_clean":"2024-09-17T15:36:22.048419+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:36:22.048419+0000","last_undegraded":"2024-09-17T15:36:22.048419+0000","last_fullsized":"2024-09-17T15:36:22.048419+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21067,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959637+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:36:21.959637+0000","last_peered":"2024-09-17T15:36:21.959637+0000","last_clean":"2024-09-17T15:36:21.959637+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:36:21.959637+0000","last_undegraded":"2024-09-17T15:36:21.959637+0000","last_fullsized":"2024-09-17T15:36:21.959637+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954544+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:36:21.954544+0000","last_peered":"2024-09-17T15:36:21.954544+0000","last_clean":"2024-09-17T15:36:21.954544+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:36:21.954544+0000","last_undegraded":"2024-09-17T15:36:21.954544+0000","last_fullsized":"2024-09-17T15:36:21.954544+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955888+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:36:21.955888+0000","last_peered":"2024-09-17T15:36:21.955888+0000","last_clean":"2024-09-17T15:36:21.955888+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:36:21.955888+0000","last_undegraded":"2024-09-17T15:36:21.955888+0000","last_fullsized":"2024-09-17T15:36:21.955888+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955084+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:36:21.955084+0000","last_peered":"2024-09-17T15:36:21.955084+0000","last_clean":"2024-09-17T15:36:21.955084+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:36:21.955084+0000","last_undegraded":"2024-09-17T15:36:21.955084+0000","last_fullsized":"2024-09-17T15:36:21.955084+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20830,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.961042+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:36:21.961042+0000","last_peered":"2024-09-17T15:36:21.961042+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:36:21.961042+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959684+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:36:21.959684+0000","last_peered":"2024-09-17T15:36:21.959684+0000","last_clean":"2024-09-17T15:36:21.959684+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:36:21.959684+0000","last_undegraded":"2024-09-17T15:36:21.959684+0000","last_fullsized":"2024-09-17T15:36:21.959684+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28827,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955167+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:36:21.955167+0000","last_peered":"2024-09-17T15:36:21.955167+0000","last_clean":"2024-09-17T15:36:21.955167+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:36:21.955167+0000","last_undegraded":"2024-09-17T15:36:21.955167+0000","last_fullsized":"2024-09-17T15:36:21.955167+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955801+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:36:21.955801+0000","last_peered":"2024-09-17T15:36:21.955801+0000","last_clean":"2024-09-17T15:36:21.955801+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:36:21.955801+0000","last_undegraded":"2024-09-17T15:36:21.955801+0000","last_fullsized":"2024-09-17T15:36:21.955801+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29164,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954760+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:36:21.954760+0000","last_peered":"2024-09-17T15:36:21.954760+0000","last_clean":"2024-09-17T15:36:21.954760+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:36:21.954760+0000","last_undegraded":"2024-09-17T15:36:21.954760+0000","last_fullsized":"2024-09-17T15:36:21.954760+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21286,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.960250+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:36:21.960250+0000","last_peered":"2024-09-17T15:36:21.960250+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:36:21.960250+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":90,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954762+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:36:21.954762+0000","last_peered":"2024-09-17T15:36:21.954762+0000","last_clean":"2024-09-17T15:36:21.954762+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:36:21.954762+0000","last_undegraded":"2024-09-17T15:36:21.954762+0000","last_fullsized":"2024-09-17T15:36:21.954762+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28078,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955344+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:36:21.955344+0000","last_peered":"2024-09-17T15:36:21.955344+0000","last_clean":"2024-09-17T15:36:21.955344+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:36:21.955344+0000","last_undegraded":"2024-09-17T15:36:21.955344+0000","last_fullsized":"2024-09-17T15:36:21.955344+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955852+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:36:21.955852+0000","last_peered":"2024-09-17T15:36:21.955852+0000","last_clean":"2024-09-17T15:36:21.955852+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:36:21.955852+0000","last_undegraded":"2024-09-17T15:36:21.955852+0000","last_fullsized":"2024-09-17T15:36:21.955852+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21207,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.959971+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:36:21.959971+0000","last_peered":"2024-09-17T15:36:21.959971+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:36:21.959971+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":90,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955263+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:36:21.955263+0000","last_peered":"2024-09-17T15:36:21.955263+0000","last_clean":"2024-09-17T15:36:21.955263+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:36:21.955263+0000","last_undegraded":"2024-09-17T15:36:21.955263+0000","last_fullsized":"2024-09-17T15:36:21.955263+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":95,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.956005+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:36:21.956005+0000","last_peered":"2024-09-17T15:36:21.956005+0000","last_clean":"2024-09-17T15:36:21.956005+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:36:21.956005+0000","last_undegraded":"2024-09-17T15:36:21.956005+0000","last_fullsized":"2024-09-17T15:36:21.956005+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29029,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954672+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:36:21.954672+0000","last_peered":"2024-09-17T15:36:21.954672+0000","last_clean":"2024-09-17T15:36:21.954672+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:36:21.954672+0000","last_undegraded":"2024-09-17T15:36:21.954672+0000","last_fullsized":"2024-09-17T15:36:21.954672+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21914,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27546,"num_bytes_recovered":41572719,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202625024,"data_stored":202505795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051723,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497300,"kb_used_data":132412,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874540,"statfs":{"total":96636764160,"available":96127528960,"internally_reserved":0,"allocated":135589888,"data_stored":135260559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724668,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487544,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884296,"statfs":{"total":96636764160,"available":96137519104,"internally_reserved":0,"allocated":499712,"data_stored":155467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":498724523},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247951,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539388,"kb_used_data":66280,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832452,"statfs":{"total":96636764160,"available":96084430848,"internally_reserved":0,"allocated":67870720,"data_stored":67537314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135147520,"data_stored":135111390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:36:40.976 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:36:41.279 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:41.280 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:36:41.292 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":414,"stamp":"2024-09-17T15:36:39.822416+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21914,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27547,"num_bytes_recovered":41572738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1524232,"kb_used_data":199180,"kb_used_omap":13,"kb_used_meta":1324978,"kb_avail":281591288,"statfs":{"total":289910292480,"available":288349478912,"internally_reserved":0,"allocated":203960320,"data_stored":202953340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1356778495},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1980,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1980,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001985"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20921,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.961364+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:36:21.961364+0000","last_peered":"2024-09-17T15:36:21.961364+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:36:21.961364+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21058,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.956094+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:36:21.956094+0000","last_peered":"2024-09-17T15:36:21.956094+0000","last_clean":"2024-09-17T15:36:21.956094+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:36:21.956094+0000","last_undegraded":"2024-09-17T15:36:21.956094+0000","last_fullsized":"2024-09-17T15:36:21.956094+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":25918,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:38.447960+0000","last_change":"2024-09-17T15:36:20.543951+0000","last_active":"2024-09-17T15:36:38.447960+0000","last_peered":"2024-09-17T15:36:38.447960+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:36:38.447960+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1752,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2417,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1752}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21063,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955651+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:36:21.955651+0000","last_peered":"2024-09-17T15:36:21.955651+0000","last_clean":"2024-09-17T15:36:21.955651+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:36:21.955651+0000","last_undegraded":"2024-09-17T15:36:21.955651+0000","last_fullsized":"2024-09-17T15:36:21.955651+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20805,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959779+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:36:21.959779+0000","last_peered":"2024-09-17T15:36:21.959779+0000","last_clean":"2024-09-17T15:36:21.959779+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:36:21.959779+0000","last_undegraded":"2024-09-17T15:36:21.959779+0000","last_fullsized":"2024-09-17T15:36:21.959779+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21565,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.959782+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:36:21.959782+0000","last_peered":"2024-09-17T15:36:21.959782+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:36:21.959782+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4191},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:22.048419+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:36:22.048419+0000","last_peered":"2024-09-17T15:36:22.048419+0000","last_clean":"2024-09-17T15:36:22.048419+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:36:22.048419+0000","last_undegraded":"2024-09-17T15:36:22.048419+0000","last_fullsized":"2024-09-17T15:36:22.048419+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21067,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959637+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:36:21.959637+0000","last_peered":"2024-09-17T15:36:21.959637+0000","last_clean":"2024-09-17T15:36:21.959637+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:36:21.959637+0000","last_undegraded":"2024-09-17T15:36:21.959637+0000","last_fullsized":"2024-09-17T15:36:21.959637+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954544+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:36:21.954544+0000","last_peered":"2024-09-17T15:36:21.954544+0000","last_clean":"2024-09-17T15:36:21.954544+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:36:21.954544+0000","last_undegraded":"2024-09-17T15:36:21.954544+0000","last_fullsized":"2024-09-17T15:36:21.954544+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955888+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:36:21.955888+0000","last_peered":"2024-09-17T15:36:21.955888+0000","last_clean":"2024-09-17T15:36:21.955888+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:36:21.955888+0000","last_undegraded":"2024-09-17T15:36:21.955888+0000","last_fullsized":"2024-09-17T15:36:21.955888+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955084+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:36:21.955084+0000","last_peered":"2024-09-17T15:36:21.955084+0000","last_clean":"2024-09-17T15:36:21.955084+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:36:21.955084+0000","last_undegraded":"2024-09-17T15:36:21.955084+0000","last_fullsized":"2024-09-17T15:36:21.955084+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20830,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.961042+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:36:21.961042+0000","last_peered":"2024-09-17T15:36:21.961042+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:36:21.961042+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959684+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:36:21.959684+0000","last_peered":"2024-09-17T15:36:21.959684+0000","last_clean":"2024-09-17T15:36:21.959684+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:36:21.959684+0000","last_undegraded":"2024-09-17T15:36:21.959684+0000","last_fullsized":"2024-09-17T15:36:21.959684+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28827,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955167+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:36:21.955167+0000","last_peered":"2024-09-17T15:36:21.955167+0000","last_clean":"2024-09-17T15:36:21.955167+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:36:21.955167+0000","last_undegraded":"2024-09-17T15:36:21.955167+0000","last_fullsized":"2024-09-17T15:36:21.955167+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955801+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:36:21.955801+0000","last_peered":"2024-09-17T15:36:21.955801+0000","last_clean":"2024-09-17T15:36:21.955801+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:36:21.955801+0000","last_undegraded":"2024-09-17T15:36:21.955801+0000","last_fullsized":"2024-09-17T15:36:21.955801+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29164,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954760+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:36:21.954760+0000","last_peered":"2024-09-17T15:36:21.954760+0000","last_clean":"2024-09-17T15:36:21.954760+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:36:21.954760+0000","last_undegraded":"2024-09-17T15:36:21.954760+0000","last_fullsized":"2024-09-17T15:36:21.954760+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21286,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.960250+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:36:21.960250+0000","last_peered":"2024-09-17T15:36:21.960250+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:36:21.960250+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":90,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954762+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:36:21.954762+0000","last_peered":"2024-09-17T15:36:21.954762+0000","last_clean":"2024-09-17T15:36:21.954762+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:36:21.954762+0000","last_undegraded":"2024-09-17T15:36:21.954762+0000","last_fullsized":"2024-09-17T15:36:21.954762+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28078,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955344+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:36:21.955344+0000","last_peered":"2024-09-17T15:36:21.955344+0000","last_clean":"2024-09-17T15:36:21.955344+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:36:21.955344+0000","last_undegraded":"2024-09-17T15:36:21.955344+0000","last_fullsized":"2024-09-17T15:36:21.955344+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955852+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:36:21.955852+0000","last_peered":"2024-09-17T15:36:21.955852+0000","last_clean":"2024-09-17T15:36:21.955852+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:36:21.955852+0000","last_undegraded":"2024-09-17T15:36:21.955852+0000","last_fullsized":"2024-09-17T15:36:21.955852+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21207,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.959971+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:36:21.959971+0000","last_peered":"2024-09-17T15:36:21.959971+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:36:21.959971+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":90,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955263+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:36:21.955263+0000","last_peered":"2024-09-17T15:36:21.955263+0000","last_clean":"2024-09-17T15:36:21.955263+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:36:21.955263+0000","last_undegraded":"2024-09-17T15:36:21.955263+0000","last_fullsized":"2024-09-17T15:36:21.955263+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":95,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.956005+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:36:21.956005+0000","last_peered":"2024-09-17T15:36:21.956005+0000","last_clean":"2024-09-17T15:36:21.956005+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:36:21.956005+0000","last_undegraded":"2024-09-17T15:36:21.956005+0000","last_fullsized":"2024-09-17T15:36:21.956005+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29029,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954672+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:36:21.954672+0000","last_peered":"2024-09-17T15:36:21.954672+0000","last_clean":"2024-09-17T15:36:21.954672+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:36:21.954672+0000","last_undegraded":"2024-09-17T15:36:21.954672+0000","last_fullsized":"2024-09-17T15:36:21.954672+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21914,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27546,"num_bytes_recovered":41572719,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202625024,"data_stored":202505795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051723,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497300,"kb_used_data":132412,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874540,"statfs":{"total":96636764160,"available":96127528960,"internally_reserved":0,"allocated":135589888,"data_stored":135260559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724668,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487544,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884296,"statfs":{"total":96636764160,"available":96137519104,"internally_reserved":0,"allocated":499712,"data_stored":155467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":498724523},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247951,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539388,"kb_used_data":66280,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832452,"statfs":{"total":96636764160,"available":96084430848,"internally_reserved":0,"allocated":67870720,"data_stored":67537314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135147520,"data_stored":135111390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:36:41.293 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:36:41.293 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'20648", 'reported_seq': 20921, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:21.961364+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:36:21.961364+0000', 'last_peered': '2024-09-17T15:36:21.961364+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:36:21.961364+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'10600", 'ondisk_log_start': "43'10600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20648, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:41.294 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:36:41.294 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'20934", 'reported_seq': 25918, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:38.447960+0000', 'last_change': '2024-09-17T15:36:20.543951+0000', 'last_active': '2024-09-17T15:36:38.447960+0000', 'last_peered': '2024-09-17T15:36:38.447960+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:36:38.447960+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "43'10900", 'ondisk_log_start': "43'10900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1752, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 19091, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2417, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1752}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:36:41.294 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:36:41.294 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'21251", 'reported_seq': 21565, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:21.959782+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:36:21.959782+0000', 'last_peered': '2024-09-17T15:36:21.959782+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:36:21.959782+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "43'11200", 'ondisk_log_start': "43'11200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17750, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4191}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:41.294 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:36:41.294 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20542", 'reported_seq': 20830, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:21.961042+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:36:21.961042+0000', 'last_peered': '2024-09-17T15:36:21.961042+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:36:21.961042+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17138, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3943}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:41.294 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:36:41.295 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'20966", 'reported_seq': 21286, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:21.960250+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:36:21.960250+0000', 'last_peered': '2024-09-17T15:36:21.960250+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:36:21.960250+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "43'10900", 'ondisk_log_start': "43'10900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17514, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4057}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:41.295 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:36:41.295 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'20886", 'reported_seq': 21207, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:21.959971+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:36:21.959971+0000', 'last_peered': '2024-09-17T15:36:21.959971+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:36:21.959971+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'10800", 'ondisk_log_start': "43'10800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17673, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3980}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:41.295 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:36:41.680 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:41.680 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 21914/24 objects degraded (91308.333%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":738,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1560813568,"bytes_avail":288349478912,"bytes_total":289910292480,"degraded_objects":21914,"degraded_total":24,"degraded_ratio":913.08333333333348,"recovering_objects_per_sec":164,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1980,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:36:41.693 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:36:41.694 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:36:42.018 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:42.018 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:36:42.031 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":415,"stamp":"2024-09-17T15:36:41.822768+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21914,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27547,"num_bytes_recovered":41572738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1524232,"kb_used_data":199180,"kb_used_omap":13,"kb_used_meta":1324978,"kb_avail":281591288,"statfs":{"total":289910292480,"available":288349478912,"internally_reserved":0,"allocated":203960320,"data_stored":202953340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1356778495},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1347,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1347,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002025"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20921,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.961364+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:36:21.961364+0000","last_peered":"2024-09-17T15:36:21.961364+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:36:21.961364+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21058,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.956094+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:36:21.956094+0000","last_peered":"2024-09-17T15:36:21.956094+0000","last_clean":"2024-09-17T15:36:21.956094+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:36:21.956094+0000","last_undegraded":"2024-09-17T15:36:21.956094+0000","last_fullsized":"2024-09-17T15:36:21.956094+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":25918,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:38.447960+0000","last_change":"2024-09-17T15:36:20.543951+0000","last_active":"2024-09-17T15:36:38.447960+0000","last_peered":"2024-09-17T15:36:38.447960+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:36:38.447960+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1752,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2417,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1752}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21063,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955651+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:36:21.955651+0000","last_peered":"2024-09-17T15:36:21.955651+0000","last_clean":"2024-09-17T15:36:21.955651+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:36:21.955651+0000","last_undegraded":"2024-09-17T15:36:21.955651+0000","last_fullsized":"2024-09-17T15:36:21.955651+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20805,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959779+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:36:21.959779+0000","last_peered":"2024-09-17T15:36:21.959779+0000","last_clean":"2024-09-17T15:36:21.959779+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:36:21.959779+0000","last_undegraded":"2024-09-17T15:36:21.959779+0000","last_fullsized":"2024-09-17T15:36:21.959779+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21565,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.959782+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:36:21.959782+0000","last_peered":"2024-09-17T15:36:21.959782+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:36:21.959782+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4191},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:22.048419+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:36:22.048419+0000","last_peered":"2024-09-17T15:36:22.048419+0000","last_clean":"2024-09-17T15:36:22.048419+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:36:22.048419+0000","last_undegraded":"2024-09-17T15:36:22.048419+0000","last_fullsized":"2024-09-17T15:36:22.048419+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21067,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959637+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:36:21.959637+0000","last_peered":"2024-09-17T15:36:21.959637+0000","last_clean":"2024-09-17T15:36:21.959637+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:36:21.959637+0000","last_undegraded":"2024-09-17T15:36:21.959637+0000","last_fullsized":"2024-09-17T15:36:21.959637+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954544+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:36:21.954544+0000","last_peered":"2024-09-17T15:36:21.954544+0000","last_clean":"2024-09-17T15:36:21.954544+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:36:21.954544+0000","last_undegraded":"2024-09-17T15:36:21.954544+0000","last_fullsized":"2024-09-17T15:36:21.954544+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955888+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:36:21.955888+0000","last_peered":"2024-09-17T15:36:21.955888+0000","last_clean":"2024-09-17T15:36:21.955888+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:36:21.955888+0000","last_undegraded":"2024-09-17T15:36:21.955888+0000","last_fullsized":"2024-09-17T15:36:21.955888+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955084+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:36:21.955084+0000","last_peered":"2024-09-17T15:36:21.955084+0000","last_clean":"2024-09-17T15:36:21.955084+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:36:21.955084+0000","last_undegraded":"2024-09-17T15:36:21.955084+0000","last_fullsized":"2024-09-17T15:36:21.955084+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20830,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.961042+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:36:21.961042+0000","last_peered":"2024-09-17T15:36:21.961042+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:36:21.961042+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959684+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:36:21.959684+0000","last_peered":"2024-09-17T15:36:21.959684+0000","last_clean":"2024-09-17T15:36:21.959684+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:36:21.959684+0000","last_undegraded":"2024-09-17T15:36:21.959684+0000","last_fullsized":"2024-09-17T15:36:21.959684+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28827,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955167+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:36:21.955167+0000","last_peered":"2024-09-17T15:36:21.955167+0000","last_clean":"2024-09-17T15:36:21.955167+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:36:21.955167+0000","last_undegraded":"2024-09-17T15:36:21.955167+0000","last_fullsized":"2024-09-17T15:36:21.955167+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955801+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:36:21.955801+0000","last_peered":"2024-09-17T15:36:21.955801+0000","last_clean":"2024-09-17T15:36:21.955801+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:36:21.955801+0000","last_undegraded":"2024-09-17T15:36:21.955801+0000","last_fullsized":"2024-09-17T15:36:21.955801+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29164,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954760+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:36:21.954760+0000","last_peered":"2024-09-17T15:36:21.954760+0000","last_clean":"2024-09-17T15:36:21.954760+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:36:21.954760+0000","last_undegraded":"2024-09-17T15:36:21.954760+0000","last_fullsized":"2024-09-17T15:36:21.954760+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21286,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.960250+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:36:21.960250+0000","last_peered":"2024-09-17T15:36:21.960250+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:36:21.960250+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":90,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954762+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:36:21.954762+0000","last_peered":"2024-09-17T15:36:21.954762+0000","last_clean":"2024-09-17T15:36:21.954762+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:36:21.954762+0000","last_undegraded":"2024-09-17T15:36:21.954762+0000","last_fullsized":"2024-09-17T15:36:21.954762+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28078,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955344+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:36:21.955344+0000","last_peered":"2024-09-17T15:36:21.955344+0000","last_clean":"2024-09-17T15:36:21.955344+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:36:21.955344+0000","last_undegraded":"2024-09-17T15:36:21.955344+0000","last_fullsized":"2024-09-17T15:36:21.955344+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955852+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:36:21.955852+0000","last_peered":"2024-09-17T15:36:21.955852+0000","last_clean":"2024-09-17T15:36:21.955852+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:36:21.955852+0000","last_undegraded":"2024-09-17T15:36:21.955852+0000","last_fullsized":"2024-09-17T15:36:21.955852+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21207,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.959971+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:36:21.959971+0000","last_peered":"2024-09-17T15:36:21.959971+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:36:21.959971+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":90,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955263+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:36:21.955263+0000","last_peered":"2024-09-17T15:36:21.955263+0000","last_clean":"2024-09-17T15:36:21.955263+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:36:21.955263+0000","last_undegraded":"2024-09-17T15:36:21.955263+0000","last_fullsized":"2024-09-17T15:36:21.955263+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":95,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.956005+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:36:21.956005+0000","last_peered":"2024-09-17T15:36:21.956005+0000","last_clean":"2024-09-17T15:36:21.956005+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:36:21.956005+0000","last_undegraded":"2024-09-17T15:36:21.956005+0000","last_fullsized":"2024-09-17T15:36:21.956005+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29029,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954672+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:36:21.954672+0000","last_peered":"2024-09-17T15:36:21.954672+0000","last_clean":"2024-09-17T15:36:21.954672+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:36:21.954672+0000","last_undegraded":"2024-09-17T15:36:21.954672+0000","last_fullsized":"2024-09-17T15:36:21.954672+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21914,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27546,"num_bytes_recovered":41572719,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202625024,"data_stored":202505795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051723,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497300,"kb_used_data":132412,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874540,"statfs":{"total":96636764160,"available":96127528960,"internally_reserved":0,"allocated":135589888,"data_stored":135260559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724668,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487544,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884296,"statfs":{"total":96636764160,"available":96137519104,"internally_reserved":0,"allocated":499712,"data_stored":155467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":498724523},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247951,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539388,"kb_used_data":66280,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832452,"statfs":{"total":96636764160,"available":96084430848,"internally_reserved":0,"allocated":67870720,"data_stored":67537314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135147520,"data_stored":135111390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:36:45.034 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:36:45.365 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:45.365 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:36:45.377 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":416,"stamp":"2024-09-17T15:36:43.823174+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28473,"num_bytes_recovered":41572738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1524232,"kb_used_data":199180,"kb_used_omap":13,"kb_used_meta":1324978,"kb_avail":281591288,"statfs":{"total":289910292480,"available":288349478912,"internally_reserved":0,"allocated":203960320,"data_stored":202953340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1356778495},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2273,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2273,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002077"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20921,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.961364+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:36:21.961364+0000","last_peered":"2024-09-17T15:36:21.961364+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:36:21.961364+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21058,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.956094+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:36:21.956094+0000","last_peered":"2024-09-17T15:36:21.956094+0000","last_clean":"2024-09-17T15:36:21.956094+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:36:21.956094+0000","last_undegraded":"2024-09-17T15:36:21.956094+0000","last_fullsized":"2024-09-17T15:36:21.956094+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":27770,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:43.448064+0000","last_change":"2024-09-17T15:36:20.543951+0000","last_active":"2024-09-17T15:36:43.448064+0000","last_peered":"2024-09-17T15:36:43.448064+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:36:43.448064+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":826,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3343,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-826}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21063,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955651+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:36:21.955651+0000","last_peered":"2024-09-17T15:36:21.955651+0000","last_clean":"2024-09-17T15:36:21.955651+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:36:21.955651+0000","last_undegraded":"2024-09-17T15:36:21.955651+0000","last_fullsized":"2024-09-17T15:36:21.955651+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20805,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959779+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:36:21.959779+0000","last_peered":"2024-09-17T15:36:21.959779+0000","last_clean":"2024-09-17T15:36:21.959779+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:36:21.959779+0000","last_undegraded":"2024-09-17T15:36:21.959779+0000","last_fullsized":"2024-09-17T15:36:21.959779+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21565,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.959782+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:36:21.959782+0000","last_peered":"2024-09-17T15:36:21.959782+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:36:21.959782+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4191},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:22.048419+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:36:22.048419+0000","last_peered":"2024-09-17T15:36:22.048419+0000","last_clean":"2024-09-17T15:36:22.048419+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:36:22.048419+0000","last_undegraded":"2024-09-17T15:36:22.048419+0000","last_fullsized":"2024-09-17T15:36:22.048419+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21067,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959637+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:36:21.959637+0000","last_peered":"2024-09-17T15:36:21.959637+0000","last_clean":"2024-09-17T15:36:21.959637+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:36:21.959637+0000","last_undegraded":"2024-09-17T15:36:21.959637+0000","last_fullsized":"2024-09-17T15:36:21.959637+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954544+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:36:21.954544+0000","last_peered":"2024-09-17T15:36:21.954544+0000","last_clean":"2024-09-17T15:36:21.954544+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:36:21.954544+0000","last_undegraded":"2024-09-17T15:36:21.954544+0000","last_fullsized":"2024-09-17T15:36:21.954544+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955888+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:36:21.955888+0000","last_peered":"2024-09-17T15:36:21.955888+0000","last_clean":"2024-09-17T15:36:21.955888+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:36:21.955888+0000","last_undegraded":"2024-09-17T15:36:21.955888+0000","last_fullsized":"2024-09-17T15:36:21.955888+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955084+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:36:21.955084+0000","last_peered":"2024-09-17T15:36:21.955084+0000","last_clean":"2024-09-17T15:36:21.955084+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:36:21.955084+0000","last_undegraded":"2024-09-17T15:36:21.955084+0000","last_fullsized":"2024-09-17T15:36:21.955084+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20830,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.961042+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:36:21.961042+0000","last_peered":"2024-09-17T15:36:21.961042+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:36:21.961042+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959684+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:36:21.959684+0000","last_peered":"2024-09-17T15:36:21.959684+0000","last_clean":"2024-09-17T15:36:21.959684+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:36:21.959684+0000","last_undegraded":"2024-09-17T15:36:21.959684+0000","last_fullsized":"2024-09-17T15:36:21.959684+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28827,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955167+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:36:21.955167+0000","last_peered":"2024-09-17T15:36:21.955167+0000","last_clean":"2024-09-17T15:36:21.955167+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:36:21.955167+0000","last_undegraded":"2024-09-17T15:36:21.955167+0000","last_fullsized":"2024-09-17T15:36:21.955167+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955801+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:36:21.955801+0000","last_peered":"2024-09-17T15:36:21.955801+0000","last_clean":"2024-09-17T15:36:21.955801+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:36:21.955801+0000","last_undegraded":"2024-09-17T15:36:21.955801+0000","last_fullsized":"2024-09-17T15:36:21.955801+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29164,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954760+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:36:21.954760+0000","last_peered":"2024-09-17T15:36:21.954760+0000","last_clean":"2024-09-17T15:36:21.954760+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:36:21.954760+0000","last_undegraded":"2024-09-17T15:36:21.954760+0000","last_fullsized":"2024-09-17T15:36:21.954760+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21286,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.960250+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:36:21.960250+0000","last_peered":"2024-09-17T15:36:21.960250+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:36:21.960250+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":90,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954762+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:36:21.954762+0000","last_peered":"2024-09-17T15:36:21.954762+0000","last_clean":"2024-09-17T15:36:21.954762+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:36:21.954762+0000","last_undegraded":"2024-09-17T15:36:21.954762+0000","last_fullsized":"2024-09-17T15:36:21.954762+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28078,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955344+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:36:21.955344+0000","last_peered":"2024-09-17T15:36:21.955344+0000","last_clean":"2024-09-17T15:36:21.955344+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:36:21.955344+0000","last_undegraded":"2024-09-17T15:36:21.955344+0000","last_fullsized":"2024-09-17T15:36:21.955344+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955852+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:36:21.955852+0000","last_peered":"2024-09-17T15:36:21.955852+0000","last_clean":"2024-09-17T15:36:21.955852+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:36:21.955852+0000","last_undegraded":"2024-09-17T15:36:21.955852+0000","last_fullsized":"2024-09-17T15:36:21.955852+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21207,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.959971+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:36:21.959971+0000","last_peered":"2024-09-17T15:36:21.959971+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:36:21.959971+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":90,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955263+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:36:21.955263+0000","last_peered":"2024-09-17T15:36:21.955263+0000","last_clean":"2024-09-17T15:36:21.955263+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:36:21.955263+0000","last_undegraded":"2024-09-17T15:36:21.955263+0000","last_fullsized":"2024-09-17T15:36:21.955263+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":95,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.956005+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:36:21.956005+0000","last_peered":"2024-09-17T15:36:21.956005+0000","last_clean":"2024-09-17T15:36:21.956005+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:36:21.956005+0000","last_undegraded":"2024-09-17T15:36:21.956005+0000","last_fullsized":"2024-09-17T15:36:21.956005+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29029,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954672+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:36:21.954672+0000","last_peered":"2024-09-17T15:36:21.954672+0000","last_clean":"2024-09-17T15:36:21.954672+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:36:21.954672+0000","last_undegraded":"2024-09-17T15:36:21.954672+0000","last_fullsized":"2024-09-17T15:36:21.954672+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28472,"num_bytes_recovered":41572719,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202625024,"data_stored":202505795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051724,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497300,"kb_used_data":132412,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874540,"statfs":{"total":96636764160,"available":96127528960,"internally_reserved":0,"allocated":135589888,"data_stored":135260559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724668,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487544,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884296,"statfs":{"total":96636764160,"available":96137519104,"internally_reserved":0,"allocated":499712,"data_stored":155467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":498724523},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247951,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539388,"kb_used_data":66280,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832452,"statfs":{"total":96636764160,"available":96084430848,"internally_reserved":0,"allocated":67870720,"data_stored":67537314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135147520,"data_stored":135111390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:36:45.379 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:36:45.703 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:45.703 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:36:45.715 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":416,"stamp":"2024-09-17T15:36:43.823174+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28473,"num_bytes_recovered":41572738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1524232,"kb_used_data":199180,"kb_used_omap":13,"kb_used_meta":1324978,"kb_avail":281591288,"statfs":{"total":289910292480,"available":288349478912,"internally_reserved":0,"allocated":203960320,"data_stored":202953340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1356778495},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2273,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2273,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002077"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20921,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.961364+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:36:21.961364+0000","last_peered":"2024-09-17T15:36:21.961364+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:36:21.961364+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21058,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.956094+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:36:21.956094+0000","last_peered":"2024-09-17T15:36:21.956094+0000","last_clean":"2024-09-17T15:36:21.956094+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:36:21.956094+0000","last_undegraded":"2024-09-17T15:36:21.956094+0000","last_fullsized":"2024-09-17T15:36:21.956094+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":27770,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:43.448064+0000","last_change":"2024-09-17T15:36:20.543951+0000","last_active":"2024-09-17T15:36:43.448064+0000","last_peered":"2024-09-17T15:36:43.448064+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:36:43.448064+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":826,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3343,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-826}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21063,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955651+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:36:21.955651+0000","last_peered":"2024-09-17T15:36:21.955651+0000","last_clean":"2024-09-17T15:36:21.955651+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:36:21.955651+0000","last_undegraded":"2024-09-17T15:36:21.955651+0000","last_fullsized":"2024-09-17T15:36:21.955651+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20805,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959779+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:36:21.959779+0000","last_peered":"2024-09-17T15:36:21.959779+0000","last_clean":"2024-09-17T15:36:21.959779+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:36:21.959779+0000","last_undegraded":"2024-09-17T15:36:21.959779+0000","last_fullsized":"2024-09-17T15:36:21.959779+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21565,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.959782+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:36:21.959782+0000","last_peered":"2024-09-17T15:36:21.959782+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:36:21.959782+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4191},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:22.048419+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:36:22.048419+0000","last_peered":"2024-09-17T15:36:22.048419+0000","last_clean":"2024-09-17T15:36:22.048419+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:36:22.048419+0000","last_undegraded":"2024-09-17T15:36:22.048419+0000","last_fullsized":"2024-09-17T15:36:22.048419+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21067,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959637+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:36:21.959637+0000","last_peered":"2024-09-17T15:36:21.959637+0000","last_clean":"2024-09-17T15:36:21.959637+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:36:21.959637+0000","last_undegraded":"2024-09-17T15:36:21.959637+0000","last_fullsized":"2024-09-17T15:36:21.959637+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954544+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:36:21.954544+0000","last_peered":"2024-09-17T15:36:21.954544+0000","last_clean":"2024-09-17T15:36:21.954544+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:36:21.954544+0000","last_undegraded":"2024-09-17T15:36:21.954544+0000","last_fullsized":"2024-09-17T15:36:21.954544+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955888+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:36:21.955888+0000","last_peered":"2024-09-17T15:36:21.955888+0000","last_clean":"2024-09-17T15:36:21.955888+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:36:21.955888+0000","last_undegraded":"2024-09-17T15:36:21.955888+0000","last_fullsized":"2024-09-17T15:36:21.955888+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955084+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:36:21.955084+0000","last_peered":"2024-09-17T15:36:21.955084+0000","last_clean":"2024-09-17T15:36:21.955084+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:36:21.955084+0000","last_undegraded":"2024-09-17T15:36:21.955084+0000","last_fullsized":"2024-09-17T15:36:21.955084+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20830,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.961042+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:36:21.961042+0000","last_peered":"2024-09-17T15:36:21.961042+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:36:21.961042+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959684+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:36:21.959684+0000","last_peered":"2024-09-17T15:36:21.959684+0000","last_clean":"2024-09-17T15:36:21.959684+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:36:21.959684+0000","last_undegraded":"2024-09-17T15:36:21.959684+0000","last_fullsized":"2024-09-17T15:36:21.959684+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28827,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955167+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:36:21.955167+0000","last_peered":"2024-09-17T15:36:21.955167+0000","last_clean":"2024-09-17T15:36:21.955167+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:36:21.955167+0000","last_undegraded":"2024-09-17T15:36:21.955167+0000","last_fullsized":"2024-09-17T15:36:21.955167+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955801+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:36:21.955801+0000","last_peered":"2024-09-17T15:36:21.955801+0000","last_clean":"2024-09-17T15:36:21.955801+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:36:21.955801+0000","last_undegraded":"2024-09-17T15:36:21.955801+0000","last_fullsized":"2024-09-17T15:36:21.955801+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29164,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954760+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:36:21.954760+0000","last_peered":"2024-09-17T15:36:21.954760+0000","last_clean":"2024-09-17T15:36:21.954760+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:36:21.954760+0000","last_undegraded":"2024-09-17T15:36:21.954760+0000","last_fullsized":"2024-09-17T15:36:21.954760+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21286,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.960250+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:36:21.960250+0000","last_peered":"2024-09-17T15:36:21.960250+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:36:21.960250+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":90,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954762+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:36:21.954762+0000","last_peered":"2024-09-17T15:36:21.954762+0000","last_clean":"2024-09-17T15:36:21.954762+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:36:21.954762+0000","last_undegraded":"2024-09-17T15:36:21.954762+0000","last_fullsized":"2024-09-17T15:36:21.954762+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28078,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955344+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:36:21.955344+0000","last_peered":"2024-09-17T15:36:21.955344+0000","last_clean":"2024-09-17T15:36:21.955344+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:36:21.955344+0000","last_undegraded":"2024-09-17T15:36:21.955344+0000","last_fullsized":"2024-09-17T15:36:21.955344+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955852+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:36:21.955852+0000","last_peered":"2024-09-17T15:36:21.955852+0000","last_clean":"2024-09-17T15:36:21.955852+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:36:21.955852+0000","last_undegraded":"2024-09-17T15:36:21.955852+0000","last_fullsized":"2024-09-17T15:36:21.955852+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21207,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.959971+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:36:21.959971+0000","last_peered":"2024-09-17T15:36:21.959971+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:36:21.959971+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":90,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955263+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:36:21.955263+0000","last_peered":"2024-09-17T15:36:21.955263+0000","last_clean":"2024-09-17T15:36:21.955263+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:36:21.955263+0000","last_undegraded":"2024-09-17T15:36:21.955263+0000","last_fullsized":"2024-09-17T15:36:21.955263+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":95,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.956005+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:36:21.956005+0000","last_peered":"2024-09-17T15:36:21.956005+0000","last_clean":"2024-09-17T15:36:21.956005+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:36:21.956005+0000","last_undegraded":"2024-09-17T15:36:21.956005+0000","last_fullsized":"2024-09-17T15:36:21.956005+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29029,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954672+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:36:21.954672+0000","last_peered":"2024-09-17T15:36:21.954672+0000","last_clean":"2024-09-17T15:36:21.954672+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:36:21.954672+0000","last_undegraded":"2024-09-17T15:36:21.954672+0000","last_fullsized":"2024-09-17T15:36:21.954672+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28472,"num_bytes_recovered":41572719,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202625024,"data_stored":202505795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051724,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497300,"kb_used_data":132412,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874540,"statfs":{"total":96636764160,"available":96127528960,"internally_reserved":0,"allocated":135589888,"data_stored":135260559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724668,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487544,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884296,"statfs":{"total":96636764160,"available":96137519104,"internally_reserved":0,"allocated":499712,"data_stored":155467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":498724523},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247951,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539388,"kb_used_data":66280,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832452,"statfs":{"total":96636764160,"available":96084430848,"internally_reserved":0,"allocated":67870720,"data_stored":67537314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135147520,"data_stored":135111390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:36:45.717 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:36:45.717 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'20648", 'reported_seq': 20921, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:21.961364+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:36:21.961364+0000', 'last_peered': '2024-09-17T15:36:21.961364+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:36:21.961364+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'10600", 'ondisk_log_start': "43'10600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20648, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:45.717 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-17T15:36:45.717 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'20934", 'reported_seq': 27770, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:43.448064+0000', 'last_change': '2024-09-17T15:36:20.543951+0000', 'last_active': '2024-09-17T15:36:43.448064+0000', 'last_peered': '2024-09-17T15:36:43.448064+0000', 'last_clean': '2024-09-17T15:26:20.158176+0000', 'last_became_active': '2024-09-17T15:26:22.681963+0000', 'last_became_peered': '2024-09-17T15:26:22.681963+0000', 'last_unstale': '2024-09-17T15:36:43.448064+0000', 'last_undegraded': '2024-09-17T15:26:22.226561+0000', 'last_fullsized': '2024-09-17T15:26:22.189646+0000', 'mapping_epoch': 40, 'log_start': "43'10900", 'ondisk_log_start': "43'10900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'1872", 'last_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:25:51.885344+0000', 'objects_scrubbed': 1839, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 826, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 19091, 'num_write_kb': 34501, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3343, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -826}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T15:36:45.717 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:36:45.717 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'21251", 'reported_seq': 21565, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:21.959782+0000', 'last_change': '2024-09-17T15:26:22.747181+0000', 'last_active': '2024-09-17T15:36:21.959782+0000', 'last_peered': '2024-09-17T15:36:21.959782+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:36:21.959782+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "43'11200", 'ondisk_log_start': "43'11200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17750, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4191}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:45.718 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:36:45.718 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20542", 'reported_seq': 20830, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:21.961042+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:36:21.961042+0000', 'last_peered': '2024-09-17T15:36:21.961042+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:36:21.961042+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17138, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3943}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:45.718 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:36:45.718 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'20966", 'reported_seq': 21286, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:21.960250+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:36:21.960250+0000', 'last_peered': '2024-09-17T15:36:21.960250+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:36:21.960250+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "43'10900", 'ondisk_log_start': "43'10900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17514, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4057}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:45.718 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:36:45.718 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'20886", 'reported_seq': 21207, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:21.959971+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:36:21.959971+0000', 'last_peered': '2024-09-17T15:36:21.959971+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:36:21.959971+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'10800", 'ondisk_log_start': "43'10800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17673, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3980}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:45.718 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:36:46.112 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:46.112 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 20988/24 objects degraded (87450.000%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":743,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1565073408,"bytes_avail":288345219072,"bytes_total":289910292480,"degraded_objects":20988,"degraded_total":24,"degraded_ratio":874.5,"recovering_objects_per_sec":138,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1661,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:36:46.128 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:36:46.128 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:36:46.452 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:46.452 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:36:46.464 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":417,"stamp":"2024-09-17T15:36:45.823581+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28473,"num_bytes_recovered":41572738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1528392,"kb_used_data":199180,"kb_used_omap":13,"kb_used_meta":1329138,"kb_avail":281587128,"statfs":{"total":289910292480,"available":288345219072,"internally_reserved":0,"allocated":203960320,"data_stored":202953340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1361038335},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1661,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1661,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002190"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20921,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.961364+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:36:21.961364+0000","last_peered":"2024-09-17T15:36:21.961364+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:36:21.961364+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21058,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.956094+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:36:21.956094+0000","last_peered":"2024-09-17T15:36:21.956094+0000","last_clean":"2024-09-17T15:36:21.956094+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:36:21.956094+0000","last_undegraded":"2024-09-17T15:36:21.956094+0000","last_fullsized":"2024-09-17T15:36:21.956094+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":27770,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:43.448064+0000","last_change":"2024-09-17T15:36:20.543951+0000","last_active":"2024-09-17T15:36:43.448064+0000","last_peered":"2024-09-17T15:36:43.448064+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:36:43.448064+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":826,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3343,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-826}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21063,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955651+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:36:21.955651+0000","last_peered":"2024-09-17T15:36:21.955651+0000","last_clean":"2024-09-17T15:36:21.955651+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:36:21.955651+0000","last_undegraded":"2024-09-17T15:36:21.955651+0000","last_fullsized":"2024-09-17T15:36:21.955651+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20805,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959779+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:36:21.959779+0000","last_peered":"2024-09-17T15:36:21.959779+0000","last_clean":"2024-09-17T15:36:21.959779+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:36:21.959779+0000","last_undegraded":"2024-09-17T15:36:21.959779+0000","last_fullsized":"2024-09-17T15:36:21.959779+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21565,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.959782+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:36:21.959782+0000","last_peered":"2024-09-17T15:36:21.959782+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:36:21.959782+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4191},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:22.048419+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:36:22.048419+0000","last_peered":"2024-09-17T15:36:22.048419+0000","last_clean":"2024-09-17T15:36:22.048419+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:36:22.048419+0000","last_undegraded":"2024-09-17T15:36:22.048419+0000","last_fullsized":"2024-09-17T15:36:22.048419+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21067,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959637+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:36:21.959637+0000","last_peered":"2024-09-17T15:36:21.959637+0000","last_clean":"2024-09-17T15:36:21.959637+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:36:21.959637+0000","last_undegraded":"2024-09-17T15:36:21.959637+0000","last_fullsized":"2024-09-17T15:36:21.959637+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954544+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:36:21.954544+0000","last_peered":"2024-09-17T15:36:21.954544+0000","last_clean":"2024-09-17T15:36:21.954544+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:36:21.954544+0000","last_undegraded":"2024-09-17T15:36:21.954544+0000","last_fullsized":"2024-09-17T15:36:21.954544+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955888+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:36:21.955888+0000","last_peered":"2024-09-17T15:36:21.955888+0000","last_clean":"2024-09-17T15:36:21.955888+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:36:21.955888+0000","last_undegraded":"2024-09-17T15:36:21.955888+0000","last_fullsized":"2024-09-17T15:36:21.955888+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955084+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:36:21.955084+0000","last_peered":"2024-09-17T15:36:21.955084+0000","last_clean":"2024-09-17T15:36:21.955084+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:36:21.955084+0000","last_undegraded":"2024-09-17T15:36:21.955084+0000","last_fullsized":"2024-09-17T15:36:21.955084+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20830,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.961042+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:36:21.961042+0000","last_peered":"2024-09-17T15:36:21.961042+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:36:21.961042+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959684+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:36:21.959684+0000","last_peered":"2024-09-17T15:36:21.959684+0000","last_clean":"2024-09-17T15:36:21.959684+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:36:21.959684+0000","last_undegraded":"2024-09-17T15:36:21.959684+0000","last_fullsized":"2024-09-17T15:36:21.959684+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28827,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955167+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:36:21.955167+0000","last_peered":"2024-09-17T15:36:21.955167+0000","last_clean":"2024-09-17T15:36:21.955167+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:36:21.955167+0000","last_undegraded":"2024-09-17T15:36:21.955167+0000","last_fullsized":"2024-09-17T15:36:21.955167+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955801+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:36:21.955801+0000","last_peered":"2024-09-17T15:36:21.955801+0000","last_clean":"2024-09-17T15:36:21.955801+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:36:21.955801+0000","last_undegraded":"2024-09-17T15:36:21.955801+0000","last_fullsized":"2024-09-17T15:36:21.955801+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29164,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954760+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:36:21.954760+0000","last_peered":"2024-09-17T15:36:21.954760+0000","last_clean":"2024-09-17T15:36:21.954760+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:36:21.954760+0000","last_undegraded":"2024-09-17T15:36:21.954760+0000","last_fullsized":"2024-09-17T15:36:21.954760+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21286,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.960250+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:36:21.960250+0000","last_peered":"2024-09-17T15:36:21.960250+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:36:21.960250+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":90,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954762+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:36:21.954762+0000","last_peered":"2024-09-17T15:36:21.954762+0000","last_clean":"2024-09-17T15:36:21.954762+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:36:21.954762+0000","last_undegraded":"2024-09-17T15:36:21.954762+0000","last_fullsized":"2024-09-17T15:36:21.954762+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28078,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955344+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:36:21.955344+0000","last_peered":"2024-09-17T15:36:21.955344+0000","last_clean":"2024-09-17T15:36:21.955344+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:36:21.955344+0000","last_undegraded":"2024-09-17T15:36:21.955344+0000","last_fullsized":"2024-09-17T15:36:21.955344+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955852+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:36:21.955852+0000","last_peered":"2024-09-17T15:36:21.955852+0000","last_clean":"2024-09-17T15:36:21.955852+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:36:21.955852+0000","last_undegraded":"2024-09-17T15:36:21.955852+0000","last_fullsized":"2024-09-17T15:36:21.955852+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21207,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.959971+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:36:21.959971+0000","last_peered":"2024-09-17T15:36:21.959971+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:36:21.959971+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":90,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955263+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:36:21.955263+0000","last_peered":"2024-09-17T15:36:21.955263+0000","last_clean":"2024-09-17T15:36:21.955263+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:36:21.955263+0000","last_undegraded":"2024-09-17T15:36:21.955263+0000","last_fullsized":"2024-09-17T15:36:21.955263+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":95,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.956005+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:36:21.956005+0000","last_peered":"2024-09-17T15:36:21.956005+0000","last_clean":"2024-09-17T15:36:21.956005+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:36:21.956005+0000","last_undegraded":"2024-09-17T15:36:21.956005+0000","last_fullsized":"2024-09-17T15:36:21.956005+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29029,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954672+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:36:21.954672+0000","last_peered":"2024-09-17T15:36:21.954672+0000","last_clean":"2024-09-17T15:36:21.954672+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:36:21.954672+0000","last_undegraded":"2024-09-17T15:36:21.954672+0000","last_fullsized":"2024-09-17T15:36:21.954672+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28472,"num_bytes_recovered":41572719,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202625024,"data_stored":202505795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051724,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497300,"kb_used_data":132412,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874540,"statfs":{"total":96636764160,"available":96127528960,"internally_reserved":0,"allocated":135589888,"data_stored":135260559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724669,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491704,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880136,"statfs":{"total":96636764160,"available":96133259264,"internally_reserved":0,"allocated":499712,"data_stored":155467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":502984363},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247952,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539388,"kb_used_data":66280,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832452,"statfs":{"total":96636764160,"available":96084430848,"internally_reserved":0,"allocated":67870720,"data_stored":67537314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135147520,"data_stored":135111390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:36:49.466 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:36:49.825 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:49.826 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:36:49.839 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":420,"stamp":"2024-09-17T15:36:48.041279+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28473,"num_bytes_recovered":41572738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1528392,"kb_used_data":199180,"kb_used_omap":13,"kb_used_meta":1329138,"kb_avail":281587128,"statfs":{"total":289910292480,"available":288345219072,"internally_reserved":0,"allocated":203960320,"data_stored":202953340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1361038335},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-926,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":926,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.218863"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20921,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.961364+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:36:21.961364+0000","last_peered":"2024-09-17T15:36:21.961364+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:36:21.961364+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21058,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.956094+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:36:21.956094+0000","last_peered":"2024-09-17T15:36:21.956094+0000","last_clean":"2024-09-17T15:36:21.956094+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:36:21.956094+0000","last_undegraded":"2024-09-17T15:36:21.956094+0000","last_fullsized":"2024-09-17T15:36:21.956094+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":27770,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:43.448064+0000","last_change":"2024-09-17T15:36:20.543951+0000","last_active":"2024-09-17T15:36:43.448064+0000","last_peered":"2024-09-17T15:36:43.448064+0000","last_clean":"2024-09-17T15:26:20.158176+0000","last_became_active":"2024-09-17T15:26:22.681963+0000","last_became_peered":"2024-09-17T15:26:22.681963+0000","last_unstale":"2024-09-17T15:36:43.448064+0000","last_undegraded":"2024-09-17T15:26:22.226561+0000","last_fullsized":"2024-09-17T15:26:22.189646+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":826,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3343,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-826}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21063,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955651+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:36:21.955651+0000","last_peered":"2024-09-17T15:36:21.955651+0000","last_clean":"2024-09-17T15:36:21.955651+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:36:21.955651+0000","last_undegraded":"2024-09-17T15:36:21.955651+0000","last_fullsized":"2024-09-17T15:36:21.955651+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20805,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959779+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:36:21.959779+0000","last_peered":"2024-09-17T15:36:21.959779+0000","last_clean":"2024-09-17T15:36:21.959779+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:36:21.959779+0000","last_undegraded":"2024-09-17T15:36:21.959779+0000","last_fullsized":"2024-09-17T15:36:21.959779+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21565,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.959782+0000","last_change":"2024-09-17T15:26:22.747181+0000","last_active":"2024-09-17T15:36:21.959782+0000","last_peered":"2024-09-17T15:36:21.959782+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:36:21.959782+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4191},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:22.048419+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:36:22.048419+0000","last_peered":"2024-09-17T15:36:22.048419+0000","last_clean":"2024-09-17T15:36:22.048419+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:36:22.048419+0000","last_undegraded":"2024-09-17T15:36:22.048419+0000","last_fullsized":"2024-09-17T15:36:22.048419+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21067,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959637+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:36:21.959637+0000","last_peered":"2024-09-17T15:36:21.959637+0000","last_clean":"2024-09-17T15:36:21.959637+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:36:21.959637+0000","last_undegraded":"2024-09-17T15:36:21.959637+0000","last_fullsized":"2024-09-17T15:36:21.959637+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954544+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:36:21.954544+0000","last_peered":"2024-09-17T15:36:21.954544+0000","last_clean":"2024-09-17T15:36:21.954544+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:36:21.954544+0000","last_undegraded":"2024-09-17T15:36:21.954544+0000","last_fullsized":"2024-09-17T15:36:21.954544+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29122,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955888+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:36:21.955888+0000","last_peered":"2024-09-17T15:36:21.955888+0000","last_clean":"2024-09-17T15:36:21.955888+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:36:21.955888+0000","last_undegraded":"2024-09-17T15:36:21.955888+0000","last_fullsized":"2024-09-17T15:36:21.955888+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955084+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:36:21.955084+0000","last_peered":"2024-09-17T15:36:21.955084+0000","last_clean":"2024-09-17T15:36:21.955084+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:36:21.955084+0000","last_undegraded":"2024-09-17T15:36:21.955084+0000","last_fullsized":"2024-09-17T15:36:21.955084+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20830,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.961042+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:36:21.961042+0000","last_peered":"2024-09-17T15:36:21.961042+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:36:21.961042+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":126,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.959684+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:36:21.959684+0000","last_peered":"2024-09-17T15:36:21.959684+0000","last_clean":"2024-09-17T15:36:21.959684+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:36:21.959684+0000","last_undegraded":"2024-09-17T15:36:21.959684+0000","last_fullsized":"2024-09-17T15:36:21.959684+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28827,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955167+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:36:21.955167+0000","last_peered":"2024-09-17T15:36:21.955167+0000","last_clean":"2024-09-17T15:36:21.955167+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:36:21.955167+0000","last_undegraded":"2024-09-17T15:36:21.955167+0000","last_fullsized":"2024-09-17T15:36:21.955167+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955801+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:36:21.955801+0000","last_peered":"2024-09-17T15:36:21.955801+0000","last_clean":"2024-09-17T15:36:21.955801+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:36:21.955801+0000","last_undegraded":"2024-09-17T15:36:21.955801+0000","last_fullsized":"2024-09-17T15:36:21.955801+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29164,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954760+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:36:21.954760+0000","last_peered":"2024-09-17T15:36:21.954760+0000","last_clean":"2024-09-17T15:36:21.954760+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:36:21.954760+0000","last_undegraded":"2024-09-17T15:36:21.954760+0000","last_fullsized":"2024-09-17T15:36:21.954760+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21286,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.960250+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:36:21.960250+0000","last_peered":"2024-09-17T15:36:21.960250+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:36:21.960250+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":90,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954762+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:36:21.954762+0000","last_peered":"2024-09-17T15:36:21.954762+0000","last_clean":"2024-09-17T15:36:21.954762+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:36:21.954762+0000","last_undegraded":"2024-09-17T15:36:21.954762+0000","last_fullsized":"2024-09-17T15:36:21.954762+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28078,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955344+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:36:21.955344+0000","last_peered":"2024-09-17T15:36:21.955344+0000","last_clean":"2024-09-17T15:36:21.955344+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:36:21.955344+0000","last_undegraded":"2024-09-17T15:36:21.955344+0000","last_fullsized":"2024-09-17T15:36:21.955344+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955852+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:36:21.955852+0000","last_peered":"2024-09-17T15:36:21.955852+0000","last_clean":"2024-09-17T15:36:21.955852+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:36:21.955852+0000","last_undegraded":"2024-09-17T15:36:21.955852+0000","last_fullsized":"2024-09-17T15:36:21.955852+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21207,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:21.959971+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:36:21.959971+0000","last_peered":"2024-09-17T15:36:21.959971+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:36:21.959971+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":90,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.955263+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:36:21.955263+0000","last_peered":"2024-09-17T15:36:21.955263+0000","last_clean":"2024-09-17T15:36:21.955263+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:36:21.955263+0000","last_undegraded":"2024-09-17T15:36:21.955263+0000","last_fullsized":"2024-09-17T15:36:21.955263+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":95,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.956005+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:36:21.956005+0000","last_peered":"2024-09-17T15:36:21.956005+0000","last_clean":"2024-09-17T15:36:21.956005+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:36:21.956005+0000","last_undegraded":"2024-09-17T15:36:21.956005+0000","last_fullsized":"2024-09-17T15:36:21.956005+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29029,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T15:36:21.954672+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:36:21.954672+0000","last_peered":"2024-09-17T15:36:21.954672+0000","last_clean":"2024-09-17T15:36:21.954672+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:36:21.954672+0000","last_undegraded":"2024-09-17T15:36:21.954672+0000","last_fullsized":"2024-09-17T15:36:21.954672+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28472,"num_bytes_recovered":41572719,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202625024,"data_stored":202505795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051724,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497300,"kb_used_data":132412,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874540,"statfs":{"total":96636764160,"available":96127528960,"internally_reserved":0,"allocated":135589888,"data_stored":135260559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724669,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491704,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880136,"statfs":{"total":96636764160,"available":96133259264,"internally_reserved":0,"allocated":499712,"data_stored":155467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":502984363},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247952,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539388,"kb_used_data":66280,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832452,"statfs":{"total":96636764160,"available":96084430848,"internally_reserved":0,"allocated":67870720,"data_stored":67537314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135147520,"data_stored":135111390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:36:49.840 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:36:50.170 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:50.170 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:36:50.182 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":421,"stamp":"2024-09-17T15:36:49.824257+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19739,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29722,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1523924,"kb_used_data":198868,"kb_used_omap":13,"kb_used_meta":1324978,"kb_avail":281591596,"statfs":{"total":289910292480,"available":288349794304,"internally_reserved":0,"allocated":203640832,"data_stored":202610723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1356778495},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2175,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001489"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20925,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.053279+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:36:48.053279+0000","last_peered":"2024-09-17T15:36:48.053279+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:36:48.053279+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21062,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043955+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:36:48.043955+0000","last_peered":"2024-09-17T15:36:48.043955+0000","last_clean":"2024-09-17T15:36:48.043955+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:36:48.043955+0000","last_undegraded":"2024-09-17T15:36:48.043955+0000","last_fullsized":"2024-09-17T15:36:48.043955+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21170,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.120714+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:36:48.120714+0000","last_peered":"2024-09-17T15:36:48.120714+0000","last_clean":"2024-09-17T15:36:48.120714+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:36:48.120714+0000","last_undegraded":"2024-09-17T15:36:48.120714+0000","last_fullsized":"2024-09-17T15:36:48.120714+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21067,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043731+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:36:48.043731+0000","last_peered":"2024-09-17T15:36:48.043731+0000","last_clean":"2024-09-17T15:36:48.043731+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:36:48.043731+0000","last_undegraded":"2024-09-17T15:36:48.043731+0000","last_fullsized":"2024-09-17T15:36:48.043731+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20809,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052901+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:36:48.052901+0000","last_peered":"2024-09-17T15:36:48.052901+0000","last_clean":"2024-09-17T15:36:48.052901+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:36:48.052901+0000","last_undegraded":"2024-09-17T15:36:48.052901+0000","last_fullsized":"2024-09-17T15:36:48.052901+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":22416,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:49.408099+0000","last_change":"2024-09-17T15:36:46.948876+0000","last_active":"2024-09-17T15:36:49.408099+0000","last_peered":"2024-09-17T15:36:49.408099+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:36:49.408099+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3770,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":501,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3769}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21104,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042819+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:36:48.042819+0000","last_peered":"2024-09-17T15:36:48.042819+0000","last_clean":"2024-09-17T15:36:48.042819+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:36:48.042819+0000","last_undegraded":"2024-09-17T15:36:48.042819+0000","last_fullsized":"2024-09-17T15:36:48.042819+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21071,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052737+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:36:48.052737+0000","last_peered":"2024-09-17T15:36:48.052737+0000","last_clean":"2024-09-17T15:36:48.052737+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:36:48.052737+0000","last_undegraded":"2024-09-17T15:36:48.052737+0000","last_fullsized":"2024-09-17T15:36:48.052737+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042197+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:36:48.042197+0000","last_peered":"2024-09-17T15:36:48.042197+0000","last_clean":"2024-09-17T15:36:48.042197+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:36:48.042197+0000","last_undegraded":"2024-09-17T15:36:48.042197+0000","last_fullsized":"2024-09-17T15:36:48.042197+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29127,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042385+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:36:48.042385+0000","last_peered":"2024-09-17T15:36:48.042385+0000","last_clean":"2024-09-17T15:36:48.042385+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:36:48.042385+0000","last_undegraded":"2024-09-17T15:36:48.042385+0000","last_fullsized":"2024-09-17T15:36:48.042385+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042912+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:36:48.042912+0000","last_peered":"2024-09-17T15:36:48.042912+0000","last_clean":"2024-09-17T15:36:48.042912+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:36:48.042912+0000","last_undegraded":"2024-09-17T15:36:48.042912+0000","last_fullsized":"2024-09-17T15:36:48.042912+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20834,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.059616+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:36:48.059616+0000","last_peered":"2024-09-17T15:36:48.059616+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:36:48.059616+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":130,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052749+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:36:48.052749+0000","last_peered":"2024-09-17T15:36:48.052749+0000","last_clean":"2024-09-17T15:36:48.052749+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:36:48.052749+0000","last_undegraded":"2024-09-17T15:36:48.052749+0000","last_fullsized":"2024-09-17T15:36:48.052749+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28831,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042985+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:36:48.042985+0000","last_peered":"2024-09-17T15:36:48.042985+0000","last_clean":"2024-09-17T15:36:48.042985+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:36:48.042985+0000","last_undegraded":"2024-09-17T15:36:48.042985+0000","last_fullsized":"2024-09-17T15:36:48.042985+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":105,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042299+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:36:48.042299+0000","last_peered":"2024-09-17T15:36:48.042299+0000","last_clean":"2024-09-17T15:36:48.042299+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:36:48.042299+0000","last_undegraded":"2024-09-17T15:36:48.042299+0000","last_fullsized":"2024-09-17T15:36:48.042299+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29168,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042529+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:36:48.042529+0000","last_peered":"2024-09-17T15:36:48.042529+0000","last_clean":"2024-09-17T15:36:48.042529+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:36:48.042529+0000","last_undegraded":"2024-09-17T15:36:48.042529+0000","last_fullsized":"2024-09-17T15:36:48.042529+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21290,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.053156+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:36:48.053156+0000","last_peered":"2024-09-17T15:36:48.053156+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:36:48.053156+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042552+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:36:48.042552+0000","last_peered":"2024-09-17T15:36:48.042552+0000","last_clean":"2024-09-17T15:36:48.042552+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:36:48.042552+0000","last_undegraded":"2024-09-17T15:36:48.042552+0000","last_fullsized":"2024-09-17T15:36:48.042552+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28082,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042306+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:36:48.042306+0000","last_peered":"2024-09-17T15:36:48.042306+0000","last_clean":"2024-09-17T15:36:48.042306+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:36:48.042306+0000","last_undegraded":"2024-09-17T15:36:48.042306+0000","last_fullsized":"2024-09-17T15:36:48.042306+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043694+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:36:48.043694+0000","last_peered":"2024-09-17T15:36:48.043694+0000","last_clean":"2024-09-17T15:36:48.043694+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:36:48.043694+0000","last_undegraded":"2024-09-17T15:36:48.043694+0000","last_fullsized":"2024-09-17T15:36:48.043694+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21211,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.053263+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:36:48.053263+0000","last_peered":"2024-09-17T15:36:48.053263+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:36:48.053263+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":94,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042177+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:36:48.042177+0000","last_peered":"2024-09-17T15:36:48.042177+0000","last_clean":"2024-09-17T15:36:48.042177+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:36:48.042177+0000","last_undegraded":"2024-09-17T15:36:48.042177+0000","last_fullsized":"2024-09-17T15:36:48.042177+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":99,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042677+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:36:48.042677+0000","last_peered":"2024-09-17T15:36:48.042677+0000","last_clean":"2024-09-17T15:36:48.042677+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:36:48.042677+0000","last_undegraded":"2024-09-17T15:36:48.042677+0000","last_fullsized":"2024-09-17T15:36:48.042677+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29033,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042443+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:36:48.042443+0000","last_peered":"2024-09-17T15:36:48.042443+0000","last_clean":"2024-09-17T15:36:48.042443+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:36:48.042443+0000","last_undegraded":"2024-09-17T15:36:48.042443+0000","last_fullsized":"2024-09-17T15:36:48.042443+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19739,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29721,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":201895936,"data_stored":201780320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051725,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":496956,"kb_used_data":132072,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874884,"statfs":{"total":96636764160,"available":96127881216,"internally_reserved":0,"allocated":135241728,"data_stored":134907059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724670,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487560,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884280,"statfs":{"total":96636764160,"available":96137502720,"internally_reserved":0,"allocated":512000,"data_stored":160667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":498724523},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247953,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539408,"kb_used_data":66296,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832432,"statfs":{"total":96636764160,"available":96084410368,"internally_reserved":0,"allocated":67887104,"data_stored":67542997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134422528,"data_stored":134386398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:36:50.184 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:36:50.184 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'20648", 'reported_seq': 20925, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:48.053279+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:36:48.053279+0000', 'last_peered': '2024-09-17T15:36:48.053279+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:36:48.053279+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'10600", 'ondisk_log_start': "43'10600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20648, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:50.184 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:36:50.184 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'21251", 'reported_seq': 22416, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:49.408099+0000', 'last_change': '2024-09-17T15:36:46.948876+0000', 'last_active': '2024-09-17T15:36:49.408099+0000', 'last_peered': '2024-09-17T15:36:49.408099+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:36:49.408099+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "43'11200", 'ondisk_log_start': "43'11200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3770, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17750, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 501, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3769}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:50.184 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:36:50.184 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20542", 'reported_seq': 20834, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:48.059616+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:36:48.059616+0000', 'last_peered': '2024-09-17T15:36:48.059616+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:36:48.059616+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17138, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3943}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:50.184 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:36:50.184 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'20966", 'reported_seq': 21290, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:48.053156+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:36:48.053156+0000', 'last_peered': '2024-09-17T15:36:48.053156+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:36:48.053156+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "43'10900", 'ondisk_log_start': "43'10900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17514, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4057}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:50.185 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:36:50.185 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'20886", 'reported_seq': 21211, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:48.053263+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:36:48.053263+0000', 'last_peered': '2024-09-17T15:36:48.053263+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:36:48.053263+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'10800", 'ondisk_log_start': "43'10800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17673, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3980}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:50.185 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:36:50.567 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:50.567 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 19739/24 objects degraded (82245.833%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":747,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1560498176,"bytes_avail":288349794304,"bytes_total":289910292480,"degraded_objects":19739,"degraded_total":24,"degraded_ratio":822.45833333333348,"recovering_objects_per_sec":271,"recovering_bytes_per_sec":60,"recovering_keys_per_sec":0,"num_objects_recovered":2175,"num_bytes_recovered":483,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:36:50.582 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:36:50.582 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:36:50.886 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:50.887 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:36:50.899 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":421,"stamp":"2024-09-17T15:36:49.824257+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19739,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29722,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1523924,"kb_used_data":198868,"kb_used_omap":13,"kb_used_meta":1324978,"kb_avail":281591596,"statfs":{"total":289910292480,"available":288349794304,"internally_reserved":0,"allocated":203640832,"data_stored":202610723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1356778495},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2175,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001489"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20925,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.053279+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:36:48.053279+0000","last_peered":"2024-09-17T15:36:48.053279+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:36:48.053279+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21062,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043955+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:36:48.043955+0000","last_peered":"2024-09-17T15:36:48.043955+0000","last_clean":"2024-09-17T15:36:48.043955+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:36:48.043955+0000","last_undegraded":"2024-09-17T15:36:48.043955+0000","last_fullsized":"2024-09-17T15:36:48.043955+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21170,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.120714+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:36:48.120714+0000","last_peered":"2024-09-17T15:36:48.120714+0000","last_clean":"2024-09-17T15:36:48.120714+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:36:48.120714+0000","last_undegraded":"2024-09-17T15:36:48.120714+0000","last_fullsized":"2024-09-17T15:36:48.120714+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21067,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043731+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:36:48.043731+0000","last_peered":"2024-09-17T15:36:48.043731+0000","last_clean":"2024-09-17T15:36:48.043731+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:36:48.043731+0000","last_undegraded":"2024-09-17T15:36:48.043731+0000","last_fullsized":"2024-09-17T15:36:48.043731+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20809,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052901+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:36:48.052901+0000","last_peered":"2024-09-17T15:36:48.052901+0000","last_clean":"2024-09-17T15:36:48.052901+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:36:48.052901+0000","last_undegraded":"2024-09-17T15:36:48.052901+0000","last_fullsized":"2024-09-17T15:36:48.052901+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":22416,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:49.408099+0000","last_change":"2024-09-17T15:36:46.948876+0000","last_active":"2024-09-17T15:36:49.408099+0000","last_peered":"2024-09-17T15:36:49.408099+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:36:49.408099+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3770,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":501,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3769}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21104,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042819+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:36:48.042819+0000","last_peered":"2024-09-17T15:36:48.042819+0000","last_clean":"2024-09-17T15:36:48.042819+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:36:48.042819+0000","last_undegraded":"2024-09-17T15:36:48.042819+0000","last_fullsized":"2024-09-17T15:36:48.042819+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21071,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052737+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:36:48.052737+0000","last_peered":"2024-09-17T15:36:48.052737+0000","last_clean":"2024-09-17T15:36:48.052737+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:36:48.052737+0000","last_undegraded":"2024-09-17T15:36:48.052737+0000","last_fullsized":"2024-09-17T15:36:48.052737+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042197+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:36:48.042197+0000","last_peered":"2024-09-17T15:36:48.042197+0000","last_clean":"2024-09-17T15:36:48.042197+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:36:48.042197+0000","last_undegraded":"2024-09-17T15:36:48.042197+0000","last_fullsized":"2024-09-17T15:36:48.042197+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29127,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042385+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:36:48.042385+0000","last_peered":"2024-09-17T15:36:48.042385+0000","last_clean":"2024-09-17T15:36:48.042385+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:36:48.042385+0000","last_undegraded":"2024-09-17T15:36:48.042385+0000","last_fullsized":"2024-09-17T15:36:48.042385+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042912+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:36:48.042912+0000","last_peered":"2024-09-17T15:36:48.042912+0000","last_clean":"2024-09-17T15:36:48.042912+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:36:48.042912+0000","last_undegraded":"2024-09-17T15:36:48.042912+0000","last_fullsized":"2024-09-17T15:36:48.042912+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20834,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.059616+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:36:48.059616+0000","last_peered":"2024-09-17T15:36:48.059616+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:36:48.059616+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":130,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052749+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:36:48.052749+0000","last_peered":"2024-09-17T15:36:48.052749+0000","last_clean":"2024-09-17T15:36:48.052749+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:36:48.052749+0000","last_undegraded":"2024-09-17T15:36:48.052749+0000","last_fullsized":"2024-09-17T15:36:48.052749+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28831,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042985+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:36:48.042985+0000","last_peered":"2024-09-17T15:36:48.042985+0000","last_clean":"2024-09-17T15:36:48.042985+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:36:48.042985+0000","last_undegraded":"2024-09-17T15:36:48.042985+0000","last_fullsized":"2024-09-17T15:36:48.042985+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":105,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042299+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:36:48.042299+0000","last_peered":"2024-09-17T15:36:48.042299+0000","last_clean":"2024-09-17T15:36:48.042299+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:36:48.042299+0000","last_undegraded":"2024-09-17T15:36:48.042299+0000","last_fullsized":"2024-09-17T15:36:48.042299+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29168,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042529+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:36:48.042529+0000","last_peered":"2024-09-17T15:36:48.042529+0000","last_clean":"2024-09-17T15:36:48.042529+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:36:48.042529+0000","last_undegraded":"2024-09-17T15:36:48.042529+0000","last_fullsized":"2024-09-17T15:36:48.042529+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21290,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.053156+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:36:48.053156+0000","last_peered":"2024-09-17T15:36:48.053156+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:36:48.053156+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042552+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:36:48.042552+0000","last_peered":"2024-09-17T15:36:48.042552+0000","last_clean":"2024-09-17T15:36:48.042552+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:36:48.042552+0000","last_undegraded":"2024-09-17T15:36:48.042552+0000","last_fullsized":"2024-09-17T15:36:48.042552+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28082,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042306+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:36:48.042306+0000","last_peered":"2024-09-17T15:36:48.042306+0000","last_clean":"2024-09-17T15:36:48.042306+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:36:48.042306+0000","last_undegraded":"2024-09-17T15:36:48.042306+0000","last_fullsized":"2024-09-17T15:36:48.042306+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043694+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:36:48.043694+0000","last_peered":"2024-09-17T15:36:48.043694+0000","last_clean":"2024-09-17T15:36:48.043694+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:36:48.043694+0000","last_undegraded":"2024-09-17T15:36:48.043694+0000","last_fullsized":"2024-09-17T15:36:48.043694+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21211,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.053263+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:36:48.053263+0000","last_peered":"2024-09-17T15:36:48.053263+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:36:48.053263+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":94,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042177+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:36:48.042177+0000","last_peered":"2024-09-17T15:36:48.042177+0000","last_clean":"2024-09-17T15:36:48.042177+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:36:48.042177+0000","last_undegraded":"2024-09-17T15:36:48.042177+0000","last_fullsized":"2024-09-17T15:36:48.042177+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":99,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042677+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:36:48.042677+0000","last_peered":"2024-09-17T15:36:48.042677+0000","last_clean":"2024-09-17T15:36:48.042677+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:36:48.042677+0000","last_undegraded":"2024-09-17T15:36:48.042677+0000","last_fullsized":"2024-09-17T15:36:48.042677+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29033,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042443+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:36:48.042443+0000","last_peered":"2024-09-17T15:36:48.042443+0000","last_clean":"2024-09-17T15:36:48.042443+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:36:48.042443+0000","last_undegraded":"2024-09-17T15:36:48.042443+0000","last_fullsized":"2024-09-17T15:36:48.042443+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19739,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29721,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":201895936,"data_stored":201780320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051725,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":496956,"kb_used_data":132072,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93874884,"statfs":{"total":96636764160,"available":96127881216,"internally_reserved":0,"allocated":135241728,"data_stored":134907059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":373616300},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724670,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487560,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884280,"statfs":{"total":96636764160,"available":96137502720,"internally_reserved":0,"allocated":512000,"data_stored":160667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":498724523},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247953,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539408,"kb_used_data":66296,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832432,"statfs":{"total":96636764160,"available":96084410368,"internally_reserved":0,"allocated":67887104,"data_stored":67542997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134422528,"data_stored":134386398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:36:53.902 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:36:54.329 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:54.329 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:36:54.342 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":423,"stamp":"2024-09-17T15:36:53.824987+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19739,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29722,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1526288,"kb_used_data":197076,"kb_used_omap":13,"kb_used_meta":1329138,"kb_avail":281589232,"statfs":{"total":289910292480,"available":288347373568,"internally_reserved":0,"allocated":201805824,"data_stored":200770354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1361038335},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1249,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1249,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001405"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20925,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.053279+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:36:48.053279+0000","last_peered":"2024-09-17T15:36:48.053279+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:36:48.053279+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21062,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043955+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:36:48.043955+0000","last_peered":"2024-09-17T15:36:48.043955+0000","last_clean":"2024-09-17T15:36:48.043955+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:36:48.043955+0000","last_undegraded":"2024-09-17T15:36:48.043955+0000","last_fullsized":"2024-09-17T15:36:48.043955+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21170,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.120714+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:36:48.120714+0000","last_peered":"2024-09-17T15:36:48.120714+0000","last_clean":"2024-09-17T15:36:48.120714+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:36:48.120714+0000","last_undegraded":"2024-09-17T15:36:48.120714+0000","last_fullsized":"2024-09-17T15:36:48.120714+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21067,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043731+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:36:48.043731+0000","last_peered":"2024-09-17T15:36:48.043731+0000","last_clean":"2024-09-17T15:36:48.043731+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:36:48.043731+0000","last_undegraded":"2024-09-17T15:36:48.043731+0000","last_fullsized":"2024-09-17T15:36:48.043731+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20809,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052901+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:36:48.052901+0000","last_peered":"2024-09-17T15:36:48.052901+0000","last_clean":"2024-09-17T15:36:48.052901+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:36:48.052901+0000","last_undegraded":"2024-09-17T15:36:48.052901+0000","last_fullsized":"2024-09-17T15:36:48.052901+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":22416,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:49.408099+0000","last_change":"2024-09-17T15:36:46.948876+0000","last_active":"2024-09-17T15:36:49.408099+0000","last_peered":"2024-09-17T15:36:49.408099+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:36:49.408099+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3770,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":501,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3769}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21104,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042819+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:36:48.042819+0000","last_peered":"2024-09-17T15:36:48.042819+0000","last_clean":"2024-09-17T15:36:48.042819+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:36:48.042819+0000","last_undegraded":"2024-09-17T15:36:48.042819+0000","last_fullsized":"2024-09-17T15:36:48.042819+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21071,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052737+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:36:48.052737+0000","last_peered":"2024-09-17T15:36:48.052737+0000","last_clean":"2024-09-17T15:36:48.052737+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:36:48.052737+0000","last_undegraded":"2024-09-17T15:36:48.052737+0000","last_fullsized":"2024-09-17T15:36:48.052737+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042197+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:36:48.042197+0000","last_peered":"2024-09-17T15:36:48.042197+0000","last_clean":"2024-09-17T15:36:48.042197+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:36:48.042197+0000","last_undegraded":"2024-09-17T15:36:48.042197+0000","last_fullsized":"2024-09-17T15:36:48.042197+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29127,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042385+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:36:48.042385+0000","last_peered":"2024-09-17T15:36:48.042385+0000","last_clean":"2024-09-17T15:36:48.042385+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:36:48.042385+0000","last_undegraded":"2024-09-17T15:36:48.042385+0000","last_fullsized":"2024-09-17T15:36:48.042385+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042912+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:36:48.042912+0000","last_peered":"2024-09-17T15:36:48.042912+0000","last_clean":"2024-09-17T15:36:48.042912+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:36:48.042912+0000","last_undegraded":"2024-09-17T15:36:48.042912+0000","last_fullsized":"2024-09-17T15:36:48.042912+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20834,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.059616+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:36:48.059616+0000","last_peered":"2024-09-17T15:36:48.059616+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:36:48.059616+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":130,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052749+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:36:48.052749+0000","last_peered":"2024-09-17T15:36:48.052749+0000","last_clean":"2024-09-17T15:36:48.052749+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:36:48.052749+0000","last_undegraded":"2024-09-17T15:36:48.052749+0000","last_fullsized":"2024-09-17T15:36:48.052749+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28831,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042985+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:36:48.042985+0000","last_peered":"2024-09-17T15:36:48.042985+0000","last_clean":"2024-09-17T15:36:48.042985+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:36:48.042985+0000","last_undegraded":"2024-09-17T15:36:48.042985+0000","last_fullsized":"2024-09-17T15:36:48.042985+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":105,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042299+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:36:48.042299+0000","last_peered":"2024-09-17T15:36:48.042299+0000","last_clean":"2024-09-17T15:36:48.042299+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:36:48.042299+0000","last_undegraded":"2024-09-17T15:36:48.042299+0000","last_fullsized":"2024-09-17T15:36:48.042299+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29168,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042529+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:36:48.042529+0000","last_peered":"2024-09-17T15:36:48.042529+0000","last_clean":"2024-09-17T15:36:48.042529+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:36:48.042529+0000","last_undegraded":"2024-09-17T15:36:48.042529+0000","last_fullsized":"2024-09-17T15:36:48.042529+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21290,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.053156+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:36:48.053156+0000","last_peered":"2024-09-17T15:36:48.053156+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:36:48.053156+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042552+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:36:48.042552+0000","last_peered":"2024-09-17T15:36:48.042552+0000","last_clean":"2024-09-17T15:36:48.042552+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:36:48.042552+0000","last_undegraded":"2024-09-17T15:36:48.042552+0000","last_fullsized":"2024-09-17T15:36:48.042552+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28082,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042306+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:36:48.042306+0000","last_peered":"2024-09-17T15:36:48.042306+0000","last_clean":"2024-09-17T15:36:48.042306+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:36:48.042306+0000","last_undegraded":"2024-09-17T15:36:48.042306+0000","last_fullsized":"2024-09-17T15:36:48.042306+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043694+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:36:48.043694+0000","last_peered":"2024-09-17T15:36:48.043694+0000","last_clean":"2024-09-17T15:36:48.043694+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:36:48.043694+0000","last_undegraded":"2024-09-17T15:36:48.043694+0000","last_fullsized":"2024-09-17T15:36:48.043694+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21211,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.053263+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:36:48.053263+0000","last_peered":"2024-09-17T15:36:48.053263+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:36:48.053263+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":94,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042177+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:36:48.042177+0000","last_peered":"2024-09-17T15:36:48.042177+0000","last_clean":"2024-09-17T15:36:48.042177+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:36:48.042177+0000","last_undegraded":"2024-09-17T15:36:48.042177+0000","last_fullsized":"2024-09-17T15:36:48.042177+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":99,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042677+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:36:48.042677+0000","last_peered":"2024-09-17T15:36:48.042677+0000","last_clean":"2024-09-17T15:36:48.042677+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:36:48.042677+0000","last_undegraded":"2024-09-17T15:36:48.042677+0000","last_fullsized":"2024-09-17T15:36:48.042677+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29033,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042443+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:36:48.042443+0000","last_peered":"2024-09-17T15:36:48.042443+0000","last_clean":"2024-09-17T15:36:48.042443+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:36:48.042443+0000","last_undegraded":"2024-09-17T15:36:48.042443+0000","last_fullsized":"2024-09-17T15:36:48.042443+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19739,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29721,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":200052736,"data_stored":199937120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051726,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":499320,"kb_used_data":130280,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93872520,"statfs":{"total":96636764160,"available":96125460480,"internally_reserved":0,"allocated":133406720,"data_stored":133066690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":377876140},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724670,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487560,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884280,"statfs":{"total":96636764160,"available":96137502720,"internally_reserved":0,"allocated":512000,"data_stored":160667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":498724523},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247953,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539408,"kb_used_data":66296,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832432,"statfs":{"total":96636764160,"available":96084410368,"internally_reserved":0,"allocated":67887104,"data_stored":67542997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":132579328,"data_stored":132543198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:36:54.344 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:36:54.701 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:54.702 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:36:54.713 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":423,"stamp":"2024-09-17T15:36:53.824987+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19739,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29722,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1526288,"kb_used_data":197076,"kb_used_omap":13,"kb_used_meta":1329138,"kb_avail":281589232,"statfs":{"total":289910292480,"available":288347373568,"internally_reserved":0,"allocated":201805824,"data_stored":200770354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1361038335},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1249,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1249,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001405"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20925,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.053279+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:36:48.053279+0000","last_peered":"2024-09-17T15:36:48.053279+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:36:48.053279+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21062,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043955+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:36:48.043955+0000","last_peered":"2024-09-17T15:36:48.043955+0000","last_clean":"2024-09-17T15:36:48.043955+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:36:48.043955+0000","last_undegraded":"2024-09-17T15:36:48.043955+0000","last_fullsized":"2024-09-17T15:36:48.043955+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21170,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.120714+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:36:48.120714+0000","last_peered":"2024-09-17T15:36:48.120714+0000","last_clean":"2024-09-17T15:36:48.120714+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:36:48.120714+0000","last_undegraded":"2024-09-17T15:36:48.120714+0000","last_fullsized":"2024-09-17T15:36:48.120714+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21067,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043731+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:36:48.043731+0000","last_peered":"2024-09-17T15:36:48.043731+0000","last_clean":"2024-09-17T15:36:48.043731+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:36:48.043731+0000","last_undegraded":"2024-09-17T15:36:48.043731+0000","last_fullsized":"2024-09-17T15:36:48.043731+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20809,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052901+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:36:48.052901+0000","last_peered":"2024-09-17T15:36:48.052901+0000","last_clean":"2024-09-17T15:36:48.052901+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:36:48.052901+0000","last_undegraded":"2024-09-17T15:36:48.052901+0000","last_fullsized":"2024-09-17T15:36:48.052901+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":22416,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:49.408099+0000","last_change":"2024-09-17T15:36:46.948876+0000","last_active":"2024-09-17T15:36:49.408099+0000","last_peered":"2024-09-17T15:36:49.408099+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:36:49.408099+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3770,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":501,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3769}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21104,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042819+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:36:48.042819+0000","last_peered":"2024-09-17T15:36:48.042819+0000","last_clean":"2024-09-17T15:36:48.042819+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:36:48.042819+0000","last_undegraded":"2024-09-17T15:36:48.042819+0000","last_fullsized":"2024-09-17T15:36:48.042819+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21071,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052737+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:36:48.052737+0000","last_peered":"2024-09-17T15:36:48.052737+0000","last_clean":"2024-09-17T15:36:48.052737+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:36:48.052737+0000","last_undegraded":"2024-09-17T15:36:48.052737+0000","last_fullsized":"2024-09-17T15:36:48.052737+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042197+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:36:48.042197+0000","last_peered":"2024-09-17T15:36:48.042197+0000","last_clean":"2024-09-17T15:36:48.042197+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:36:48.042197+0000","last_undegraded":"2024-09-17T15:36:48.042197+0000","last_fullsized":"2024-09-17T15:36:48.042197+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29127,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042385+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:36:48.042385+0000","last_peered":"2024-09-17T15:36:48.042385+0000","last_clean":"2024-09-17T15:36:48.042385+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:36:48.042385+0000","last_undegraded":"2024-09-17T15:36:48.042385+0000","last_fullsized":"2024-09-17T15:36:48.042385+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042912+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:36:48.042912+0000","last_peered":"2024-09-17T15:36:48.042912+0000","last_clean":"2024-09-17T15:36:48.042912+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:36:48.042912+0000","last_undegraded":"2024-09-17T15:36:48.042912+0000","last_fullsized":"2024-09-17T15:36:48.042912+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20834,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.059616+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:36:48.059616+0000","last_peered":"2024-09-17T15:36:48.059616+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:36:48.059616+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":130,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052749+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:36:48.052749+0000","last_peered":"2024-09-17T15:36:48.052749+0000","last_clean":"2024-09-17T15:36:48.052749+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:36:48.052749+0000","last_undegraded":"2024-09-17T15:36:48.052749+0000","last_fullsized":"2024-09-17T15:36:48.052749+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28831,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042985+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:36:48.042985+0000","last_peered":"2024-09-17T15:36:48.042985+0000","last_clean":"2024-09-17T15:36:48.042985+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:36:48.042985+0000","last_undegraded":"2024-09-17T15:36:48.042985+0000","last_fullsized":"2024-09-17T15:36:48.042985+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":105,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042299+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:36:48.042299+0000","last_peered":"2024-09-17T15:36:48.042299+0000","last_clean":"2024-09-17T15:36:48.042299+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:36:48.042299+0000","last_undegraded":"2024-09-17T15:36:48.042299+0000","last_fullsized":"2024-09-17T15:36:48.042299+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29168,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042529+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:36:48.042529+0000","last_peered":"2024-09-17T15:36:48.042529+0000","last_clean":"2024-09-17T15:36:48.042529+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:36:48.042529+0000","last_undegraded":"2024-09-17T15:36:48.042529+0000","last_fullsized":"2024-09-17T15:36:48.042529+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21290,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.053156+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:36:48.053156+0000","last_peered":"2024-09-17T15:36:48.053156+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:36:48.053156+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042552+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:36:48.042552+0000","last_peered":"2024-09-17T15:36:48.042552+0000","last_clean":"2024-09-17T15:36:48.042552+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:36:48.042552+0000","last_undegraded":"2024-09-17T15:36:48.042552+0000","last_fullsized":"2024-09-17T15:36:48.042552+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28082,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042306+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:36:48.042306+0000","last_peered":"2024-09-17T15:36:48.042306+0000","last_clean":"2024-09-17T15:36:48.042306+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:36:48.042306+0000","last_undegraded":"2024-09-17T15:36:48.042306+0000","last_fullsized":"2024-09-17T15:36:48.042306+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043694+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:36:48.043694+0000","last_peered":"2024-09-17T15:36:48.043694+0000","last_clean":"2024-09-17T15:36:48.043694+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:36:48.043694+0000","last_undegraded":"2024-09-17T15:36:48.043694+0000","last_fullsized":"2024-09-17T15:36:48.043694+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21211,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.053263+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:36:48.053263+0000","last_peered":"2024-09-17T15:36:48.053263+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:36:48.053263+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":94,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042177+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:36:48.042177+0000","last_peered":"2024-09-17T15:36:48.042177+0000","last_clean":"2024-09-17T15:36:48.042177+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:36:48.042177+0000","last_undegraded":"2024-09-17T15:36:48.042177+0000","last_fullsized":"2024-09-17T15:36:48.042177+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":99,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042677+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:36:48.042677+0000","last_peered":"2024-09-17T15:36:48.042677+0000","last_clean":"2024-09-17T15:36:48.042677+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:36:48.042677+0000","last_undegraded":"2024-09-17T15:36:48.042677+0000","last_fullsized":"2024-09-17T15:36:48.042677+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29033,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042443+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:36:48.042443+0000","last_peered":"2024-09-17T15:36:48.042443+0000","last_clean":"2024-09-17T15:36:48.042443+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:36:48.042443+0000","last_undegraded":"2024-09-17T15:36:48.042443+0000","last_fullsized":"2024-09-17T15:36:48.042443+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19739,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29721,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":200052736,"data_stored":199937120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051726,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":499320,"kb_used_data":130280,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93872520,"statfs":{"total":96636764160,"available":96125460480,"internally_reserved":0,"allocated":133406720,"data_stored":133066690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":377876140},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724670,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487560,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884280,"statfs":{"total":96636764160,"available":96137502720,"internally_reserved":0,"allocated":512000,"data_stored":160667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":498724523},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247953,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539408,"kb_used_data":66296,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832432,"statfs":{"total":96636764160,"available":96084410368,"internally_reserved":0,"allocated":67887104,"data_stored":67542997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":132579328,"data_stored":132543198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:36:54.714 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:36:54.714 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'20648", 'reported_seq': 20925, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:48.053279+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:36:48.053279+0000', 'last_peered': '2024-09-17T15:36:48.053279+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:36:48.053279+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'10600", 'ondisk_log_start': "43'10600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20648, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:54.715 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:36:54.715 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'21251", 'reported_seq': 22416, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:49.408099+0000', 'last_change': '2024-09-17T15:36:46.948876+0000', 'last_active': '2024-09-17T15:36:49.408099+0000', 'last_peered': '2024-09-17T15:36:49.408099+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:36:49.408099+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "43'11200", 'ondisk_log_start': "43'11200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3770, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17750, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 501, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3769}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:54.715 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:36:54.715 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20542", 'reported_seq': 20834, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:48.059616+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:36:48.059616+0000', 'last_peered': '2024-09-17T15:36:48.059616+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:36:48.059616+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17138, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3943}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:54.715 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:36:54.715 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'20966", 'reported_seq': 21290, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:48.053156+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:36:48.053156+0000', 'last_peered': '2024-09-17T15:36:48.053156+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:36:48.053156+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "43'10900", 'ondisk_log_start': "43'10900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17514, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4057}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:54.715 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:36:54.715 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'20886", 'reported_seq': 21211, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:48.053263+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:36:48.053263+0000', 'last_peered': '2024-09-17T15:36:48.053263+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:36:48.053263+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'10800", 'ondisk_log_start': "43'10800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17673, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3980}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:54.716 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:36:55.118 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:55.118 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 19739/24 objects degraded (82245.833%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":752,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1562918912,"bytes_avail":288347373568,"bytes_total":289910292480,"degraded_objects":19739,"degraded_total":24,"degraded_ratio":822.45833333333348,"recovering_objects_per_sec":156,"recovering_bytes_per_sec":60,"recovering_keys_per_sec":0,"num_objects_recovered":1249,"num_bytes_recovered":483,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:36:55.131 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:36:55.132 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:36:55.441 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:55.442 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:36:55.454 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":423,"stamp":"2024-09-17T15:36:53.824987+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19739,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29722,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1526288,"kb_used_data":197076,"kb_used_omap":13,"kb_used_meta":1329138,"kb_avail":281589232,"statfs":{"total":289910292480,"available":288347373568,"internally_reserved":0,"allocated":201805824,"data_stored":200770354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1361038335},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1249,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1249,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001405"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20925,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.053279+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:36:48.053279+0000","last_peered":"2024-09-17T15:36:48.053279+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:36:48.053279+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21062,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043955+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:36:48.043955+0000","last_peered":"2024-09-17T15:36:48.043955+0000","last_clean":"2024-09-17T15:36:48.043955+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:36:48.043955+0000","last_undegraded":"2024-09-17T15:36:48.043955+0000","last_fullsized":"2024-09-17T15:36:48.043955+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21170,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.120714+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:36:48.120714+0000","last_peered":"2024-09-17T15:36:48.120714+0000","last_clean":"2024-09-17T15:36:48.120714+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:36:48.120714+0000","last_undegraded":"2024-09-17T15:36:48.120714+0000","last_fullsized":"2024-09-17T15:36:48.120714+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21067,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043731+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:36:48.043731+0000","last_peered":"2024-09-17T15:36:48.043731+0000","last_clean":"2024-09-17T15:36:48.043731+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:36:48.043731+0000","last_undegraded":"2024-09-17T15:36:48.043731+0000","last_fullsized":"2024-09-17T15:36:48.043731+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20809,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052901+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:36:48.052901+0000","last_peered":"2024-09-17T15:36:48.052901+0000","last_clean":"2024-09-17T15:36:48.052901+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:36:48.052901+0000","last_undegraded":"2024-09-17T15:36:48.052901+0000","last_fullsized":"2024-09-17T15:36:48.052901+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":22416,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:49.408099+0000","last_change":"2024-09-17T15:36:46.948876+0000","last_active":"2024-09-17T15:36:49.408099+0000","last_peered":"2024-09-17T15:36:49.408099+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:36:49.408099+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3770,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":501,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3769}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21104,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042819+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:36:48.042819+0000","last_peered":"2024-09-17T15:36:48.042819+0000","last_clean":"2024-09-17T15:36:48.042819+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:36:48.042819+0000","last_undegraded":"2024-09-17T15:36:48.042819+0000","last_fullsized":"2024-09-17T15:36:48.042819+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21071,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052737+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:36:48.052737+0000","last_peered":"2024-09-17T15:36:48.052737+0000","last_clean":"2024-09-17T15:36:48.052737+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:36:48.052737+0000","last_undegraded":"2024-09-17T15:36:48.052737+0000","last_fullsized":"2024-09-17T15:36:48.052737+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042197+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:36:48.042197+0000","last_peered":"2024-09-17T15:36:48.042197+0000","last_clean":"2024-09-17T15:36:48.042197+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:36:48.042197+0000","last_undegraded":"2024-09-17T15:36:48.042197+0000","last_fullsized":"2024-09-17T15:36:48.042197+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29127,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042385+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:36:48.042385+0000","last_peered":"2024-09-17T15:36:48.042385+0000","last_clean":"2024-09-17T15:36:48.042385+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:36:48.042385+0000","last_undegraded":"2024-09-17T15:36:48.042385+0000","last_fullsized":"2024-09-17T15:36:48.042385+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042912+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:36:48.042912+0000","last_peered":"2024-09-17T15:36:48.042912+0000","last_clean":"2024-09-17T15:36:48.042912+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:36:48.042912+0000","last_undegraded":"2024-09-17T15:36:48.042912+0000","last_fullsized":"2024-09-17T15:36:48.042912+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20834,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.059616+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:36:48.059616+0000","last_peered":"2024-09-17T15:36:48.059616+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:36:48.059616+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":130,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052749+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:36:48.052749+0000","last_peered":"2024-09-17T15:36:48.052749+0000","last_clean":"2024-09-17T15:36:48.052749+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:36:48.052749+0000","last_undegraded":"2024-09-17T15:36:48.052749+0000","last_fullsized":"2024-09-17T15:36:48.052749+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28831,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042985+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:36:48.042985+0000","last_peered":"2024-09-17T15:36:48.042985+0000","last_clean":"2024-09-17T15:36:48.042985+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:36:48.042985+0000","last_undegraded":"2024-09-17T15:36:48.042985+0000","last_fullsized":"2024-09-17T15:36:48.042985+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":105,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042299+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:36:48.042299+0000","last_peered":"2024-09-17T15:36:48.042299+0000","last_clean":"2024-09-17T15:36:48.042299+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:36:48.042299+0000","last_undegraded":"2024-09-17T15:36:48.042299+0000","last_fullsized":"2024-09-17T15:36:48.042299+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29168,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042529+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:36:48.042529+0000","last_peered":"2024-09-17T15:36:48.042529+0000","last_clean":"2024-09-17T15:36:48.042529+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:36:48.042529+0000","last_undegraded":"2024-09-17T15:36:48.042529+0000","last_fullsized":"2024-09-17T15:36:48.042529+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21290,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.053156+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:36:48.053156+0000","last_peered":"2024-09-17T15:36:48.053156+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:36:48.053156+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042552+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:36:48.042552+0000","last_peered":"2024-09-17T15:36:48.042552+0000","last_clean":"2024-09-17T15:36:48.042552+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:36:48.042552+0000","last_undegraded":"2024-09-17T15:36:48.042552+0000","last_fullsized":"2024-09-17T15:36:48.042552+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28082,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042306+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:36:48.042306+0000","last_peered":"2024-09-17T15:36:48.042306+0000","last_clean":"2024-09-17T15:36:48.042306+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:36:48.042306+0000","last_undegraded":"2024-09-17T15:36:48.042306+0000","last_fullsized":"2024-09-17T15:36:48.042306+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043694+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:36:48.043694+0000","last_peered":"2024-09-17T15:36:48.043694+0000","last_clean":"2024-09-17T15:36:48.043694+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:36:48.043694+0000","last_undegraded":"2024-09-17T15:36:48.043694+0000","last_fullsized":"2024-09-17T15:36:48.043694+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21211,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.053263+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:36:48.053263+0000","last_peered":"2024-09-17T15:36:48.053263+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:36:48.053263+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":94,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042177+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:36:48.042177+0000","last_peered":"2024-09-17T15:36:48.042177+0000","last_clean":"2024-09-17T15:36:48.042177+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:36:48.042177+0000","last_undegraded":"2024-09-17T15:36:48.042177+0000","last_fullsized":"2024-09-17T15:36:48.042177+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":99,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042677+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:36:48.042677+0000","last_peered":"2024-09-17T15:36:48.042677+0000","last_clean":"2024-09-17T15:36:48.042677+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:36:48.042677+0000","last_undegraded":"2024-09-17T15:36:48.042677+0000","last_fullsized":"2024-09-17T15:36:48.042677+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29033,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042443+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:36:48.042443+0000","last_peered":"2024-09-17T15:36:48.042443+0000","last_clean":"2024-09-17T15:36:48.042443+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:36:48.042443+0000","last_undegraded":"2024-09-17T15:36:48.042443+0000","last_fullsized":"2024-09-17T15:36:48.042443+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19739,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29721,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":200052736,"data_stored":199937120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051726,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":499320,"kb_used_data":130280,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93872520,"statfs":{"total":96636764160,"available":96125460480,"internally_reserved":0,"allocated":133406720,"data_stored":133066690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":377876140},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724670,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487560,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884280,"statfs":{"total":96636764160,"available":96137502720,"internally_reserved":0,"allocated":512000,"data_stored":160667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":498724523},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247953,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539408,"kb_used_data":66296,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832432,"statfs":{"total":96636764160,"available":96084410368,"internally_reserved":0,"allocated":67887104,"data_stored":67542997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":132579328,"data_stored":132543198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:36:58.456 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:36:58.784 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:58.784 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:36:58.796 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":425,"stamp":"2024-09-17T15:36:57.825644+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19146,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30315,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1530444,"kb_used_data":197076,"kb_used_omap":13,"kb_used_meta":1333298,"kb_avail":281585076,"statfs":{"total":289910292480,"available":288343117824,"internally_reserved":0,"allocated":201805824,"data_stored":200770354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1365298175},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1842,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1842,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001680"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20925,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.053279+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:36:48.053279+0000","last_peered":"2024-09-17T15:36:48.053279+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:36:48.053279+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21062,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043955+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:36:48.043955+0000","last_peered":"2024-09-17T15:36:48.043955+0000","last_clean":"2024-09-17T15:36:48.043955+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:36:48.043955+0000","last_undegraded":"2024-09-17T15:36:48.043955+0000","last_fullsized":"2024-09-17T15:36:48.043955+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21170,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.120714+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:36:48.120714+0000","last_peered":"2024-09-17T15:36:48.120714+0000","last_clean":"2024-09-17T15:36:48.120714+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:36:48.120714+0000","last_undegraded":"2024-09-17T15:36:48.120714+0000","last_fullsized":"2024-09-17T15:36:48.120714+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21067,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043731+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:36:48.043731+0000","last_peered":"2024-09-17T15:36:48.043731+0000","last_clean":"2024-09-17T15:36:48.043731+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:36:48.043731+0000","last_undegraded":"2024-09-17T15:36:48.043731+0000","last_fullsized":"2024-09-17T15:36:48.043731+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20809,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052901+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:36:48.052901+0000","last_peered":"2024-09-17T15:36:48.052901+0000","last_clean":"2024-09-17T15:36:48.052901+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:36:48.052901+0000","last_undegraded":"2024-09-17T15:36:48.052901+0000","last_fullsized":"2024-09-17T15:36:48.052901+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":23602,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:54.408499+0000","last_change":"2024-09-17T15:36:46.948876+0000","last_active":"2024-09-17T15:36:54.408499+0000","last_peered":"2024-09-17T15:36:54.408499+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:36:54.408499+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3177,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1094,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3176}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21104,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042819+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:36:48.042819+0000","last_peered":"2024-09-17T15:36:48.042819+0000","last_clean":"2024-09-17T15:36:48.042819+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:36:48.042819+0000","last_undegraded":"2024-09-17T15:36:48.042819+0000","last_fullsized":"2024-09-17T15:36:48.042819+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21071,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052737+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:36:48.052737+0000","last_peered":"2024-09-17T15:36:48.052737+0000","last_clean":"2024-09-17T15:36:48.052737+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:36:48.052737+0000","last_undegraded":"2024-09-17T15:36:48.052737+0000","last_fullsized":"2024-09-17T15:36:48.052737+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042197+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:36:48.042197+0000","last_peered":"2024-09-17T15:36:48.042197+0000","last_clean":"2024-09-17T15:36:48.042197+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:36:48.042197+0000","last_undegraded":"2024-09-17T15:36:48.042197+0000","last_fullsized":"2024-09-17T15:36:48.042197+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29127,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042385+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:36:48.042385+0000","last_peered":"2024-09-17T15:36:48.042385+0000","last_clean":"2024-09-17T15:36:48.042385+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:36:48.042385+0000","last_undegraded":"2024-09-17T15:36:48.042385+0000","last_fullsized":"2024-09-17T15:36:48.042385+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042912+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:36:48.042912+0000","last_peered":"2024-09-17T15:36:48.042912+0000","last_clean":"2024-09-17T15:36:48.042912+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:36:48.042912+0000","last_undegraded":"2024-09-17T15:36:48.042912+0000","last_fullsized":"2024-09-17T15:36:48.042912+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20834,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.059616+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:36:48.059616+0000","last_peered":"2024-09-17T15:36:48.059616+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:36:48.059616+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":130,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052749+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:36:48.052749+0000","last_peered":"2024-09-17T15:36:48.052749+0000","last_clean":"2024-09-17T15:36:48.052749+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:36:48.052749+0000","last_undegraded":"2024-09-17T15:36:48.052749+0000","last_fullsized":"2024-09-17T15:36:48.052749+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28831,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042985+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:36:48.042985+0000","last_peered":"2024-09-17T15:36:48.042985+0000","last_clean":"2024-09-17T15:36:48.042985+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:36:48.042985+0000","last_undegraded":"2024-09-17T15:36:48.042985+0000","last_fullsized":"2024-09-17T15:36:48.042985+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":105,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042299+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:36:48.042299+0000","last_peered":"2024-09-17T15:36:48.042299+0000","last_clean":"2024-09-17T15:36:48.042299+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:36:48.042299+0000","last_undegraded":"2024-09-17T15:36:48.042299+0000","last_fullsized":"2024-09-17T15:36:48.042299+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29168,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042529+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:36:48.042529+0000","last_peered":"2024-09-17T15:36:48.042529+0000","last_clean":"2024-09-17T15:36:48.042529+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:36:48.042529+0000","last_undegraded":"2024-09-17T15:36:48.042529+0000","last_fullsized":"2024-09-17T15:36:48.042529+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21290,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.053156+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:36:48.053156+0000","last_peered":"2024-09-17T15:36:48.053156+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:36:48.053156+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042552+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:36:48.042552+0000","last_peered":"2024-09-17T15:36:48.042552+0000","last_clean":"2024-09-17T15:36:48.042552+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:36:48.042552+0000","last_undegraded":"2024-09-17T15:36:48.042552+0000","last_fullsized":"2024-09-17T15:36:48.042552+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28082,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042306+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:36:48.042306+0000","last_peered":"2024-09-17T15:36:48.042306+0000","last_clean":"2024-09-17T15:36:48.042306+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:36:48.042306+0000","last_undegraded":"2024-09-17T15:36:48.042306+0000","last_fullsized":"2024-09-17T15:36:48.042306+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043694+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:36:48.043694+0000","last_peered":"2024-09-17T15:36:48.043694+0000","last_clean":"2024-09-17T15:36:48.043694+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:36:48.043694+0000","last_undegraded":"2024-09-17T15:36:48.043694+0000","last_fullsized":"2024-09-17T15:36:48.043694+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21211,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.053263+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:36:48.053263+0000","last_peered":"2024-09-17T15:36:48.053263+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:36:48.053263+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":94,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042177+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:36:48.042177+0000","last_peered":"2024-09-17T15:36:48.042177+0000","last_clean":"2024-09-17T15:36:48.042177+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:36:48.042177+0000","last_undegraded":"2024-09-17T15:36:48.042177+0000","last_fullsized":"2024-09-17T15:36:48.042177+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":99,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042677+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:36:48.042677+0000","last_peered":"2024-09-17T15:36:48.042677+0000","last_clean":"2024-09-17T15:36:48.042677+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:36:48.042677+0000","last_undegraded":"2024-09-17T15:36:48.042677+0000","last_fullsized":"2024-09-17T15:36:48.042677+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29033,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042443+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:36:48.042443+0000","last_peered":"2024-09-17T15:36:48.042443+0000","last_clean":"2024-09-17T15:36:48.042443+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:36:48.042443+0000","last_undegraded":"2024-09-17T15:36:48.042443+0000","last_fullsized":"2024-09-17T15:36:48.042443+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19146,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30314,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":200052736,"data_stored":199937120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051726,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":499320,"kb_used_data":130280,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93872520,"statfs":{"total":96636764160,"available":96125460480,"internally_reserved":0,"allocated":133406720,"data_stored":133066690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":377876140},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724671,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491716,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880124,"statfs":{"total":96636764160,"available":96133246976,"internally_reserved":0,"allocated":512000,"data_stored":160667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":502984363},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247954,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539408,"kb_used_data":66296,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832432,"statfs":{"total":96636764160,"available":96084410368,"internally_reserved":0,"allocated":67887104,"data_stored":67542997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":132579328,"data_stored":132543198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:36:58.798 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:36:59.129 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:59.130 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:36:59.143 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":425,"stamp":"2024-09-17T15:36:57.825644+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19146,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30315,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1530444,"kb_used_data":197076,"kb_used_omap":13,"kb_used_meta":1333298,"kb_avail":281585076,"statfs":{"total":289910292480,"available":288343117824,"internally_reserved":0,"allocated":201805824,"data_stored":200770354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1365298175},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1842,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1842,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001680"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20925,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.053279+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:36:48.053279+0000","last_peered":"2024-09-17T15:36:48.053279+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:36:48.053279+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21062,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043955+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:36:48.043955+0000","last_peered":"2024-09-17T15:36:48.043955+0000","last_clean":"2024-09-17T15:36:48.043955+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:36:48.043955+0000","last_undegraded":"2024-09-17T15:36:48.043955+0000","last_fullsized":"2024-09-17T15:36:48.043955+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21170,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.120714+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:36:48.120714+0000","last_peered":"2024-09-17T15:36:48.120714+0000","last_clean":"2024-09-17T15:36:48.120714+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:36:48.120714+0000","last_undegraded":"2024-09-17T15:36:48.120714+0000","last_fullsized":"2024-09-17T15:36:48.120714+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21067,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043731+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:36:48.043731+0000","last_peered":"2024-09-17T15:36:48.043731+0000","last_clean":"2024-09-17T15:36:48.043731+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:36:48.043731+0000","last_undegraded":"2024-09-17T15:36:48.043731+0000","last_fullsized":"2024-09-17T15:36:48.043731+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20809,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052901+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:36:48.052901+0000","last_peered":"2024-09-17T15:36:48.052901+0000","last_clean":"2024-09-17T15:36:48.052901+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:36:48.052901+0000","last_undegraded":"2024-09-17T15:36:48.052901+0000","last_fullsized":"2024-09-17T15:36:48.052901+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":23602,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:54.408499+0000","last_change":"2024-09-17T15:36:46.948876+0000","last_active":"2024-09-17T15:36:54.408499+0000","last_peered":"2024-09-17T15:36:54.408499+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:36:54.408499+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3177,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1094,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3176}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21104,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042819+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:36:48.042819+0000","last_peered":"2024-09-17T15:36:48.042819+0000","last_clean":"2024-09-17T15:36:48.042819+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:36:48.042819+0000","last_undegraded":"2024-09-17T15:36:48.042819+0000","last_fullsized":"2024-09-17T15:36:48.042819+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21071,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052737+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:36:48.052737+0000","last_peered":"2024-09-17T15:36:48.052737+0000","last_clean":"2024-09-17T15:36:48.052737+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:36:48.052737+0000","last_undegraded":"2024-09-17T15:36:48.052737+0000","last_fullsized":"2024-09-17T15:36:48.052737+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042197+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:36:48.042197+0000","last_peered":"2024-09-17T15:36:48.042197+0000","last_clean":"2024-09-17T15:36:48.042197+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:36:48.042197+0000","last_undegraded":"2024-09-17T15:36:48.042197+0000","last_fullsized":"2024-09-17T15:36:48.042197+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29127,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042385+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:36:48.042385+0000","last_peered":"2024-09-17T15:36:48.042385+0000","last_clean":"2024-09-17T15:36:48.042385+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:36:48.042385+0000","last_undegraded":"2024-09-17T15:36:48.042385+0000","last_fullsized":"2024-09-17T15:36:48.042385+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042912+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:36:48.042912+0000","last_peered":"2024-09-17T15:36:48.042912+0000","last_clean":"2024-09-17T15:36:48.042912+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:36:48.042912+0000","last_undegraded":"2024-09-17T15:36:48.042912+0000","last_fullsized":"2024-09-17T15:36:48.042912+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20834,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.059616+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:36:48.059616+0000","last_peered":"2024-09-17T15:36:48.059616+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:36:48.059616+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":130,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052749+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:36:48.052749+0000","last_peered":"2024-09-17T15:36:48.052749+0000","last_clean":"2024-09-17T15:36:48.052749+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:36:48.052749+0000","last_undegraded":"2024-09-17T15:36:48.052749+0000","last_fullsized":"2024-09-17T15:36:48.052749+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28831,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042985+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:36:48.042985+0000","last_peered":"2024-09-17T15:36:48.042985+0000","last_clean":"2024-09-17T15:36:48.042985+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:36:48.042985+0000","last_undegraded":"2024-09-17T15:36:48.042985+0000","last_fullsized":"2024-09-17T15:36:48.042985+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":105,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042299+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:36:48.042299+0000","last_peered":"2024-09-17T15:36:48.042299+0000","last_clean":"2024-09-17T15:36:48.042299+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:36:48.042299+0000","last_undegraded":"2024-09-17T15:36:48.042299+0000","last_fullsized":"2024-09-17T15:36:48.042299+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29168,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042529+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:36:48.042529+0000","last_peered":"2024-09-17T15:36:48.042529+0000","last_clean":"2024-09-17T15:36:48.042529+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:36:48.042529+0000","last_undegraded":"2024-09-17T15:36:48.042529+0000","last_fullsized":"2024-09-17T15:36:48.042529+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21290,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.053156+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:36:48.053156+0000","last_peered":"2024-09-17T15:36:48.053156+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:36:48.053156+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042552+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:36:48.042552+0000","last_peered":"2024-09-17T15:36:48.042552+0000","last_clean":"2024-09-17T15:36:48.042552+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:36:48.042552+0000","last_undegraded":"2024-09-17T15:36:48.042552+0000","last_fullsized":"2024-09-17T15:36:48.042552+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28082,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042306+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:36:48.042306+0000","last_peered":"2024-09-17T15:36:48.042306+0000","last_clean":"2024-09-17T15:36:48.042306+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:36:48.042306+0000","last_undegraded":"2024-09-17T15:36:48.042306+0000","last_fullsized":"2024-09-17T15:36:48.042306+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043694+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:36:48.043694+0000","last_peered":"2024-09-17T15:36:48.043694+0000","last_clean":"2024-09-17T15:36:48.043694+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:36:48.043694+0000","last_undegraded":"2024-09-17T15:36:48.043694+0000","last_fullsized":"2024-09-17T15:36:48.043694+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21211,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.053263+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:36:48.053263+0000","last_peered":"2024-09-17T15:36:48.053263+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:36:48.053263+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":94,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042177+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:36:48.042177+0000","last_peered":"2024-09-17T15:36:48.042177+0000","last_clean":"2024-09-17T15:36:48.042177+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:36:48.042177+0000","last_undegraded":"2024-09-17T15:36:48.042177+0000","last_fullsized":"2024-09-17T15:36:48.042177+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":99,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042677+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:36:48.042677+0000","last_peered":"2024-09-17T15:36:48.042677+0000","last_clean":"2024-09-17T15:36:48.042677+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:36:48.042677+0000","last_undegraded":"2024-09-17T15:36:48.042677+0000","last_fullsized":"2024-09-17T15:36:48.042677+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29033,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042443+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:36:48.042443+0000","last_peered":"2024-09-17T15:36:48.042443+0000","last_clean":"2024-09-17T15:36:48.042443+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:36:48.042443+0000","last_undegraded":"2024-09-17T15:36:48.042443+0000","last_fullsized":"2024-09-17T15:36:48.042443+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19146,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30314,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":200052736,"data_stored":199937120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051726,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":499320,"kb_used_data":130280,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93872520,"statfs":{"total":96636764160,"available":96125460480,"internally_reserved":0,"allocated":133406720,"data_stored":133066690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":377876140},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724671,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491716,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880124,"statfs":{"total":96636764160,"available":96133246976,"internally_reserved":0,"allocated":512000,"data_stored":160667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":502984363},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247954,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539408,"kb_used_data":66296,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832432,"statfs":{"total":96636764160,"available":96084410368,"internally_reserved":0,"allocated":67887104,"data_stored":67542997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":132579328,"data_stored":132543198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:36:59.144 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:36:59.144 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'20648", 'reported_seq': 20925, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:48.053279+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:36:48.053279+0000', 'last_peered': '2024-09-17T15:36:48.053279+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:36:48.053279+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'10600", 'ondisk_log_start': "43'10600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20648, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:59.144 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:36:59.144 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'21251", 'reported_seq': 23602, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:54.408499+0000', 'last_change': '2024-09-17T15:36:46.948876+0000', 'last_active': '2024-09-17T15:36:54.408499+0000', 'last_peered': '2024-09-17T15:36:54.408499+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:36:54.408499+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "43'11200", 'ondisk_log_start': "43'11200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3177, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17750, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1094, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3176}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:59.144 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:36:59.145 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20542", 'reported_seq': 20834, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:48.059616+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:36:48.059616+0000', 'last_peered': '2024-09-17T15:36:48.059616+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:36:48.059616+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17138, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3943}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:59.145 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:36:59.145 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'20966", 'reported_seq': 21290, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:48.053156+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:36:48.053156+0000', 'last_peered': '2024-09-17T15:36:48.053156+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:36:48.053156+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "43'10900", 'ondisk_log_start': "43'10900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17514, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4057}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:59.145 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:36:59.145 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'20886", 'reported_seq': 21211, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:48.053263+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:36:48.053263+0000', 'last_peered': '2024-09-17T15:36:48.053263+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:36:48.053263+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'10800", 'ondisk_log_start': "43'10800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17673, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3980}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:36:59.145 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:36:59.547 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:59.547 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 19146/24 objects degraded (79775.000%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":756,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1567174656,"bytes_avail":288343117824,"bytes_total":289910292480,"degraded_objects":19146,"degraded_total":24,"degraded_ratio":797.75,"recovering_objects_per_sec":184,"recovering_bytes_per_sec":48,"recovering_keys_per_sec":0,"num_objects_recovered":1842,"num_bytes_recovered":483,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:36:59.564 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:36:59.564 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:36:59.872 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:36:59.873 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:36:59.885 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":426,"stamp":"2024-09-17T15:36:59.825937+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18544,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30917,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1516164,"kb_used_data":195276,"kb_used_omap":13,"kb_used_meta":1320818,"kb_avail":281599356,"statfs":{"total":289910292480,"available":288357740544,"internally_reserved":0,"allocated":199962624,"data_stored":198927154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1352518655},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2444,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.784657"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20925,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.053279+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:36:48.053279+0000","last_peered":"2024-09-17T15:36:48.053279+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:36:48.053279+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21062,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043955+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:36:48.043955+0000","last_peered":"2024-09-17T15:36:48.043955+0000","last_clean":"2024-09-17T15:36:48.043955+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:36:48.043955+0000","last_undegraded":"2024-09-17T15:36:48.043955+0000","last_fullsized":"2024-09-17T15:36:48.043955+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21170,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.120714+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:36:48.120714+0000","last_peered":"2024-09-17T15:36:48.120714+0000","last_clean":"2024-09-17T15:36:48.120714+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:36:48.120714+0000","last_undegraded":"2024-09-17T15:36:48.120714+0000","last_fullsized":"2024-09-17T15:36:48.120714+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21067,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043731+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:36:48.043731+0000","last_peered":"2024-09-17T15:36:48.043731+0000","last_clean":"2024-09-17T15:36:48.043731+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:36:48.043731+0000","last_undegraded":"2024-09-17T15:36:48.043731+0000","last_fullsized":"2024-09-17T15:36:48.043731+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20809,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052901+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:36:48.052901+0000","last_peered":"2024-09-17T15:36:48.052901+0000","last_clean":"2024-09-17T15:36:48.052901+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:36:48.052901+0000","last_undegraded":"2024-09-17T15:36:48.052901+0000","last_fullsized":"2024-09-17T15:36:48.052901+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":24806,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:59.408673+0000","last_change":"2024-09-17T15:36:46.948876+0000","last_active":"2024-09-17T15:36:59.408673+0000","last_peered":"2024-09-17T15:36:59.408673+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:36:59.408673+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2575,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1696,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2574}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21104,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042819+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:36:48.042819+0000","last_peered":"2024-09-17T15:36:48.042819+0000","last_clean":"2024-09-17T15:36:48.042819+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:36:48.042819+0000","last_undegraded":"2024-09-17T15:36:48.042819+0000","last_fullsized":"2024-09-17T15:36:48.042819+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21071,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052737+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:36:48.052737+0000","last_peered":"2024-09-17T15:36:48.052737+0000","last_clean":"2024-09-17T15:36:48.052737+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:36:48.052737+0000","last_undegraded":"2024-09-17T15:36:48.052737+0000","last_fullsized":"2024-09-17T15:36:48.052737+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042197+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:36:48.042197+0000","last_peered":"2024-09-17T15:36:48.042197+0000","last_clean":"2024-09-17T15:36:48.042197+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:36:48.042197+0000","last_undegraded":"2024-09-17T15:36:48.042197+0000","last_fullsized":"2024-09-17T15:36:48.042197+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29127,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042385+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:36:48.042385+0000","last_peered":"2024-09-17T15:36:48.042385+0000","last_clean":"2024-09-17T15:36:48.042385+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:36:48.042385+0000","last_undegraded":"2024-09-17T15:36:48.042385+0000","last_fullsized":"2024-09-17T15:36:48.042385+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042912+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:36:48.042912+0000","last_peered":"2024-09-17T15:36:48.042912+0000","last_clean":"2024-09-17T15:36:48.042912+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:36:48.042912+0000","last_undegraded":"2024-09-17T15:36:48.042912+0000","last_fullsized":"2024-09-17T15:36:48.042912+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20834,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.059616+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:36:48.059616+0000","last_peered":"2024-09-17T15:36:48.059616+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:36:48.059616+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":130,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052749+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:36:48.052749+0000","last_peered":"2024-09-17T15:36:48.052749+0000","last_clean":"2024-09-17T15:36:48.052749+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:36:48.052749+0000","last_undegraded":"2024-09-17T15:36:48.052749+0000","last_fullsized":"2024-09-17T15:36:48.052749+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28831,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042985+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:36:48.042985+0000","last_peered":"2024-09-17T15:36:48.042985+0000","last_clean":"2024-09-17T15:36:48.042985+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:36:48.042985+0000","last_undegraded":"2024-09-17T15:36:48.042985+0000","last_fullsized":"2024-09-17T15:36:48.042985+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":105,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042299+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:36:48.042299+0000","last_peered":"2024-09-17T15:36:48.042299+0000","last_clean":"2024-09-17T15:36:48.042299+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:36:48.042299+0000","last_undegraded":"2024-09-17T15:36:48.042299+0000","last_fullsized":"2024-09-17T15:36:48.042299+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29168,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042529+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:36:48.042529+0000","last_peered":"2024-09-17T15:36:48.042529+0000","last_clean":"2024-09-17T15:36:48.042529+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:36:48.042529+0000","last_undegraded":"2024-09-17T15:36:48.042529+0000","last_fullsized":"2024-09-17T15:36:48.042529+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21290,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.053156+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:36:48.053156+0000","last_peered":"2024-09-17T15:36:48.053156+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:36:48.053156+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042552+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:36:48.042552+0000","last_peered":"2024-09-17T15:36:48.042552+0000","last_clean":"2024-09-17T15:36:48.042552+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:36:48.042552+0000","last_undegraded":"2024-09-17T15:36:48.042552+0000","last_fullsized":"2024-09-17T15:36:48.042552+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28082,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042306+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:36:48.042306+0000","last_peered":"2024-09-17T15:36:48.042306+0000","last_clean":"2024-09-17T15:36:48.042306+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:36:48.042306+0000","last_undegraded":"2024-09-17T15:36:48.042306+0000","last_fullsized":"2024-09-17T15:36:48.042306+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043694+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:36:48.043694+0000","last_peered":"2024-09-17T15:36:48.043694+0000","last_clean":"2024-09-17T15:36:48.043694+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:36:48.043694+0000","last_undegraded":"2024-09-17T15:36:48.043694+0000","last_fullsized":"2024-09-17T15:36:48.043694+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21211,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.053263+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:36:48.053263+0000","last_peered":"2024-09-17T15:36:48.053263+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:36:48.053263+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042251+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:36:48.042251+0000","last_peered":"2024-09-17T15:36:48.042251+0000","last_clean":"2024-09-17T15:36:48.042251+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:36:48.042251+0000","last_undegraded":"2024-09-17T15:36:48.042251+0000","last_fullsized":"2024-09-17T15:36:48.042251+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":99,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042677+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:36:48.042677+0000","last_peered":"2024-09-17T15:36:48.042677+0000","last_clean":"2024-09-17T15:36:48.042677+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:36:48.042677+0000","last_undegraded":"2024-09-17T15:36:48.042677+0000","last_fullsized":"2024-09-17T15:36:48.042677+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29033,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042443+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:36:48.042443+0000","last_peered":"2024-09-17T15:36:48.042443+0000","last_clean":"2024-09-17T15:36:48.042443+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:36:48.042443+0000","last_undegraded":"2024-09-17T15:36:48.042443+0000","last_fullsized":"2024-09-17T15:36:48.042443+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18544,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30916,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":198209536,"data_stored":198093920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051727,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497520,"kb_used_data":128480,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93874320,"statfs":{"total":96636764160,"available":96127303680,"internally_reserved":0,"allocated":131563520,"data_stored":131223490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":377876140},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724672,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479236,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892604,"statfs":{"total":96636764160,"available":96146026496,"internally_reserved":0,"allocated":512000,"data_stored":160667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":490204843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247955,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539408,"kb_used_data":66296,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832432,"statfs":{"total":96636764160,"available":96084410368,"internally_reserved":0,"allocated":67887104,"data_stored":67542997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130736128,"data_stored":130699998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:37:02.888 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:37:03.191 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:03.191 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:37:03.203 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":427,"stamp":"2024-09-17T15:37:01.826285+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18544,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30917,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1516164,"kb_used_data":195276,"kb_used_omap":13,"kb_used_meta":1320818,"kb_avail":281599356,"statfs":{"total":289910292480,"available":288357740544,"internally_reserved":0,"allocated":199962624,"data_stored":198927154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1352518655},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002027"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20925,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.053279+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:36:48.053279+0000","last_peered":"2024-09-17T15:36:48.053279+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:36:48.053279+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21062,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043955+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:36:48.043955+0000","last_peered":"2024-09-17T15:36:48.043955+0000","last_clean":"2024-09-17T15:36:48.043955+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:36:48.043955+0000","last_undegraded":"2024-09-17T15:36:48.043955+0000","last_fullsized":"2024-09-17T15:36:48.043955+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21170,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.120714+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:36:48.120714+0000","last_peered":"2024-09-17T15:36:48.120714+0000","last_clean":"2024-09-17T15:36:48.120714+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:36:48.120714+0000","last_undegraded":"2024-09-17T15:36:48.120714+0000","last_fullsized":"2024-09-17T15:36:48.120714+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21067,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043731+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:36:48.043731+0000","last_peered":"2024-09-17T15:36:48.043731+0000","last_clean":"2024-09-17T15:36:48.043731+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:36:48.043731+0000","last_undegraded":"2024-09-17T15:36:48.043731+0000","last_fullsized":"2024-09-17T15:36:48.043731+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20809,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052901+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:36:48.052901+0000","last_peered":"2024-09-17T15:36:48.052901+0000","last_clean":"2024-09-17T15:36:48.052901+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:36:48.052901+0000","last_undegraded":"2024-09-17T15:36:48.052901+0000","last_fullsized":"2024-09-17T15:36:48.052901+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":24806,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:59.408673+0000","last_change":"2024-09-17T15:36:46.948876+0000","last_active":"2024-09-17T15:36:59.408673+0000","last_peered":"2024-09-17T15:36:59.408673+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:36:59.408673+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2575,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1696,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2574}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21104,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042819+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:36:48.042819+0000","last_peered":"2024-09-17T15:36:48.042819+0000","last_clean":"2024-09-17T15:36:48.042819+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:36:48.042819+0000","last_undegraded":"2024-09-17T15:36:48.042819+0000","last_fullsized":"2024-09-17T15:36:48.042819+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21071,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052737+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:36:48.052737+0000","last_peered":"2024-09-17T15:36:48.052737+0000","last_clean":"2024-09-17T15:36:48.052737+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:36:48.052737+0000","last_undegraded":"2024-09-17T15:36:48.052737+0000","last_fullsized":"2024-09-17T15:36:48.052737+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042197+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:36:48.042197+0000","last_peered":"2024-09-17T15:36:48.042197+0000","last_clean":"2024-09-17T15:36:48.042197+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:36:48.042197+0000","last_undegraded":"2024-09-17T15:36:48.042197+0000","last_fullsized":"2024-09-17T15:36:48.042197+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29127,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042385+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:36:48.042385+0000","last_peered":"2024-09-17T15:36:48.042385+0000","last_clean":"2024-09-17T15:36:48.042385+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:36:48.042385+0000","last_undegraded":"2024-09-17T15:36:48.042385+0000","last_fullsized":"2024-09-17T15:36:48.042385+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042912+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:36:48.042912+0000","last_peered":"2024-09-17T15:36:48.042912+0000","last_clean":"2024-09-17T15:36:48.042912+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:36:48.042912+0000","last_undegraded":"2024-09-17T15:36:48.042912+0000","last_fullsized":"2024-09-17T15:36:48.042912+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20834,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.059616+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:36:48.059616+0000","last_peered":"2024-09-17T15:36:48.059616+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:36:48.059616+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":130,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052749+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:36:48.052749+0000","last_peered":"2024-09-17T15:36:48.052749+0000","last_clean":"2024-09-17T15:36:48.052749+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:36:48.052749+0000","last_undegraded":"2024-09-17T15:36:48.052749+0000","last_fullsized":"2024-09-17T15:36:48.052749+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28831,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042985+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:36:48.042985+0000","last_peered":"2024-09-17T15:36:48.042985+0000","last_clean":"2024-09-17T15:36:48.042985+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:36:48.042985+0000","last_undegraded":"2024-09-17T15:36:48.042985+0000","last_fullsized":"2024-09-17T15:36:48.042985+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":105,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042299+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:36:48.042299+0000","last_peered":"2024-09-17T15:36:48.042299+0000","last_clean":"2024-09-17T15:36:48.042299+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:36:48.042299+0000","last_undegraded":"2024-09-17T15:36:48.042299+0000","last_fullsized":"2024-09-17T15:36:48.042299+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29168,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042529+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:36:48.042529+0000","last_peered":"2024-09-17T15:36:48.042529+0000","last_clean":"2024-09-17T15:36:48.042529+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:36:48.042529+0000","last_undegraded":"2024-09-17T15:36:48.042529+0000","last_fullsized":"2024-09-17T15:36:48.042529+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21290,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.053156+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:36:48.053156+0000","last_peered":"2024-09-17T15:36:48.053156+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:36:48.053156+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042552+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:36:48.042552+0000","last_peered":"2024-09-17T15:36:48.042552+0000","last_clean":"2024-09-17T15:36:48.042552+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:36:48.042552+0000","last_undegraded":"2024-09-17T15:36:48.042552+0000","last_fullsized":"2024-09-17T15:36:48.042552+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28082,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042306+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:36:48.042306+0000","last_peered":"2024-09-17T15:36:48.042306+0000","last_clean":"2024-09-17T15:36:48.042306+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:36:48.042306+0000","last_undegraded":"2024-09-17T15:36:48.042306+0000","last_fullsized":"2024-09-17T15:36:48.042306+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043694+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:36:48.043694+0000","last_peered":"2024-09-17T15:36:48.043694+0000","last_clean":"2024-09-17T15:36:48.043694+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:36:48.043694+0000","last_undegraded":"2024-09-17T15:36:48.043694+0000","last_fullsized":"2024-09-17T15:36:48.043694+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21211,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.053263+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:36:48.053263+0000","last_peered":"2024-09-17T15:36:48.053263+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:36:48.053263+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042251+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:36:48.042251+0000","last_peered":"2024-09-17T15:36:48.042251+0000","last_clean":"2024-09-17T15:36:48.042251+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:36:48.042251+0000","last_undegraded":"2024-09-17T15:36:48.042251+0000","last_fullsized":"2024-09-17T15:36:48.042251+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":99,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042677+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:36:48.042677+0000","last_peered":"2024-09-17T15:36:48.042677+0000","last_clean":"2024-09-17T15:36:48.042677+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:36:48.042677+0000","last_undegraded":"2024-09-17T15:36:48.042677+0000","last_fullsized":"2024-09-17T15:36:48.042677+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29033,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042443+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:36:48.042443+0000","last_peered":"2024-09-17T15:36:48.042443+0000","last_clean":"2024-09-17T15:36:48.042443+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:36:48.042443+0000","last_undegraded":"2024-09-17T15:36:48.042443+0000","last_fullsized":"2024-09-17T15:36:48.042443+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18544,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30916,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":198209536,"data_stored":198093920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051727,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497520,"kb_used_data":128480,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93874320,"statfs":{"total":96636764160,"available":96127303680,"internally_reserved":0,"allocated":131563520,"data_stored":131223490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":377876140},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724672,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479236,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892604,"statfs":{"total":96636764160,"available":96146026496,"internally_reserved":0,"allocated":512000,"data_stored":160667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":490204843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247955,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539408,"kb_used_data":66296,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832432,"statfs":{"total":96636764160,"available":96084410368,"internally_reserved":0,"allocated":67887104,"data_stored":67542997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130736128,"data_stored":130699998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:37:03.205 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:37:03.565 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:03.566 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:37:03.578 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":427,"stamp":"2024-09-17T15:37:01.826285+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18544,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30917,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1516164,"kb_used_data":195276,"kb_used_omap":13,"kb_used_meta":1320818,"kb_avail":281599356,"statfs":{"total":289910292480,"available":288357740544,"internally_reserved":0,"allocated":199962624,"data_stored":198927154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1352518655},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002027"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20925,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.053279+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:36:48.053279+0000","last_peered":"2024-09-17T15:36:48.053279+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:36:48.053279+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21062,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043955+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:36:48.043955+0000","last_peered":"2024-09-17T15:36:48.043955+0000","last_clean":"2024-09-17T15:36:48.043955+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:36:48.043955+0000","last_undegraded":"2024-09-17T15:36:48.043955+0000","last_fullsized":"2024-09-17T15:36:48.043955+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21170,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.120714+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:36:48.120714+0000","last_peered":"2024-09-17T15:36:48.120714+0000","last_clean":"2024-09-17T15:36:48.120714+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:36:48.120714+0000","last_undegraded":"2024-09-17T15:36:48.120714+0000","last_fullsized":"2024-09-17T15:36:48.120714+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21067,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043731+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:36:48.043731+0000","last_peered":"2024-09-17T15:36:48.043731+0000","last_clean":"2024-09-17T15:36:48.043731+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:36:48.043731+0000","last_undegraded":"2024-09-17T15:36:48.043731+0000","last_fullsized":"2024-09-17T15:36:48.043731+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20809,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052901+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:36:48.052901+0000","last_peered":"2024-09-17T15:36:48.052901+0000","last_clean":"2024-09-17T15:36:48.052901+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:36:48.052901+0000","last_undegraded":"2024-09-17T15:36:48.052901+0000","last_fullsized":"2024-09-17T15:36:48.052901+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":24806,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:59.408673+0000","last_change":"2024-09-17T15:36:46.948876+0000","last_active":"2024-09-17T15:36:59.408673+0000","last_peered":"2024-09-17T15:36:59.408673+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:36:59.408673+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2575,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1696,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2574}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21104,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042819+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:36:48.042819+0000","last_peered":"2024-09-17T15:36:48.042819+0000","last_clean":"2024-09-17T15:36:48.042819+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:36:48.042819+0000","last_undegraded":"2024-09-17T15:36:48.042819+0000","last_fullsized":"2024-09-17T15:36:48.042819+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21071,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052737+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:36:48.052737+0000","last_peered":"2024-09-17T15:36:48.052737+0000","last_clean":"2024-09-17T15:36:48.052737+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:36:48.052737+0000","last_undegraded":"2024-09-17T15:36:48.052737+0000","last_fullsized":"2024-09-17T15:36:48.052737+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042197+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:36:48.042197+0000","last_peered":"2024-09-17T15:36:48.042197+0000","last_clean":"2024-09-17T15:36:48.042197+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:36:48.042197+0000","last_undegraded":"2024-09-17T15:36:48.042197+0000","last_fullsized":"2024-09-17T15:36:48.042197+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29127,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042385+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:36:48.042385+0000","last_peered":"2024-09-17T15:36:48.042385+0000","last_clean":"2024-09-17T15:36:48.042385+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:36:48.042385+0000","last_undegraded":"2024-09-17T15:36:48.042385+0000","last_fullsized":"2024-09-17T15:36:48.042385+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042912+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:36:48.042912+0000","last_peered":"2024-09-17T15:36:48.042912+0000","last_clean":"2024-09-17T15:36:48.042912+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:36:48.042912+0000","last_undegraded":"2024-09-17T15:36:48.042912+0000","last_fullsized":"2024-09-17T15:36:48.042912+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20834,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.059616+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:36:48.059616+0000","last_peered":"2024-09-17T15:36:48.059616+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:36:48.059616+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":130,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052749+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:36:48.052749+0000","last_peered":"2024-09-17T15:36:48.052749+0000","last_clean":"2024-09-17T15:36:48.052749+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:36:48.052749+0000","last_undegraded":"2024-09-17T15:36:48.052749+0000","last_fullsized":"2024-09-17T15:36:48.052749+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28831,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042985+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:36:48.042985+0000","last_peered":"2024-09-17T15:36:48.042985+0000","last_clean":"2024-09-17T15:36:48.042985+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:36:48.042985+0000","last_undegraded":"2024-09-17T15:36:48.042985+0000","last_fullsized":"2024-09-17T15:36:48.042985+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":105,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042299+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:36:48.042299+0000","last_peered":"2024-09-17T15:36:48.042299+0000","last_clean":"2024-09-17T15:36:48.042299+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:36:48.042299+0000","last_undegraded":"2024-09-17T15:36:48.042299+0000","last_fullsized":"2024-09-17T15:36:48.042299+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29168,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042529+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:36:48.042529+0000","last_peered":"2024-09-17T15:36:48.042529+0000","last_clean":"2024-09-17T15:36:48.042529+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:36:48.042529+0000","last_undegraded":"2024-09-17T15:36:48.042529+0000","last_fullsized":"2024-09-17T15:36:48.042529+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21290,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.053156+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:36:48.053156+0000","last_peered":"2024-09-17T15:36:48.053156+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:36:48.053156+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042552+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:36:48.042552+0000","last_peered":"2024-09-17T15:36:48.042552+0000","last_clean":"2024-09-17T15:36:48.042552+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:36:48.042552+0000","last_undegraded":"2024-09-17T15:36:48.042552+0000","last_fullsized":"2024-09-17T15:36:48.042552+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28082,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042306+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:36:48.042306+0000","last_peered":"2024-09-17T15:36:48.042306+0000","last_clean":"2024-09-17T15:36:48.042306+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:36:48.042306+0000","last_undegraded":"2024-09-17T15:36:48.042306+0000","last_fullsized":"2024-09-17T15:36:48.042306+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043694+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:36:48.043694+0000","last_peered":"2024-09-17T15:36:48.043694+0000","last_clean":"2024-09-17T15:36:48.043694+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:36:48.043694+0000","last_undegraded":"2024-09-17T15:36:48.043694+0000","last_fullsized":"2024-09-17T15:36:48.043694+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21211,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.053263+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:36:48.053263+0000","last_peered":"2024-09-17T15:36:48.053263+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:36:48.053263+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042251+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:36:48.042251+0000","last_peered":"2024-09-17T15:36:48.042251+0000","last_clean":"2024-09-17T15:36:48.042251+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:36:48.042251+0000","last_undegraded":"2024-09-17T15:36:48.042251+0000","last_fullsized":"2024-09-17T15:36:48.042251+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":99,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042677+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:36:48.042677+0000","last_peered":"2024-09-17T15:36:48.042677+0000","last_clean":"2024-09-17T15:36:48.042677+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:36:48.042677+0000","last_undegraded":"2024-09-17T15:36:48.042677+0000","last_fullsized":"2024-09-17T15:36:48.042677+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29033,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042443+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:36:48.042443+0000","last_peered":"2024-09-17T15:36:48.042443+0000","last_clean":"2024-09-17T15:36:48.042443+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:36:48.042443+0000","last_undegraded":"2024-09-17T15:36:48.042443+0000","last_fullsized":"2024-09-17T15:36:48.042443+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18544,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30916,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":198209536,"data_stored":198093920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051727,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497520,"kb_used_data":128480,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93874320,"statfs":{"total":96636764160,"available":96127303680,"internally_reserved":0,"allocated":131563520,"data_stored":131223490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":377876140},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724672,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479236,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892604,"statfs":{"total":96636764160,"available":96146026496,"internally_reserved":0,"allocated":512000,"data_stored":160667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":490204843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247955,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539408,"kb_used_data":66296,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832432,"statfs":{"total":96636764160,"available":96084410368,"internally_reserved":0,"allocated":67887104,"data_stored":67542997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130736128,"data_stored":130699998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:37:03.580 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:37:03.580 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'20648", 'reported_seq': 20925, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:48.053279+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:36:48.053279+0000', 'last_peered': '2024-09-17T15:36:48.053279+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:36:48.053279+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'10600", 'ondisk_log_start': "43'10600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20648, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:03.580 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:37:03.580 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'21251", 'reported_seq': 24806, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:59.408673+0000', 'last_change': '2024-09-17T15:36:46.948876+0000', 'last_active': '2024-09-17T15:36:59.408673+0000', 'last_peered': '2024-09-17T15:36:59.408673+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:36:59.408673+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "43'11200", 'ondisk_log_start': "43'11200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2575, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17750, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1696, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2574}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:03.580 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:37:03.580 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20542", 'reported_seq': 20834, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:48.059616+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:36:48.059616+0000', 'last_peered': '2024-09-17T15:36:48.059616+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:36:48.059616+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17138, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3943}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:03.580 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:37:03.580 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'20966", 'reported_seq': 21290, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:48.053156+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:36:48.053156+0000', 'last_peered': '2024-09-17T15:36:48.053156+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:36:48.053156+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "43'10900", 'ondisk_log_start': "43'10900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17514, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4057}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:03.581 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:37:03.581 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'20886", 'reported_seq': 21211, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:48.053263+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:36:48.053263+0000', 'last_peered': '2024-09-17T15:36:48.053263+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:36:48.053263+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'10800", 'ondisk_log_start': "43'10800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17673, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3980}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:03.581 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:37:03.984 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:03.984 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 18544/24 objects degraded (77266.667%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":760,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1552551936,"bytes_avail":288357740544,"bytes_total":289910292480,"degraded_objects":18544,"degraded_total":24,"degraded_ratio":772.66666666666652,"recovering_objects_per_sec":99,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1195,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:37:03.997 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:37:03.997 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:37:04.327 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:04.328 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:37:04.340 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":428,"stamp":"2024-09-17T15:37:03.826631+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18544,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30917,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1514364,"kb_used_data":193476,"kb_used_omap":13,"kb_used_meta":1320818,"kb_avail":281601156,"statfs":{"total":289910292480,"available":288359583744,"internally_reserved":0,"allocated":198119424,"data_stored":197083954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1352518655},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002038"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20925,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.053279+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:36:48.053279+0000","last_peered":"2024-09-17T15:36:48.053279+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:36:48.053279+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21062,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043955+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:36:48.043955+0000","last_peered":"2024-09-17T15:36:48.043955+0000","last_clean":"2024-09-17T15:36:48.043955+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:36:48.043955+0000","last_undegraded":"2024-09-17T15:36:48.043955+0000","last_fullsized":"2024-09-17T15:36:48.043955+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21170,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.120714+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:36:48.120714+0000","last_peered":"2024-09-17T15:36:48.120714+0000","last_clean":"2024-09-17T15:36:48.120714+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:36:48.120714+0000","last_undegraded":"2024-09-17T15:36:48.120714+0000","last_fullsized":"2024-09-17T15:36:48.120714+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21067,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043731+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:36:48.043731+0000","last_peered":"2024-09-17T15:36:48.043731+0000","last_clean":"2024-09-17T15:36:48.043731+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:36:48.043731+0000","last_undegraded":"2024-09-17T15:36:48.043731+0000","last_fullsized":"2024-09-17T15:36:48.043731+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20809,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052901+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:36:48.052901+0000","last_peered":"2024-09-17T15:36:48.052901+0000","last_clean":"2024-09-17T15:36:48.052901+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:36:48.052901+0000","last_undegraded":"2024-09-17T15:36:48.052901+0000","last_fullsized":"2024-09-17T15:36:48.052901+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":24806,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:59.408673+0000","last_change":"2024-09-17T15:36:46.948876+0000","last_active":"2024-09-17T15:36:59.408673+0000","last_peered":"2024-09-17T15:36:59.408673+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:36:59.408673+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2575,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1696,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2574}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21104,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042819+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:36:48.042819+0000","last_peered":"2024-09-17T15:36:48.042819+0000","last_clean":"2024-09-17T15:36:48.042819+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:36:48.042819+0000","last_undegraded":"2024-09-17T15:36:48.042819+0000","last_fullsized":"2024-09-17T15:36:48.042819+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21071,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052737+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:36:48.052737+0000","last_peered":"2024-09-17T15:36:48.052737+0000","last_clean":"2024-09-17T15:36:48.052737+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:36:48.052737+0000","last_undegraded":"2024-09-17T15:36:48.052737+0000","last_fullsized":"2024-09-17T15:36:48.052737+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042197+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:36:48.042197+0000","last_peered":"2024-09-17T15:36:48.042197+0000","last_clean":"2024-09-17T15:36:48.042197+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:36:48.042197+0000","last_undegraded":"2024-09-17T15:36:48.042197+0000","last_fullsized":"2024-09-17T15:36:48.042197+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29127,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042385+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:36:48.042385+0000","last_peered":"2024-09-17T15:36:48.042385+0000","last_clean":"2024-09-17T15:36:48.042385+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:36:48.042385+0000","last_undegraded":"2024-09-17T15:36:48.042385+0000","last_fullsized":"2024-09-17T15:36:48.042385+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042912+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:36:48.042912+0000","last_peered":"2024-09-17T15:36:48.042912+0000","last_clean":"2024-09-17T15:36:48.042912+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:36:48.042912+0000","last_undegraded":"2024-09-17T15:36:48.042912+0000","last_fullsized":"2024-09-17T15:36:48.042912+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20834,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.059616+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:36:48.059616+0000","last_peered":"2024-09-17T15:36:48.059616+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:36:48.059616+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":130,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052749+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:36:48.052749+0000","last_peered":"2024-09-17T15:36:48.052749+0000","last_clean":"2024-09-17T15:36:48.052749+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:36:48.052749+0000","last_undegraded":"2024-09-17T15:36:48.052749+0000","last_fullsized":"2024-09-17T15:36:48.052749+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28831,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042985+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:36:48.042985+0000","last_peered":"2024-09-17T15:36:48.042985+0000","last_clean":"2024-09-17T15:36:48.042985+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:36:48.042985+0000","last_undegraded":"2024-09-17T15:36:48.042985+0000","last_fullsized":"2024-09-17T15:36:48.042985+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":105,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042299+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:36:48.042299+0000","last_peered":"2024-09-17T15:36:48.042299+0000","last_clean":"2024-09-17T15:36:48.042299+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:36:48.042299+0000","last_undegraded":"2024-09-17T15:36:48.042299+0000","last_fullsized":"2024-09-17T15:36:48.042299+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29168,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042529+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:36:48.042529+0000","last_peered":"2024-09-17T15:36:48.042529+0000","last_clean":"2024-09-17T15:36:48.042529+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:36:48.042529+0000","last_undegraded":"2024-09-17T15:36:48.042529+0000","last_fullsized":"2024-09-17T15:36:48.042529+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21290,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.053156+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:36:48.053156+0000","last_peered":"2024-09-17T15:36:48.053156+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:36:48.053156+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042552+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:36:48.042552+0000","last_peered":"2024-09-17T15:36:48.042552+0000","last_clean":"2024-09-17T15:36:48.042552+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:36:48.042552+0000","last_undegraded":"2024-09-17T15:36:48.042552+0000","last_fullsized":"2024-09-17T15:36:48.042552+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28082,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042306+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:36:48.042306+0000","last_peered":"2024-09-17T15:36:48.042306+0000","last_clean":"2024-09-17T15:36:48.042306+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:36:48.042306+0000","last_undegraded":"2024-09-17T15:36:48.042306+0000","last_fullsized":"2024-09-17T15:36:48.042306+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043694+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:36:48.043694+0000","last_peered":"2024-09-17T15:36:48.043694+0000","last_clean":"2024-09-17T15:36:48.043694+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:36:48.043694+0000","last_undegraded":"2024-09-17T15:36:48.043694+0000","last_fullsized":"2024-09-17T15:36:48.043694+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21211,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.053263+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:36:48.053263+0000","last_peered":"2024-09-17T15:36:48.053263+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:36:48.053263+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042251+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:36:48.042251+0000","last_peered":"2024-09-17T15:36:48.042251+0000","last_clean":"2024-09-17T15:36:48.042251+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:36:48.042251+0000","last_undegraded":"2024-09-17T15:36:48.042251+0000","last_fullsized":"2024-09-17T15:36:48.042251+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":99,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042677+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:36:48.042677+0000","last_peered":"2024-09-17T15:36:48.042677+0000","last_clean":"2024-09-17T15:36:48.042677+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:36:48.042677+0000","last_undegraded":"2024-09-17T15:36:48.042677+0000","last_fullsized":"2024-09-17T15:36:48.042677+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29033,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042443+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:36:48.042443+0000","last_peered":"2024-09-17T15:36:48.042443+0000","last_clean":"2024-09-17T15:36:48.042443+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:36:48.042443+0000","last_undegraded":"2024-09-17T15:36:48.042443+0000","last_fullsized":"2024-09-17T15:36:48.042443+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18544,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30916,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":196366336,"data_stored":196250720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051728,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":495720,"kb_used_data":126680,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93876120,"statfs":{"total":96636764160,"available":96129146880,"internally_reserved":0,"allocated":129720320,"data_stored":129380290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":377876140},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724672,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479236,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892604,"statfs":{"total":96636764160,"available":96146026496,"internally_reserved":0,"allocated":512000,"data_stored":160667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":490204843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247955,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539408,"kb_used_data":66296,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832432,"statfs":{"total":96636764160,"available":96084410368,"internally_reserved":0,"allocated":67887104,"data_stored":67542997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":128892928,"data_stored":128856798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:37:07.342 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:37:07.643 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:07.643 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:37:07.655 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":429,"stamp":"2024-09-17T15:37:05.826938+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17854,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31607,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1518524,"kb_used_data":193476,"kb_used_omap":13,"kb_used_meta":1324978,"kb_avail":281596996,"statfs":{"total":289910292480,"available":288355323904,"internally_reserved":0,"allocated":198119424,"data_stored":197083954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1356778495},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1885,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1885,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001951"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20925,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.053279+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:36:48.053279+0000","last_peered":"2024-09-17T15:36:48.053279+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:36:48.053279+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21062,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043955+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:36:48.043955+0000","last_peered":"2024-09-17T15:36:48.043955+0000","last_clean":"2024-09-17T15:36:48.043955+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:36:48.043955+0000","last_undegraded":"2024-09-17T15:36:48.043955+0000","last_fullsized":"2024-09-17T15:36:48.043955+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21170,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.120714+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:36:48.120714+0000","last_peered":"2024-09-17T15:36:48.120714+0000","last_clean":"2024-09-17T15:36:48.120714+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:36:48.120714+0000","last_undegraded":"2024-09-17T15:36:48.120714+0000","last_fullsized":"2024-09-17T15:36:48.120714+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21067,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043731+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:36:48.043731+0000","last_peered":"2024-09-17T15:36:48.043731+0000","last_clean":"2024-09-17T15:36:48.043731+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:36:48.043731+0000","last_undegraded":"2024-09-17T15:36:48.043731+0000","last_fullsized":"2024-09-17T15:36:48.043731+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20809,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052901+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:36:48.052901+0000","last_peered":"2024-09-17T15:36:48.052901+0000","last_clean":"2024-09-17T15:36:48.052901+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:36:48.052901+0000","last_undegraded":"2024-09-17T15:36:48.052901+0000","last_fullsized":"2024-09-17T15:36:48.052901+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":26186,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:04.407665+0000","last_change":"2024-09-17T15:36:46.948876+0000","last_active":"2024-09-17T15:37:04.407665+0000","last_peered":"2024-09-17T15:37:04.407665+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:37:04.407665+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1885,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2386,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1884}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21104,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042819+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:36:48.042819+0000","last_peered":"2024-09-17T15:36:48.042819+0000","last_clean":"2024-09-17T15:36:48.042819+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:36:48.042819+0000","last_undegraded":"2024-09-17T15:36:48.042819+0000","last_fullsized":"2024-09-17T15:36:48.042819+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21071,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052737+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:36:48.052737+0000","last_peered":"2024-09-17T15:36:48.052737+0000","last_clean":"2024-09-17T15:36:48.052737+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:36:48.052737+0000","last_undegraded":"2024-09-17T15:36:48.052737+0000","last_fullsized":"2024-09-17T15:36:48.052737+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042197+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:36:48.042197+0000","last_peered":"2024-09-17T15:36:48.042197+0000","last_clean":"2024-09-17T15:36:48.042197+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:36:48.042197+0000","last_undegraded":"2024-09-17T15:36:48.042197+0000","last_fullsized":"2024-09-17T15:36:48.042197+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29127,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042385+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:36:48.042385+0000","last_peered":"2024-09-17T15:36:48.042385+0000","last_clean":"2024-09-17T15:36:48.042385+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:36:48.042385+0000","last_undegraded":"2024-09-17T15:36:48.042385+0000","last_fullsized":"2024-09-17T15:36:48.042385+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042912+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:36:48.042912+0000","last_peered":"2024-09-17T15:36:48.042912+0000","last_clean":"2024-09-17T15:36:48.042912+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:36:48.042912+0000","last_undegraded":"2024-09-17T15:36:48.042912+0000","last_fullsized":"2024-09-17T15:36:48.042912+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20834,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.059616+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:36:48.059616+0000","last_peered":"2024-09-17T15:36:48.059616+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:36:48.059616+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":130,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052749+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:36:48.052749+0000","last_peered":"2024-09-17T15:36:48.052749+0000","last_clean":"2024-09-17T15:36:48.052749+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:36:48.052749+0000","last_undegraded":"2024-09-17T15:36:48.052749+0000","last_fullsized":"2024-09-17T15:36:48.052749+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28831,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042985+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:36:48.042985+0000","last_peered":"2024-09-17T15:36:48.042985+0000","last_clean":"2024-09-17T15:36:48.042985+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:36:48.042985+0000","last_undegraded":"2024-09-17T15:36:48.042985+0000","last_fullsized":"2024-09-17T15:36:48.042985+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":105,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042299+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:36:48.042299+0000","last_peered":"2024-09-17T15:36:48.042299+0000","last_clean":"2024-09-17T15:36:48.042299+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:36:48.042299+0000","last_undegraded":"2024-09-17T15:36:48.042299+0000","last_fullsized":"2024-09-17T15:36:48.042299+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29168,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042529+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:36:48.042529+0000","last_peered":"2024-09-17T15:36:48.042529+0000","last_clean":"2024-09-17T15:36:48.042529+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:36:48.042529+0000","last_undegraded":"2024-09-17T15:36:48.042529+0000","last_fullsized":"2024-09-17T15:36:48.042529+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21291,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.054117+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:36:48.054117+0000","last_peered":"2024-09-17T15:36:48.054117+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:36:48.054117+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042552+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:36:48.042552+0000","last_peered":"2024-09-17T15:36:48.042552+0000","last_clean":"2024-09-17T15:36:48.042552+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:36:48.042552+0000","last_undegraded":"2024-09-17T15:36:48.042552+0000","last_fullsized":"2024-09-17T15:36:48.042552+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28082,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042306+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:36:48.042306+0000","last_peered":"2024-09-17T15:36:48.042306+0000","last_clean":"2024-09-17T15:36:48.042306+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:36:48.042306+0000","last_undegraded":"2024-09-17T15:36:48.042306+0000","last_fullsized":"2024-09-17T15:36:48.042306+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043694+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:36:48.043694+0000","last_peered":"2024-09-17T15:36:48.043694+0000","last_clean":"2024-09-17T15:36:48.043694+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:36:48.043694+0000","last_undegraded":"2024-09-17T15:36:48.043694+0000","last_fullsized":"2024-09-17T15:36:48.043694+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21211,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.053263+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:36:48.053263+0000","last_peered":"2024-09-17T15:36:48.053263+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:36:48.053263+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042251+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:36:48.042251+0000","last_peered":"2024-09-17T15:36:48.042251+0000","last_clean":"2024-09-17T15:36:48.042251+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:36:48.042251+0000","last_undegraded":"2024-09-17T15:36:48.042251+0000","last_fullsized":"2024-09-17T15:36:48.042251+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":99,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042677+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:36:48.042677+0000","last_peered":"2024-09-17T15:36:48.042677+0000","last_clean":"2024-09-17T15:36:48.042677+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:36:48.042677+0000","last_undegraded":"2024-09-17T15:36:48.042677+0000","last_fullsized":"2024-09-17T15:36:48.042677+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29033,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042443+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:36:48.042443+0000","last_peered":"2024-09-17T15:36:48.042443+0000","last_clean":"2024-09-17T15:36:48.042443+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:36:48.042443+0000","last_undegraded":"2024-09-17T15:36:48.042443+0000","last_fullsized":"2024-09-17T15:36:48.042443+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17854,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31606,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":196366336,"data_stored":196250720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051728,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":495720,"kb_used_data":126680,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93876120,"statfs":{"total":96636764160,"available":96129146880,"internally_reserved":0,"allocated":129720320,"data_stored":129380290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":377876140},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724673,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483396,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888444,"statfs":{"total":96636764160,"available":96141766656,"internally_reserved":0,"allocated":512000,"data_stored":160667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":494464683},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247956,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539408,"kb_used_data":66296,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832432,"statfs":{"total":96636764160,"available":96084410368,"internally_reserved":0,"allocated":67887104,"data_stored":67542997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":128892928,"data_stored":128856798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:37:07.657 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:37:08.001 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:08.001 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:37:08.013 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":430,"stamp":"2024-09-17T15:37:07.827275+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17854,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31607,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1518524,"kb_used_data":193476,"kb_used_omap":13,"kb_used_meta":1324978,"kb_avail":281596996,"statfs":{"total":289910292480,"available":288355323904,"internally_reserved":0,"allocated":198119424,"data_stored":197083954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1356778495},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1292,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001979"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20925,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.053279+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:36:48.053279+0000","last_peered":"2024-09-17T15:36:48.053279+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:36:48.053279+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21062,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043955+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:36:48.043955+0000","last_peered":"2024-09-17T15:36:48.043955+0000","last_clean":"2024-09-17T15:36:48.043955+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:36:48.043955+0000","last_undegraded":"2024-09-17T15:36:48.043955+0000","last_fullsized":"2024-09-17T15:36:48.043955+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21170,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.120714+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:36:48.120714+0000","last_peered":"2024-09-17T15:36:48.120714+0000","last_clean":"2024-09-17T15:36:48.120714+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:36:48.120714+0000","last_undegraded":"2024-09-17T15:36:48.120714+0000","last_fullsized":"2024-09-17T15:36:48.120714+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21067,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043731+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:36:48.043731+0000","last_peered":"2024-09-17T15:36:48.043731+0000","last_clean":"2024-09-17T15:36:48.043731+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:36:48.043731+0000","last_undegraded":"2024-09-17T15:36:48.043731+0000","last_fullsized":"2024-09-17T15:36:48.043731+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20809,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052901+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:36:48.052901+0000","last_peered":"2024-09-17T15:36:48.052901+0000","last_clean":"2024-09-17T15:36:48.052901+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:36:48.052901+0000","last_undegraded":"2024-09-17T15:36:48.052901+0000","last_fullsized":"2024-09-17T15:36:48.052901+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":26186,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:04.407665+0000","last_change":"2024-09-17T15:36:46.948876+0000","last_active":"2024-09-17T15:37:04.407665+0000","last_peered":"2024-09-17T15:37:04.407665+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:37:04.407665+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1885,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2386,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1884}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21104,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042819+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:36:48.042819+0000","last_peered":"2024-09-17T15:36:48.042819+0000","last_clean":"2024-09-17T15:36:48.042819+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:36:48.042819+0000","last_undegraded":"2024-09-17T15:36:48.042819+0000","last_fullsized":"2024-09-17T15:36:48.042819+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21071,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052737+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:36:48.052737+0000","last_peered":"2024-09-17T15:36:48.052737+0000","last_clean":"2024-09-17T15:36:48.052737+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:36:48.052737+0000","last_undegraded":"2024-09-17T15:36:48.052737+0000","last_fullsized":"2024-09-17T15:36:48.052737+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042197+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:36:48.042197+0000","last_peered":"2024-09-17T15:36:48.042197+0000","last_clean":"2024-09-17T15:36:48.042197+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:36:48.042197+0000","last_undegraded":"2024-09-17T15:36:48.042197+0000","last_fullsized":"2024-09-17T15:36:48.042197+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29127,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042385+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:36:48.042385+0000","last_peered":"2024-09-17T15:36:48.042385+0000","last_clean":"2024-09-17T15:36:48.042385+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:36:48.042385+0000","last_undegraded":"2024-09-17T15:36:48.042385+0000","last_fullsized":"2024-09-17T15:36:48.042385+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042912+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:36:48.042912+0000","last_peered":"2024-09-17T15:36:48.042912+0000","last_clean":"2024-09-17T15:36:48.042912+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:36:48.042912+0000","last_undegraded":"2024-09-17T15:36:48.042912+0000","last_fullsized":"2024-09-17T15:36:48.042912+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20834,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.059616+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:36:48.059616+0000","last_peered":"2024-09-17T15:36:48.059616+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:36:48.059616+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":130,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052749+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:36:48.052749+0000","last_peered":"2024-09-17T15:36:48.052749+0000","last_clean":"2024-09-17T15:36:48.052749+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:36:48.052749+0000","last_undegraded":"2024-09-17T15:36:48.052749+0000","last_fullsized":"2024-09-17T15:36:48.052749+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28831,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042985+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:36:48.042985+0000","last_peered":"2024-09-17T15:36:48.042985+0000","last_clean":"2024-09-17T15:36:48.042985+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:36:48.042985+0000","last_undegraded":"2024-09-17T15:36:48.042985+0000","last_fullsized":"2024-09-17T15:36:48.042985+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":105,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042299+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:36:48.042299+0000","last_peered":"2024-09-17T15:36:48.042299+0000","last_clean":"2024-09-17T15:36:48.042299+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:36:48.042299+0000","last_undegraded":"2024-09-17T15:36:48.042299+0000","last_fullsized":"2024-09-17T15:36:48.042299+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29168,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042529+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:36:48.042529+0000","last_peered":"2024-09-17T15:36:48.042529+0000","last_clean":"2024-09-17T15:36:48.042529+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:36:48.042529+0000","last_undegraded":"2024-09-17T15:36:48.042529+0000","last_fullsized":"2024-09-17T15:36:48.042529+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21291,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.054117+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:36:48.054117+0000","last_peered":"2024-09-17T15:36:48.054117+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:36:48.054117+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042552+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:36:48.042552+0000","last_peered":"2024-09-17T15:36:48.042552+0000","last_clean":"2024-09-17T15:36:48.042552+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:36:48.042552+0000","last_undegraded":"2024-09-17T15:36:48.042552+0000","last_fullsized":"2024-09-17T15:36:48.042552+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28082,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042306+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:36:48.042306+0000","last_peered":"2024-09-17T15:36:48.042306+0000","last_clean":"2024-09-17T15:36:48.042306+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:36:48.042306+0000","last_undegraded":"2024-09-17T15:36:48.042306+0000","last_fullsized":"2024-09-17T15:36:48.042306+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043694+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:36:48.043694+0000","last_peered":"2024-09-17T15:36:48.043694+0000","last_clean":"2024-09-17T15:36:48.043694+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:36:48.043694+0000","last_undegraded":"2024-09-17T15:36:48.043694+0000","last_fullsized":"2024-09-17T15:36:48.043694+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21211,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.053263+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:36:48.053263+0000","last_peered":"2024-09-17T15:36:48.053263+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:36:48.053263+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042251+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:36:48.042251+0000","last_peered":"2024-09-17T15:36:48.042251+0000","last_clean":"2024-09-17T15:36:48.042251+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:36:48.042251+0000","last_undegraded":"2024-09-17T15:36:48.042251+0000","last_fullsized":"2024-09-17T15:36:48.042251+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":99,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042677+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:36:48.042677+0000","last_peered":"2024-09-17T15:36:48.042677+0000","last_clean":"2024-09-17T15:36:48.042677+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:36:48.042677+0000","last_undegraded":"2024-09-17T15:36:48.042677+0000","last_fullsized":"2024-09-17T15:36:48.042677+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29033,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042443+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:36:48.042443+0000","last_peered":"2024-09-17T15:36:48.042443+0000","last_clean":"2024-09-17T15:36:48.042443+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:36:48.042443+0000","last_undegraded":"2024-09-17T15:36:48.042443+0000","last_fullsized":"2024-09-17T15:36:48.042443+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17854,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31606,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":196366336,"data_stored":196250720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051728,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":495720,"kb_used_data":126680,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93876120,"statfs":{"total":96636764160,"available":96129146880,"internally_reserved":0,"allocated":129720320,"data_stored":129380290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":377876140},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724673,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483396,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888444,"statfs":{"total":96636764160,"available":96141766656,"internally_reserved":0,"allocated":512000,"data_stored":160667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":494464683},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247956,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539408,"kb_used_data":66296,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832432,"statfs":{"total":96636764160,"available":96084410368,"internally_reserved":0,"allocated":67887104,"data_stored":67542997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":128892928,"data_stored":128856798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:37:08.014 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:37:08.014 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'20648", 'reported_seq': 20925, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:48.053279+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:36:48.053279+0000', 'last_peered': '2024-09-17T15:36:48.053279+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:36:48.053279+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'10600", 'ondisk_log_start': "43'10600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20648, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:08.015 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:37:08.015 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'21251", 'reported_seq': 26186, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:37:04.407665+0000', 'last_change': '2024-09-17T15:36:46.948876+0000', 'last_active': '2024-09-17T15:37:04.407665+0000', 'last_peered': '2024-09-17T15:37:04.407665+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:37:04.407665+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "43'11200", 'ondisk_log_start': "43'11200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1885, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17750, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2386, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1884}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:08.015 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:37:08.015 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20542", 'reported_seq': 20834, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:48.059616+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:36:48.059616+0000', 'last_peered': '2024-09-17T15:36:48.059616+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:36:48.059616+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17138, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3943}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:08.015 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:37:08.015 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'20966", 'reported_seq': 21291, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:48.054117+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:36:48.054117+0000', 'last_peered': '2024-09-17T15:36:48.054117+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:36:48.054117+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "43'10900", 'ondisk_log_start': "43'10900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17514, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4057}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:08.015 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:37:08.015 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'20886", 'reported_seq': 21211, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:48.053263+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:36:48.053263+0000', 'last_peered': '2024-09-17T15:36:48.053263+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:36:48.053263+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'10800", 'ondisk_log_start': "43'10800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17673, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3980}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:08.016 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:37:08.394 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:08.394 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 17854/24 objects degraded (74391.667%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":765,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1554968576,"bytes_avail":288355323904,"bytes_total":289910292480,"degraded_objects":17854,"degraded_total":24,"degraded_ratio":743.91666666666652,"recovering_objects_per_sec":107,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1292,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:37:08.407 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:37:08.407 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:37:08.724 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:08.724 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:37:08.736 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":430,"stamp":"2024-09-17T15:37:07.827275+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17854,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31607,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1518524,"kb_used_data":193476,"kb_used_omap":13,"kb_used_meta":1324978,"kb_avail":281596996,"statfs":{"total":289910292480,"available":288355323904,"internally_reserved":0,"allocated":198119424,"data_stored":197083954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1356778495},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1292,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001979"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20925,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.053279+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:36:48.053279+0000","last_peered":"2024-09-17T15:36:48.053279+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:36:48.053279+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21062,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043955+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:36:48.043955+0000","last_peered":"2024-09-17T15:36:48.043955+0000","last_clean":"2024-09-17T15:36:48.043955+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:36:48.043955+0000","last_undegraded":"2024-09-17T15:36:48.043955+0000","last_fullsized":"2024-09-17T15:36:48.043955+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21170,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.120714+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:36:48.120714+0000","last_peered":"2024-09-17T15:36:48.120714+0000","last_clean":"2024-09-17T15:36:48.120714+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:36:48.120714+0000","last_undegraded":"2024-09-17T15:36:48.120714+0000","last_fullsized":"2024-09-17T15:36:48.120714+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21067,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043731+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:36:48.043731+0000","last_peered":"2024-09-17T15:36:48.043731+0000","last_clean":"2024-09-17T15:36:48.043731+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:36:48.043731+0000","last_undegraded":"2024-09-17T15:36:48.043731+0000","last_fullsized":"2024-09-17T15:36:48.043731+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20809,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052901+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:36:48.052901+0000","last_peered":"2024-09-17T15:36:48.052901+0000","last_clean":"2024-09-17T15:36:48.052901+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:36:48.052901+0000","last_undegraded":"2024-09-17T15:36:48.052901+0000","last_fullsized":"2024-09-17T15:36:48.052901+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":26186,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:04.407665+0000","last_change":"2024-09-17T15:36:46.948876+0000","last_active":"2024-09-17T15:37:04.407665+0000","last_peered":"2024-09-17T15:37:04.407665+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:37:04.407665+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1885,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2386,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1884}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21104,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042819+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:36:48.042819+0000","last_peered":"2024-09-17T15:36:48.042819+0000","last_clean":"2024-09-17T15:36:48.042819+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:36:48.042819+0000","last_undegraded":"2024-09-17T15:36:48.042819+0000","last_fullsized":"2024-09-17T15:36:48.042819+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21071,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052737+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:36:48.052737+0000","last_peered":"2024-09-17T15:36:48.052737+0000","last_clean":"2024-09-17T15:36:48.052737+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:36:48.052737+0000","last_undegraded":"2024-09-17T15:36:48.052737+0000","last_fullsized":"2024-09-17T15:36:48.052737+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042197+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:36:48.042197+0000","last_peered":"2024-09-17T15:36:48.042197+0000","last_clean":"2024-09-17T15:36:48.042197+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:36:48.042197+0000","last_undegraded":"2024-09-17T15:36:48.042197+0000","last_fullsized":"2024-09-17T15:36:48.042197+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29127,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042385+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:36:48.042385+0000","last_peered":"2024-09-17T15:36:48.042385+0000","last_clean":"2024-09-17T15:36:48.042385+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:36:48.042385+0000","last_undegraded":"2024-09-17T15:36:48.042385+0000","last_fullsized":"2024-09-17T15:36:48.042385+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042912+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:36:48.042912+0000","last_peered":"2024-09-17T15:36:48.042912+0000","last_clean":"2024-09-17T15:36:48.042912+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:36:48.042912+0000","last_undegraded":"2024-09-17T15:36:48.042912+0000","last_fullsized":"2024-09-17T15:36:48.042912+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20834,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.059616+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:36:48.059616+0000","last_peered":"2024-09-17T15:36:48.059616+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:36:48.059616+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":130,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052749+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:36:48.052749+0000","last_peered":"2024-09-17T15:36:48.052749+0000","last_clean":"2024-09-17T15:36:48.052749+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:36:48.052749+0000","last_undegraded":"2024-09-17T15:36:48.052749+0000","last_fullsized":"2024-09-17T15:36:48.052749+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28831,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042985+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:36:48.042985+0000","last_peered":"2024-09-17T15:36:48.042985+0000","last_clean":"2024-09-17T15:36:48.042985+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:36:48.042985+0000","last_undegraded":"2024-09-17T15:36:48.042985+0000","last_fullsized":"2024-09-17T15:36:48.042985+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":105,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042299+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:36:48.042299+0000","last_peered":"2024-09-17T15:36:48.042299+0000","last_clean":"2024-09-17T15:36:48.042299+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:36:48.042299+0000","last_undegraded":"2024-09-17T15:36:48.042299+0000","last_fullsized":"2024-09-17T15:36:48.042299+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29168,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042529+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:36:48.042529+0000","last_peered":"2024-09-17T15:36:48.042529+0000","last_clean":"2024-09-17T15:36:48.042529+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:36:48.042529+0000","last_undegraded":"2024-09-17T15:36:48.042529+0000","last_fullsized":"2024-09-17T15:36:48.042529+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21291,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.054117+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:36:48.054117+0000","last_peered":"2024-09-17T15:36:48.054117+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:36:48.054117+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042552+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:36:48.042552+0000","last_peered":"2024-09-17T15:36:48.042552+0000","last_clean":"2024-09-17T15:36:48.042552+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:36:48.042552+0000","last_undegraded":"2024-09-17T15:36:48.042552+0000","last_fullsized":"2024-09-17T15:36:48.042552+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28082,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042306+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:36:48.042306+0000","last_peered":"2024-09-17T15:36:48.042306+0000","last_clean":"2024-09-17T15:36:48.042306+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:36:48.042306+0000","last_undegraded":"2024-09-17T15:36:48.042306+0000","last_fullsized":"2024-09-17T15:36:48.042306+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043694+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:36:48.043694+0000","last_peered":"2024-09-17T15:36:48.043694+0000","last_clean":"2024-09-17T15:36:48.043694+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:36:48.043694+0000","last_undegraded":"2024-09-17T15:36:48.043694+0000","last_fullsized":"2024-09-17T15:36:48.043694+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21211,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.053263+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:36:48.053263+0000","last_peered":"2024-09-17T15:36:48.053263+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:36:48.053263+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042251+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:36:48.042251+0000","last_peered":"2024-09-17T15:36:48.042251+0000","last_clean":"2024-09-17T15:36:48.042251+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:36:48.042251+0000","last_undegraded":"2024-09-17T15:36:48.042251+0000","last_fullsized":"2024-09-17T15:36:48.042251+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":99,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042677+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:36:48.042677+0000","last_peered":"2024-09-17T15:36:48.042677+0000","last_clean":"2024-09-17T15:36:48.042677+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:36:48.042677+0000","last_undegraded":"2024-09-17T15:36:48.042677+0000","last_fullsized":"2024-09-17T15:36:48.042677+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29033,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042443+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:36:48.042443+0000","last_peered":"2024-09-17T15:36:48.042443+0000","last_clean":"2024-09-17T15:36:48.042443+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:36:48.042443+0000","last_undegraded":"2024-09-17T15:36:48.042443+0000","last_fullsized":"2024-09-17T15:36:48.042443+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17854,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31606,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":196366336,"data_stored":196250720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051728,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":495720,"kb_used_data":126680,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93876120,"statfs":{"total":96636764160,"available":96129146880,"internally_reserved":0,"allocated":129720320,"data_stored":129380290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":377876140},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724673,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483396,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888444,"statfs":{"total":96636764160,"available":96141766656,"internally_reserved":0,"allocated":512000,"data_stored":160667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":494464683},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247956,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539408,"kb_used_data":66296,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832432,"statfs":{"total":96636764160,"available":96084410368,"internally_reserved":0,"allocated":67887104,"data_stored":67542997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":128892928,"data_stored":128856798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:37:11.738 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:37:12.042 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:12.043 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:37:12.054 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":432,"stamp":"2024-09-17T15:37:11.827919+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17000,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32461,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1525044,"kb_used_data":191676,"kb_used_omap":13,"kb_used_meta":1333298,"kb_avail":281590476,"statfs":{"total":289910292480,"available":288348647424,"internally_reserved":0,"allocated":196276224,"data_stored":195247980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1365298175},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1544,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1544,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001982"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20925,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.053279+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:36:48.053279+0000","last_peered":"2024-09-17T15:36:48.053279+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:36:48.053279+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21062,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043955+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:36:48.043955+0000","last_peered":"2024-09-17T15:36:48.043955+0000","last_clean":"2024-09-17T15:36:48.043955+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:36:48.043955+0000","last_undegraded":"2024-09-17T15:36:48.043955+0000","last_fullsized":"2024-09-17T15:36:48.043955+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21170,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.120714+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:36:48.120714+0000","last_peered":"2024-09-17T15:36:48.120714+0000","last_clean":"2024-09-17T15:36:48.120714+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:36:48.120714+0000","last_undegraded":"2024-09-17T15:36:48.120714+0000","last_fullsized":"2024-09-17T15:36:48.120714+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21067,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043731+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:36:48.043731+0000","last_peered":"2024-09-17T15:36:48.043731+0000","last_clean":"2024-09-17T15:36:48.043731+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:36:48.043731+0000","last_undegraded":"2024-09-17T15:36:48.043731+0000","last_fullsized":"2024-09-17T15:36:48.043731+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20809,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052901+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:36:48.052901+0000","last_peered":"2024-09-17T15:36:48.052901+0000","last_clean":"2024-09-17T15:36:48.052901+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:36:48.052901+0000","last_undegraded":"2024-09-17T15:36:48.052901+0000","last_fullsized":"2024-09-17T15:36:48.052901+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":27894,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:09.411196+0000","last_change":"2024-09-17T15:36:46.948876+0000","last_active":"2024-09-17T15:37:09.411196+0000","last_peered":"2024-09-17T15:37:09.411196+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:37:09.411196+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3240,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1030}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21104,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042819+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:36:48.042819+0000","last_peered":"2024-09-17T15:36:48.042819+0000","last_clean":"2024-09-17T15:36:48.042819+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:36:48.042819+0000","last_undegraded":"2024-09-17T15:36:48.042819+0000","last_fullsized":"2024-09-17T15:36:48.042819+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21071,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052737+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:36:48.052737+0000","last_peered":"2024-09-17T15:36:48.052737+0000","last_clean":"2024-09-17T15:36:48.052737+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:36:48.052737+0000","last_undegraded":"2024-09-17T15:36:48.052737+0000","last_fullsized":"2024-09-17T15:36:48.052737+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042197+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:36:48.042197+0000","last_peered":"2024-09-17T15:36:48.042197+0000","last_clean":"2024-09-17T15:36:48.042197+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:36:48.042197+0000","last_undegraded":"2024-09-17T15:36:48.042197+0000","last_fullsized":"2024-09-17T15:36:48.042197+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29127,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042385+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:36:48.042385+0000","last_peered":"2024-09-17T15:36:48.042385+0000","last_clean":"2024-09-17T15:36:48.042385+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:36:48.042385+0000","last_undegraded":"2024-09-17T15:36:48.042385+0000","last_fullsized":"2024-09-17T15:36:48.042385+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042912+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:36:48.042912+0000","last_peered":"2024-09-17T15:36:48.042912+0000","last_clean":"2024-09-17T15:36:48.042912+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:36:48.042912+0000","last_undegraded":"2024-09-17T15:36:48.042912+0000","last_fullsized":"2024-09-17T15:36:48.042912+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20834,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.059616+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:36:48.059616+0000","last_peered":"2024-09-17T15:36:48.059616+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:36:48.059616+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":130,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052749+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:36:48.052749+0000","last_peered":"2024-09-17T15:36:48.052749+0000","last_clean":"2024-09-17T15:36:48.052749+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:36:48.052749+0000","last_undegraded":"2024-09-17T15:36:48.052749+0000","last_fullsized":"2024-09-17T15:36:48.052749+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28831,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042985+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:36:48.042985+0000","last_peered":"2024-09-17T15:36:48.042985+0000","last_clean":"2024-09-17T15:36:48.042985+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:36:48.042985+0000","last_undegraded":"2024-09-17T15:36:48.042985+0000","last_fullsized":"2024-09-17T15:36:48.042985+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":105,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042299+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:36:48.042299+0000","last_peered":"2024-09-17T15:36:48.042299+0000","last_clean":"2024-09-17T15:36:48.042299+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:36:48.042299+0000","last_undegraded":"2024-09-17T15:36:48.042299+0000","last_fullsized":"2024-09-17T15:36:48.042299+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29168,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042529+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:36:48.042529+0000","last_peered":"2024-09-17T15:36:48.042529+0000","last_clean":"2024-09-17T15:36:48.042529+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:36:48.042529+0000","last_undegraded":"2024-09-17T15:36:48.042529+0000","last_fullsized":"2024-09-17T15:36:48.042529+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21291,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.054117+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:36:48.054117+0000","last_peered":"2024-09-17T15:36:48.054117+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:36:48.054117+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042552+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:36:48.042552+0000","last_peered":"2024-09-17T15:36:48.042552+0000","last_clean":"2024-09-17T15:36:48.042552+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:36:48.042552+0000","last_undegraded":"2024-09-17T15:36:48.042552+0000","last_fullsized":"2024-09-17T15:36:48.042552+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28082,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042306+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:36:48.042306+0000","last_peered":"2024-09-17T15:36:48.042306+0000","last_clean":"2024-09-17T15:36:48.042306+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:36:48.042306+0000","last_undegraded":"2024-09-17T15:36:48.042306+0000","last_fullsized":"2024-09-17T15:36:48.042306+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043694+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:36:48.043694+0000","last_peered":"2024-09-17T15:36:48.043694+0000","last_clean":"2024-09-17T15:36:48.043694+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:36:48.043694+0000","last_undegraded":"2024-09-17T15:36:48.043694+0000","last_fullsized":"2024-09-17T15:36:48.043694+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21211,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.053263+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:36:48.053263+0000","last_peered":"2024-09-17T15:36:48.053263+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:36:48.053263+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042251+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:36:48.042251+0000","last_peered":"2024-09-17T15:36:48.042251+0000","last_clean":"2024-09-17T15:36:48.042251+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:36:48.042251+0000","last_undegraded":"2024-09-17T15:36:48.042251+0000","last_fullsized":"2024-09-17T15:36:48.042251+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":99,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042677+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:36:48.042677+0000","last_peered":"2024-09-17T15:36:48.042677+0000","last_clean":"2024-09-17T15:36:48.042677+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:36:48.042677+0000","last_undegraded":"2024-09-17T15:36:48.042677+0000","last_fullsized":"2024-09-17T15:36:48.042677+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29033,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042443+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:36:48.042443+0000","last_peered":"2024-09-17T15:36:48.042443+0000","last_clean":"2024-09-17T15:36:48.042443+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:36:48.042443+0000","last_undegraded":"2024-09-17T15:36:48.042443+0000","last_fullsized":"2024-09-17T15:36:48.042443+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17000,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32460,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":194523136,"data_stored":194414746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051729,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493920,"kb_used_data":124880,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93877920,"statfs":{"total":96636764160,"available":96130990080,"internally_reserved":0,"allocated":127877120,"data_stored":127544316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":377876140},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724674,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491716,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880124,"statfs":{"total":96636764160,"available":96133246976,"internally_reserved":0,"allocated":512000,"data_stored":160667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":502984363},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247957,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539408,"kb_used_data":66296,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832432,"statfs":{"total":96636764160,"available":96084410368,"internally_reserved":0,"allocated":67887104,"data_stored":67542997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127049728,"data_stored":127020824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:37:12.056 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:37:12.374 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:12.375 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:37:12.387 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":432,"stamp":"2024-09-17T15:37:11.827919+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17000,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32461,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1525044,"kb_used_data":191676,"kb_used_omap":13,"kb_used_meta":1333298,"kb_avail":281590476,"statfs":{"total":289910292480,"available":288348647424,"internally_reserved":0,"allocated":196276224,"data_stored":195247980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1365298175},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1544,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1544,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001982"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20925,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.053279+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:36:48.053279+0000","last_peered":"2024-09-17T15:36:48.053279+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:36:48.053279+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21062,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043955+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:36:48.043955+0000","last_peered":"2024-09-17T15:36:48.043955+0000","last_clean":"2024-09-17T15:36:48.043955+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:36:48.043955+0000","last_undegraded":"2024-09-17T15:36:48.043955+0000","last_fullsized":"2024-09-17T15:36:48.043955+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21170,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.120714+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:36:48.120714+0000","last_peered":"2024-09-17T15:36:48.120714+0000","last_clean":"2024-09-17T15:36:48.120714+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:36:48.120714+0000","last_undegraded":"2024-09-17T15:36:48.120714+0000","last_fullsized":"2024-09-17T15:36:48.120714+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21067,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043731+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:36:48.043731+0000","last_peered":"2024-09-17T15:36:48.043731+0000","last_clean":"2024-09-17T15:36:48.043731+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:36:48.043731+0000","last_undegraded":"2024-09-17T15:36:48.043731+0000","last_fullsized":"2024-09-17T15:36:48.043731+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20809,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052901+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:36:48.052901+0000","last_peered":"2024-09-17T15:36:48.052901+0000","last_clean":"2024-09-17T15:36:48.052901+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:36:48.052901+0000","last_undegraded":"2024-09-17T15:36:48.052901+0000","last_fullsized":"2024-09-17T15:36:48.052901+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":27894,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:09.411196+0000","last_change":"2024-09-17T15:36:46.948876+0000","last_active":"2024-09-17T15:37:09.411196+0000","last_peered":"2024-09-17T15:37:09.411196+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:37:09.411196+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3240,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1030}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21104,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042819+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:36:48.042819+0000","last_peered":"2024-09-17T15:36:48.042819+0000","last_clean":"2024-09-17T15:36:48.042819+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:36:48.042819+0000","last_undegraded":"2024-09-17T15:36:48.042819+0000","last_fullsized":"2024-09-17T15:36:48.042819+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21071,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052737+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:36:48.052737+0000","last_peered":"2024-09-17T15:36:48.052737+0000","last_clean":"2024-09-17T15:36:48.052737+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:36:48.052737+0000","last_undegraded":"2024-09-17T15:36:48.052737+0000","last_fullsized":"2024-09-17T15:36:48.052737+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042197+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:36:48.042197+0000","last_peered":"2024-09-17T15:36:48.042197+0000","last_clean":"2024-09-17T15:36:48.042197+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:36:48.042197+0000","last_undegraded":"2024-09-17T15:36:48.042197+0000","last_fullsized":"2024-09-17T15:36:48.042197+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29127,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042385+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:36:48.042385+0000","last_peered":"2024-09-17T15:36:48.042385+0000","last_clean":"2024-09-17T15:36:48.042385+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:36:48.042385+0000","last_undegraded":"2024-09-17T15:36:48.042385+0000","last_fullsized":"2024-09-17T15:36:48.042385+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042912+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:36:48.042912+0000","last_peered":"2024-09-17T15:36:48.042912+0000","last_clean":"2024-09-17T15:36:48.042912+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:36:48.042912+0000","last_undegraded":"2024-09-17T15:36:48.042912+0000","last_fullsized":"2024-09-17T15:36:48.042912+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20834,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.059616+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:36:48.059616+0000","last_peered":"2024-09-17T15:36:48.059616+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:36:48.059616+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":130,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052749+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:36:48.052749+0000","last_peered":"2024-09-17T15:36:48.052749+0000","last_clean":"2024-09-17T15:36:48.052749+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:36:48.052749+0000","last_undegraded":"2024-09-17T15:36:48.052749+0000","last_fullsized":"2024-09-17T15:36:48.052749+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28831,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042985+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:36:48.042985+0000","last_peered":"2024-09-17T15:36:48.042985+0000","last_clean":"2024-09-17T15:36:48.042985+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:36:48.042985+0000","last_undegraded":"2024-09-17T15:36:48.042985+0000","last_fullsized":"2024-09-17T15:36:48.042985+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":105,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042299+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:36:48.042299+0000","last_peered":"2024-09-17T15:36:48.042299+0000","last_clean":"2024-09-17T15:36:48.042299+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:36:48.042299+0000","last_undegraded":"2024-09-17T15:36:48.042299+0000","last_fullsized":"2024-09-17T15:36:48.042299+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29168,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042529+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:36:48.042529+0000","last_peered":"2024-09-17T15:36:48.042529+0000","last_clean":"2024-09-17T15:36:48.042529+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:36:48.042529+0000","last_undegraded":"2024-09-17T15:36:48.042529+0000","last_fullsized":"2024-09-17T15:36:48.042529+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21291,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.054117+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:36:48.054117+0000","last_peered":"2024-09-17T15:36:48.054117+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:36:48.054117+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042552+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:36:48.042552+0000","last_peered":"2024-09-17T15:36:48.042552+0000","last_clean":"2024-09-17T15:36:48.042552+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:36:48.042552+0000","last_undegraded":"2024-09-17T15:36:48.042552+0000","last_fullsized":"2024-09-17T15:36:48.042552+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28082,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042306+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:36:48.042306+0000","last_peered":"2024-09-17T15:36:48.042306+0000","last_clean":"2024-09-17T15:36:48.042306+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:36:48.042306+0000","last_undegraded":"2024-09-17T15:36:48.042306+0000","last_fullsized":"2024-09-17T15:36:48.042306+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043694+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:36:48.043694+0000","last_peered":"2024-09-17T15:36:48.043694+0000","last_clean":"2024-09-17T15:36:48.043694+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:36:48.043694+0000","last_undegraded":"2024-09-17T15:36:48.043694+0000","last_fullsized":"2024-09-17T15:36:48.043694+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21211,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.053263+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:36:48.053263+0000","last_peered":"2024-09-17T15:36:48.053263+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:36:48.053263+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042251+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:36:48.042251+0000","last_peered":"2024-09-17T15:36:48.042251+0000","last_clean":"2024-09-17T15:36:48.042251+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:36:48.042251+0000","last_undegraded":"2024-09-17T15:36:48.042251+0000","last_fullsized":"2024-09-17T15:36:48.042251+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":99,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042677+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:36:48.042677+0000","last_peered":"2024-09-17T15:36:48.042677+0000","last_clean":"2024-09-17T15:36:48.042677+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:36:48.042677+0000","last_undegraded":"2024-09-17T15:36:48.042677+0000","last_fullsized":"2024-09-17T15:36:48.042677+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29033,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042443+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:36:48.042443+0000","last_peered":"2024-09-17T15:36:48.042443+0000","last_clean":"2024-09-17T15:36:48.042443+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:36:48.042443+0000","last_undegraded":"2024-09-17T15:36:48.042443+0000","last_fullsized":"2024-09-17T15:36:48.042443+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17000,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32460,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":194523136,"data_stored":194414746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051729,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493920,"kb_used_data":124880,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93877920,"statfs":{"total":96636764160,"available":96130990080,"internally_reserved":0,"allocated":127877120,"data_stored":127544316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":377876140},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724674,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491716,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880124,"statfs":{"total":96636764160,"available":96133246976,"internally_reserved":0,"allocated":512000,"data_stored":160667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":502984363},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247957,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539408,"kb_used_data":66296,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832432,"statfs":{"total":96636764160,"available":96084410368,"internally_reserved":0,"allocated":67887104,"data_stored":67542997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127049728,"data_stored":127020824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:37:12.389 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:37:12.389 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'20648", 'reported_seq': 20925, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:48.053279+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:36:48.053279+0000', 'last_peered': '2024-09-17T15:36:48.053279+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:36:48.053279+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'10600", 'ondisk_log_start': "43'10600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20648, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:12.389 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:37:12.389 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'21251", 'reported_seq': 27894, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:37:09.411196+0000', 'last_change': '2024-09-17T15:36:46.948876+0000', 'last_active': '2024-09-17T15:37:09.411196+0000', 'last_peered': '2024-09-17T15:37:09.411196+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:37:09.411196+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "43'11200", 'ondisk_log_start': "43'11200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17750, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3240, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1030}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:12.389 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:37:12.389 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20542", 'reported_seq': 20834, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:48.059616+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:36:48.059616+0000', 'last_peered': '2024-09-17T15:36:48.059616+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:36:48.059616+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17138, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3943}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:12.389 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:37:12.389 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'20966", 'reported_seq': 21291, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:48.054117+0000', 'last_change': '2024-09-17T15:26:22.748739+0000', 'last_active': '2024-09-17T15:36:48.054117+0000', 'last_peered': '2024-09-17T15:36:48.054117+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:36:48.054117+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "43'10900", 'ondisk_log_start': "43'10900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17514, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4057}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:12.390 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:37:12.390 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'20886", 'reported_seq': 21211, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:36:48.053263+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:36:48.053263+0000', 'last_peered': '2024-09-17T15:36:48.053263+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:36:48.053263+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'10800", 'ondisk_log_start': "43'10800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17673, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3980}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:12.390 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:37:12.773 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:12.773 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 17000/24 objects degraded (70833.333%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":769,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1561645056,"bytes_avail":288348647424,"bytes_total":289910292480,"degraded_objects":17000,"degraded_total":24,"degraded_ratio":708.33333333333348,"recovering_objects_per_sec":128,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1544,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:37:12.786 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:37:12.786 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:37:13.096 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:13.097 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:37:13.110 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":432,"stamp":"2024-09-17T15:37:11.827919+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17000,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32461,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1525044,"kb_used_data":191676,"kb_used_omap":13,"kb_used_meta":1333298,"kb_avail":281590476,"statfs":{"total":289910292480,"available":288348647424,"internally_reserved":0,"allocated":196276224,"data_stored":195247980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1365298175},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1544,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1544,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001982"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20925,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.053279+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:36:48.053279+0000","last_peered":"2024-09-17T15:36:48.053279+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:36:48.053279+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21062,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043955+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:36:48.043955+0000","last_peered":"2024-09-17T15:36:48.043955+0000","last_clean":"2024-09-17T15:36:48.043955+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:36:48.043955+0000","last_undegraded":"2024-09-17T15:36:48.043955+0000","last_fullsized":"2024-09-17T15:36:48.043955+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21170,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.120714+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:36:48.120714+0000","last_peered":"2024-09-17T15:36:48.120714+0000","last_clean":"2024-09-17T15:36:48.120714+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:36:48.120714+0000","last_undegraded":"2024-09-17T15:36:48.120714+0000","last_fullsized":"2024-09-17T15:36:48.120714+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21067,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043731+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:36:48.043731+0000","last_peered":"2024-09-17T15:36:48.043731+0000","last_clean":"2024-09-17T15:36:48.043731+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:36:48.043731+0000","last_undegraded":"2024-09-17T15:36:48.043731+0000","last_fullsized":"2024-09-17T15:36:48.043731+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20809,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052901+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:36:48.052901+0000","last_peered":"2024-09-17T15:36:48.052901+0000","last_clean":"2024-09-17T15:36:48.052901+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:36:48.052901+0000","last_undegraded":"2024-09-17T15:36:48.052901+0000","last_fullsized":"2024-09-17T15:36:48.052901+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":27894,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:09.411196+0000","last_change":"2024-09-17T15:36:46.948876+0000","last_active":"2024-09-17T15:37:09.411196+0000","last_peered":"2024-09-17T15:37:09.411196+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:37:09.411196+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3240,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1030}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21104,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042819+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:36:48.042819+0000","last_peered":"2024-09-17T15:36:48.042819+0000","last_clean":"2024-09-17T15:36:48.042819+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:36:48.042819+0000","last_undegraded":"2024-09-17T15:36:48.042819+0000","last_fullsized":"2024-09-17T15:36:48.042819+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21071,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052737+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:36:48.052737+0000","last_peered":"2024-09-17T15:36:48.052737+0000","last_clean":"2024-09-17T15:36:48.052737+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:36:48.052737+0000","last_undegraded":"2024-09-17T15:36:48.052737+0000","last_fullsized":"2024-09-17T15:36:48.052737+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042197+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:36:48.042197+0000","last_peered":"2024-09-17T15:36:48.042197+0000","last_clean":"2024-09-17T15:36:48.042197+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:36:48.042197+0000","last_undegraded":"2024-09-17T15:36:48.042197+0000","last_fullsized":"2024-09-17T15:36:48.042197+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29127,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042385+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:36:48.042385+0000","last_peered":"2024-09-17T15:36:48.042385+0000","last_clean":"2024-09-17T15:36:48.042385+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:36:48.042385+0000","last_undegraded":"2024-09-17T15:36:48.042385+0000","last_fullsized":"2024-09-17T15:36:48.042385+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042912+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:36:48.042912+0000","last_peered":"2024-09-17T15:36:48.042912+0000","last_clean":"2024-09-17T15:36:48.042912+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:36:48.042912+0000","last_undegraded":"2024-09-17T15:36:48.042912+0000","last_fullsized":"2024-09-17T15:36:48.042912+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20834,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.059616+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:36:48.059616+0000","last_peered":"2024-09-17T15:36:48.059616+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:36:48.059616+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":130,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.052749+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:36:48.052749+0000","last_peered":"2024-09-17T15:36:48.052749+0000","last_clean":"2024-09-17T15:36:48.052749+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:36:48.052749+0000","last_undegraded":"2024-09-17T15:36:48.052749+0000","last_fullsized":"2024-09-17T15:36:48.052749+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28831,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042985+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:36:48.042985+0000","last_peered":"2024-09-17T15:36:48.042985+0000","last_clean":"2024-09-17T15:36:48.042985+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:36:48.042985+0000","last_undegraded":"2024-09-17T15:36:48.042985+0000","last_fullsized":"2024-09-17T15:36:48.042985+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":105,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042299+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:36:48.042299+0000","last_peered":"2024-09-17T15:36:48.042299+0000","last_clean":"2024-09-17T15:36:48.042299+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:36:48.042299+0000","last_undegraded":"2024-09-17T15:36:48.042299+0000","last_fullsized":"2024-09-17T15:36:48.042299+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29168,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042529+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:36:48.042529+0000","last_peered":"2024-09-17T15:36:48.042529+0000","last_clean":"2024-09-17T15:36:48.042529+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:36:48.042529+0000","last_undegraded":"2024-09-17T15:36:48.042529+0000","last_fullsized":"2024-09-17T15:36:48.042529+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21291,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.054117+0000","last_change":"2024-09-17T15:26:22.748739+0000","last_active":"2024-09-17T15:36:48.054117+0000","last_peered":"2024-09-17T15:36:48.054117+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:36:48.054117+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042552+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:36:48.042552+0000","last_peered":"2024-09-17T15:36:48.042552+0000","last_clean":"2024-09-17T15:36:48.042552+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:36:48.042552+0000","last_undegraded":"2024-09-17T15:36:48.042552+0000","last_fullsized":"2024-09-17T15:36:48.042552+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28082,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042306+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:36:48.042306+0000","last_peered":"2024-09-17T15:36:48.042306+0000","last_clean":"2024-09-17T15:36:48.042306+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:36:48.042306+0000","last_undegraded":"2024-09-17T15:36:48.042306+0000","last_fullsized":"2024-09-17T15:36:48.042306+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043694+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:36:48.043694+0000","last_peered":"2024-09-17T15:36:48.043694+0000","last_clean":"2024-09-17T15:36:48.043694+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:36:48.043694+0000","last_undegraded":"2024-09-17T15:36:48.043694+0000","last_fullsized":"2024-09-17T15:36:48.043694+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21211,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:36:48.053263+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:36:48.053263+0000","last_peered":"2024-09-17T15:36:48.053263+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:36:48.053263+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042251+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:36:48.042251+0000","last_peered":"2024-09-17T15:36:48.042251+0000","last_clean":"2024-09-17T15:36:48.042251+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:36:48.042251+0000","last_undegraded":"2024-09-17T15:36:48.042251+0000","last_fullsized":"2024-09-17T15:36:48.042251+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":99,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042677+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:36:48.042677+0000","last_peered":"2024-09-17T15:36:48.042677+0000","last_clean":"2024-09-17T15:36:48.042677+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:36:48.042677+0000","last_undegraded":"2024-09-17T15:36:48.042677+0000","last_fullsized":"2024-09-17T15:36:48.042677+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29033,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042443+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:36:48.042443+0000","last_peered":"2024-09-17T15:36:48.042443+0000","last_clean":"2024-09-17T15:36:48.042443+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:36:48.042443+0000","last_undegraded":"2024-09-17T15:36:48.042443+0000","last_fullsized":"2024-09-17T15:36:48.042443+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17000,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32460,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":194523136,"data_stored":194414746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051729,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493920,"kb_used_data":124880,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93877920,"statfs":{"total":96636764160,"available":96130990080,"internally_reserved":0,"allocated":127877120,"data_stored":127544316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":377876140},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724674,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491716,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880124,"statfs":{"total":96636764160,"available":96133246976,"internally_reserved":0,"allocated":512000,"data_stored":160667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":502984363},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247957,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539408,"kb_used_data":66296,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832432,"statfs":{"total":96636764160,"available":96084410368,"internally_reserved":0,"allocated":67887104,"data_stored":67542997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127049728,"data_stored":127020824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:37:16.112 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:37:16.432 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:16.432 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:37:16.446 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":436,"stamp":"2024-09-17T15:37:15.828523+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16929,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32532,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1532972,"kb_used_data":189876,"kb_used_omap":13,"kb_used_meta":1343026,"kb_avail":281582548,"statfs":{"total":289910292480,"available":288340529152,"internally_reserved":0,"allocated":194433024,"data_stored":193404780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1375259647},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-925,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":925,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001247"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20927,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:14.111651+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:37:14.111651+0000","last_peered":"2024-09-17T15:37:14.111651+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:37:14.111651+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21062,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043955+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:36:48.043955+0000","last_peered":"2024-09-17T15:36:48.043955+0000","last_clean":"2024-09-17T15:36:48.043955+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:36:48.043955+0000","last_undegraded":"2024-09-17T15:36:48.043955+0000","last_fullsized":"2024-09-17T15:36:48.043955+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21170,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.120714+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:36:48.120714+0000","last_peered":"2024-09-17T15:36:48.120714+0000","last_clean":"2024-09-17T15:36:48.120714+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:36:48.120714+0000","last_undegraded":"2024-09-17T15:36:48.120714+0000","last_fullsized":"2024-09-17T15:36:48.120714+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21067,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043731+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:36:48.043731+0000","last_peered":"2024-09-17T15:36:48.043731+0000","last_clean":"2024-09-17T15:36:48.043731+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:36:48.043731+0000","last_undegraded":"2024-09-17T15:36:48.043731+0000","last_fullsized":"2024-09-17T15:36:48.043731+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20812,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T15:37:14.111074+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:37:14.111074+0000","last_peered":"2024-09-17T15:37:14.111074+0000","last_clean":"2024-09-17T15:37:14.111074+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:37:14.111074+0000","last_undegraded":"2024-09-17T15:37:14.111074+0000","last_fullsized":"2024-09-17T15:37:14.111074+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":27894,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:09.411196+0000","last_change":"2024-09-17T15:36:46.948876+0000","last_active":"2024-09-17T15:37:09.411196+0000","last_peered":"2024-09-17T15:37:09.411196+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:37:09.411196+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3240,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1030}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21104,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042819+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:36:48.042819+0000","last_peered":"2024-09-17T15:36:48.042819+0000","last_clean":"2024-09-17T15:36:48.042819+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:36:48.042819+0000","last_undegraded":"2024-09-17T15:36:48.042819+0000","last_fullsized":"2024-09-17T15:36:48.042819+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21073,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T15:37:14.110437+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:37:14.110437+0000","last_peered":"2024-09-17T15:37:14.110437+0000","last_clean":"2024-09-17T15:37:14.110437+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:37:14.110437+0000","last_undegraded":"2024-09-17T15:37:14.110437+0000","last_fullsized":"2024-09-17T15:37:14.110437+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042197+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:36:48.042197+0000","last_peered":"2024-09-17T15:36:48.042197+0000","last_clean":"2024-09-17T15:36:48.042197+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:36:48.042197+0000","last_undegraded":"2024-09-17T15:36:48.042197+0000","last_fullsized":"2024-09-17T15:36:48.042197+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29127,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042385+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:36:48.042385+0000","last_peered":"2024-09-17T15:36:48.042385+0000","last_clean":"2024-09-17T15:36:48.042385+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:36:48.042385+0000","last_undegraded":"2024-09-17T15:36:48.042385+0000","last_fullsized":"2024-09-17T15:36:48.042385+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042912+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:36:48.042912+0000","last_peered":"2024-09-17T15:36:48.042912+0000","last_clean":"2024-09-17T15:36:48.042912+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:36:48.042912+0000","last_undegraded":"2024-09-17T15:36:48.042912+0000","last_fullsized":"2024-09-17T15:36:48.042912+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20836,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:14.116950+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:37:14.116950+0000","last_peered":"2024-09-17T15:37:14.116950+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:37:14.116950+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":132,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T15:37:14.110523+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:37:14.110523+0000","last_peered":"2024-09-17T15:37:14.110523+0000","last_clean":"2024-09-17T15:37:14.110523+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:37:14.110523+0000","last_undegraded":"2024-09-17T15:37:14.110523+0000","last_fullsized":"2024-09-17T15:37:14.110523+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28831,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042985+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:36:48.042985+0000","last_peered":"2024-09-17T15:36:48.042985+0000","last_clean":"2024-09-17T15:36:48.042985+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:36:48.042985+0000","last_undegraded":"2024-09-17T15:36:48.042985+0000","last_fullsized":"2024-09-17T15:36:48.042985+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":105,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042299+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:36:48.042299+0000","last_peered":"2024-09-17T15:36:48.042299+0000","last_clean":"2024-09-17T15:36:48.042299+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:36:48.042299+0000","last_undegraded":"2024-09-17T15:36:48.042299+0000","last_fullsized":"2024-09-17T15:36:48.042299+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29168,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042529+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:36:48.042529+0000","last_peered":"2024-09-17T15:36:48.042529+0000","last_clean":"2024-09-17T15:36:48.042529+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:36:48.042529+0000","last_undegraded":"2024-09-17T15:36:48.042529+0000","last_fullsized":"2024-09-17T15:36:48.042529+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21436,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:14.408957+0000","last_change":"2024-09-17T15:37:13.729028+0000","last_active":"2024-09-17T15:37:14.408957+0000","last_peered":"2024-09-17T15:37:14.408957+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:37:14.408957+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3987,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3987}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042552+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:36:48.042552+0000","last_peered":"2024-09-17T15:36:48.042552+0000","last_clean":"2024-09-17T15:36:48.042552+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:36:48.042552+0000","last_undegraded":"2024-09-17T15:36:48.042552+0000","last_fullsized":"2024-09-17T15:36:48.042552+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28082,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042306+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:36:48.042306+0000","last_peered":"2024-09-17T15:36:48.042306+0000","last_clean":"2024-09-17T15:36:48.042306+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:36:48.042306+0000","last_undegraded":"2024-09-17T15:36:48.042306+0000","last_fullsized":"2024-09-17T15:36:48.042306+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043694+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:36:48.043694+0000","last_peered":"2024-09-17T15:36:48.043694+0000","last_clean":"2024-09-17T15:36:48.043694+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:36:48.043694+0000","last_undegraded":"2024-09-17T15:36:48.043694+0000","last_fullsized":"2024-09-17T15:36:48.043694+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21213,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:14.111073+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:37:14.111073+0000","last_peered":"2024-09-17T15:37:14.111073+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:37:14.111073+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042251+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:36:48.042251+0000","last_peered":"2024-09-17T15:36:48.042251+0000","last_clean":"2024-09-17T15:36:48.042251+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:36:48.042251+0000","last_undegraded":"2024-09-17T15:36:48.042251+0000","last_fullsized":"2024-09-17T15:36:48.042251+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":100,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042712+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:36:48.042712+0000","last_peered":"2024-09-17T15:36:48.042712+0000","last_clean":"2024-09-17T15:36:48.042712+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:36:48.042712+0000","last_undegraded":"2024-09-17T15:36:48.042712+0000","last_fullsized":"2024-09-17T15:36:48.042712+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29034,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042480+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:36:48.042480+0000","last_peered":"2024-09-17T15:36:48.042480+0000","last_clean":"2024-09-17T15:36:48.042480+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:36:48.042480+0000","last_undegraded":"2024-09-17T15:36:48.042480+0000","last_fullsized":"2024-09-17T15:36:48.042480+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16929,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32531,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":192675840,"data_stored":192571063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051730,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492120,"kb_used_data":123080,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93879720,"statfs":{"total":96636764160,"available":96132833280,"internally_reserved":0,"allocated":126033920,"data_stored":125701116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":377876140},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724675,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501444,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870396,"statfs":{"total":96636764160,"available":96123285504,"internally_reserved":0,"allocated":512000,"data_stored":160667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":512945835},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247958,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539408,"kb_used_data":66296,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832432,"statfs":{"total":96636764160,"available":96084410368,"internally_reserved":0,"allocated":67887104,"data_stored":67542997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":125206528,"data_stored":125177624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:37:16.447 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:37:16.765 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:16.765 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:37:16.778 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":436,"stamp":"2024-09-17T15:37:15.828523+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16929,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32532,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1532972,"kb_used_data":189876,"kb_used_omap":13,"kb_used_meta":1343026,"kb_avail":281582548,"statfs":{"total":289910292480,"available":288340529152,"internally_reserved":0,"allocated":194433024,"data_stored":193404780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1375259647},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-925,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":925,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001247"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20927,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:14.111651+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:37:14.111651+0000","last_peered":"2024-09-17T15:37:14.111651+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:37:14.111651+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21062,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043955+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:36:48.043955+0000","last_peered":"2024-09-17T15:36:48.043955+0000","last_clean":"2024-09-17T15:36:48.043955+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:36:48.043955+0000","last_undegraded":"2024-09-17T15:36:48.043955+0000","last_fullsized":"2024-09-17T15:36:48.043955+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21170,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.120714+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:36:48.120714+0000","last_peered":"2024-09-17T15:36:48.120714+0000","last_clean":"2024-09-17T15:36:48.120714+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:36:48.120714+0000","last_undegraded":"2024-09-17T15:36:48.120714+0000","last_fullsized":"2024-09-17T15:36:48.120714+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21067,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043731+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:36:48.043731+0000","last_peered":"2024-09-17T15:36:48.043731+0000","last_clean":"2024-09-17T15:36:48.043731+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:36:48.043731+0000","last_undegraded":"2024-09-17T15:36:48.043731+0000","last_fullsized":"2024-09-17T15:36:48.043731+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20812,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T15:37:14.111074+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:37:14.111074+0000","last_peered":"2024-09-17T15:37:14.111074+0000","last_clean":"2024-09-17T15:37:14.111074+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:37:14.111074+0000","last_undegraded":"2024-09-17T15:37:14.111074+0000","last_fullsized":"2024-09-17T15:37:14.111074+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":27894,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:09.411196+0000","last_change":"2024-09-17T15:36:46.948876+0000","last_active":"2024-09-17T15:37:09.411196+0000","last_peered":"2024-09-17T15:37:09.411196+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:37:09.411196+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3240,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1030}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21104,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042819+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:36:48.042819+0000","last_peered":"2024-09-17T15:36:48.042819+0000","last_clean":"2024-09-17T15:36:48.042819+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:36:48.042819+0000","last_undegraded":"2024-09-17T15:36:48.042819+0000","last_fullsized":"2024-09-17T15:36:48.042819+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21073,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T15:37:14.110437+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:37:14.110437+0000","last_peered":"2024-09-17T15:37:14.110437+0000","last_clean":"2024-09-17T15:37:14.110437+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:37:14.110437+0000","last_undegraded":"2024-09-17T15:37:14.110437+0000","last_fullsized":"2024-09-17T15:37:14.110437+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042197+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:36:48.042197+0000","last_peered":"2024-09-17T15:36:48.042197+0000","last_clean":"2024-09-17T15:36:48.042197+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:36:48.042197+0000","last_undegraded":"2024-09-17T15:36:48.042197+0000","last_fullsized":"2024-09-17T15:36:48.042197+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29127,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042385+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:36:48.042385+0000","last_peered":"2024-09-17T15:36:48.042385+0000","last_clean":"2024-09-17T15:36:48.042385+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:36:48.042385+0000","last_undegraded":"2024-09-17T15:36:48.042385+0000","last_fullsized":"2024-09-17T15:36:48.042385+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042912+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:36:48.042912+0000","last_peered":"2024-09-17T15:36:48.042912+0000","last_clean":"2024-09-17T15:36:48.042912+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:36:48.042912+0000","last_undegraded":"2024-09-17T15:36:48.042912+0000","last_fullsized":"2024-09-17T15:36:48.042912+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20836,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:14.116950+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:37:14.116950+0000","last_peered":"2024-09-17T15:37:14.116950+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:37:14.116950+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":132,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T15:37:14.110523+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:37:14.110523+0000","last_peered":"2024-09-17T15:37:14.110523+0000","last_clean":"2024-09-17T15:37:14.110523+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:37:14.110523+0000","last_undegraded":"2024-09-17T15:37:14.110523+0000","last_fullsized":"2024-09-17T15:37:14.110523+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28831,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042985+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:36:48.042985+0000","last_peered":"2024-09-17T15:36:48.042985+0000","last_clean":"2024-09-17T15:36:48.042985+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:36:48.042985+0000","last_undegraded":"2024-09-17T15:36:48.042985+0000","last_fullsized":"2024-09-17T15:36:48.042985+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":105,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042299+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:36:48.042299+0000","last_peered":"2024-09-17T15:36:48.042299+0000","last_clean":"2024-09-17T15:36:48.042299+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:36:48.042299+0000","last_undegraded":"2024-09-17T15:36:48.042299+0000","last_fullsized":"2024-09-17T15:36:48.042299+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29168,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042529+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:36:48.042529+0000","last_peered":"2024-09-17T15:36:48.042529+0000","last_clean":"2024-09-17T15:36:48.042529+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:36:48.042529+0000","last_undegraded":"2024-09-17T15:36:48.042529+0000","last_fullsized":"2024-09-17T15:36:48.042529+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21436,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:14.408957+0000","last_change":"2024-09-17T15:37:13.729028+0000","last_active":"2024-09-17T15:37:14.408957+0000","last_peered":"2024-09-17T15:37:14.408957+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:37:14.408957+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3987,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3987}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042552+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:36:48.042552+0000","last_peered":"2024-09-17T15:36:48.042552+0000","last_clean":"2024-09-17T15:36:48.042552+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:36:48.042552+0000","last_undegraded":"2024-09-17T15:36:48.042552+0000","last_fullsized":"2024-09-17T15:36:48.042552+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28082,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042306+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:36:48.042306+0000","last_peered":"2024-09-17T15:36:48.042306+0000","last_clean":"2024-09-17T15:36:48.042306+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:36:48.042306+0000","last_undegraded":"2024-09-17T15:36:48.042306+0000","last_fullsized":"2024-09-17T15:36:48.042306+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043694+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:36:48.043694+0000","last_peered":"2024-09-17T15:36:48.043694+0000","last_clean":"2024-09-17T15:36:48.043694+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:36:48.043694+0000","last_undegraded":"2024-09-17T15:36:48.043694+0000","last_fullsized":"2024-09-17T15:36:48.043694+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21213,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:14.111073+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:37:14.111073+0000","last_peered":"2024-09-17T15:37:14.111073+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:37:14.111073+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042251+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:36:48.042251+0000","last_peered":"2024-09-17T15:36:48.042251+0000","last_clean":"2024-09-17T15:36:48.042251+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:36:48.042251+0000","last_undegraded":"2024-09-17T15:36:48.042251+0000","last_fullsized":"2024-09-17T15:36:48.042251+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":100,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042712+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:36:48.042712+0000","last_peered":"2024-09-17T15:36:48.042712+0000","last_clean":"2024-09-17T15:36:48.042712+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:36:48.042712+0000","last_undegraded":"2024-09-17T15:36:48.042712+0000","last_fullsized":"2024-09-17T15:36:48.042712+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29034,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042480+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:36:48.042480+0000","last_peered":"2024-09-17T15:36:48.042480+0000","last_clean":"2024-09-17T15:36:48.042480+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:36:48.042480+0000","last_undegraded":"2024-09-17T15:36:48.042480+0000","last_fullsized":"2024-09-17T15:36:48.042480+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16929,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32531,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":192675840,"data_stored":192571063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051730,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492120,"kb_used_data":123080,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93879720,"statfs":{"total":96636764160,"available":96132833280,"internally_reserved":0,"allocated":126033920,"data_stored":125701116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":377876140},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724675,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501444,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870396,"statfs":{"total":96636764160,"available":96123285504,"internally_reserved":0,"allocated":512000,"data_stored":160667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":512945835},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247958,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539408,"kb_used_data":66296,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832432,"statfs":{"total":96636764160,"available":96084410368,"internally_reserved":0,"allocated":67887104,"data_stored":67542997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":125206528,"data_stored":125177624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:37:16.779 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:37:16.780 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'20648", 'reported_seq': 20927, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:37:14.111651+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:37:14.111651+0000', 'last_peered': '2024-09-17T15:37:14.111651+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:37:14.111651+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'10600", 'ondisk_log_start': "43'10600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20648, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:16.780 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-17T15:37:16.780 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'21251", 'reported_seq': 27894, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:37:09.411196+0000', 'last_change': '2024-09-17T15:36:46.948876+0000', 'last_active': '2024-09-17T15:37:09.411196+0000', 'last_peered': '2024-09-17T15:37:09.411196+0000', 'last_clean': '2024-09-17T15:26:20.173246+0000', 'last_became_active': '2024-09-17T15:26:22.739839+0000', 'last_became_peered': '2024-09-17T15:26:22.739839+0000', 'last_unstale': '2024-09-17T15:37:09.411196+0000', 'last_undegraded': '2024-09-17T15:26:22.317688+0000', 'last_fullsized': '2024-09-17T15:26:22.272909+0000', 'mapping_epoch': 40, 'log_start': "43'11200", 'ondisk_log_start': "43'11200", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3550", 'last_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:08.867394+0000', 'objects_scrubbed': 3497, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1244, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17750, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3240, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1030}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:16.780 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:37:16.780 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20542", 'reported_seq': 20836, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:37:14.116950+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:37:14.116950+0000', 'last_peered': '2024-09-17T15:37:14.116950+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:37:14.116950+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17138, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3943}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:16.780 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:37:16.780 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'20966", 'reported_seq': 21436, 'reported_epoch': 56, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:37:14.408957+0000', 'last_change': '2024-09-17T15:37:13.729028+0000', 'last_active': '2024-09-17T15:37:14.408957+0000', 'last_peered': '2024-09-17T15:37:14.408957+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:37:14.408957+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "43'10900", 'ondisk_log_start': "43'10900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3987, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17514, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 152, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3987}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:16.780 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:37:16.781 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'20886", 'reported_seq': 21213, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:37:14.111073+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:37:14.111073+0000', 'last_peered': '2024-09-17T15:37:14.111073+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:37:14.111073+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'10800", 'ondisk_log_start': "43'10800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17673, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3980}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:16.781 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:37:17.157 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:17.157 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 16929/24 objects degraded (70537.500%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":774,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":2}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1569763328,"bytes_avail":288340529152,"bytes_total":289910292480,"degraded_objects":16929,"degraded_total":24,"degraded_ratio":705.375,"recovering_objects_per_sec":115,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":925,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:37:17.170 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:37:17.171 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:37:17.497 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:17.497 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:37:17.511 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":436,"stamp":"2024-09-17T15:37:15.828523+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16929,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32532,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1532972,"kb_used_data":189876,"kb_used_omap":13,"kb_used_meta":1343026,"kb_avail":281582548,"statfs":{"total":289910292480,"available":288340529152,"internally_reserved":0,"allocated":194433024,"data_stored":193404780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1375259647},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-925,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":925,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001247"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20927,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:14.111651+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:37:14.111651+0000","last_peered":"2024-09-17T15:37:14.111651+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:37:14.111651+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21062,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043955+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:36:48.043955+0000","last_peered":"2024-09-17T15:36:48.043955+0000","last_clean":"2024-09-17T15:36:48.043955+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:36:48.043955+0000","last_undegraded":"2024-09-17T15:36:48.043955+0000","last_fullsized":"2024-09-17T15:36:48.043955+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21170,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.120714+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:36:48.120714+0000","last_peered":"2024-09-17T15:36:48.120714+0000","last_clean":"2024-09-17T15:36:48.120714+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:36:48.120714+0000","last_undegraded":"2024-09-17T15:36:48.120714+0000","last_fullsized":"2024-09-17T15:36:48.120714+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21067,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043731+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:36:48.043731+0000","last_peered":"2024-09-17T15:36:48.043731+0000","last_clean":"2024-09-17T15:36:48.043731+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:36:48.043731+0000","last_undegraded":"2024-09-17T15:36:48.043731+0000","last_fullsized":"2024-09-17T15:36:48.043731+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20812,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T15:37:14.111074+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:37:14.111074+0000","last_peered":"2024-09-17T15:37:14.111074+0000","last_clean":"2024-09-17T15:37:14.111074+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:37:14.111074+0000","last_undegraded":"2024-09-17T15:37:14.111074+0000","last_fullsized":"2024-09-17T15:37:14.111074+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":27894,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:09.411196+0000","last_change":"2024-09-17T15:36:46.948876+0000","last_active":"2024-09-17T15:37:09.411196+0000","last_peered":"2024-09-17T15:37:09.411196+0000","last_clean":"2024-09-17T15:26:20.173246+0000","last_became_active":"2024-09-17T15:26:22.739839+0000","last_became_peered":"2024-09-17T15:26:22.739839+0000","last_unstale":"2024-09-17T15:37:09.411196+0000","last_undegraded":"2024-09-17T15:26:22.317688+0000","last_fullsized":"2024-09-17T15:26:22.272909+0000","mapping_epoch":40,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3240,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1030}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21104,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042819+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:36:48.042819+0000","last_peered":"2024-09-17T15:36:48.042819+0000","last_clean":"2024-09-17T15:36:48.042819+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:36:48.042819+0000","last_undegraded":"2024-09-17T15:36:48.042819+0000","last_fullsized":"2024-09-17T15:36:48.042819+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21073,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T15:37:14.110437+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:37:14.110437+0000","last_peered":"2024-09-17T15:37:14.110437+0000","last_clean":"2024-09-17T15:37:14.110437+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:37:14.110437+0000","last_undegraded":"2024-09-17T15:37:14.110437+0000","last_fullsized":"2024-09-17T15:37:14.110437+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042197+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:36:48.042197+0000","last_peered":"2024-09-17T15:36:48.042197+0000","last_clean":"2024-09-17T15:36:48.042197+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:36:48.042197+0000","last_undegraded":"2024-09-17T15:36:48.042197+0000","last_fullsized":"2024-09-17T15:36:48.042197+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29127,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042385+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:36:48.042385+0000","last_peered":"2024-09-17T15:36:48.042385+0000","last_clean":"2024-09-17T15:36:48.042385+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:36:48.042385+0000","last_undegraded":"2024-09-17T15:36:48.042385+0000","last_fullsized":"2024-09-17T15:36:48.042385+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042912+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:36:48.042912+0000","last_peered":"2024-09-17T15:36:48.042912+0000","last_clean":"2024-09-17T15:36:48.042912+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:36:48.042912+0000","last_undegraded":"2024-09-17T15:36:48.042912+0000","last_fullsized":"2024-09-17T15:36:48.042912+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20836,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:14.116950+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:37:14.116950+0000","last_peered":"2024-09-17T15:37:14.116950+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:37:14.116950+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":132,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T15:37:14.110523+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:37:14.110523+0000","last_peered":"2024-09-17T15:37:14.110523+0000","last_clean":"2024-09-17T15:37:14.110523+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:37:14.110523+0000","last_undegraded":"2024-09-17T15:37:14.110523+0000","last_fullsized":"2024-09-17T15:37:14.110523+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28831,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042985+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:36:48.042985+0000","last_peered":"2024-09-17T15:36:48.042985+0000","last_clean":"2024-09-17T15:36:48.042985+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:36:48.042985+0000","last_undegraded":"2024-09-17T15:36:48.042985+0000","last_fullsized":"2024-09-17T15:36:48.042985+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":105,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042299+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:36:48.042299+0000","last_peered":"2024-09-17T15:36:48.042299+0000","last_clean":"2024-09-17T15:36:48.042299+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:36:48.042299+0000","last_undegraded":"2024-09-17T15:36:48.042299+0000","last_fullsized":"2024-09-17T15:36:48.042299+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29168,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042529+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:36:48.042529+0000","last_peered":"2024-09-17T15:36:48.042529+0000","last_clean":"2024-09-17T15:36:48.042529+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:36:48.042529+0000","last_undegraded":"2024-09-17T15:36:48.042529+0000","last_fullsized":"2024-09-17T15:36:48.042529+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21436,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:14.408957+0000","last_change":"2024-09-17T15:37:13.729028+0000","last_active":"2024-09-17T15:37:14.408957+0000","last_peered":"2024-09-17T15:37:14.408957+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:37:14.408957+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3987,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3987}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042552+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:36:48.042552+0000","last_peered":"2024-09-17T15:36:48.042552+0000","last_clean":"2024-09-17T15:36:48.042552+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:36:48.042552+0000","last_undegraded":"2024-09-17T15:36:48.042552+0000","last_fullsized":"2024-09-17T15:36:48.042552+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28082,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042306+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:36:48.042306+0000","last_peered":"2024-09-17T15:36:48.042306+0000","last_clean":"2024-09-17T15:36:48.042306+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:36:48.042306+0000","last_undegraded":"2024-09-17T15:36:48.042306+0000","last_fullsized":"2024-09-17T15:36:48.042306+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.043694+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:36:48.043694+0000","last_peered":"2024-09-17T15:36:48.043694+0000","last_clean":"2024-09-17T15:36:48.043694+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:36:48.043694+0000","last_undegraded":"2024-09-17T15:36:48.043694+0000","last_fullsized":"2024-09-17T15:36:48.043694+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21213,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:14.111073+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:37:14.111073+0000","last_peered":"2024-09-17T15:37:14.111073+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:37:14.111073+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042251+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:36:48.042251+0000","last_peered":"2024-09-17T15:36:48.042251+0000","last_clean":"2024-09-17T15:36:48.042251+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:36:48.042251+0000","last_undegraded":"2024-09-17T15:36:48.042251+0000","last_fullsized":"2024-09-17T15:36:48.042251+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":100,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042712+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:36:48.042712+0000","last_peered":"2024-09-17T15:36:48.042712+0000","last_clean":"2024-09-17T15:36:48.042712+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:36:48.042712+0000","last_undegraded":"2024-09-17T15:36:48.042712+0000","last_fullsized":"2024-09-17T15:36:48.042712+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29034,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T15:36:48.042480+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:36:48.042480+0000","last_peered":"2024-09-17T15:36:48.042480+0000","last_clean":"2024-09-17T15:36:48.042480+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:36:48.042480+0000","last_undegraded":"2024-09-17T15:36:48.042480+0000","last_fullsized":"2024-09-17T15:36:48.042480+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16929,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32531,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":192675840,"data_stored":192571063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051730,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492120,"kb_used_data":123080,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93879720,"statfs":{"total":96636764160,"available":96132833280,"internally_reserved":0,"allocated":126033920,"data_stored":125701116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":377876140},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724675,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501444,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870396,"statfs":{"total":96636764160,"available":96123285504,"internally_reserved":0,"allocated":512000,"data_stored":160667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":512945835},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247958,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539408,"kb_used_data":66296,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832432,"statfs":{"total":96636764160,"available":96084410368,"internally_reserved":0,"allocated":67887104,"data_stored":67542997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":125206528,"data_stored":125177624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:37:20.513 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:37:20.813 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:20.813 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:37:20.825 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":438,"stamp":"2024-09-17T15:37:19.829103+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15364,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34097,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1535020,"kb_used_data":187764,"kb_used_omap":13,"kb_used_meta":1347186,"kb_avail":281580500,"statfs":{"total":289910292480,"available":288338432000,"internally_reserved":0,"allocated":192270336,"data_stored":191213452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1379519487},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1636,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1636,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001183"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20929,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.112415+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:37:15.112415+0000","last_peered":"2024-09-17T15:37:15.112415+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:37:15.112415+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21066,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352281+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:37:15.352281+0000","last_peered":"2024-09-17T15:37:15.352281+0000","last_clean":"2024-09-17T15:37:15.352281+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:37:15.352281+0000","last_undegraded":"2024-09-17T15:37:15.352281+0000","last_fullsized":"2024-09-17T15:37:15.352281+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21174,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112900+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:37:15.112900+0000","last_peered":"2024-09-17T15:37:15.112900+0000","last_clean":"2024-09-17T15:37:15.112900+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:37:15.112900+0000","last_undegraded":"2024-09-17T15:37:15.112900+0000","last_fullsized":"2024-09-17T15:37:15.112900+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21071,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352626+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:37:15.352626+0000","last_peered":"2024-09-17T15:37:15.352626+0000","last_clean":"2024-09-17T15:37:15.352626+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:37:15.352626+0000","last_undegraded":"2024-09-17T15:37:15.352626+0000","last_fullsized":"2024-09-17T15:37:15.352626+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20814,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112605+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:37:15.112605+0000","last_peered":"2024-09-17T15:37:15.112605+0000","last_clean":"2024-09-17T15:37:15.112605+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:37:15.112605+0000","last_undegraded":"2024-09-17T15:37:15.112605+0000","last_fullsized":"2024-09-17T15:37:15.112605+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21565,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.188303+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:37:15.188303+0000","last_peered":"2024-09-17T15:37:15.188303+0000","last_clean":"2024-09-17T15:37:15.188303+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:37:15.188303+0000","last_undegraded":"2024-09-17T15:37:15.188303+0000","last_fullsized":"2024-09-17T15:37:15.188303+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21108,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.111962+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:37:15.111962+0000","last_peered":"2024-09-17T15:37:15.111962+0000","last_clean":"2024-09-17T15:37:15.111962+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:37:15.111962+0000","last_undegraded":"2024-09-17T15:37:15.111962+0000","last_fullsized":"2024-09-17T15:37:15.111962+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21075,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112248+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:37:15.112248+0000","last_peered":"2024-09-17T15:37:15.112248+0000","last_clean":"2024-09-17T15:37:15.112248+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:37:15.112248+0000","last_undegraded":"2024-09-17T15:37:15.112248+0000","last_fullsized":"2024-09-17T15:37:15.112248+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.111832+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:37:15.111832+0000","last_peered":"2024-09-17T15:37:15.111832+0000","last_clean":"2024-09-17T15:37:15.111832+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:37:15.111832+0000","last_undegraded":"2024-09-17T15:37:15.111832+0000","last_fullsized":"2024-09-17T15:37:15.111832+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29131,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112153+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:37:15.112153+0000","last_peered":"2024-09-17T15:37:15.112153+0000","last_clean":"2024-09-17T15:37:15.112153+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:37:15.112153+0000","last_undegraded":"2024-09-17T15:37:15.112153+0000","last_fullsized":"2024-09-17T15:37:15.112153+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352003+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:37:15.352003+0000","last_peered":"2024-09-17T15:37:15.352003+0000","last_clean":"2024-09-17T15:37:15.352003+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:37:15.352003+0000","last_undegraded":"2024-09-17T15:37:15.352003+0000","last_fullsized":"2024-09-17T15:37:15.352003+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20838,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.112848+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:37:15.112848+0000","last_peered":"2024-09-17T15:37:15.112848+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:37:15.112848+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112615+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:37:15.112615+0000","last_peered":"2024-09-17T15:37:15.112615+0000","last_clean":"2024-09-17T15:37:15.112615+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:37:15.112615+0000","last_undegraded":"2024-09-17T15:37:15.112615+0000","last_fullsized":"2024-09-17T15:37:15.112615+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28835,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352081+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:37:15.352081+0000","last_peered":"2024-09-17T15:37:15.352081+0000","last_clean":"2024-09-17T15:37:15.352081+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:37:15.352081+0000","last_undegraded":"2024-09-17T15:37:15.352081+0000","last_fullsized":"2024-09-17T15:37:15.352081+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":109,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112066+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:37:15.112066+0000","last_peered":"2024-09-17T15:37:15.112066+0000","last_clean":"2024-09-17T15:37:15.112066+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:37:15.112066+0000","last_undegraded":"2024-09-17T15:37:15.112066+0000","last_fullsized":"2024-09-17T15:37:15.112066+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29172,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.351970+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:37:15.351970+0000","last_peered":"2024-09-17T15:37:15.351970+0000","last_clean":"2024-09-17T15:37:15.351970+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:37:15.351970+0000","last_undegraded":"2024-09-17T15:37:15.351970+0000","last_fullsized":"2024-09-17T15:37:15.351970+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":22506,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:19.410260+0000","last_change":"2024-09-17T15:37:13.729028+0000","last_active":"2024-09-17T15:37:19.410260+0000","last_peered":"2024-09-17T15:37:19.410260+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:37:19.410260+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":686,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3453}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112352+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:37:15.112352+0000","last_peered":"2024-09-17T15:37:15.112352+0000","last_clean":"2024-09-17T15:37:15.112352+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:37:15.112352+0000","last_undegraded":"2024-09-17T15:37:15.112352+0000","last_fullsized":"2024-09-17T15:37:15.112352+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28086,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112447+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:37:15.112447+0000","last_peered":"2024-09-17T15:37:15.112447+0000","last_clean":"2024-09-17T15:37:15.112447+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:37:15.112447+0000","last_undegraded":"2024-09-17T15:37:15.112447+0000","last_fullsized":"2024-09-17T15:37:15.112447+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352224+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:37:15.352224+0000","last_peered":"2024-09-17T15:37:15.352224+0000","last_clean":"2024-09-17T15:37:15.352224+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:37:15.352224+0000","last_undegraded":"2024-09-17T15:37:15.352224+0000","last_fullsized":"2024-09-17T15:37:15.352224+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21215,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.113213+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:37:15.113213+0000","last_peered":"2024-09-17T15:37:15.113213+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:37:15.113213+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112357+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:37:15.112357+0000","last_peered":"2024-09-17T15:37:15.112357+0000","last_clean":"2024-09-17T15:37:15.112357+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:37:15.112357+0000","last_undegraded":"2024-09-17T15:37:15.112357+0000","last_fullsized":"2024-09-17T15:37:15.112357+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":104,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112592+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:37:15.112592+0000","last_peered":"2024-09-17T15:37:15.112592+0000","last_clean":"2024-09-17T15:37:15.112592+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:37:15.112592+0000","last_undegraded":"2024-09-17T15:37:15.112592+0000","last_fullsized":"2024-09-17T15:37:15.112592+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29038,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112255+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:37:15.112255+0000","last_peered":"2024-09-17T15:37:15.112255+0000","last_clean":"2024-09-17T15:37:15.112255+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:37:15.112255+0000","last_undegraded":"2024-09-17T15:37:15.112255+0000","last_fullsized":"2024-09-17T15:37:15.112255+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15364,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34096,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":190345216,"data_stored":190240439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051731,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":489980,"kb_used_data":120940,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93881860,"statfs":{"total":96636764160,"available":96135024640,"internally_reserved":0,"allocated":123842560,"data_stored":123499005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":377876140},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724676,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505616,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866224,"statfs":{"total":96636764160,"available":96119013376,"internally_reserved":0,"allocated":524288,"data_stored":165817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":517205675},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247959,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539424,"kb_used_data":66312,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832416,"statfs":{"total":96636764160,"available":96084393984,"internally_reserved":0,"allocated":67903488,"data_stored":67548630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":122875904,"data_stored":122847000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:37:20.826 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:37:21.146 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:21.147 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:37:21.161 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":438,"stamp":"2024-09-17T15:37:19.829103+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15364,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34097,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1535020,"kb_used_data":187764,"kb_used_omap":13,"kb_used_meta":1347186,"kb_avail":281580500,"statfs":{"total":289910292480,"available":288338432000,"internally_reserved":0,"allocated":192270336,"data_stored":191213452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1379519487},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1636,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1636,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001183"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20929,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.112415+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:37:15.112415+0000","last_peered":"2024-09-17T15:37:15.112415+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:37:15.112415+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21066,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352281+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:37:15.352281+0000","last_peered":"2024-09-17T15:37:15.352281+0000","last_clean":"2024-09-17T15:37:15.352281+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:37:15.352281+0000","last_undegraded":"2024-09-17T15:37:15.352281+0000","last_fullsized":"2024-09-17T15:37:15.352281+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21174,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112900+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:37:15.112900+0000","last_peered":"2024-09-17T15:37:15.112900+0000","last_clean":"2024-09-17T15:37:15.112900+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:37:15.112900+0000","last_undegraded":"2024-09-17T15:37:15.112900+0000","last_fullsized":"2024-09-17T15:37:15.112900+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21071,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352626+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:37:15.352626+0000","last_peered":"2024-09-17T15:37:15.352626+0000","last_clean":"2024-09-17T15:37:15.352626+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:37:15.352626+0000","last_undegraded":"2024-09-17T15:37:15.352626+0000","last_fullsized":"2024-09-17T15:37:15.352626+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20814,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112605+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:37:15.112605+0000","last_peered":"2024-09-17T15:37:15.112605+0000","last_clean":"2024-09-17T15:37:15.112605+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:37:15.112605+0000","last_undegraded":"2024-09-17T15:37:15.112605+0000","last_fullsized":"2024-09-17T15:37:15.112605+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21565,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.188303+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:37:15.188303+0000","last_peered":"2024-09-17T15:37:15.188303+0000","last_clean":"2024-09-17T15:37:15.188303+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:37:15.188303+0000","last_undegraded":"2024-09-17T15:37:15.188303+0000","last_fullsized":"2024-09-17T15:37:15.188303+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21108,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.111962+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:37:15.111962+0000","last_peered":"2024-09-17T15:37:15.111962+0000","last_clean":"2024-09-17T15:37:15.111962+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:37:15.111962+0000","last_undegraded":"2024-09-17T15:37:15.111962+0000","last_fullsized":"2024-09-17T15:37:15.111962+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21075,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112248+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:37:15.112248+0000","last_peered":"2024-09-17T15:37:15.112248+0000","last_clean":"2024-09-17T15:37:15.112248+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:37:15.112248+0000","last_undegraded":"2024-09-17T15:37:15.112248+0000","last_fullsized":"2024-09-17T15:37:15.112248+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.111832+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:37:15.111832+0000","last_peered":"2024-09-17T15:37:15.111832+0000","last_clean":"2024-09-17T15:37:15.111832+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:37:15.111832+0000","last_undegraded":"2024-09-17T15:37:15.111832+0000","last_fullsized":"2024-09-17T15:37:15.111832+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29131,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112153+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:37:15.112153+0000","last_peered":"2024-09-17T15:37:15.112153+0000","last_clean":"2024-09-17T15:37:15.112153+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:37:15.112153+0000","last_undegraded":"2024-09-17T15:37:15.112153+0000","last_fullsized":"2024-09-17T15:37:15.112153+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352003+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:37:15.352003+0000","last_peered":"2024-09-17T15:37:15.352003+0000","last_clean":"2024-09-17T15:37:15.352003+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:37:15.352003+0000","last_undegraded":"2024-09-17T15:37:15.352003+0000","last_fullsized":"2024-09-17T15:37:15.352003+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20838,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.112848+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:37:15.112848+0000","last_peered":"2024-09-17T15:37:15.112848+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:37:15.112848+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112615+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:37:15.112615+0000","last_peered":"2024-09-17T15:37:15.112615+0000","last_clean":"2024-09-17T15:37:15.112615+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:37:15.112615+0000","last_undegraded":"2024-09-17T15:37:15.112615+0000","last_fullsized":"2024-09-17T15:37:15.112615+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28835,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352081+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:37:15.352081+0000","last_peered":"2024-09-17T15:37:15.352081+0000","last_clean":"2024-09-17T15:37:15.352081+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:37:15.352081+0000","last_undegraded":"2024-09-17T15:37:15.352081+0000","last_fullsized":"2024-09-17T15:37:15.352081+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":109,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112066+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:37:15.112066+0000","last_peered":"2024-09-17T15:37:15.112066+0000","last_clean":"2024-09-17T15:37:15.112066+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:37:15.112066+0000","last_undegraded":"2024-09-17T15:37:15.112066+0000","last_fullsized":"2024-09-17T15:37:15.112066+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29172,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.351970+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:37:15.351970+0000","last_peered":"2024-09-17T15:37:15.351970+0000","last_clean":"2024-09-17T15:37:15.351970+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:37:15.351970+0000","last_undegraded":"2024-09-17T15:37:15.351970+0000","last_fullsized":"2024-09-17T15:37:15.351970+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":22506,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:19.410260+0000","last_change":"2024-09-17T15:37:13.729028+0000","last_active":"2024-09-17T15:37:19.410260+0000","last_peered":"2024-09-17T15:37:19.410260+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:37:19.410260+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":686,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3453}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112352+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:37:15.112352+0000","last_peered":"2024-09-17T15:37:15.112352+0000","last_clean":"2024-09-17T15:37:15.112352+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:37:15.112352+0000","last_undegraded":"2024-09-17T15:37:15.112352+0000","last_fullsized":"2024-09-17T15:37:15.112352+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28086,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112447+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:37:15.112447+0000","last_peered":"2024-09-17T15:37:15.112447+0000","last_clean":"2024-09-17T15:37:15.112447+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:37:15.112447+0000","last_undegraded":"2024-09-17T15:37:15.112447+0000","last_fullsized":"2024-09-17T15:37:15.112447+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352224+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:37:15.352224+0000","last_peered":"2024-09-17T15:37:15.352224+0000","last_clean":"2024-09-17T15:37:15.352224+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:37:15.352224+0000","last_undegraded":"2024-09-17T15:37:15.352224+0000","last_fullsized":"2024-09-17T15:37:15.352224+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21215,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.113213+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:37:15.113213+0000","last_peered":"2024-09-17T15:37:15.113213+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:37:15.113213+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112357+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:37:15.112357+0000","last_peered":"2024-09-17T15:37:15.112357+0000","last_clean":"2024-09-17T15:37:15.112357+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:37:15.112357+0000","last_undegraded":"2024-09-17T15:37:15.112357+0000","last_fullsized":"2024-09-17T15:37:15.112357+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":104,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112592+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:37:15.112592+0000","last_peered":"2024-09-17T15:37:15.112592+0000","last_clean":"2024-09-17T15:37:15.112592+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:37:15.112592+0000","last_undegraded":"2024-09-17T15:37:15.112592+0000","last_fullsized":"2024-09-17T15:37:15.112592+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29038,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112255+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:37:15.112255+0000","last_peered":"2024-09-17T15:37:15.112255+0000","last_clean":"2024-09-17T15:37:15.112255+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:37:15.112255+0000","last_undegraded":"2024-09-17T15:37:15.112255+0000","last_fullsized":"2024-09-17T15:37:15.112255+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15364,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34096,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":190345216,"data_stored":190240439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051731,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":489980,"kb_used_data":120940,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93881860,"statfs":{"total":96636764160,"available":96135024640,"internally_reserved":0,"allocated":123842560,"data_stored":123499005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":377876140},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724676,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505616,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866224,"statfs":{"total":96636764160,"available":96119013376,"internally_reserved":0,"allocated":524288,"data_stored":165817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":517205675},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247959,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539424,"kb_used_data":66312,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832416,"statfs":{"total":96636764160,"available":96084393984,"internally_reserved":0,"allocated":67903488,"data_stored":67548630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":122875904,"data_stored":122847000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:37:21.162 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:37:21.162 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'20648", 'reported_seq': 20929, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:37:15.112415+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:37:15.112415+0000', 'last_peered': '2024-09-17T15:37:15.112415+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:37:15.112415+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'10600", 'ondisk_log_start': "43'10600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20648, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:21.162 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:37:21.162 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20542", 'reported_seq': 20838, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:37:15.112848+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:37:15.112848+0000', 'last_peered': '2024-09-17T15:37:15.112848+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:37:15.112848+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17138, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3943}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:21.163 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:37:21.163 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'20966", 'reported_seq': 22506, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:37:19.410260+0000', 'last_change': '2024-09-17T15:37:13.729028+0000', 'last_active': '2024-09-17T15:37:19.410260+0000', 'last_peered': '2024-09-17T15:37:19.410260+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:37:19.410260+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "43'10900", 'ondisk_log_start': "43'10900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17514, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 686, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3453}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:21.163 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:37:21.163 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'20886", 'reported_seq': 21215, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:37:15.113213+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:37:15.113213+0000', 'last_peered': '2024-09-17T15:37:15.113213+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:37:15.113213+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'10800", 'ondisk_log_start': "43'10800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17673, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3980}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:21.163 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:37:21.534 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:21.534 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 15364/24 objects degraded (64016.667%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":778,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1571860480,"bytes_avail":288338432000,"bytes_total":289910292480,"degraded_objects":15364,"degraded_total":24,"degraded_ratio":640.16666666666663,"recovering_objects_per_sec":204,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1636,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:37:21.549 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:37:21.550 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:37:21.919 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:21.919 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:37:21.932 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":439,"stamp":"2024-09-17T15:37:21.829473+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15364,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34097,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1535020,"kb_used_data":187764,"kb_used_omap":13,"kb_used_meta":1347186,"kb_avail":281580500,"statfs":{"total":289910292480,"available":288338432000,"internally_reserved":0,"allocated":192270336,"data_stored":191213452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1379519487},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1636,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1636,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001256"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20929,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.112415+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:37:15.112415+0000","last_peered":"2024-09-17T15:37:15.112415+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:37:15.112415+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21066,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352281+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:37:15.352281+0000","last_peered":"2024-09-17T15:37:15.352281+0000","last_clean":"2024-09-17T15:37:15.352281+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:37:15.352281+0000","last_undegraded":"2024-09-17T15:37:15.352281+0000","last_fullsized":"2024-09-17T15:37:15.352281+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21174,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112900+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:37:15.112900+0000","last_peered":"2024-09-17T15:37:15.112900+0000","last_clean":"2024-09-17T15:37:15.112900+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:37:15.112900+0000","last_undegraded":"2024-09-17T15:37:15.112900+0000","last_fullsized":"2024-09-17T15:37:15.112900+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21071,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352626+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:37:15.352626+0000","last_peered":"2024-09-17T15:37:15.352626+0000","last_clean":"2024-09-17T15:37:15.352626+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:37:15.352626+0000","last_undegraded":"2024-09-17T15:37:15.352626+0000","last_fullsized":"2024-09-17T15:37:15.352626+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20814,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112605+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:37:15.112605+0000","last_peered":"2024-09-17T15:37:15.112605+0000","last_clean":"2024-09-17T15:37:15.112605+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:37:15.112605+0000","last_undegraded":"2024-09-17T15:37:15.112605+0000","last_fullsized":"2024-09-17T15:37:15.112605+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21565,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.188303+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:37:15.188303+0000","last_peered":"2024-09-17T15:37:15.188303+0000","last_clean":"2024-09-17T15:37:15.188303+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:37:15.188303+0000","last_undegraded":"2024-09-17T15:37:15.188303+0000","last_fullsized":"2024-09-17T15:37:15.188303+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21108,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.111962+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:37:15.111962+0000","last_peered":"2024-09-17T15:37:15.111962+0000","last_clean":"2024-09-17T15:37:15.111962+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:37:15.111962+0000","last_undegraded":"2024-09-17T15:37:15.111962+0000","last_fullsized":"2024-09-17T15:37:15.111962+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21075,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112248+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:37:15.112248+0000","last_peered":"2024-09-17T15:37:15.112248+0000","last_clean":"2024-09-17T15:37:15.112248+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:37:15.112248+0000","last_undegraded":"2024-09-17T15:37:15.112248+0000","last_fullsized":"2024-09-17T15:37:15.112248+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.111832+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:37:15.111832+0000","last_peered":"2024-09-17T15:37:15.111832+0000","last_clean":"2024-09-17T15:37:15.111832+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:37:15.111832+0000","last_undegraded":"2024-09-17T15:37:15.111832+0000","last_fullsized":"2024-09-17T15:37:15.111832+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29131,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112153+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:37:15.112153+0000","last_peered":"2024-09-17T15:37:15.112153+0000","last_clean":"2024-09-17T15:37:15.112153+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:37:15.112153+0000","last_undegraded":"2024-09-17T15:37:15.112153+0000","last_fullsized":"2024-09-17T15:37:15.112153+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352003+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:37:15.352003+0000","last_peered":"2024-09-17T15:37:15.352003+0000","last_clean":"2024-09-17T15:37:15.352003+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:37:15.352003+0000","last_undegraded":"2024-09-17T15:37:15.352003+0000","last_fullsized":"2024-09-17T15:37:15.352003+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20838,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.112848+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:37:15.112848+0000","last_peered":"2024-09-17T15:37:15.112848+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:37:15.112848+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112615+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:37:15.112615+0000","last_peered":"2024-09-17T15:37:15.112615+0000","last_clean":"2024-09-17T15:37:15.112615+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:37:15.112615+0000","last_undegraded":"2024-09-17T15:37:15.112615+0000","last_fullsized":"2024-09-17T15:37:15.112615+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28835,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352081+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:37:15.352081+0000","last_peered":"2024-09-17T15:37:15.352081+0000","last_clean":"2024-09-17T15:37:15.352081+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:37:15.352081+0000","last_undegraded":"2024-09-17T15:37:15.352081+0000","last_fullsized":"2024-09-17T15:37:15.352081+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":109,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112066+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:37:15.112066+0000","last_peered":"2024-09-17T15:37:15.112066+0000","last_clean":"2024-09-17T15:37:15.112066+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:37:15.112066+0000","last_undegraded":"2024-09-17T15:37:15.112066+0000","last_fullsized":"2024-09-17T15:37:15.112066+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29172,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.351970+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:37:15.351970+0000","last_peered":"2024-09-17T15:37:15.351970+0000","last_clean":"2024-09-17T15:37:15.351970+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:37:15.351970+0000","last_undegraded":"2024-09-17T15:37:15.351970+0000","last_fullsized":"2024-09-17T15:37:15.351970+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":22506,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:19.410260+0000","last_change":"2024-09-17T15:37:13.729028+0000","last_active":"2024-09-17T15:37:19.410260+0000","last_peered":"2024-09-17T15:37:19.410260+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:37:19.410260+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":686,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3453}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112352+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:37:15.112352+0000","last_peered":"2024-09-17T15:37:15.112352+0000","last_clean":"2024-09-17T15:37:15.112352+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:37:15.112352+0000","last_undegraded":"2024-09-17T15:37:15.112352+0000","last_fullsized":"2024-09-17T15:37:15.112352+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28086,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112447+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:37:15.112447+0000","last_peered":"2024-09-17T15:37:15.112447+0000","last_clean":"2024-09-17T15:37:15.112447+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:37:15.112447+0000","last_undegraded":"2024-09-17T15:37:15.112447+0000","last_fullsized":"2024-09-17T15:37:15.112447+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352224+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:37:15.352224+0000","last_peered":"2024-09-17T15:37:15.352224+0000","last_clean":"2024-09-17T15:37:15.352224+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:37:15.352224+0000","last_undegraded":"2024-09-17T15:37:15.352224+0000","last_fullsized":"2024-09-17T15:37:15.352224+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21215,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.113213+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:37:15.113213+0000","last_peered":"2024-09-17T15:37:15.113213+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:37:15.113213+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112357+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:37:15.112357+0000","last_peered":"2024-09-17T15:37:15.112357+0000","last_clean":"2024-09-17T15:37:15.112357+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:37:15.112357+0000","last_undegraded":"2024-09-17T15:37:15.112357+0000","last_fullsized":"2024-09-17T15:37:15.112357+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":104,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112592+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:37:15.112592+0000","last_peered":"2024-09-17T15:37:15.112592+0000","last_clean":"2024-09-17T15:37:15.112592+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:37:15.112592+0000","last_undegraded":"2024-09-17T15:37:15.112592+0000","last_fullsized":"2024-09-17T15:37:15.112592+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29038,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112255+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:37:15.112255+0000","last_peered":"2024-09-17T15:37:15.112255+0000","last_clean":"2024-09-17T15:37:15.112255+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:37:15.112255+0000","last_undegraded":"2024-09-17T15:37:15.112255+0000","last_fullsized":"2024-09-17T15:37:15.112255+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15364,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34096,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":190345216,"data_stored":190240439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051731,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":489980,"kb_used_data":120940,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93881860,"statfs":{"total":96636764160,"available":96135024640,"internally_reserved":0,"allocated":123842560,"data_stored":123499005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":377876140},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724676,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505616,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866224,"statfs":{"total":96636764160,"available":96119013376,"internally_reserved":0,"allocated":524288,"data_stored":165817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":517205675},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247959,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539424,"kb_used_data":66312,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832416,"statfs":{"total":96636764160,"available":96084393984,"internally_reserved":0,"allocated":67903488,"data_stored":67548630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":122875904,"data_stored":122847000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:37:24.935 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:37:25.270 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:25.271 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:37:25.282 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":440,"stamp":"2024-09-17T15:37:23.829769+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15364,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34097,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1532620,"kb_used_data":185364,"kb_used_omap":13,"kb_used_meta":1347186,"kb_avail":281582900,"statfs":{"total":289910292480,"available":288340889600,"internally_reserved":0,"allocated":189812736,"data_stored":188755852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1379519487},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1636,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1636,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.721031"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20929,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.112415+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:37:15.112415+0000","last_peered":"2024-09-17T15:37:15.112415+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:37:15.112415+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21066,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352281+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:37:15.352281+0000","last_peered":"2024-09-17T15:37:15.352281+0000","last_clean":"2024-09-17T15:37:15.352281+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:37:15.352281+0000","last_undegraded":"2024-09-17T15:37:15.352281+0000","last_fullsized":"2024-09-17T15:37:15.352281+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21174,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112900+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:37:15.112900+0000","last_peered":"2024-09-17T15:37:15.112900+0000","last_clean":"2024-09-17T15:37:15.112900+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:37:15.112900+0000","last_undegraded":"2024-09-17T15:37:15.112900+0000","last_fullsized":"2024-09-17T15:37:15.112900+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21071,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352626+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:37:15.352626+0000","last_peered":"2024-09-17T15:37:15.352626+0000","last_clean":"2024-09-17T15:37:15.352626+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:37:15.352626+0000","last_undegraded":"2024-09-17T15:37:15.352626+0000","last_fullsized":"2024-09-17T15:37:15.352626+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20814,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112605+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:37:15.112605+0000","last_peered":"2024-09-17T15:37:15.112605+0000","last_clean":"2024-09-17T15:37:15.112605+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:37:15.112605+0000","last_undegraded":"2024-09-17T15:37:15.112605+0000","last_fullsized":"2024-09-17T15:37:15.112605+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21565,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.188303+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:37:15.188303+0000","last_peered":"2024-09-17T15:37:15.188303+0000","last_clean":"2024-09-17T15:37:15.188303+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:37:15.188303+0000","last_undegraded":"2024-09-17T15:37:15.188303+0000","last_fullsized":"2024-09-17T15:37:15.188303+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21108,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.111962+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:37:15.111962+0000","last_peered":"2024-09-17T15:37:15.111962+0000","last_clean":"2024-09-17T15:37:15.111962+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:37:15.111962+0000","last_undegraded":"2024-09-17T15:37:15.111962+0000","last_fullsized":"2024-09-17T15:37:15.111962+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21075,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112248+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:37:15.112248+0000","last_peered":"2024-09-17T15:37:15.112248+0000","last_clean":"2024-09-17T15:37:15.112248+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:37:15.112248+0000","last_undegraded":"2024-09-17T15:37:15.112248+0000","last_fullsized":"2024-09-17T15:37:15.112248+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.111832+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:37:15.111832+0000","last_peered":"2024-09-17T15:37:15.111832+0000","last_clean":"2024-09-17T15:37:15.111832+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:37:15.111832+0000","last_undegraded":"2024-09-17T15:37:15.111832+0000","last_fullsized":"2024-09-17T15:37:15.111832+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29131,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112153+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:37:15.112153+0000","last_peered":"2024-09-17T15:37:15.112153+0000","last_clean":"2024-09-17T15:37:15.112153+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:37:15.112153+0000","last_undegraded":"2024-09-17T15:37:15.112153+0000","last_fullsized":"2024-09-17T15:37:15.112153+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352003+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:37:15.352003+0000","last_peered":"2024-09-17T15:37:15.352003+0000","last_clean":"2024-09-17T15:37:15.352003+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:37:15.352003+0000","last_undegraded":"2024-09-17T15:37:15.352003+0000","last_fullsized":"2024-09-17T15:37:15.352003+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20838,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.112848+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:37:15.112848+0000","last_peered":"2024-09-17T15:37:15.112848+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:37:15.112848+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112615+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:37:15.112615+0000","last_peered":"2024-09-17T15:37:15.112615+0000","last_clean":"2024-09-17T15:37:15.112615+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:37:15.112615+0000","last_undegraded":"2024-09-17T15:37:15.112615+0000","last_fullsized":"2024-09-17T15:37:15.112615+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28835,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352081+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:37:15.352081+0000","last_peered":"2024-09-17T15:37:15.352081+0000","last_clean":"2024-09-17T15:37:15.352081+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:37:15.352081+0000","last_undegraded":"2024-09-17T15:37:15.352081+0000","last_fullsized":"2024-09-17T15:37:15.352081+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":109,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112066+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:37:15.112066+0000","last_peered":"2024-09-17T15:37:15.112066+0000","last_clean":"2024-09-17T15:37:15.112066+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:37:15.112066+0000","last_undegraded":"2024-09-17T15:37:15.112066+0000","last_fullsized":"2024-09-17T15:37:15.112066+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29173,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352021+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:37:15.352021+0000","last_peered":"2024-09-17T15:37:15.352021+0000","last_clean":"2024-09-17T15:37:15.352021+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:37:15.352021+0000","last_undegraded":"2024-09-17T15:37:15.352021+0000","last_fullsized":"2024-09-17T15:37:15.352021+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":22506,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:19.410260+0000","last_change":"2024-09-17T15:37:13.729028+0000","last_active":"2024-09-17T15:37:19.410260+0000","last_peered":"2024-09-17T15:37:19.410260+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:37:19.410260+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":686,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3453}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112352+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:37:15.112352+0000","last_peered":"2024-09-17T15:37:15.112352+0000","last_clean":"2024-09-17T15:37:15.112352+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:37:15.112352+0000","last_undegraded":"2024-09-17T15:37:15.112352+0000","last_fullsized":"2024-09-17T15:37:15.112352+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28086,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112447+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:37:15.112447+0000","last_peered":"2024-09-17T15:37:15.112447+0000","last_clean":"2024-09-17T15:37:15.112447+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:37:15.112447+0000","last_undegraded":"2024-09-17T15:37:15.112447+0000","last_fullsized":"2024-09-17T15:37:15.112447+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352224+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:37:15.352224+0000","last_peered":"2024-09-17T15:37:15.352224+0000","last_clean":"2024-09-17T15:37:15.352224+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:37:15.352224+0000","last_undegraded":"2024-09-17T15:37:15.352224+0000","last_fullsized":"2024-09-17T15:37:15.352224+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21215,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.113213+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:37:15.113213+0000","last_peered":"2024-09-17T15:37:15.113213+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:37:15.113213+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112357+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:37:15.112357+0000","last_peered":"2024-09-17T15:37:15.112357+0000","last_clean":"2024-09-17T15:37:15.112357+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:37:15.112357+0000","last_undegraded":"2024-09-17T15:37:15.112357+0000","last_fullsized":"2024-09-17T15:37:15.112357+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":104,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112592+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:37:15.112592+0000","last_peered":"2024-09-17T15:37:15.112592+0000","last_clean":"2024-09-17T15:37:15.112592+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:37:15.112592+0000","last_undegraded":"2024-09-17T15:37:15.112592+0000","last_fullsized":"2024-09-17T15:37:15.112592+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29038,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112255+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:37:15.112255+0000","last_peered":"2024-09-17T15:37:15.112255+0000","last_clean":"2024-09-17T15:37:15.112255+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:37:15.112255+0000","last_undegraded":"2024-09-17T15:37:15.112255+0000","last_fullsized":"2024-09-17T15:37:15.112255+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15364,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34096,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":187887616,"data_stored":187782839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051732,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487580,"kb_used_data":118540,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93884260,"statfs":{"total":96636764160,"available":96137482240,"internally_reserved":0,"allocated":121384960,"data_stored":121041405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":377876140},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724676,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505616,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866224,"statfs":{"total":96636764160,"available":96119013376,"internally_reserved":0,"allocated":524288,"data_stored":165817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":517205675},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247959,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539424,"kb_used_data":66312,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832416,"statfs":{"total":96636764160,"available":96084393984,"internally_reserved":0,"allocated":67903488,"data_stored":67548630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":120418304,"data_stored":120389400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:37:25.283 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:37:25.593 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:25.594 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:37:25.607 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":440,"stamp":"2024-09-17T15:37:23.829769+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15364,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34097,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1532620,"kb_used_data":185364,"kb_used_omap":13,"kb_used_meta":1347186,"kb_avail":281582900,"statfs":{"total":289910292480,"available":288340889600,"internally_reserved":0,"allocated":189812736,"data_stored":188755852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1379519487},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1636,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1636,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.721031"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20929,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.112415+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:37:15.112415+0000","last_peered":"2024-09-17T15:37:15.112415+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:37:15.112415+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21066,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352281+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:37:15.352281+0000","last_peered":"2024-09-17T15:37:15.352281+0000","last_clean":"2024-09-17T15:37:15.352281+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:37:15.352281+0000","last_undegraded":"2024-09-17T15:37:15.352281+0000","last_fullsized":"2024-09-17T15:37:15.352281+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21174,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112900+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:37:15.112900+0000","last_peered":"2024-09-17T15:37:15.112900+0000","last_clean":"2024-09-17T15:37:15.112900+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:37:15.112900+0000","last_undegraded":"2024-09-17T15:37:15.112900+0000","last_fullsized":"2024-09-17T15:37:15.112900+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21071,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352626+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:37:15.352626+0000","last_peered":"2024-09-17T15:37:15.352626+0000","last_clean":"2024-09-17T15:37:15.352626+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:37:15.352626+0000","last_undegraded":"2024-09-17T15:37:15.352626+0000","last_fullsized":"2024-09-17T15:37:15.352626+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20814,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112605+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:37:15.112605+0000","last_peered":"2024-09-17T15:37:15.112605+0000","last_clean":"2024-09-17T15:37:15.112605+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:37:15.112605+0000","last_undegraded":"2024-09-17T15:37:15.112605+0000","last_fullsized":"2024-09-17T15:37:15.112605+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21565,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.188303+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:37:15.188303+0000","last_peered":"2024-09-17T15:37:15.188303+0000","last_clean":"2024-09-17T15:37:15.188303+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:37:15.188303+0000","last_undegraded":"2024-09-17T15:37:15.188303+0000","last_fullsized":"2024-09-17T15:37:15.188303+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21108,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.111962+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:37:15.111962+0000","last_peered":"2024-09-17T15:37:15.111962+0000","last_clean":"2024-09-17T15:37:15.111962+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:37:15.111962+0000","last_undegraded":"2024-09-17T15:37:15.111962+0000","last_fullsized":"2024-09-17T15:37:15.111962+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21075,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112248+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:37:15.112248+0000","last_peered":"2024-09-17T15:37:15.112248+0000","last_clean":"2024-09-17T15:37:15.112248+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:37:15.112248+0000","last_undegraded":"2024-09-17T15:37:15.112248+0000","last_fullsized":"2024-09-17T15:37:15.112248+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.111832+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:37:15.111832+0000","last_peered":"2024-09-17T15:37:15.111832+0000","last_clean":"2024-09-17T15:37:15.111832+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:37:15.111832+0000","last_undegraded":"2024-09-17T15:37:15.111832+0000","last_fullsized":"2024-09-17T15:37:15.111832+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29131,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112153+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:37:15.112153+0000","last_peered":"2024-09-17T15:37:15.112153+0000","last_clean":"2024-09-17T15:37:15.112153+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:37:15.112153+0000","last_undegraded":"2024-09-17T15:37:15.112153+0000","last_fullsized":"2024-09-17T15:37:15.112153+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352003+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:37:15.352003+0000","last_peered":"2024-09-17T15:37:15.352003+0000","last_clean":"2024-09-17T15:37:15.352003+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:37:15.352003+0000","last_undegraded":"2024-09-17T15:37:15.352003+0000","last_fullsized":"2024-09-17T15:37:15.352003+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20838,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.112848+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:37:15.112848+0000","last_peered":"2024-09-17T15:37:15.112848+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:37:15.112848+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112615+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:37:15.112615+0000","last_peered":"2024-09-17T15:37:15.112615+0000","last_clean":"2024-09-17T15:37:15.112615+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:37:15.112615+0000","last_undegraded":"2024-09-17T15:37:15.112615+0000","last_fullsized":"2024-09-17T15:37:15.112615+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28835,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352081+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:37:15.352081+0000","last_peered":"2024-09-17T15:37:15.352081+0000","last_clean":"2024-09-17T15:37:15.352081+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:37:15.352081+0000","last_undegraded":"2024-09-17T15:37:15.352081+0000","last_fullsized":"2024-09-17T15:37:15.352081+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":109,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112066+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:37:15.112066+0000","last_peered":"2024-09-17T15:37:15.112066+0000","last_clean":"2024-09-17T15:37:15.112066+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:37:15.112066+0000","last_undegraded":"2024-09-17T15:37:15.112066+0000","last_fullsized":"2024-09-17T15:37:15.112066+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29173,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352021+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:37:15.352021+0000","last_peered":"2024-09-17T15:37:15.352021+0000","last_clean":"2024-09-17T15:37:15.352021+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:37:15.352021+0000","last_undegraded":"2024-09-17T15:37:15.352021+0000","last_fullsized":"2024-09-17T15:37:15.352021+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":22506,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:19.410260+0000","last_change":"2024-09-17T15:37:13.729028+0000","last_active":"2024-09-17T15:37:19.410260+0000","last_peered":"2024-09-17T15:37:19.410260+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:37:19.410260+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":686,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3453}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112352+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:37:15.112352+0000","last_peered":"2024-09-17T15:37:15.112352+0000","last_clean":"2024-09-17T15:37:15.112352+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:37:15.112352+0000","last_undegraded":"2024-09-17T15:37:15.112352+0000","last_fullsized":"2024-09-17T15:37:15.112352+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28086,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112447+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:37:15.112447+0000","last_peered":"2024-09-17T15:37:15.112447+0000","last_clean":"2024-09-17T15:37:15.112447+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:37:15.112447+0000","last_undegraded":"2024-09-17T15:37:15.112447+0000","last_fullsized":"2024-09-17T15:37:15.112447+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352224+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:37:15.352224+0000","last_peered":"2024-09-17T15:37:15.352224+0000","last_clean":"2024-09-17T15:37:15.352224+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:37:15.352224+0000","last_undegraded":"2024-09-17T15:37:15.352224+0000","last_fullsized":"2024-09-17T15:37:15.352224+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21215,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.113213+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:37:15.113213+0000","last_peered":"2024-09-17T15:37:15.113213+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:37:15.113213+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112357+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:37:15.112357+0000","last_peered":"2024-09-17T15:37:15.112357+0000","last_clean":"2024-09-17T15:37:15.112357+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:37:15.112357+0000","last_undegraded":"2024-09-17T15:37:15.112357+0000","last_fullsized":"2024-09-17T15:37:15.112357+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":104,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112592+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:37:15.112592+0000","last_peered":"2024-09-17T15:37:15.112592+0000","last_clean":"2024-09-17T15:37:15.112592+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:37:15.112592+0000","last_undegraded":"2024-09-17T15:37:15.112592+0000","last_fullsized":"2024-09-17T15:37:15.112592+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29038,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112255+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:37:15.112255+0000","last_peered":"2024-09-17T15:37:15.112255+0000","last_clean":"2024-09-17T15:37:15.112255+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:37:15.112255+0000","last_undegraded":"2024-09-17T15:37:15.112255+0000","last_fullsized":"2024-09-17T15:37:15.112255+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15364,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34096,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":187887616,"data_stored":187782839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051732,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487580,"kb_used_data":118540,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93884260,"statfs":{"total":96636764160,"available":96137482240,"internally_reserved":0,"allocated":121384960,"data_stored":121041405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":377876140},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724676,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505616,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866224,"statfs":{"total":96636764160,"available":96119013376,"internally_reserved":0,"allocated":524288,"data_stored":165817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":517205675},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247959,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539424,"kb_used_data":66312,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832416,"statfs":{"total":96636764160,"available":96084393984,"internally_reserved":0,"allocated":67903488,"data_stored":67548630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":120418304,"data_stored":120389400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:37:25.609 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:37:25.609 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'20648", 'reported_seq': 20929, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:37:15.112415+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:37:15.112415+0000', 'last_peered': '2024-09-17T15:37:15.112415+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:37:15.112415+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'10600", 'ondisk_log_start': "43'10600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20648, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:25.609 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:37:25.609 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20542", 'reported_seq': 20838, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:37:15.112848+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:37:15.112848+0000', 'last_peered': '2024-09-17T15:37:15.112848+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:37:15.112848+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17138, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3943}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:25.609 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:37:25.609 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'20966", 'reported_seq': 22506, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:37:19.410260+0000', 'last_change': '2024-09-17T15:37:13.729028+0000', 'last_active': '2024-09-17T15:37:19.410260+0000', 'last_peered': '2024-09-17T15:37:19.410260+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:37:19.410260+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "43'10900", 'ondisk_log_start': "43'10900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17514, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 686, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3453}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:25.610 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:37:25.610 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'20886", 'reported_seq': 21215, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:37:15.113213+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:37:15.113213+0000', 'last_peered': '2024-09-17T15:37:15.113213+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:37:15.113213+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'10800", 'ondisk_log_start': "43'10800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17673, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3980}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:25.610 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:37:26.003 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:26.003 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 15364/24 objects degraded (64016.667%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":783,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1569402880,"bytes_avail":288340889600,"bytes_total":289910292480,"degraded_objects":15364,"degraded_total":24,"degraded_ratio":640.16666666666663,"recovering_objects_per_sec":168,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1636,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:37:26.017 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:37:26.018 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:37:26.324 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:26.324 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:37:26.338 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":441,"stamp":"2024-09-17T15:37:25.830084+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14783,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34678,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1536780,"kb_used_data":185364,"kb_used_omap":13,"kb_used_meta":1351346,"kb_avail":281578740,"statfs":{"total":289910292480,"available":288336629760,"internally_reserved":0,"allocated":189812736,"data_stored":188755852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1383779327},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2146,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2146,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.719094"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20929,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.112415+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:37:15.112415+0000","last_peered":"2024-09-17T15:37:15.112415+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:37:15.112415+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21066,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352281+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:37:15.352281+0000","last_peered":"2024-09-17T15:37:15.352281+0000","last_clean":"2024-09-17T15:37:15.352281+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:37:15.352281+0000","last_undegraded":"2024-09-17T15:37:15.352281+0000","last_fullsized":"2024-09-17T15:37:15.352281+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21174,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112900+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:37:15.112900+0000","last_peered":"2024-09-17T15:37:15.112900+0000","last_clean":"2024-09-17T15:37:15.112900+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:37:15.112900+0000","last_undegraded":"2024-09-17T15:37:15.112900+0000","last_fullsized":"2024-09-17T15:37:15.112900+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21071,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352626+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:37:15.352626+0000","last_peered":"2024-09-17T15:37:15.352626+0000","last_clean":"2024-09-17T15:37:15.352626+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:37:15.352626+0000","last_undegraded":"2024-09-17T15:37:15.352626+0000","last_fullsized":"2024-09-17T15:37:15.352626+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20814,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112605+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:37:15.112605+0000","last_peered":"2024-09-17T15:37:15.112605+0000","last_clean":"2024-09-17T15:37:15.112605+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:37:15.112605+0000","last_undegraded":"2024-09-17T15:37:15.112605+0000","last_fullsized":"2024-09-17T15:37:15.112605+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21565,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.188303+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:37:15.188303+0000","last_peered":"2024-09-17T15:37:15.188303+0000","last_clean":"2024-09-17T15:37:15.188303+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:37:15.188303+0000","last_undegraded":"2024-09-17T15:37:15.188303+0000","last_fullsized":"2024-09-17T15:37:15.188303+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21108,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.111962+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:37:15.111962+0000","last_peered":"2024-09-17T15:37:15.111962+0000","last_clean":"2024-09-17T15:37:15.111962+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:37:15.111962+0000","last_undegraded":"2024-09-17T15:37:15.111962+0000","last_fullsized":"2024-09-17T15:37:15.111962+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21075,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112248+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:37:15.112248+0000","last_peered":"2024-09-17T15:37:15.112248+0000","last_clean":"2024-09-17T15:37:15.112248+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:37:15.112248+0000","last_undegraded":"2024-09-17T15:37:15.112248+0000","last_fullsized":"2024-09-17T15:37:15.112248+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.111832+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:37:15.111832+0000","last_peered":"2024-09-17T15:37:15.111832+0000","last_clean":"2024-09-17T15:37:15.111832+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:37:15.111832+0000","last_undegraded":"2024-09-17T15:37:15.111832+0000","last_fullsized":"2024-09-17T15:37:15.111832+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29131,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112153+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:37:15.112153+0000","last_peered":"2024-09-17T15:37:15.112153+0000","last_clean":"2024-09-17T15:37:15.112153+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:37:15.112153+0000","last_undegraded":"2024-09-17T15:37:15.112153+0000","last_fullsized":"2024-09-17T15:37:15.112153+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352003+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:37:15.352003+0000","last_peered":"2024-09-17T15:37:15.352003+0000","last_clean":"2024-09-17T15:37:15.352003+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:37:15.352003+0000","last_undegraded":"2024-09-17T15:37:15.352003+0000","last_fullsized":"2024-09-17T15:37:15.352003+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20838,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.112848+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:37:15.112848+0000","last_peered":"2024-09-17T15:37:15.112848+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:37:15.112848+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112615+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:37:15.112615+0000","last_peered":"2024-09-17T15:37:15.112615+0000","last_clean":"2024-09-17T15:37:15.112615+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:37:15.112615+0000","last_undegraded":"2024-09-17T15:37:15.112615+0000","last_fullsized":"2024-09-17T15:37:15.112615+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28835,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352081+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:37:15.352081+0000","last_peered":"2024-09-17T15:37:15.352081+0000","last_clean":"2024-09-17T15:37:15.352081+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:37:15.352081+0000","last_undegraded":"2024-09-17T15:37:15.352081+0000","last_fullsized":"2024-09-17T15:37:15.352081+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":109,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112066+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:37:15.112066+0000","last_peered":"2024-09-17T15:37:15.112066+0000","last_clean":"2024-09-17T15:37:15.112066+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:37:15.112066+0000","last_undegraded":"2024-09-17T15:37:15.112066+0000","last_fullsized":"2024-09-17T15:37:15.112066+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29173,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352021+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:37:15.352021+0000","last_peered":"2024-09-17T15:37:15.352021+0000","last_clean":"2024-09-17T15:37:15.352021+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:37:15.352021+0000","last_undegraded":"2024-09-17T15:37:15.352021+0000","last_fullsized":"2024-09-17T15:37:15.352021+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":23668,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:24.412629+0000","last_change":"2024-09-17T15:37:13.729028+0000","last_active":"2024-09-17T15:37:24.412629+0000","last_peered":"2024-09-17T15:37:24.412629+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:37:24.412629+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2872,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1267,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2872}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112352+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:37:15.112352+0000","last_peered":"2024-09-17T15:37:15.112352+0000","last_clean":"2024-09-17T15:37:15.112352+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:37:15.112352+0000","last_undegraded":"2024-09-17T15:37:15.112352+0000","last_fullsized":"2024-09-17T15:37:15.112352+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28086,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112447+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:37:15.112447+0000","last_peered":"2024-09-17T15:37:15.112447+0000","last_clean":"2024-09-17T15:37:15.112447+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:37:15.112447+0000","last_undegraded":"2024-09-17T15:37:15.112447+0000","last_fullsized":"2024-09-17T15:37:15.112447+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352224+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:37:15.352224+0000","last_peered":"2024-09-17T15:37:15.352224+0000","last_clean":"2024-09-17T15:37:15.352224+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:37:15.352224+0000","last_undegraded":"2024-09-17T15:37:15.352224+0000","last_fullsized":"2024-09-17T15:37:15.352224+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21215,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.113213+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:37:15.113213+0000","last_peered":"2024-09-17T15:37:15.113213+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:37:15.113213+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112357+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:37:15.112357+0000","last_peered":"2024-09-17T15:37:15.112357+0000","last_clean":"2024-09-17T15:37:15.112357+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:37:15.112357+0000","last_undegraded":"2024-09-17T15:37:15.112357+0000","last_fullsized":"2024-09-17T15:37:15.112357+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":104,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112592+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:37:15.112592+0000","last_peered":"2024-09-17T15:37:15.112592+0000","last_clean":"2024-09-17T15:37:15.112592+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:37:15.112592+0000","last_undegraded":"2024-09-17T15:37:15.112592+0000","last_fullsized":"2024-09-17T15:37:15.112592+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29038,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112255+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:37:15.112255+0000","last_peered":"2024-09-17T15:37:15.112255+0000","last_clean":"2024-09-17T15:37:15.112255+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:37:15.112255+0000","last_undegraded":"2024-09-17T15:37:15.112255+0000","last_fullsized":"2024-09-17T15:37:15.112255+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14783,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34677,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":187887616,"data_stored":187782839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051732,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487580,"kb_used_data":118540,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93884260,"statfs":{"total":96636764160,"available":96137482240,"internally_reserved":0,"allocated":121384960,"data_stored":121041405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":377876140},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724677,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509776,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862064,"statfs":{"total":96636764160,"available":96114753536,"internally_reserved":0,"allocated":524288,"data_stored":165817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":521465515},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247960,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539424,"kb_used_data":66312,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832416,"statfs":{"total":96636764160,"available":96084393984,"internally_reserved":0,"allocated":67903488,"data_stored":67548630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":120418304,"data_stored":120389400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:37:29.340 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:37:29.652 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:29.653 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:37:29.666 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":442,"stamp":"2024-09-17T15:37:27.830502+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14783,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34678,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1536780,"kb_used_data":185364,"kb_used_omap":13,"kb_used_meta":1351346,"kb_avail":281578740,"statfs":{"total":289910292480,"available":288336629760,"internally_reserved":0,"allocated":189812736,"data_stored":188755852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1383779327},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2146,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2146,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001978"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20929,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.112415+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:37:15.112415+0000","last_peered":"2024-09-17T15:37:15.112415+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:37:15.112415+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21066,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352281+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:37:15.352281+0000","last_peered":"2024-09-17T15:37:15.352281+0000","last_clean":"2024-09-17T15:37:15.352281+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:37:15.352281+0000","last_undegraded":"2024-09-17T15:37:15.352281+0000","last_fullsized":"2024-09-17T15:37:15.352281+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21174,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112900+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:37:15.112900+0000","last_peered":"2024-09-17T15:37:15.112900+0000","last_clean":"2024-09-17T15:37:15.112900+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:37:15.112900+0000","last_undegraded":"2024-09-17T15:37:15.112900+0000","last_fullsized":"2024-09-17T15:37:15.112900+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21071,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352626+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:37:15.352626+0000","last_peered":"2024-09-17T15:37:15.352626+0000","last_clean":"2024-09-17T15:37:15.352626+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:37:15.352626+0000","last_undegraded":"2024-09-17T15:37:15.352626+0000","last_fullsized":"2024-09-17T15:37:15.352626+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20814,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112605+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:37:15.112605+0000","last_peered":"2024-09-17T15:37:15.112605+0000","last_clean":"2024-09-17T15:37:15.112605+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:37:15.112605+0000","last_undegraded":"2024-09-17T15:37:15.112605+0000","last_fullsized":"2024-09-17T15:37:15.112605+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21565,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.188303+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:37:15.188303+0000","last_peered":"2024-09-17T15:37:15.188303+0000","last_clean":"2024-09-17T15:37:15.188303+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:37:15.188303+0000","last_undegraded":"2024-09-17T15:37:15.188303+0000","last_fullsized":"2024-09-17T15:37:15.188303+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21108,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.111962+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:37:15.111962+0000","last_peered":"2024-09-17T15:37:15.111962+0000","last_clean":"2024-09-17T15:37:15.111962+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:37:15.111962+0000","last_undegraded":"2024-09-17T15:37:15.111962+0000","last_fullsized":"2024-09-17T15:37:15.111962+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21075,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112248+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:37:15.112248+0000","last_peered":"2024-09-17T15:37:15.112248+0000","last_clean":"2024-09-17T15:37:15.112248+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:37:15.112248+0000","last_undegraded":"2024-09-17T15:37:15.112248+0000","last_fullsized":"2024-09-17T15:37:15.112248+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.111832+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:37:15.111832+0000","last_peered":"2024-09-17T15:37:15.111832+0000","last_clean":"2024-09-17T15:37:15.111832+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:37:15.111832+0000","last_undegraded":"2024-09-17T15:37:15.111832+0000","last_fullsized":"2024-09-17T15:37:15.111832+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29131,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112153+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:37:15.112153+0000","last_peered":"2024-09-17T15:37:15.112153+0000","last_clean":"2024-09-17T15:37:15.112153+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:37:15.112153+0000","last_undegraded":"2024-09-17T15:37:15.112153+0000","last_fullsized":"2024-09-17T15:37:15.112153+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352003+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:37:15.352003+0000","last_peered":"2024-09-17T15:37:15.352003+0000","last_clean":"2024-09-17T15:37:15.352003+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:37:15.352003+0000","last_undegraded":"2024-09-17T15:37:15.352003+0000","last_fullsized":"2024-09-17T15:37:15.352003+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20838,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.112848+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:37:15.112848+0000","last_peered":"2024-09-17T15:37:15.112848+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:37:15.112848+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112615+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:37:15.112615+0000","last_peered":"2024-09-17T15:37:15.112615+0000","last_clean":"2024-09-17T15:37:15.112615+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:37:15.112615+0000","last_undegraded":"2024-09-17T15:37:15.112615+0000","last_fullsized":"2024-09-17T15:37:15.112615+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28835,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352081+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:37:15.352081+0000","last_peered":"2024-09-17T15:37:15.352081+0000","last_clean":"2024-09-17T15:37:15.352081+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:37:15.352081+0000","last_undegraded":"2024-09-17T15:37:15.352081+0000","last_fullsized":"2024-09-17T15:37:15.352081+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":109,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112066+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:37:15.112066+0000","last_peered":"2024-09-17T15:37:15.112066+0000","last_clean":"2024-09-17T15:37:15.112066+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:37:15.112066+0000","last_undegraded":"2024-09-17T15:37:15.112066+0000","last_fullsized":"2024-09-17T15:37:15.112066+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29173,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352021+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:37:15.352021+0000","last_peered":"2024-09-17T15:37:15.352021+0000","last_clean":"2024-09-17T15:37:15.352021+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:37:15.352021+0000","last_undegraded":"2024-09-17T15:37:15.352021+0000","last_fullsized":"2024-09-17T15:37:15.352021+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":23668,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:24.412629+0000","last_change":"2024-09-17T15:37:13.729028+0000","last_active":"2024-09-17T15:37:24.412629+0000","last_peered":"2024-09-17T15:37:24.412629+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:37:24.412629+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2872,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1267,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2872}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112352+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:37:15.112352+0000","last_peered":"2024-09-17T15:37:15.112352+0000","last_clean":"2024-09-17T15:37:15.112352+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:37:15.112352+0000","last_undegraded":"2024-09-17T15:37:15.112352+0000","last_fullsized":"2024-09-17T15:37:15.112352+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28086,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112447+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:37:15.112447+0000","last_peered":"2024-09-17T15:37:15.112447+0000","last_clean":"2024-09-17T15:37:15.112447+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:37:15.112447+0000","last_undegraded":"2024-09-17T15:37:15.112447+0000","last_fullsized":"2024-09-17T15:37:15.112447+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352224+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:37:15.352224+0000","last_peered":"2024-09-17T15:37:15.352224+0000","last_clean":"2024-09-17T15:37:15.352224+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:37:15.352224+0000","last_undegraded":"2024-09-17T15:37:15.352224+0000","last_fullsized":"2024-09-17T15:37:15.352224+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21215,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.113213+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:37:15.113213+0000","last_peered":"2024-09-17T15:37:15.113213+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:37:15.113213+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112357+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:37:15.112357+0000","last_peered":"2024-09-17T15:37:15.112357+0000","last_clean":"2024-09-17T15:37:15.112357+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:37:15.112357+0000","last_undegraded":"2024-09-17T15:37:15.112357+0000","last_fullsized":"2024-09-17T15:37:15.112357+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":104,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112592+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:37:15.112592+0000","last_peered":"2024-09-17T15:37:15.112592+0000","last_clean":"2024-09-17T15:37:15.112592+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:37:15.112592+0000","last_undegraded":"2024-09-17T15:37:15.112592+0000","last_fullsized":"2024-09-17T15:37:15.112592+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29038,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112255+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:37:15.112255+0000","last_peered":"2024-09-17T15:37:15.112255+0000","last_clean":"2024-09-17T15:37:15.112255+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:37:15.112255+0000","last_undegraded":"2024-09-17T15:37:15.112255+0000","last_fullsized":"2024-09-17T15:37:15.112255+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14783,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34677,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":187887616,"data_stored":187782839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051732,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487580,"kb_used_data":118540,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93884260,"statfs":{"total":96636764160,"available":96137482240,"internally_reserved":0,"allocated":121384960,"data_stored":121041405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":377876140},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724677,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509776,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862064,"statfs":{"total":96636764160,"available":96114753536,"internally_reserved":0,"allocated":524288,"data_stored":165817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":521465515},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247960,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539424,"kb_used_data":66312,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832416,"statfs":{"total":96636764160,"available":96084393984,"internally_reserved":0,"allocated":67903488,"data_stored":67548630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":120418304,"data_stored":120389400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:37:29.667 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:37:29.968 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:29.969 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:37:29.980 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":443,"stamp":"2024-09-17T15:37:29.830869+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35353,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1521900,"kb_used_data":182964,"kb_used_omap":13,"kb_used_meta":1338866,"kb_avail":281593620,"statfs":{"total":289910292480,"available":288351866880,"internally_reserved":0,"allocated":187355136,"data_stored":186298252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1370999807},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2821,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2821,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002057"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20929,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.112415+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:37:15.112415+0000","last_peered":"2024-09-17T15:37:15.112415+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:37:15.112415+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21066,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352281+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:37:15.352281+0000","last_peered":"2024-09-17T15:37:15.352281+0000","last_clean":"2024-09-17T15:37:15.352281+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:37:15.352281+0000","last_undegraded":"2024-09-17T15:37:15.352281+0000","last_fullsized":"2024-09-17T15:37:15.352281+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21174,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112900+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:37:15.112900+0000","last_peered":"2024-09-17T15:37:15.112900+0000","last_clean":"2024-09-17T15:37:15.112900+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:37:15.112900+0000","last_undegraded":"2024-09-17T15:37:15.112900+0000","last_fullsized":"2024-09-17T15:37:15.112900+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21071,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352626+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:37:15.352626+0000","last_peered":"2024-09-17T15:37:15.352626+0000","last_clean":"2024-09-17T15:37:15.352626+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:37:15.352626+0000","last_undegraded":"2024-09-17T15:37:15.352626+0000","last_fullsized":"2024-09-17T15:37:15.352626+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20814,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112605+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:37:15.112605+0000","last_peered":"2024-09-17T15:37:15.112605+0000","last_clean":"2024-09-17T15:37:15.112605+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:37:15.112605+0000","last_undegraded":"2024-09-17T15:37:15.112605+0000","last_fullsized":"2024-09-17T15:37:15.112605+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21565,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.188303+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:37:15.188303+0000","last_peered":"2024-09-17T15:37:15.188303+0000","last_clean":"2024-09-17T15:37:15.188303+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:37:15.188303+0000","last_undegraded":"2024-09-17T15:37:15.188303+0000","last_fullsized":"2024-09-17T15:37:15.188303+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21108,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.111962+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:37:15.111962+0000","last_peered":"2024-09-17T15:37:15.111962+0000","last_clean":"2024-09-17T15:37:15.111962+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:37:15.111962+0000","last_undegraded":"2024-09-17T15:37:15.111962+0000","last_fullsized":"2024-09-17T15:37:15.111962+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21075,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112248+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:37:15.112248+0000","last_peered":"2024-09-17T15:37:15.112248+0000","last_clean":"2024-09-17T15:37:15.112248+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:37:15.112248+0000","last_undegraded":"2024-09-17T15:37:15.112248+0000","last_fullsized":"2024-09-17T15:37:15.112248+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.111832+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:37:15.111832+0000","last_peered":"2024-09-17T15:37:15.111832+0000","last_clean":"2024-09-17T15:37:15.111832+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:37:15.111832+0000","last_undegraded":"2024-09-17T15:37:15.111832+0000","last_fullsized":"2024-09-17T15:37:15.111832+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29131,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112153+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:37:15.112153+0000","last_peered":"2024-09-17T15:37:15.112153+0000","last_clean":"2024-09-17T15:37:15.112153+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:37:15.112153+0000","last_undegraded":"2024-09-17T15:37:15.112153+0000","last_fullsized":"2024-09-17T15:37:15.112153+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352003+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:37:15.352003+0000","last_peered":"2024-09-17T15:37:15.352003+0000","last_clean":"2024-09-17T15:37:15.352003+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:37:15.352003+0000","last_undegraded":"2024-09-17T15:37:15.352003+0000","last_fullsized":"2024-09-17T15:37:15.352003+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20838,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.112848+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:37:15.112848+0000","last_peered":"2024-09-17T15:37:15.112848+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:37:15.112848+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112615+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:37:15.112615+0000","last_peered":"2024-09-17T15:37:15.112615+0000","last_clean":"2024-09-17T15:37:15.112615+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:37:15.112615+0000","last_undegraded":"2024-09-17T15:37:15.112615+0000","last_fullsized":"2024-09-17T15:37:15.112615+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28835,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352081+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:37:15.352081+0000","last_peered":"2024-09-17T15:37:15.352081+0000","last_clean":"2024-09-17T15:37:15.352081+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:37:15.352081+0000","last_undegraded":"2024-09-17T15:37:15.352081+0000","last_fullsized":"2024-09-17T15:37:15.352081+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":109,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112066+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:37:15.112066+0000","last_peered":"2024-09-17T15:37:15.112066+0000","last_clean":"2024-09-17T15:37:15.112066+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:37:15.112066+0000","last_undegraded":"2024-09-17T15:37:15.112066+0000","last_fullsized":"2024-09-17T15:37:15.112066+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29173,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352021+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:37:15.352021+0000","last_peered":"2024-09-17T15:37:15.352021+0000","last_clean":"2024-09-17T15:37:15.352021+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:37:15.352021+0000","last_undegraded":"2024-09-17T15:37:15.352021+0000","last_fullsized":"2024-09-17T15:37:15.352021+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":25018,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:29.414955+0000","last_change":"2024-09-17T15:37:13.729028+0000","last_active":"2024-09-17T15:37:29.414955+0000","last_peered":"2024-09-17T15:37:29.414955+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:37:29.414955+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2197,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1942,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2197}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112352+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:37:15.112352+0000","last_peered":"2024-09-17T15:37:15.112352+0000","last_clean":"2024-09-17T15:37:15.112352+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:37:15.112352+0000","last_undegraded":"2024-09-17T15:37:15.112352+0000","last_fullsized":"2024-09-17T15:37:15.112352+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28086,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112447+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:37:15.112447+0000","last_peered":"2024-09-17T15:37:15.112447+0000","last_clean":"2024-09-17T15:37:15.112447+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:37:15.112447+0000","last_undegraded":"2024-09-17T15:37:15.112447+0000","last_fullsized":"2024-09-17T15:37:15.112447+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352224+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:37:15.352224+0000","last_peered":"2024-09-17T15:37:15.352224+0000","last_clean":"2024-09-17T15:37:15.352224+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:37:15.352224+0000","last_undegraded":"2024-09-17T15:37:15.352224+0000","last_fullsized":"2024-09-17T15:37:15.352224+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21215,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.113213+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:37:15.113213+0000","last_peered":"2024-09-17T15:37:15.113213+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:37:15.113213+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112357+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:37:15.112357+0000","last_peered":"2024-09-17T15:37:15.112357+0000","last_clean":"2024-09-17T15:37:15.112357+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:37:15.112357+0000","last_undegraded":"2024-09-17T15:37:15.112357+0000","last_fullsized":"2024-09-17T15:37:15.112357+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":104,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112592+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:37:15.112592+0000","last_peered":"2024-09-17T15:37:15.112592+0000","last_clean":"2024-09-17T15:37:15.112592+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:37:15.112592+0000","last_undegraded":"2024-09-17T15:37:15.112592+0000","last_fullsized":"2024-09-17T15:37:15.112592+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29038,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112255+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:37:15.112255+0000","last_peered":"2024-09-17T15:37:15.112255+0000","last_clean":"2024-09-17T15:37:15.112255+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:37:15.112255+0000","last_undegraded":"2024-09-17T15:37:15.112255+0000","last_fullsized":"2024-09-17T15:37:15.112255+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35352,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":185430016,"data_stored":185325239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051733,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485180,"kb_used_data":116140,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93886660,"statfs":{"total":96636764160,"available":96139939840,"internally_reserved":0,"allocated":118927360,"data_stored":118583805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":377876140},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724678,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497296,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874544,"statfs":{"total":96636764160,"available":96127533056,"internally_reserved":0,"allocated":524288,"data_stored":165817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":508685995},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247961,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539424,"kb_used_data":66312,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832416,"statfs":{"total":96636764160,"available":96084393984,"internally_reserved":0,"allocated":67903488,"data_stored":67548630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117960704,"data_stored":117931800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:37:29.982 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:37:29.982 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'20648", 'reported_seq': 20929, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:37:15.112415+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:37:15.112415+0000', 'last_peered': '2024-09-17T15:37:15.112415+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:37:15.112415+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'10600", 'ondisk_log_start': "43'10600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20648, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:29.982 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:37:29.982 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20542", 'reported_seq': 20838, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:37:15.112848+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:37:15.112848+0000', 'last_peered': '2024-09-17T15:37:15.112848+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:37:15.112848+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17138, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3943}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:29.982 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:37:29.982 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'20966", 'reported_seq': 25018, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:37:29.414955+0000', 'last_change': '2024-09-17T15:37:13.729028+0000', 'last_active': '2024-09-17T15:37:29.414955+0000', 'last_peered': '2024-09-17T15:37:29.414955+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:37:29.414955+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "43'10900", 'ondisk_log_start': "43'10900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2197, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17514, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1942, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2197}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:29.982 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:37:29.982 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'20886", 'reported_seq': 21215, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:37:15.113213+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:37:15.113213+0000', 'last_peered': '2024-09-17T15:37:15.113213+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:37:15.113213+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'10800", 'ondisk_log_start': "43'10800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17673, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3980}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:29.983 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:37:30.447 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:30.447 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 14108/24 objects degraded (58783.333%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":787,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1558425600,"bytes_avail":288351866880,"bytes_total":289910292480,"degraded_objects":14108,"degraded_total":24,"degraded_ratio":587.83333333333337,"recovering_objects_per_sec":235,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2821,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:37:30.461 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:37:30.461 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:37:30.758 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:30.759 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:37:30.771 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":443,"stamp":"2024-09-17T15:37:29.830869+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35353,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1521900,"kb_used_data":182964,"kb_used_omap":13,"kb_used_meta":1338866,"kb_avail":281593620,"statfs":{"total":289910292480,"available":288351866880,"internally_reserved":0,"allocated":187355136,"data_stored":186298252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1370999807},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2821,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2821,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002057"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20929,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.112415+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:37:15.112415+0000","last_peered":"2024-09-17T15:37:15.112415+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:37:15.112415+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21066,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352281+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:37:15.352281+0000","last_peered":"2024-09-17T15:37:15.352281+0000","last_clean":"2024-09-17T15:37:15.352281+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:37:15.352281+0000","last_undegraded":"2024-09-17T15:37:15.352281+0000","last_fullsized":"2024-09-17T15:37:15.352281+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21174,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112900+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:37:15.112900+0000","last_peered":"2024-09-17T15:37:15.112900+0000","last_clean":"2024-09-17T15:37:15.112900+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:37:15.112900+0000","last_undegraded":"2024-09-17T15:37:15.112900+0000","last_fullsized":"2024-09-17T15:37:15.112900+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21071,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352626+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:37:15.352626+0000","last_peered":"2024-09-17T15:37:15.352626+0000","last_clean":"2024-09-17T15:37:15.352626+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:37:15.352626+0000","last_undegraded":"2024-09-17T15:37:15.352626+0000","last_fullsized":"2024-09-17T15:37:15.352626+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20814,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112605+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:37:15.112605+0000","last_peered":"2024-09-17T15:37:15.112605+0000","last_clean":"2024-09-17T15:37:15.112605+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:37:15.112605+0000","last_undegraded":"2024-09-17T15:37:15.112605+0000","last_fullsized":"2024-09-17T15:37:15.112605+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21565,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.188303+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:37:15.188303+0000","last_peered":"2024-09-17T15:37:15.188303+0000","last_clean":"2024-09-17T15:37:15.188303+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:37:15.188303+0000","last_undegraded":"2024-09-17T15:37:15.188303+0000","last_fullsized":"2024-09-17T15:37:15.188303+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21108,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.111962+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:37:15.111962+0000","last_peered":"2024-09-17T15:37:15.111962+0000","last_clean":"2024-09-17T15:37:15.111962+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:37:15.111962+0000","last_undegraded":"2024-09-17T15:37:15.111962+0000","last_fullsized":"2024-09-17T15:37:15.111962+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21075,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112248+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:37:15.112248+0000","last_peered":"2024-09-17T15:37:15.112248+0000","last_clean":"2024-09-17T15:37:15.112248+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:37:15.112248+0000","last_undegraded":"2024-09-17T15:37:15.112248+0000","last_fullsized":"2024-09-17T15:37:15.112248+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.111832+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:37:15.111832+0000","last_peered":"2024-09-17T15:37:15.111832+0000","last_clean":"2024-09-17T15:37:15.111832+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:37:15.111832+0000","last_undegraded":"2024-09-17T15:37:15.111832+0000","last_fullsized":"2024-09-17T15:37:15.111832+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29131,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112153+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:37:15.112153+0000","last_peered":"2024-09-17T15:37:15.112153+0000","last_clean":"2024-09-17T15:37:15.112153+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:37:15.112153+0000","last_undegraded":"2024-09-17T15:37:15.112153+0000","last_fullsized":"2024-09-17T15:37:15.112153+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352003+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:37:15.352003+0000","last_peered":"2024-09-17T15:37:15.352003+0000","last_clean":"2024-09-17T15:37:15.352003+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:37:15.352003+0000","last_undegraded":"2024-09-17T15:37:15.352003+0000","last_fullsized":"2024-09-17T15:37:15.352003+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20838,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.112848+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:37:15.112848+0000","last_peered":"2024-09-17T15:37:15.112848+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:37:15.112848+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112615+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:37:15.112615+0000","last_peered":"2024-09-17T15:37:15.112615+0000","last_clean":"2024-09-17T15:37:15.112615+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:37:15.112615+0000","last_undegraded":"2024-09-17T15:37:15.112615+0000","last_fullsized":"2024-09-17T15:37:15.112615+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28835,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352081+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:37:15.352081+0000","last_peered":"2024-09-17T15:37:15.352081+0000","last_clean":"2024-09-17T15:37:15.352081+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:37:15.352081+0000","last_undegraded":"2024-09-17T15:37:15.352081+0000","last_fullsized":"2024-09-17T15:37:15.352081+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":109,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112066+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:37:15.112066+0000","last_peered":"2024-09-17T15:37:15.112066+0000","last_clean":"2024-09-17T15:37:15.112066+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:37:15.112066+0000","last_undegraded":"2024-09-17T15:37:15.112066+0000","last_fullsized":"2024-09-17T15:37:15.112066+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29173,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352021+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:37:15.352021+0000","last_peered":"2024-09-17T15:37:15.352021+0000","last_clean":"2024-09-17T15:37:15.352021+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:37:15.352021+0000","last_undegraded":"2024-09-17T15:37:15.352021+0000","last_fullsized":"2024-09-17T15:37:15.352021+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":25018,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:29.414955+0000","last_change":"2024-09-17T15:37:13.729028+0000","last_active":"2024-09-17T15:37:29.414955+0000","last_peered":"2024-09-17T15:37:29.414955+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:37:29.414955+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2197,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1942,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2197}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112352+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:37:15.112352+0000","last_peered":"2024-09-17T15:37:15.112352+0000","last_clean":"2024-09-17T15:37:15.112352+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:37:15.112352+0000","last_undegraded":"2024-09-17T15:37:15.112352+0000","last_fullsized":"2024-09-17T15:37:15.112352+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28086,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112447+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:37:15.112447+0000","last_peered":"2024-09-17T15:37:15.112447+0000","last_clean":"2024-09-17T15:37:15.112447+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:37:15.112447+0000","last_undegraded":"2024-09-17T15:37:15.112447+0000","last_fullsized":"2024-09-17T15:37:15.112447+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352224+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:37:15.352224+0000","last_peered":"2024-09-17T15:37:15.352224+0000","last_clean":"2024-09-17T15:37:15.352224+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:37:15.352224+0000","last_undegraded":"2024-09-17T15:37:15.352224+0000","last_fullsized":"2024-09-17T15:37:15.352224+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21215,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.113213+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:37:15.113213+0000","last_peered":"2024-09-17T15:37:15.113213+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:37:15.113213+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112357+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:37:15.112357+0000","last_peered":"2024-09-17T15:37:15.112357+0000","last_clean":"2024-09-17T15:37:15.112357+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:37:15.112357+0000","last_undegraded":"2024-09-17T15:37:15.112357+0000","last_fullsized":"2024-09-17T15:37:15.112357+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":104,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112592+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:37:15.112592+0000","last_peered":"2024-09-17T15:37:15.112592+0000","last_clean":"2024-09-17T15:37:15.112592+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:37:15.112592+0000","last_undegraded":"2024-09-17T15:37:15.112592+0000","last_fullsized":"2024-09-17T15:37:15.112592+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29038,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112255+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:37:15.112255+0000","last_peered":"2024-09-17T15:37:15.112255+0000","last_clean":"2024-09-17T15:37:15.112255+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:37:15.112255+0000","last_undegraded":"2024-09-17T15:37:15.112255+0000","last_fullsized":"2024-09-17T15:37:15.112255+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35352,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":185430016,"data_stored":185325239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051733,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485180,"kb_used_data":116140,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93886660,"statfs":{"total":96636764160,"available":96139939840,"internally_reserved":0,"allocated":118927360,"data_stored":118583805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":377876140},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724678,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497296,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874544,"statfs":{"total":96636764160,"available":96127533056,"internally_reserved":0,"allocated":524288,"data_stored":165817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":508685995},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247961,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539424,"kb_used_data":66312,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832416,"statfs":{"total":96636764160,"available":96084393984,"internally_reserved":0,"allocated":67903488,"data_stored":67548630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117960704,"data_stored":117931800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:37:33.773 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:37:34.143 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:34.143 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:37:34.157 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":445,"stamp":"2024-09-17T15:37:33.831610+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35353,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1519140,"kb_used_data":180204,"kb_used_omap":13,"kb_used_meta":1338866,"kb_avail":281596380,"statfs":{"total":289910292480,"available":288354693120,"internally_reserved":0,"allocated":184528896,"data_stored":183472012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1370999807},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1256,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1256,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002136"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20929,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.112415+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:37:15.112415+0000","last_peered":"2024-09-17T15:37:15.112415+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:37:15.112415+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21066,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352281+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:37:15.352281+0000","last_peered":"2024-09-17T15:37:15.352281+0000","last_clean":"2024-09-17T15:37:15.352281+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:37:15.352281+0000","last_undegraded":"2024-09-17T15:37:15.352281+0000","last_fullsized":"2024-09-17T15:37:15.352281+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21174,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112900+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:37:15.112900+0000","last_peered":"2024-09-17T15:37:15.112900+0000","last_clean":"2024-09-17T15:37:15.112900+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:37:15.112900+0000","last_undegraded":"2024-09-17T15:37:15.112900+0000","last_fullsized":"2024-09-17T15:37:15.112900+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21071,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352626+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:37:15.352626+0000","last_peered":"2024-09-17T15:37:15.352626+0000","last_clean":"2024-09-17T15:37:15.352626+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:37:15.352626+0000","last_undegraded":"2024-09-17T15:37:15.352626+0000","last_fullsized":"2024-09-17T15:37:15.352626+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20814,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112605+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:37:15.112605+0000","last_peered":"2024-09-17T15:37:15.112605+0000","last_clean":"2024-09-17T15:37:15.112605+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:37:15.112605+0000","last_undegraded":"2024-09-17T15:37:15.112605+0000","last_fullsized":"2024-09-17T15:37:15.112605+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21565,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.188303+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:37:15.188303+0000","last_peered":"2024-09-17T15:37:15.188303+0000","last_clean":"2024-09-17T15:37:15.188303+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:37:15.188303+0000","last_undegraded":"2024-09-17T15:37:15.188303+0000","last_fullsized":"2024-09-17T15:37:15.188303+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21108,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.111962+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:37:15.111962+0000","last_peered":"2024-09-17T15:37:15.111962+0000","last_clean":"2024-09-17T15:37:15.111962+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:37:15.111962+0000","last_undegraded":"2024-09-17T15:37:15.111962+0000","last_fullsized":"2024-09-17T15:37:15.111962+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21075,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112248+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:37:15.112248+0000","last_peered":"2024-09-17T15:37:15.112248+0000","last_clean":"2024-09-17T15:37:15.112248+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:37:15.112248+0000","last_undegraded":"2024-09-17T15:37:15.112248+0000","last_fullsized":"2024-09-17T15:37:15.112248+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.111832+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:37:15.111832+0000","last_peered":"2024-09-17T15:37:15.111832+0000","last_clean":"2024-09-17T15:37:15.111832+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:37:15.111832+0000","last_undegraded":"2024-09-17T15:37:15.111832+0000","last_fullsized":"2024-09-17T15:37:15.111832+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29131,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112153+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:37:15.112153+0000","last_peered":"2024-09-17T15:37:15.112153+0000","last_clean":"2024-09-17T15:37:15.112153+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:37:15.112153+0000","last_undegraded":"2024-09-17T15:37:15.112153+0000","last_fullsized":"2024-09-17T15:37:15.112153+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352003+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:37:15.352003+0000","last_peered":"2024-09-17T15:37:15.352003+0000","last_clean":"2024-09-17T15:37:15.352003+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:37:15.352003+0000","last_undegraded":"2024-09-17T15:37:15.352003+0000","last_fullsized":"2024-09-17T15:37:15.352003+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20838,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.112848+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:37:15.112848+0000","last_peered":"2024-09-17T15:37:15.112848+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:37:15.112848+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112615+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:37:15.112615+0000","last_peered":"2024-09-17T15:37:15.112615+0000","last_clean":"2024-09-17T15:37:15.112615+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:37:15.112615+0000","last_undegraded":"2024-09-17T15:37:15.112615+0000","last_fullsized":"2024-09-17T15:37:15.112615+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28835,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352081+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:37:15.352081+0000","last_peered":"2024-09-17T15:37:15.352081+0000","last_clean":"2024-09-17T15:37:15.352081+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:37:15.352081+0000","last_undegraded":"2024-09-17T15:37:15.352081+0000","last_fullsized":"2024-09-17T15:37:15.352081+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":109,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112066+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:37:15.112066+0000","last_peered":"2024-09-17T15:37:15.112066+0000","last_clean":"2024-09-17T15:37:15.112066+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:37:15.112066+0000","last_undegraded":"2024-09-17T15:37:15.112066+0000","last_fullsized":"2024-09-17T15:37:15.112066+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29173,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352021+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:37:15.352021+0000","last_peered":"2024-09-17T15:37:15.352021+0000","last_clean":"2024-09-17T15:37:15.352021+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:37:15.352021+0000","last_undegraded":"2024-09-17T15:37:15.352021+0000","last_fullsized":"2024-09-17T15:37:15.352021+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":25018,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:29.414955+0000","last_change":"2024-09-17T15:37:13.729028+0000","last_active":"2024-09-17T15:37:29.414955+0000","last_peered":"2024-09-17T15:37:29.414955+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:37:29.414955+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2197,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1942,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2197}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112352+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:37:15.112352+0000","last_peered":"2024-09-17T15:37:15.112352+0000","last_clean":"2024-09-17T15:37:15.112352+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:37:15.112352+0000","last_undegraded":"2024-09-17T15:37:15.112352+0000","last_fullsized":"2024-09-17T15:37:15.112352+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28086,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112447+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:37:15.112447+0000","last_peered":"2024-09-17T15:37:15.112447+0000","last_clean":"2024-09-17T15:37:15.112447+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:37:15.112447+0000","last_undegraded":"2024-09-17T15:37:15.112447+0000","last_fullsized":"2024-09-17T15:37:15.112447+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352224+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:37:15.352224+0000","last_peered":"2024-09-17T15:37:15.352224+0000","last_clean":"2024-09-17T15:37:15.352224+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:37:15.352224+0000","last_undegraded":"2024-09-17T15:37:15.352224+0000","last_fullsized":"2024-09-17T15:37:15.352224+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21215,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.113213+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:37:15.113213+0000","last_peered":"2024-09-17T15:37:15.113213+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:37:15.113213+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112357+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:37:15.112357+0000","last_peered":"2024-09-17T15:37:15.112357+0000","last_clean":"2024-09-17T15:37:15.112357+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:37:15.112357+0000","last_undegraded":"2024-09-17T15:37:15.112357+0000","last_fullsized":"2024-09-17T15:37:15.112357+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":104,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112592+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:37:15.112592+0000","last_peered":"2024-09-17T15:37:15.112592+0000","last_clean":"2024-09-17T15:37:15.112592+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:37:15.112592+0000","last_undegraded":"2024-09-17T15:37:15.112592+0000","last_fullsized":"2024-09-17T15:37:15.112592+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29038,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112255+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:37:15.112255+0000","last_peered":"2024-09-17T15:37:15.112255+0000","last_clean":"2024-09-17T15:37:15.112255+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:37:15.112255+0000","last_undegraded":"2024-09-17T15:37:15.112255+0000","last_fullsized":"2024-09-17T15:37:15.112255+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35352,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":182972416,"data_stored":182867639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051734,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":482420,"kb_used_data":113380,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93889420,"statfs":{"total":96636764160,"available":96142766080,"internally_reserved":0,"allocated":116101120,"data_stored":115757565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":377876140},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724678,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497296,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874544,"statfs":{"total":96636764160,"available":96127533056,"internally_reserved":0,"allocated":524288,"data_stored":165817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":508685995},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247961,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539424,"kb_used_data":66312,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832416,"statfs":{"total":96636764160,"available":96084393984,"internally_reserved":0,"allocated":67903488,"data_stored":67548630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":115503104,"data_stored":115474200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:37:34.158 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:37:34.497 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:34.497 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:37:34.510 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":445,"stamp":"2024-09-17T15:37:33.831610+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35353,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1519140,"kb_used_data":180204,"kb_used_omap":13,"kb_used_meta":1338866,"kb_avail":281596380,"statfs":{"total":289910292480,"available":288354693120,"internally_reserved":0,"allocated":184528896,"data_stored":183472012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1370999807},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1256,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1256,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002136"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20929,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.112415+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:37:15.112415+0000","last_peered":"2024-09-17T15:37:15.112415+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:37:15.112415+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21066,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352281+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:37:15.352281+0000","last_peered":"2024-09-17T15:37:15.352281+0000","last_clean":"2024-09-17T15:37:15.352281+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:37:15.352281+0000","last_undegraded":"2024-09-17T15:37:15.352281+0000","last_fullsized":"2024-09-17T15:37:15.352281+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21174,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112900+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:37:15.112900+0000","last_peered":"2024-09-17T15:37:15.112900+0000","last_clean":"2024-09-17T15:37:15.112900+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:37:15.112900+0000","last_undegraded":"2024-09-17T15:37:15.112900+0000","last_fullsized":"2024-09-17T15:37:15.112900+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21071,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352626+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:37:15.352626+0000","last_peered":"2024-09-17T15:37:15.352626+0000","last_clean":"2024-09-17T15:37:15.352626+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:37:15.352626+0000","last_undegraded":"2024-09-17T15:37:15.352626+0000","last_fullsized":"2024-09-17T15:37:15.352626+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20814,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112605+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:37:15.112605+0000","last_peered":"2024-09-17T15:37:15.112605+0000","last_clean":"2024-09-17T15:37:15.112605+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:37:15.112605+0000","last_undegraded":"2024-09-17T15:37:15.112605+0000","last_fullsized":"2024-09-17T15:37:15.112605+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21565,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.188303+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:37:15.188303+0000","last_peered":"2024-09-17T15:37:15.188303+0000","last_clean":"2024-09-17T15:37:15.188303+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:37:15.188303+0000","last_undegraded":"2024-09-17T15:37:15.188303+0000","last_fullsized":"2024-09-17T15:37:15.188303+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21108,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.111962+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:37:15.111962+0000","last_peered":"2024-09-17T15:37:15.111962+0000","last_clean":"2024-09-17T15:37:15.111962+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:37:15.111962+0000","last_undegraded":"2024-09-17T15:37:15.111962+0000","last_fullsized":"2024-09-17T15:37:15.111962+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21075,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112248+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:37:15.112248+0000","last_peered":"2024-09-17T15:37:15.112248+0000","last_clean":"2024-09-17T15:37:15.112248+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:37:15.112248+0000","last_undegraded":"2024-09-17T15:37:15.112248+0000","last_fullsized":"2024-09-17T15:37:15.112248+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.111832+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:37:15.111832+0000","last_peered":"2024-09-17T15:37:15.111832+0000","last_clean":"2024-09-17T15:37:15.111832+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:37:15.111832+0000","last_undegraded":"2024-09-17T15:37:15.111832+0000","last_fullsized":"2024-09-17T15:37:15.111832+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29131,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112153+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:37:15.112153+0000","last_peered":"2024-09-17T15:37:15.112153+0000","last_clean":"2024-09-17T15:37:15.112153+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:37:15.112153+0000","last_undegraded":"2024-09-17T15:37:15.112153+0000","last_fullsized":"2024-09-17T15:37:15.112153+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352003+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:37:15.352003+0000","last_peered":"2024-09-17T15:37:15.352003+0000","last_clean":"2024-09-17T15:37:15.352003+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:37:15.352003+0000","last_undegraded":"2024-09-17T15:37:15.352003+0000","last_fullsized":"2024-09-17T15:37:15.352003+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20838,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.112848+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:37:15.112848+0000","last_peered":"2024-09-17T15:37:15.112848+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:37:15.112848+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112615+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:37:15.112615+0000","last_peered":"2024-09-17T15:37:15.112615+0000","last_clean":"2024-09-17T15:37:15.112615+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:37:15.112615+0000","last_undegraded":"2024-09-17T15:37:15.112615+0000","last_fullsized":"2024-09-17T15:37:15.112615+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28835,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352081+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:37:15.352081+0000","last_peered":"2024-09-17T15:37:15.352081+0000","last_clean":"2024-09-17T15:37:15.352081+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:37:15.352081+0000","last_undegraded":"2024-09-17T15:37:15.352081+0000","last_fullsized":"2024-09-17T15:37:15.352081+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":109,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112066+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:37:15.112066+0000","last_peered":"2024-09-17T15:37:15.112066+0000","last_clean":"2024-09-17T15:37:15.112066+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:37:15.112066+0000","last_undegraded":"2024-09-17T15:37:15.112066+0000","last_fullsized":"2024-09-17T15:37:15.112066+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29173,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352021+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:37:15.352021+0000","last_peered":"2024-09-17T15:37:15.352021+0000","last_clean":"2024-09-17T15:37:15.352021+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:37:15.352021+0000","last_undegraded":"2024-09-17T15:37:15.352021+0000","last_fullsized":"2024-09-17T15:37:15.352021+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":25018,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:29.414955+0000","last_change":"2024-09-17T15:37:13.729028+0000","last_active":"2024-09-17T15:37:29.414955+0000","last_peered":"2024-09-17T15:37:29.414955+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:37:29.414955+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2197,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1942,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2197}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112352+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:37:15.112352+0000","last_peered":"2024-09-17T15:37:15.112352+0000","last_clean":"2024-09-17T15:37:15.112352+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:37:15.112352+0000","last_undegraded":"2024-09-17T15:37:15.112352+0000","last_fullsized":"2024-09-17T15:37:15.112352+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28086,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112447+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:37:15.112447+0000","last_peered":"2024-09-17T15:37:15.112447+0000","last_clean":"2024-09-17T15:37:15.112447+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:37:15.112447+0000","last_undegraded":"2024-09-17T15:37:15.112447+0000","last_fullsized":"2024-09-17T15:37:15.112447+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352224+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:37:15.352224+0000","last_peered":"2024-09-17T15:37:15.352224+0000","last_clean":"2024-09-17T15:37:15.352224+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:37:15.352224+0000","last_undegraded":"2024-09-17T15:37:15.352224+0000","last_fullsized":"2024-09-17T15:37:15.352224+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21215,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.113213+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:37:15.113213+0000","last_peered":"2024-09-17T15:37:15.113213+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:37:15.113213+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112357+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:37:15.112357+0000","last_peered":"2024-09-17T15:37:15.112357+0000","last_clean":"2024-09-17T15:37:15.112357+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:37:15.112357+0000","last_undegraded":"2024-09-17T15:37:15.112357+0000","last_fullsized":"2024-09-17T15:37:15.112357+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":104,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112592+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:37:15.112592+0000","last_peered":"2024-09-17T15:37:15.112592+0000","last_clean":"2024-09-17T15:37:15.112592+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:37:15.112592+0000","last_undegraded":"2024-09-17T15:37:15.112592+0000","last_fullsized":"2024-09-17T15:37:15.112592+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29038,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112255+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:37:15.112255+0000","last_peered":"2024-09-17T15:37:15.112255+0000","last_clean":"2024-09-17T15:37:15.112255+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:37:15.112255+0000","last_undegraded":"2024-09-17T15:37:15.112255+0000","last_fullsized":"2024-09-17T15:37:15.112255+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35352,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":182972416,"data_stored":182867639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051734,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":482420,"kb_used_data":113380,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93889420,"statfs":{"total":96636764160,"available":96142766080,"internally_reserved":0,"allocated":116101120,"data_stored":115757565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":377876140},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724678,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497296,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874544,"statfs":{"total":96636764160,"available":96127533056,"internally_reserved":0,"allocated":524288,"data_stored":165817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":508685995},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247961,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539424,"kb_used_data":66312,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832416,"statfs":{"total":96636764160,"available":96084393984,"internally_reserved":0,"allocated":67903488,"data_stored":67548630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":115503104,"data_stored":115474200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:37:34.511 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:37:34.511 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'20648", 'reported_seq': 20929, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:37:15.112415+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:37:15.112415+0000', 'last_peered': '2024-09-17T15:37:15.112415+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:37:15.112415+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'10600", 'ondisk_log_start': "43'10600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20648, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:34.512 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:37:34.512 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20542", 'reported_seq': 20838, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:37:15.112848+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:37:15.112848+0000', 'last_peered': '2024-09-17T15:37:15.112848+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:37:15.112848+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17138, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3943}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:34.512 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:37:34.512 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'20966", 'reported_seq': 25018, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:37:29.414955+0000', 'last_change': '2024-09-17T15:37:13.729028+0000', 'last_active': '2024-09-17T15:37:29.414955+0000', 'last_peered': '2024-09-17T15:37:29.414955+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:37:29.414955+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "43'10900", 'ondisk_log_start': "43'10900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2197, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17514, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1942, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2197}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:34.512 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:37:34.512 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'20886", 'reported_seq': 21215, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:37:15.113213+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:37:15.113213+0000', 'last_peered': '2024-09-17T15:37:15.113213+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:37:15.113213+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'10800", 'ondisk_log_start': "43'10800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17673, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3980}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:34.512 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:37:34.914 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:34.914 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 14108/24 objects degraded (58783.333%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":791,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1555599360,"bytes_avail":288354693120,"bytes_total":289910292480,"degraded_objects":14108,"degraded_total":24,"degraded_ratio":587.83333333333337,"recovering_objects_per_sec":104,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1256,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:37:34.928 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:37:34.928 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:37:35.250 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:35.250 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:37:35.263 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":445,"stamp":"2024-09-17T15:37:33.831610+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35353,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1519140,"kb_used_data":180204,"kb_used_omap":13,"kb_used_meta":1338866,"kb_avail":281596380,"statfs":{"total":289910292480,"available":288354693120,"internally_reserved":0,"allocated":184528896,"data_stored":183472012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1370999807},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1256,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1256,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002136"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20929,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.112415+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:37:15.112415+0000","last_peered":"2024-09-17T15:37:15.112415+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:37:15.112415+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21066,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352281+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:37:15.352281+0000","last_peered":"2024-09-17T15:37:15.352281+0000","last_clean":"2024-09-17T15:37:15.352281+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:37:15.352281+0000","last_undegraded":"2024-09-17T15:37:15.352281+0000","last_fullsized":"2024-09-17T15:37:15.352281+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21174,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112900+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:37:15.112900+0000","last_peered":"2024-09-17T15:37:15.112900+0000","last_clean":"2024-09-17T15:37:15.112900+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:37:15.112900+0000","last_undegraded":"2024-09-17T15:37:15.112900+0000","last_fullsized":"2024-09-17T15:37:15.112900+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21071,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352626+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:37:15.352626+0000","last_peered":"2024-09-17T15:37:15.352626+0000","last_clean":"2024-09-17T15:37:15.352626+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:37:15.352626+0000","last_undegraded":"2024-09-17T15:37:15.352626+0000","last_fullsized":"2024-09-17T15:37:15.352626+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20814,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112605+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:37:15.112605+0000","last_peered":"2024-09-17T15:37:15.112605+0000","last_clean":"2024-09-17T15:37:15.112605+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:37:15.112605+0000","last_undegraded":"2024-09-17T15:37:15.112605+0000","last_fullsized":"2024-09-17T15:37:15.112605+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21565,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.188303+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:37:15.188303+0000","last_peered":"2024-09-17T15:37:15.188303+0000","last_clean":"2024-09-17T15:37:15.188303+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:37:15.188303+0000","last_undegraded":"2024-09-17T15:37:15.188303+0000","last_fullsized":"2024-09-17T15:37:15.188303+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21108,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.111962+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:37:15.111962+0000","last_peered":"2024-09-17T15:37:15.111962+0000","last_clean":"2024-09-17T15:37:15.111962+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:37:15.111962+0000","last_undegraded":"2024-09-17T15:37:15.111962+0000","last_fullsized":"2024-09-17T15:37:15.111962+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21075,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112248+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:37:15.112248+0000","last_peered":"2024-09-17T15:37:15.112248+0000","last_clean":"2024-09-17T15:37:15.112248+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:37:15.112248+0000","last_undegraded":"2024-09-17T15:37:15.112248+0000","last_fullsized":"2024-09-17T15:37:15.112248+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.111832+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:37:15.111832+0000","last_peered":"2024-09-17T15:37:15.111832+0000","last_clean":"2024-09-17T15:37:15.111832+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:37:15.111832+0000","last_undegraded":"2024-09-17T15:37:15.111832+0000","last_fullsized":"2024-09-17T15:37:15.111832+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29131,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112153+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:37:15.112153+0000","last_peered":"2024-09-17T15:37:15.112153+0000","last_clean":"2024-09-17T15:37:15.112153+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:37:15.112153+0000","last_undegraded":"2024-09-17T15:37:15.112153+0000","last_fullsized":"2024-09-17T15:37:15.112153+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352003+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:37:15.352003+0000","last_peered":"2024-09-17T15:37:15.352003+0000","last_clean":"2024-09-17T15:37:15.352003+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:37:15.352003+0000","last_undegraded":"2024-09-17T15:37:15.352003+0000","last_fullsized":"2024-09-17T15:37:15.352003+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20838,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.112848+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:37:15.112848+0000","last_peered":"2024-09-17T15:37:15.112848+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:37:15.112848+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112615+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:37:15.112615+0000","last_peered":"2024-09-17T15:37:15.112615+0000","last_clean":"2024-09-17T15:37:15.112615+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:37:15.112615+0000","last_undegraded":"2024-09-17T15:37:15.112615+0000","last_fullsized":"2024-09-17T15:37:15.112615+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28835,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352081+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:37:15.352081+0000","last_peered":"2024-09-17T15:37:15.352081+0000","last_clean":"2024-09-17T15:37:15.352081+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:37:15.352081+0000","last_undegraded":"2024-09-17T15:37:15.352081+0000","last_fullsized":"2024-09-17T15:37:15.352081+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":109,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112066+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:37:15.112066+0000","last_peered":"2024-09-17T15:37:15.112066+0000","last_clean":"2024-09-17T15:37:15.112066+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:37:15.112066+0000","last_undegraded":"2024-09-17T15:37:15.112066+0000","last_fullsized":"2024-09-17T15:37:15.112066+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29173,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352021+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:37:15.352021+0000","last_peered":"2024-09-17T15:37:15.352021+0000","last_clean":"2024-09-17T15:37:15.352021+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:37:15.352021+0000","last_undegraded":"2024-09-17T15:37:15.352021+0000","last_fullsized":"2024-09-17T15:37:15.352021+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":25018,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:29.414955+0000","last_change":"2024-09-17T15:37:13.729028+0000","last_active":"2024-09-17T15:37:29.414955+0000","last_peered":"2024-09-17T15:37:29.414955+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:37:29.414955+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2197,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1942,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2197}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112352+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:37:15.112352+0000","last_peered":"2024-09-17T15:37:15.112352+0000","last_clean":"2024-09-17T15:37:15.112352+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:37:15.112352+0000","last_undegraded":"2024-09-17T15:37:15.112352+0000","last_fullsized":"2024-09-17T15:37:15.112352+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28086,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112447+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:37:15.112447+0000","last_peered":"2024-09-17T15:37:15.112447+0000","last_clean":"2024-09-17T15:37:15.112447+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:37:15.112447+0000","last_undegraded":"2024-09-17T15:37:15.112447+0000","last_fullsized":"2024-09-17T15:37:15.112447+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352224+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:37:15.352224+0000","last_peered":"2024-09-17T15:37:15.352224+0000","last_clean":"2024-09-17T15:37:15.352224+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:37:15.352224+0000","last_undegraded":"2024-09-17T15:37:15.352224+0000","last_fullsized":"2024-09-17T15:37:15.352224+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21215,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.113213+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:37:15.113213+0000","last_peered":"2024-09-17T15:37:15.113213+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:37:15.113213+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112357+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:37:15.112357+0000","last_peered":"2024-09-17T15:37:15.112357+0000","last_clean":"2024-09-17T15:37:15.112357+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:37:15.112357+0000","last_undegraded":"2024-09-17T15:37:15.112357+0000","last_fullsized":"2024-09-17T15:37:15.112357+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":104,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112592+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:37:15.112592+0000","last_peered":"2024-09-17T15:37:15.112592+0000","last_clean":"2024-09-17T15:37:15.112592+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:37:15.112592+0000","last_undegraded":"2024-09-17T15:37:15.112592+0000","last_fullsized":"2024-09-17T15:37:15.112592+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29038,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112255+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:37:15.112255+0000","last_peered":"2024-09-17T15:37:15.112255+0000","last_clean":"2024-09-17T15:37:15.112255+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:37:15.112255+0000","last_undegraded":"2024-09-17T15:37:15.112255+0000","last_fullsized":"2024-09-17T15:37:15.112255+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35352,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":182972416,"data_stored":182867639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051734,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":482420,"kb_used_data":113380,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93889420,"statfs":{"total":96636764160,"available":96142766080,"internally_reserved":0,"allocated":116101120,"data_stored":115757565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":377876140},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724678,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497296,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874544,"statfs":{"total":96636764160,"available":96127533056,"internally_reserved":0,"allocated":524288,"data_stored":165817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":508685995},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247961,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539424,"kb_used_data":66312,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832416,"statfs":{"total":96636764160,"available":96084393984,"internally_reserved":0,"allocated":67903488,"data_stored":67548630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":115503104,"data_stored":115474200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:37:38.266 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:37:38.570 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:38.571 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:37:38.583 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":447,"stamp":"2024-09-17T15:37:37.832355+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13338,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36123,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1527460,"kb_used_data":180204,"kb_used_omap":13,"kb_used_meta":1347186,"kb_avail":281588060,"statfs":{"total":289910292480,"available":288346173440,"internally_reserved":0,"allocated":184528896,"data_stored":183472012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1379519487},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1445,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1445,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002271"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20929,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.112415+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:37:15.112415+0000","last_peered":"2024-09-17T15:37:15.112415+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:37:15.112415+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21066,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352281+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:37:15.352281+0000","last_peered":"2024-09-17T15:37:15.352281+0000","last_clean":"2024-09-17T15:37:15.352281+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:37:15.352281+0000","last_undegraded":"2024-09-17T15:37:15.352281+0000","last_fullsized":"2024-09-17T15:37:15.352281+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21174,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112900+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:37:15.112900+0000","last_peered":"2024-09-17T15:37:15.112900+0000","last_clean":"2024-09-17T15:37:15.112900+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:37:15.112900+0000","last_undegraded":"2024-09-17T15:37:15.112900+0000","last_fullsized":"2024-09-17T15:37:15.112900+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21071,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352626+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:37:15.352626+0000","last_peered":"2024-09-17T15:37:15.352626+0000","last_clean":"2024-09-17T15:37:15.352626+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:37:15.352626+0000","last_undegraded":"2024-09-17T15:37:15.352626+0000","last_fullsized":"2024-09-17T15:37:15.352626+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20814,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112605+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:37:15.112605+0000","last_peered":"2024-09-17T15:37:15.112605+0000","last_clean":"2024-09-17T15:37:15.112605+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:37:15.112605+0000","last_undegraded":"2024-09-17T15:37:15.112605+0000","last_fullsized":"2024-09-17T15:37:15.112605+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21565,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.188303+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:37:15.188303+0000","last_peered":"2024-09-17T15:37:15.188303+0000","last_clean":"2024-09-17T15:37:15.188303+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:37:15.188303+0000","last_undegraded":"2024-09-17T15:37:15.188303+0000","last_fullsized":"2024-09-17T15:37:15.188303+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21108,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.111962+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:37:15.111962+0000","last_peered":"2024-09-17T15:37:15.111962+0000","last_clean":"2024-09-17T15:37:15.111962+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:37:15.111962+0000","last_undegraded":"2024-09-17T15:37:15.111962+0000","last_fullsized":"2024-09-17T15:37:15.111962+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21075,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112248+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:37:15.112248+0000","last_peered":"2024-09-17T15:37:15.112248+0000","last_clean":"2024-09-17T15:37:15.112248+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:37:15.112248+0000","last_undegraded":"2024-09-17T15:37:15.112248+0000","last_fullsized":"2024-09-17T15:37:15.112248+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.111832+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:37:15.111832+0000","last_peered":"2024-09-17T15:37:15.111832+0000","last_clean":"2024-09-17T15:37:15.111832+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:37:15.111832+0000","last_undegraded":"2024-09-17T15:37:15.111832+0000","last_fullsized":"2024-09-17T15:37:15.111832+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29131,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112153+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:37:15.112153+0000","last_peered":"2024-09-17T15:37:15.112153+0000","last_clean":"2024-09-17T15:37:15.112153+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:37:15.112153+0000","last_undegraded":"2024-09-17T15:37:15.112153+0000","last_fullsized":"2024-09-17T15:37:15.112153+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352003+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:37:15.352003+0000","last_peered":"2024-09-17T15:37:15.352003+0000","last_clean":"2024-09-17T15:37:15.352003+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:37:15.352003+0000","last_undegraded":"2024-09-17T15:37:15.352003+0000","last_fullsized":"2024-09-17T15:37:15.352003+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20838,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.112848+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:37:15.112848+0000","last_peered":"2024-09-17T15:37:15.112848+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:37:15.112848+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112615+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:37:15.112615+0000","last_peered":"2024-09-17T15:37:15.112615+0000","last_clean":"2024-09-17T15:37:15.112615+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:37:15.112615+0000","last_undegraded":"2024-09-17T15:37:15.112615+0000","last_fullsized":"2024-09-17T15:37:15.112615+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28835,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352081+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:37:15.352081+0000","last_peered":"2024-09-17T15:37:15.352081+0000","last_clean":"2024-09-17T15:37:15.352081+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:37:15.352081+0000","last_undegraded":"2024-09-17T15:37:15.352081+0000","last_fullsized":"2024-09-17T15:37:15.352081+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":109,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112066+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:37:15.112066+0000","last_peered":"2024-09-17T15:37:15.112066+0000","last_clean":"2024-09-17T15:37:15.112066+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:37:15.112066+0000","last_undegraded":"2024-09-17T15:37:15.112066+0000","last_fullsized":"2024-09-17T15:37:15.112066+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29173,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352021+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:37:15.352021+0000","last_peered":"2024-09-17T15:37:15.352021+0000","last_clean":"2024-09-17T15:37:15.352021+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:37:15.352021+0000","last_undegraded":"2024-09-17T15:37:15.352021+0000","last_fullsized":"2024-09-17T15:37:15.352021+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":26558,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:34.417537+0000","last_change":"2024-09-17T15:37:13.729028+0000","last_active":"2024-09-17T15:37:34.417537+0000","last_peered":"2024-09-17T15:37:34.417537+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:37:34.417537+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1427,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2712,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1427}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112352+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:37:15.112352+0000","last_peered":"2024-09-17T15:37:15.112352+0000","last_clean":"2024-09-17T15:37:15.112352+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:37:15.112352+0000","last_undegraded":"2024-09-17T15:37:15.112352+0000","last_fullsized":"2024-09-17T15:37:15.112352+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28086,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112447+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:37:15.112447+0000","last_peered":"2024-09-17T15:37:15.112447+0000","last_clean":"2024-09-17T15:37:15.112447+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:37:15.112447+0000","last_undegraded":"2024-09-17T15:37:15.112447+0000","last_fullsized":"2024-09-17T15:37:15.112447+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352224+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:37:15.352224+0000","last_peered":"2024-09-17T15:37:15.352224+0000","last_clean":"2024-09-17T15:37:15.352224+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:37:15.352224+0000","last_undegraded":"2024-09-17T15:37:15.352224+0000","last_fullsized":"2024-09-17T15:37:15.352224+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21215,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.113213+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:37:15.113213+0000","last_peered":"2024-09-17T15:37:15.113213+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:37:15.113213+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112357+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:37:15.112357+0000","last_peered":"2024-09-17T15:37:15.112357+0000","last_clean":"2024-09-17T15:37:15.112357+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:37:15.112357+0000","last_undegraded":"2024-09-17T15:37:15.112357+0000","last_fullsized":"2024-09-17T15:37:15.112357+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":104,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112592+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:37:15.112592+0000","last_peered":"2024-09-17T15:37:15.112592+0000","last_clean":"2024-09-17T15:37:15.112592+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:37:15.112592+0000","last_undegraded":"2024-09-17T15:37:15.112592+0000","last_fullsized":"2024-09-17T15:37:15.112592+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29038,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112255+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:37:15.112255+0000","last_peered":"2024-09-17T15:37:15.112255+0000","last_clean":"2024-09-17T15:37:15.112255+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:37:15.112255+0000","last_undegraded":"2024-09-17T15:37:15.112255+0000","last_fullsized":"2024-09-17T15:37:15.112255+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13338,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36122,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":182972416,"data_stored":182867639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051734,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":482420,"kb_used_data":113380,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93889420,"statfs":{"total":96636764160,"available":96142766080,"internally_reserved":0,"allocated":116101120,"data_stored":115757565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":377876140},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724679,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505616,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866224,"statfs":{"total":96636764160,"available":96119013376,"internally_reserved":0,"allocated":524288,"data_stored":165817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":517205675},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247962,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539424,"kb_used_data":66312,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832416,"statfs":{"total":96636764160,"available":96084393984,"internally_reserved":0,"allocated":67903488,"data_stored":67548630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":115503104,"data_stored":115474200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:37:38.584 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:37:38.895 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:38.895 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:37:38.908 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":447,"stamp":"2024-09-17T15:37:37.832355+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13338,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36123,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1527460,"kb_used_data":180204,"kb_used_omap":13,"kb_used_meta":1347186,"kb_avail":281588060,"statfs":{"total":289910292480,"available":288346173440,"internally_reserved":0,"allocated":184528896,"data_stored":183472012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1379519487},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1445,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1445,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002271"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20929,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.112415+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:37:15.112415+0000","last_peered":"2024-09-17T15:37:15.112415+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:37:15.112415+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21066,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352281+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:37:15.352281+0000","last_peered":"2024-09-17T15:37:15.352281+0000","last_clean":"2024-09-17T15:37:15.352281+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:37:15.352281+0000","last_undegraded":"2024-09-17T15:37:15.352281+0000","last_fullsized":"2024-09-17T15:37:15.352281+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21174,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112900+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:37:15.112900+0000","last_peered":"2024-09-17T15:37:15.112900+0000","last_clean":"2024-09-17T15:37:15.112900+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:37:15.112900+0000","last_undegraded":"2024-09-17T15:37:15.112900+0000","last_fullsized":"2024-09-17T15:37:15.112900+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21071,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352626+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:37:15.352626+0000","last_peered":"2024-09-17T15:37:15.352626+0000","last_clean":"2024-09-17T15:37:15.352626+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:37:15.352626+0000","last_undegraded":"2024-09-17T15:37:15.352626+0000","last_fullsized":"2024-09-17T15:37:15.352626+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20814,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112605+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:37:15.112605+0000","last_peered":"2024-09-17T15:37:15.112605+0000","last_clean":"2024-09-17T15:37:15.112605+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:37:15.112605+0000","last_undegraded":"2024-09-17T15:37:15.112605+0000","last_fullsized":"2024-09-17T15:37:15.112605+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21565,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.188303+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:37:15.188303+0000","last_peered":"2024-09-17T15:37:15.188303+0000","last_clean":"2024-09-17T15:37:15.188303+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:37:15.188303+0000","last_undegraded":"2024-09-17T15:37:15.188303+0000","last_fullsized":"2024-09-17T15:37:15.188303+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21108,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.111962+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:37:15.111962+0000","last_peered":"2024-09-17T15:37:15.111962+0000","last_clean":"2024-09-17T15:37:15.111962+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:37:15.111962+0000","last_undegraded":"2024-09-17T15:37:15.111962+0000","last_fullsized":"2024-09-17T15:37:15.111962+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21075,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112248+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:37:15.112248+0000","last_peered":"2024-09-17T15:37:15.112248+0000","last_clean":"2024-09-17T15:37:15.112248+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:37:15.112248+0000","last_undegraded":"2024-09-17T15:37:15.112248+0000","last_fullsized":"2024-09-17T15:37:15.112248+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.111832+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:37:15.111832+0000","last_peered":"2024-09-17T15:37:15.111832+0000","last_clean":"2024-09-17T15:37:15.111832+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:37:15.111832+0000","last_undegraded":"2024-09-17T15:37:15.111832+0000","last_fullsized":"2024-09-17T15:37:15.111832+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29131,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112153+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:37:15.112153+0000","last_peered":"2024-09-17T15:37:15.112153+0000","last_clean":"2024-09-17T15:37:15.112153+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:37:15.112153+0000","last_undegraded":"2024-09-17T15:37:15.112153+0000","last_fullsized":"2024-09-17T15:37:15.112153+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352003+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:37:15.352003+0000","last_peered":"2024-09-17T15:37:15.352003+0000","last_clean":"2024-09-17T15:37:15.352003+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:37:15.352003+0000","last_undegraded":"2024-09-17T15:37:15.352003+0000","last_fullsized":"2024-09-17T15:37:15.352003+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20838,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.112848+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:37:15.112848+0000","last_peered":"2024-09-17T15:37:15.112848+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:37:15.112848+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112615+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:37:15.112615+0000","last_peered":"2024-09-17T15:37:15.112615+0000","last_clean":"2024-09-17T15:37:15.112615+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:37:15.112615+0000","last_undegraded":"2024-09-17T15:37:15.112615+0000","last_fullsized":"2024-09-17T15:37:15.112615+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28835,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352081+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:37:15.352081+0000","last_peered":"2024-09-17T15:37:15.352081+0000","last_clean":"2024-09-17T15:37:15.352081+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:37:15.352081+0000","last_undegraded":"2024-09-17T15:37:15.352081+0000","last_fullsized":"2024-09-17T15:37:15.352081+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":109,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112066+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:37:15.112066+0000","last_peered":"2024-09-17T15:37:15.112066+0000","last_clean":"2024-09-17T15:37:15.112066+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:37:15.112066+0000","last_undegraded":"2024-09-17T15:37:15.112066+0000","last_fullsized":"2024-09-17T15:37:15.112066+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29173,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352021+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:37:15.352021+0000","last_peered":"2024-09-17T15:37:15.352021+0000","last_clean":"2024-09-17T15:37:15.352021+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:37:15.352021+0000","last_undegraded":"2024-09-17T15:37:15.352021+0000","last_fullsized":"2024-09-17T15:37:15.352021+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":26558,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:34.417537+0000","last_change":"2024-09-17T15:37:13.729028+0000","last_active":"2024-09-17T15:37:34.417537+0000","last_peered":"2024-09-17T15:37:34.417537+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:37:34.417537+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1427,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2712,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1427}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112352+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:37:15.112352+0000","last_peered":"2024-09-17T15:37:15.112352+0000","last_clean":"2024-09-17T15:37:15.112352+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:37:15.112352+0000","last_undegraded":"2024-09-17T15:37:15.112352+0000","last_fullsized":"2024-09-17T15:37:15.112352+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28086,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112447+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:37:15.112447+0000","last_peered":"2024-09-17T15:37:15.112447+0000","last_clean":"2024-09-17T15:37:15.112447+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:37:15.112447+0000","last_undegraded":"2024-09-17T15:37:15.112447+0000","last_fullsized":"2024-09-17T15:37:15.112447+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352224+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:37:15.352224+0000","last_peered":"2024-09-17T15:37:15.352224+0000","last_clean":"2024-09-17T15:37:15.352224+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:37:15.352224+0000","last_undegraded":"2024-09-17T15:37:15.352224+0000","last_fullsized":"2024-09-17T15:37:15.352224+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21215,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.113213+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:37:15.113213+0000","last_peered":"2024-09-17T15:37:15.113213+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:37:15.113213+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112357+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:37:15.112357+0000","last_peered":"2024-09-17T15:37:15.112357+0000","last_clean":"2024-09-17T15:37:15.112357+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:37:15.112357+0000","last_undegraded":"2024-09-17T15:37:15.112357+0000","last_fullsized":"2024-09-17T15:37:15.112357+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":104,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112592+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:37:15.112592+0000","last_peered":"2024-09-17T15:37:15.112592+0000","last_clean":"2024-09-17T15:37:15.112592+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:37:15.112592+0000","last_undegraded":"2024-09-17T15:37:15.112592+0000","last_fullsized":"2024-09-17T15:37:15.112592+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29038,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112255+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:37:15.112255+0000","last_peered":"2024-09-17T15:37:15.112255+0000","last_clean":"2024-09-17T15:37:15.112255+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:37:15.112255+0000","last_undegraded":"2024-09-17T15:37:15.112255+0000","last_fullsized":"2024-09-17T15:37:15.112255+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13338,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36122,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":182972416,"data_stored":182867639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051734,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":482420,"kb_used_data":113380,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93889420,"statfs":{"total":96636764160,"available":96142766080,"internally_reserved":0,"allocated":116101120,"data_stored":115757565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":377876140},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724679,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505616,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866224,"statfs":{"total":96636764160,"available":96119013376,"internally_reserved":0,"allocated":524288,"data_stored":165817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":517205675},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247962,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539424,"kb_used_data":66312,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832416,"statfs":{"total":96636764160,"available":96084393984,"internally_reserved":0,"allocated":67903488,"data_stored":67548630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":115503104,"data_stored":115474200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:37:38.909 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:37:38.909 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'20648", 'reported_seq': 20929, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:37:15.112415+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:37:15.112415+0000', 'last_peered': '2024-09-17T15:37:15.112415+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:37:15.112415+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'10600", 'ondisk_log_start': "43'10600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20648, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:38.909 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:37:38.909 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20542", 'reported_seq': 20838, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:37:15.112848+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:37:15.112848+0000', 'last_peered': '2024-09-17T15:37:15.112848+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:37:15.112848+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17138, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3943}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:38.909 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:37:38.909 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'20966", 'reported_seq': 26558, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:37:34.417537+0000', 'last_change': '2024-09-17T15:37:13.729028+0000', 'last_active': '2024-09-17T15:37:34.417537+0000', 'last_peered': '2024-09-17T15:37:34.417537+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:37:34.417537+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "43'10900", 'ondisk_log_start': "43'10900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1427, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17514, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2712, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1427}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:38.910 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:37:38.910 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'20886", 'reported_seq': 21215, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:37:15.113213+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:37:15.113213+0000', 'last_peered': '2024-09-17T15:37:15.113213+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:37:15.113213+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'10800", 'ondisk_log_start': "43'10800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17673, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3980}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:38.910 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:37:39.315 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:39.315 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 13338/24 objects degraded (55575.000%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":796,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1564119040,"bytes_avail":288346173440,"bytes_total":289910292480,"degraded_objects":13338,"degraded_total":24,"degraded_ratio":555.75,"recovering_objects_per_sec":120,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1445,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:37:39.327 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:37:39.328 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:37:39.675 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:39.676 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:37:39.688 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":447,"stamp":"2024-09-17T15:37:37.832355+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13338,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36123,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1527460,"kb_used_data":180204,"kb_used_omap":13,"kb_used_meta":1347186,"kb_avail":281588060,"statfs":{"total":289910292480,"available":288346173440,"internally_reserved":0,"allocated":184528896,"data_stored":183472012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1379519487},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1445,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1445,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002271"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20929,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.112415+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:37:15.112415+0000","last_peered":"2024-09-17T15:37:15.112415+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:37:15.112415+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21066,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352281+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:37:15.352281+0000","last_peered":"2024-09-17T15:37:15.352281+0000","last_clean":"2024-09-17T15:37:15.352281+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:37:15.352281+0000","last_undegraded":"2024-09-17T15:37:15.352281+0000","last_fullsized":"2024-09-17T15:37:15.352281+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21174,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112900+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:37:15.112900+0000","last_peered":"2024-09-17T15:37:15.112900+0000","last_clean":"2024-09-17T15:37:15.112900+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:37:15.112900+0000","last_undegraded":"2024-09-17T15:37:15.112900+0000","last_fullsized":"2024-09-17T15:37:15.112900+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21071,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352626+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:37:15.352626+0000","last_peered":"2024-09-17T15:37:15.352626+0000","last_clean":"2024-09-17T15:37:15.352626+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:37:15.352626+0000","last_undegraded":"2024-09-17T15:37:15.352626+0000","last_fullsized":"2024-09-17T15:37:15.352626+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20814,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112605+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:37:15.112605+0000","last_peered":"2024-09-17T15:37:15.112605+0000","last_clean":"2024-09-17T15:37:15.112605+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:37:15.112605+0000","last_undegraded":"2024-09-17T15:37:15.112605+0000","last_fullsized":"2024-09-17T15:37:15.112605+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21565,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.188303+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:37:15.188303+0000","last_peered":"2024-09-17T15:37:15.188303+0000","last_clean":"2024-09-17T15:37:15.188303+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:37:15.188303+0000","last_undegraded":"2024-09-17T15:37:15.188303+0000","last_fullsized":"2024-09-17T15:37:15.188303+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21108,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.111962+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:37:15.111962+0000","last_peered":"2024-09-17T15:37:15.111962+0000","last_clean":"2024-09-17T15:37:15.111962+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:37:15.111962+0000","last_undegraded":"2024-09-17T15:37:15.111962+0000","last_fullsized":"2024-09-17T15:37:15.111962+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21075,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112248+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:37:15.112248+0000","last_peered":"2024-09-17T15:37:15.112248+0000","last_clean":"2024-09-17T15:37:15.112248+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:37:15.112248+0000","last_undegraded":"2024-09-17T15:37:15.112248+0000","last_fullsized":"2024-09-17T15:37:15.112248+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.111832+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:37:15.111832+0000","last_peered":"2024-09-17T15:37:15.111832+0000","last_clean":"2024-09-17T15:37:15.111832+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:37:15.111832+0000","last_undegraded":"2024-09-17T15:37:15.111832+0000","last_fullsized":"2024-09-17T15:37:15.111832+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29131,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112153+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:37:15.112153+0000","last_peered":"2024-09-17T15:37:15.112153+0000","last_clean":"2024-09-17T15:37:15.112153+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:37:15.112153+0000","last_undegraded":"2024-09-17T15:37:15.112153+0000","last_fullsized":"2024-09-17T15:37:15.112153+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352003+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:37:15.352003+0000","last_peered":"2024-09-17T15:37:15.352003+0000","last_clean":"2024-09-17T15:37:15.352003+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:37:15.352003+0000","last_undegraded":"2024-09-17T15:37:15.352003+0000","last_fullsized":"2024-09-17T15:37:15.352003+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20838,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.112848+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:37:15.112848+0000","last_peered":"2024-09-17T15:37:15.112848+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:37:15.112848+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112615+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:37:15.112615+0000","last_peered":"2024-09-17T15:37:15.112615+0000","last_clean":"2024-09-17T15:37:15.112615+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:37:15.112615+0000","last_undegraded":"2024-09-17T15:37:15.112615+0000","last_fullsized":"2024-09-17T15:37:15.112615+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28835,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352081+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:37:15.352081+0000","last_peered":"2024-09-17T15:37:15.352081+0000","last_clean":"2024-09-17T15:37:15.352081+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:37:15.352081+0000","last_undegraded":"2024-09-17T15:37:15.352081+0000","last_fullsized":"2024-09-17T15:37:15.352081+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":109,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112066+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:37:15.112066+0000","last_peered":"2024-09-17T15:37:15.112066+0000","last_clean":"2024-09-17T15:37:15.112066+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:37:15.112066+0000","last_undegraded":"2024-09-17T15:37:15.112066+0000","last_fullsized":"2024-09-17T15:37:15.112066+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29173,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352021+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:37:15.352021+0000","last_peered":"2024-09-17T15:37:15.352021+0000","last_clean":"2024-09-17T15:37:15.352021+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:37:15.352021+0000","last_undegraded":"2024-09-17T15:37:15.352021+0000","last_fullsized":"2024-09-17T15:37:15.352021+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":26558,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:34.417537+0000","last_change":"2024-09-17T15:37:13.729028+0000","last_active":"2024-09-17T15:37:34.417537+0000","last_peered":"2024-09-17T15:37:34.417537+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:37:34.417537+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1427,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2712,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1427}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112352+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:37:15.112352+0000","last_peered":"2024-09-17T15:37:15.112352+0000","last_clean":"2024-09-17T15:37:15.112352+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:37:15.112352+0000","last_undegraded":"2024-09-17T15:37:15.112352+0000","last_fullsized":"2024-09-17T15:37:15.112352+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28086,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112447+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:37:15.112447+0000","last_peered":"2024-09-17T15:37:15.112447+0000","last_clean":"2024-09-17T15:37:15.112447+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:37:15.112447+0000","last_undegraded":"2024-09-17T15:37:15.112447+0000","last_fullsized":"2024-09-17T15:37:15.112447+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352224+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:37:15.352224+0000","last_peered":"2024-09-17T15:37:15.352224+0000","last_clean":"2024-09-17T15:37:15.352224+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:37:15.352224+0000","last_undegraded":"2024-09-17T15:37:15.352224+0000","last_fullsized":"2024-09-17T15:37:15.352224+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21215,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.113213+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:37:15.113213+0000","last_peered":"2024-09-17T15:37:15.113213+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:37:15.113213+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112357+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:37:15.112357+0000","last_peered":"2024-09-17T15:37:15.112357+0000","last_clean":"2024-09-17T15:37:15.112357+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:37:15.112357+0000","last_undegraded":"2024-09-17T15:37:15.112357+0000","last_fullsized":"2024-09-17T15:37:15.112357+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":104,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112592+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:37:15.112592+0000","last_peered":"2024-09-17T15:37:15.112592+0000","last_clean":"2024-09-17T15:37:15.112592+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:37:15.112592+0000","last_undegraded":"2024-09-17T15:37:15.112592+0000","last_fullsized":"2024-09-17T15:37:15.112592+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29038,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112255+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:37:15.112255+0000","last_peered":"2024-09-17T15:37:15.112255+0000","last_clean":"2024-09-17T15:37:15.112255+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:37:15.112255+0000","last_undegraded":"2024-09-17T15:37:15.112255+0000","last_fullsized":"2024-09-17T15:37:15.112255+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13338,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36122,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":182972416,"data_stored":182867639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051734,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":482420,"kb_used_data":113380,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93889420,"statfs":{"total":96636764160,"available":96142766080,"internally_reserved":0,"allocated":116101120,"data_stored":115757565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":377876140},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724679,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505616,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866224,"statfs":{"total":96636764160,"available":96119013376,"internally_reserved":0,"allocated":524288,"data_stored":165817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":517205675},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247962,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539424,"kb_used_data":66312,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832416,"statfs":{"total":96636764160,"available":96084393984,"internally_reserved":0,"allocated":67903488,"data_stored":67548630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":115503104,"data_stored":115474200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:37:42.690 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:37:43.084 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:43.084 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:37:43.094 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":451,"stamp":"2024-09-17T15:37:42.203198+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37121,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1500100,"kb_used_data":177804,"kb_used_omap":13,"kb_used_meta":1322226,"kb_avail":281615420,"statfs":{"total":289910292480,"available":288374190080,"internally_reserved":0,"allocated":182071296,"data_stored":181014412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1353960447},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1768,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1768,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.371588"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20929,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.112415+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:37:15.112415+0000","last_peered":"2024-09-17T15:37:15.112415+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:37:15.112415+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21066,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352281+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:37:15.352281+0000","last_peered":"2024-09-17T15:37:15.352281+0000","last_clean":"2024-09-17T15:37:15.352281+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:37:15.352281+0000","last_undegraded":"2024-09-17T15:37:15.352281+0000","last_fullsized":"2024-09-17T15:37:15.352281+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21174,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112900+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:37:15.112900+0000","last_peered":"2024-09-17T15:37:15.112900+0000","last_clean":"2024-09-17T15:37:15.112900+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:37:15.112900+0000","last_undegraded":"2024-09-17T15:37:15.112900+0000","last_fullsized":"2024-09-17T15:37:15.112900+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21071,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352626+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:37:15.352626+0000","last_peered":"2024-09-17T15:37:15.352626+0000","last_clean":"2024-09-17T15:37:15.352626+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:37:15.352626+0000","last_undegraded":"2024-09-17T15:37:15.352626+0000","last_fullsized":"2024-09-17T15:37:15.352626+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20814,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112605+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:37:15.112605+0000","last_peered":"2024-09-17T15:37:15.112605+0000","last_clean":"2024-09-17T15:37:15.112605+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:37:15.112605+0000","last_undegraded":"2024-09-17T15:37:15.112605+0000","last_fullsized":"2024-09-17T15:37:15.112605+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21565,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.188303+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:37:15.188303+0000","last_peered":"2024-09-17T15:37:15.188303+0000","last_clean":"2024-09-17T15:37:15.188303+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:37:15.188303+0000","last_undegraded":"2024-09-17T15:37:15.188303+0000","last_fullsized":"2024-09-17T15:37:15.188303+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21108,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.111962+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:37:15.111962+0000","last_peered":"2024-09-17T15:37:15.111962+0000","last_clean":"2024-09-17T15:37:15.111962+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:37:15.111962+0000","last_undegraded":"2024-09-17T15:37:15.111962+0000","last_fullsized":"2024-09-17T15:37:15.111962+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21075,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112248+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:37:15.112248+0000","last_peered":"2024-09-17T15:37:15.112248+0000","last_clean":"2024-09-17T15:37:15.112248+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:37:15.112248+0000","last_undegraded":"2024-09-17T15:37:15.112248+0000","last_fullsized":"2024-09-17T15:37:15.112248+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.111832+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:37:15.111832+0000","last_peered":"2024-09-17T15:37:15.111832+0000","last_clean":"2024-09-17T15:37:15.111832+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:37:15.111832+0000","last_undegraded":"2024-09-17T15:37:15.111832+0000","last_fullsized":"2024-09-17T15:37:15.111832+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29131,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112153+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:37:15.112153+0000","last_peered":"2024-09-17T15:37:15.112153+0000","last_clean":"2024-09-17T15:37:15.112153+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:37:15.112153+0000","last_undegraded":"2024-09-17T15:37:15.112153+0000","last_fullsized":"2024-09-17T15:37:15.112153+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352003+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:37:15.352003+0000","last_peered":"2024-09-17T15:37:15.352003+0000","last_clean":"2024-09-17T15:37:15.352003+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:37:15.352003+0000","last_undegraded":"2024-09-17T15:37:15.352003+0000","last_fullsized":"2024-09-17T15:37:15.352003+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20838,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.112848+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:37:15.112848+0000","last_peered":"2024-09-17T15:37:15.112848+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:37:15.112848+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":135,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112656+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:37:15.112656+0000","last_peered":"2024-09-17T15:37:15.112656+0000","last_clean":"2024-09-17T15:37:15.112656+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:37:15.112656+0000","last_undegraded":"2024-09-17T15:37:15.112656+0000","last_fullsized":"2024-09-17T15:37:15.112656+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28835,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352081+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:37:15.352081+0000","last_peered":"2024-09-17T15:37:15.352081+0000","last_clean":"2024-09-17T15:37:15.352081+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:37:15.352081+0000","last_undegraded":"2024-09-17T15:37:15.352081+0000","last_fullsized":"2024-09-17T15:37:15.352081+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":109,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112066+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:37:15.112066+0000","last_peered":"2024-09-17T15:37:15.112066+0000","last_clean":"2024-09-17T15:37:15.112066+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:37:15.112066+0000","last_undegraded":"2024-09-17T15:37:15.112066+0000","last_fullsized":"2024-09-17T15:37:15.112066+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29173,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352021+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:37:15.352021+0000","last_peered":"2024-09-17T15:37:15.352021+0000","last_clean":"2024-09-17T15:37:15.352021+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:37:15.352021+0000","last_undegraded":"2024-09-17T15:37:15.352021+0000","last_fullsized":"2024-09-17T15:37:15.352021+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":28554,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:39.416590+0000","last_change":"2024-09-17T15:37:13.729028+0000","last_active":"2024-09-17T15:37:39.416590+0000","last_peered":"2024-09-17T15:37:39.416590+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:37:39.416590+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":429,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3710,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-429}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112352+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:37:15.112352+0000","last_peered":"2024-09-17T15:37:15.112352+0000","last_clean":"2024-09-17T15:37:15.112352+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:37:15.112352+0000","last_undegraded":"2024-09-17T15:37:15.112352+0000","last_fullsized":"2024-09-17T15:37:15.112352+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28086,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112447+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:37:15.112447+0000","last_peered":"2024-09-17T15:37:15.112447+0000","last_clean":"2024-09-17T15:37:15.112447+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:37:15.112447+0000","last_undegraded":"2024-09-17T15:37:15.112447+0000","last_fullsized":"2024-09-17T15:37:15.112447+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352224+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:37:15.352224+0000","last_peered":"2024-09-17T15:37:15.352224+0000","last_clean":"2024-09-17T15:37:15.352224+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:37:15.352224+0000","last_undegraded":"2024-09-17T15:37:15.352224+0000","last_fullsized":"2024-09-17T15:37:15.352224+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21215,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.113213+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:37:15.113213+0000","last_peered":"2024-09-17T15:37:15.113213+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:37:15.113213+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112357+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:37:15.112357+0000","last_peered":"2024-09-17T15:37:15.112357+0000","last_clean":"2024-09-17T15:37:15.112357+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:37:15.112357+0000","last_undegraded":"2024-09-17T15:37:15.112357+0000","last_fullsized":"2024-09-17T15:37:15.112357+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":104,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112592+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:37:15.112592+0000","last_peered":"2024-09-17T15:37:15.112592+0000","last_clean":"2024-09-17T15:37:15.112592+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:37:15.112592+0000","last_undegraded":"2024-09-17T15:37:15.112592+0000","last_fullsized":"2024-09-17T15:37:15.112592+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29038,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112255+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:37:15.112255+0000","last_peered":"2024-09-17T15:37:15.112255+0000","last_clean":"2024-09-17T15:37:15.112255+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:37:15.112255+0000","last_undegraded":"2024-09-17T15:37:15.112255+0000","last_fullsized":"2024-09-17T15:37:15.112255+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37120,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":180514816,"data_stored":180410039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051735,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":463380,"kb_used_data":110980,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93908460,"statfs":{"total":96636764160,"available":96162263040,"internally_reserved":0,"allocated":113643520,"data_stored":113299965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724680,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497296,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874544,"statfs":{"total":96636764160,"available":96127533056,"internally_reserved":0,"allocated":524288,"data_stored":165817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":508685995},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247963,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539424,"kb_used_data":66312,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832416,"statfs":{"total":96636764160,"available":96084393984,"internally_reserved":0,"allocated":67903488,"data_stored":67548630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":113045504,"data_stored":113016600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:37:43.095 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:37:43.422 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:43.422 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:37:43.436 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":451,"stamp":"2024-09-17T15:37:42.203198+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37121,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1500100,"kb_used_data":177804,"kb_used_omap":13,"kb_used_meta":1322226,"kb_avail":281615420,"statfs":{"total":289910292480,"available":288374190080,"internally_reserved":0,"allocated":182071296,"data_stored":181014412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1353960447},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1768,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1768,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.371588"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20929,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.112415+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:37:15.112415+0000","last_peered":"2024-09-17T15:37:15.112415+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:37:15.112415+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21066,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352281+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:37:15.352281+0000","last_peered":"2024-09-17T15:37:15.352281+0000","last_clean":"2024-09-17T15:37:15.352281+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:37:15.352281+0000","last_undegraded":"2024-09-17T15:37:15.352281+0000","last_fullsized":"2024-09-17T15:37:15.352281+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21174,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112900+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:37:15.112900+0000","last_peered":"2024-09-17T15:37:15.112900+0000","last_clean":"2024-09-17T15:37:15.112900+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:37:15.112900+0000","last_undegraded":"2024-09-17T15:37:15.112900+0000","last_fullsized":"2024-09-17T15:37:15.112900+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21071,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352626+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:37:15.352626+0000","last_peered":"2024-09-17T15:37:15.352626+0000","last_clean":"2024-09-17T15:37:15.352626+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:37:15.352626+0000","last_undegraded":"2024-09-17T15:37:15.352626+0000","last_fullsized":"2024-09-17T15:37:15.352626+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20814,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112605+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:37:15.112605+0000","last_peered":"2024-09-17T15:37:15.112605+0000","last_clean":"2024-09-17T15:37:15.112605+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:37:15.112605+0000","last_undegraded":"2024-09-17T15:37:15.112605+0000","last_fullsized":"2024-09-17T15:37:15.112605+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21565,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.188303+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:37:15.188303+0000","last_peered":"2024-09-17T15:37:15.188303+0000","last_clean":"2024-09-17T15:37:15.188303+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:37:15.188303+0000","last_undegraded":"2024-09-17T15:37:15.188303+0000","last_fullsized":"2024-09-17T15:37:15.188303+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21108,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.111962+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:37:15.111962+0000","last_peered":"2024-09-17T15:37:15.111962+0000","last_clean":"2024-09-17T15:37:15.111962+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:37:15.111962+0000","last_undegraded":"2024-09-17T15:37:15.111962+0000","last_fullsized":"2024-09-17T15:37:15.111962+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21075,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112248+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:37:15.112248+0000","last_peered":"2024-09-17T15:37:15.112248+0000","last_clean":"2024-09-17T15:37:15.112248+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:37:15.112248+0000","last_undegraded":"2024-09-17T15:37:15.112248+0000","last_fullsized":"2024-09-17T15:37:15.112248+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.111832+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:37:15.111832+0000","last_peered":"2024-09-17T15:37:15.111832+0000","last_clean":"2024-09-17T15:37:15.111832+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:37:15.111832+0000","last_undegraded":"2024-09-17T15:37:15.111832+0000","last_fullsized":"2024-09-17T15:37:15.111832+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29131,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112153+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:37:15.112153+0000","last_peered":"2024-09-17T15:37:15.112153+0000","last_clean":"2024-09-17T15:37:15.112153+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:37:15.112153+0000","last_undegraded":"2024-09-17T15:37:15.112153+0000","last_fullsized":"2024-09-17T15:37:15.112153+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352003+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:37:15.352003+0000","last_peered":"2024-09-17T15:37:15.352003+0000","last_clean":"2024-09-17T15:37:15.352003+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:37:15.352003+0000","last_undegraded":"2024-09-17T15:37:15.352003+0000","last_fullsized":"2024-09-17T15:37:15.352003+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20838,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.112848+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:37:15.112848+0000","last_peered":"2024-09-17T15:37:15.112848+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:37:15.112848+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":135,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112656+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:37:15.112656+0000","last_peered":"2024-09-17T15:37:15.112656+0000","last_clean":"2024-09-17T15:37:15.112656+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:37:15.112656+0000","last_undegraded":"2024-09-17T15:37:15.112656+0000","last_fullsized":"2024-09-17T15:37:15.112656+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28835,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352081+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:37:15.352081+0000","last_peered":"2024-09-17T15:37:15.352081+0000","last_clean":"2024-09-17T15:37:15.352081+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:37:15.352081+0000","last_undegraded":"2024-09-17T15:37:15.352081+0000","last_fullsized":"2024-09-17T15:37:15.352081+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":109,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112066+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:37:15.112066+0000","last_peered":"2024-09-17T15:37:15.112066+0000","last_clean":"2024-09-17T15:37:15.112066+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:37:15.112066+0000","last_undegraded":"2024-09-17T15:37:15.112066+0000","last_fullsized":"2024-09-17T15:37:15.112066+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29173,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352021+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:37:15.352021+0000","last_peered":"2024-09-17T15:37:15.352021+0000","last_clean":"2024-09-17T15:37:15.352021+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:37:15.352021+0000","last_undegraded":"2024-09-17T15:37:15.352021+0000","last_fullsized":"2024-09-17T15:37:15.352021+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":28554,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:39.416590+0000","last_change":"2024-09-17T15:37:13.729028+0000","last_active":"2024-09-17T15:37:39.416590+0000","last_peered":"2024-09-17T15:37:39.416590+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:37:39.416590+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":429,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3710,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-429}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112352+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:37:15.112352+0000","last_peered":"2024-09-17T15:37:15.112352+0000","last_clean":"2024-09-17T15:37:15.112352+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:37:15.112352+0000","last_undegraded":"2024-09-17T15:37:15.112352+0000","last_fullsized":"2024-09-17T15:37:15.112352+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28086,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112447+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:37:15.112447+0000","last_peered":"2024-09-17T15:37:15.112447+0000","last_clean":"2024-09-17T15:37:15.112447+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:37:15.112447+0000","last_undegraded":"2024-09-17T15:37:15.112447+0000","last_fullsized":"2024-09-17T15:37:15.112447+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":101,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.352224+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:37:15.352224+0000","last_peered":"2024-09-17T15:37:15.352224+0000","last_clean":"2024-09-17T15:37:15.352224+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:37:15.352224+0000","last_undegraded":"2024-09-17T15:37:15.352224+0000","last_fullsized":"2024-09-17T15:37:15.352224+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21215,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.113213+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:37:15.113213+0000","last_peered":"2024-09-17T15:37:15.113213+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:37:15.113213+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112357+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:37:15.112357+0000","last_peered":"2024-09-17T15:37:15.112357+0000","last_clean":"2024-09-17T15:37:15.112357+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:37:15.112357+0000","last_undegraded":"2024-09-17T15:37:15.112357+0000","last_fullsized":"2024-09-17T15:37:15.112357+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":104,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112592+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:37:15.112592+0000","last_peered":"2024-09-17T15:37:15.112592+0000","last_clean":"2024-09-17T15:37:15.112592+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:37:15.112592+0000","last_undegraded":"2024-09-17T15:37:15.112592+0000","last_fullsized":"2024-09-17T15:37:15.112592+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29038,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112255+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:37:15.112255+0000","last_peered":"2024-09-17T15:37:15.112255+0000","last_clean":"2024-09-17T15:37:15.112255+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:37:15.112255+0000","last_undegraded":"2024-09-17T15:37:15.112255+0000","last_fullsized":"2024-09-17T15:37:15.112255+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37120,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":180514816,"data_stored":180410039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051735,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":463380,"kb_used_data":110980,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93908460,"statfs":{"total":96636764160,"available":96162263040,"internally_reserved":0,"allocated":113643520,"data_stored":113299965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724680,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497296,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874544,"statfs":{"total":96636764160,"available":96127533056,"internally_reserved":0,"allocated":524288,"data_stored":165817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":508685995},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247963,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539424,"kb_used_data":66312,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832416,"statfs":{"total":96636764160,"available":96084393984,"internally_reserved":0,"allocated":67903488,"data_stored":67548630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":113045504,"data_stored":113016600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:37:43.438 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:37:43.438 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'20648", 'reported_seq': 20929, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:37:15.112415+0000', 'last_change': '2024-09-17T15:26:22.750473+0000', 'last_active': '2024-09-17T15:37:15.112415+0000', 'last_peered': '2024-09-17T15:37:15.112415+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:37:15.112415+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'10600", 'ondisk_log_start': "43'10600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3984, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20648, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:43.438 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:37:43.438 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20542", 'reported_seq': 20838, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:37:15.112848+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:37:15.112848+0000', 'last_peered': '2024-09-17T15:37:15.112848+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:37:15.112848+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17138, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3943}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:43.438 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-17T15:37:43.438 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'20966", 'reported_seq': 28554, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:37:39.416590+0000', 'last_change': '2024-09-17T15:37:13.729028+0000', 'last_active': '2024-09-17T15:37:39.416590+0000', 'last_peered': '2024-09-17T15:37:39.416590+0000', 'last_clean': '2024-09-17T15:26:20.167381+0000', 'last_became_active': '2024-09-17T15:26:22.740839+0000', 'last_became_peered': '2024-09-17T15:26:22.740839+0000', 'last_unstale': '2024-09-17T15:37:39.416590+0000', 'last_undegraded': '2024-09-17T15:26:22.316891+0000', 'last_fullsized': '2024-09-17T15:26:22.194685+0000', 'mapping_epoch': 40, 'log_start': "43'10900", 'ondisk_log_start': "43'10900", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3494", 'last_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:09.987306+0000', 'objects_scrubbed': 3450, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1314, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 429, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17514, 'num_write_kb': 28128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3710, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -429}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:43.438 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:37:43.439 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'20886", 'reported_seq': 21215, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:37:15.113213+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:37:15.113213+0000', 'last_peered': '2024-09-17T15:37:15.113213+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:37:15.113213+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'10800", 'ondisk_log_start': "43'10800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17673, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3980}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:43.439 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:37:43.846 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:43.846 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 12340/24 objects degraded (51416.667%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":800,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1536102400,"bytes_avail":288374190080,"bytes_total":289910292480,"degraded_objects":12340,"degraded_total":24,"degraded_ratio":514.16666666666663,"recovering_objects_per_sec":176,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1768,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:37:43.860 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:37:43.860 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:37:44.278 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:44.279 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:37:44.291 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":452,"stamp":"2024-09-17T15:37:43.833326+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37121,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1497968,"kb_used_data":175664,"kb_used_omap":13,"kb_used_meta":1322226,"kb_avail":281617552,"statfs":{"total":289910292480,"available":288376373248,"internally_reserved":0,"allocated":179879936,"data_stored":178812251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1353960447},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-998,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":998,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001392"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":20929,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.112415+0000","last_change":"2024-09-17T15:26:22.750473+0000","last_active":"2024-09-17T15:37:15.112415+0000","last_peered":"2024-09-17T15:37:15.112415+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:37:15.112415+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21070,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207284+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:37:42.207284+0000","last_peered":"2024-09-17T15:37:42.207284+0000","last_clean":"2024-09-17T15:37:42.207284+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:37:42.207284+0000","last_undegraded":"2024-09-17T15:37:42.207284+0000","last_fullsized":"2024-09-17T15:37:42.207284+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21174,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112900+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:37:15.112900+0000","last_peered":"2024-09-17T15:37:15.112900+0000","last_clean":"2024-09-17T15:37:15.112900+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:37:15.112900+0000","last_undegraded":"2024-09-17T15:37:15.112900+0000","last_fullsized":"2024-09-17T15:37:15.112900+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21075,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207161+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:37:42.207161+0000","last_peered":"2024-09-17T15:37:42.207161+0000","last_clean":"2024-09-17T15:37:42.207161+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:37:42.207161+0000","last_undegraded":"2024-09-17T15:37:42.207161+0000","last_fullsized":"2024-09-17T15:37:42.207161+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20814,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112605+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:37:15.112605+0000","last_peered":"2024-09-17T15:37:15.112605+0000","last_clean":"2024-09-17T15:37:15.112605+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:37:15.112605+0000","last_undegraded":"2024-09-17T15:37:15.112605+0000","last_fullsized":"2024-09-17T15:37:15.112605+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21565,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.188303+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:37:15.188303+0000","last_peered":"2024-09-17T15:37:15.188303+0000","last_clean":"2024-09-17T15:37:15.188303+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:37:15.188303+0000","last_undegraded":"2024-09-17T15:37:15.188303+0000","last_fullsized":"2024-09-17T15:37:15.188303+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21108,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.111962+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:37:15.111962+0000","last_peered":"2024-09-17T15:37:15.111962+0000","last_clean":"2024-09-17T15:37:15.111962+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:37:15.111962+0000","last_undegraded":"2024-09-17T15:37:15.111962+0000","last_fullsized":"2024-09-17T15:37:15.111962+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21075,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112248+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:37:15.112248+0000","last_peered":"2024-09-17T15:37:15.112248+0000","last_clean":"2024-09-17T15:37:15.112248+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:37:15.112248+0000","last_undegraded":"2024-09-17T15:37:15.112248+0000","last_fullsized":"2024-09-17T15:37:15.112248+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.111832+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:37:15.111832+0000","last_peered":"2024-09-17T15:37:15.111832+0000","last_clean":"2024-09-17T15:37:15.111832+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:37:15.111832+0000","last_undegraded":"2024-09-17T15:37:15.111832+0000","last_fullsized":"2024-09-17T15:37:15.111832+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29131,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112153+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:37:15.112153+0000","last_peered":"2024-09-17T15:37:15.112153+0000","last_clean":"2024-09-17T15:37:15.112153+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:37:15.112153+0000","last_undegraded":"2024-09-17T15:37:15.112153+0000","last_fullsized":"2024-09-17T15:37:15.112153+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.206976+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:37:42.206976+0000","last_peered":"2024-09-17T15:37:42.206976+0000","last_clean":"2024-09-17T15:37:42.206976+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:37:42.206976+0000","last_undegraded":"2024-09-17T15:37:42.206976+0000","last_fullsized":"2024-09-17T15:37:42.206976+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20838,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.112848+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:37:15.112848+0000","last_peered":"2024-09-17T15:37:15.112848+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:37:15.112848+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":135,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112656+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:37:15.112656+0000","last_peered":"2024-09-17T15:37:15.112656+0000","last_clean":"2024-09-17T15:37:15.112656+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:37:15.112656+0000","last_undegraded":"2024-09-17T15:37:15.112656+0000","last_fullsized":"2024-09-17T15:37:15.112656+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28839,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.206983+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:37:42.206983+0000","last_peered":"2024-09-17T15:37:42.206983+0000","last_clean":"2024-09-17T15:37:42.206983+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:37:42.206983+0000","last_undegraded":"2024-09-17T15:37:42.206983+0000","last_fullsized":"2024-09-17T15:37:42.206983+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":109,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112066+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:37:15.112066+0000","last_peered":"2024-09-17T15:37:15.112066+0000","last_clean":"2024-09-17T15:37:15.112066+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:37:15.112066+0000","last_undegraded":"2024-09-17T15:37:15.112066+0000","last_fullsized":"2024-09-17T15:37:15.112066+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29177,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.206850+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:37:42.206850+0000","last_peered":"2024-09-17T15:37:42.206850+0000","last_clean":"2024-09-17T15:37:42.206850+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:37:42.206850+0000","last_undegraded":"2024-09-17T15:37:42.206850+0000","last_fullsized":"2024-09-17T15:37:42.206850+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":28554,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:39.416590+0000","last_change":"2024-09-17T15:37:13.729028+0000","last_active":"2024-09-17T15:37:39.416590+0000","last_peered":"2024-09-17T15:37:39.416590+0000","last_clean":"2024-09-17T15:26:20.167381+0000","last_became_active":"2024-09-17T15:26:22.740839+0000","last_became_peered":"2024-09-17T15:26:22.740839+0000","last_unstale":"2024-09-17T15:37:39.416590+0000","last_undegraded":"2024-09-17T15:26:22.316891+0000","last_fullsized":"2024-09-17T15:26:22.194685+0000","mapping_epoch":40,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":429,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3710,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-429}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112352+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:37:15.112352+0000","last_peered":"2024-09-17T15:37:15.112352+0000","last_clean":"2024-09-17T15:37:15.112352+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:37:15.112352+0000","last_undegraded":"2024-09-17T15:37:15.112352+0000","last_fullsized":"2024-09-17T15:37:15.112352+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28086,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112447+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:37:15.112447+0000","last_peered":"2024-09-17T15:37:15.112447+0000","last_clean":"2024-09-17T15:37:15.112447+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:37:15.112447+0000","last_undegraded":"2024-09-17T15:37:15.112447+0000","last_fullsized":"2024-09-17T15:37:15.112447+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":105,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207240+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:37:42.207240+0000","last_peered":"2024-09-17T15:37:42.207240+0000","last_clean":"2024-09-17T15:37:42.207240+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:37:42.207240+0000","last_undegraded":"2024-09-17T15:37:42.207240+0000","last_fullsized":"2024-09-17T15:37:42.207240+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21215,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:15.113213+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:37:15.113213+0000","last_peered":"2024-09-17T15:37:15.113213+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:37:15.113213+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112357+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:37:15.112357+0000","last_peered":"2024-09-17T15:37:15.112357+0000","last_clean":"2024-09-17T15:37:15.112357+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:37:15.112357+0000","last_undegraded":"2024-09-17T15:37:15.112357+0000","last_fullsized":"2024-09-17T15:37:15.112357+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":104,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112592+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:37:15.112592+0000","last_peered":"2024-09-17T15:37:15.112592+0000","last_clean":"2024-09-17T15:37:15.112592+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:37:15.112592+0000","last_undegraded":"2024-09-17T15:37:15.112592+0000","last_fullsized":"2024-09-17T15:37:15.112592+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29038,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T15:37:15.112255+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:37:15.112255+0000","last_peered":"2024-09-17T15:37:15.112255+0000","last_clean":"2024-09-17T15:37:15.112255+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:37:15.112255+0000","last_undegraded":"2024-09-17T15:37:15.112255+0000","last_fullsized":"2024-09-17T15:37:15.112255+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37120,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":177815552,"data_stored":177710775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051736,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461248,"kb_used_data":108840,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93910592,"statfs":{"total":96636764160,"available":96164446208,"internally_reserved":0,"allocated":111452160,"data_stored":111097804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724680,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497296,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874544,"statfs":{"total":96636764160,"available":96127533056,"internally_reserved":0,"allocated":524288,"data_stored":165817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":508685995},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247963,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539424,"kb_used_data":66312,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832416,"statfs":{"total":96636764160,"available":96084393984,"internally_reserved":0,"allocated":67903488,"data_stored":67548630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110346240,"data_stored":110317336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:37:47.293 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:37:47.597 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:47.598 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:37:47.611 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":453,"stamp":"2024-09-17T15:37:45.833736+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11374,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38087,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1506324,"kb_used_data":175696,"kb_used_omap":13,"kb_used_meta":1330546,"kb_avail":281609196,"statfs":{"total":289910292480,"available":288367816704,"internally_reserved":0,"allocated":179912704,"data_stored":178823417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1362480127},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1964,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001381"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":22008,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:44.418451+0000","last_change":"2024-09-17T15:37:41.185578+0000","last_active":"2024-09-17T15:37:44.418451+0000","last_peered":"2024-09-17T15:37:44.418451+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:37:44.418451+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":623,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3447}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21070,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207284+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:37:42.207284+0000","last_peered":"2024-09-17T15:37:42.207284+0000","last_clean":"2024-09-17T15:37:42.207284+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:37:42.207284+0000","last_undegraded":"2024-09-17T15:37:42.207284+0000","last_fullsized":"2024-09-17T15:37:42.207284+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21178,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204749+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:37:42.204749+0000","last_peered":"2024-09-17T15:37:42.204749+0000","last_clean":"2024-09-17T15:37:42.204749+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:37:42.204749+0000","last_undegraded":"2024-09-17T15:37:42.204749+0000","last_fullsized":"2024-09-17T15:37:42.204749+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21075,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207161+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:37:42.207161+0000","last_peered":"2024-09-17T15:37:42.207161+0000","last_clean":"2024-09-17T15:37:42.207161+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:37:42.207161+0000","last_undegraded":"2024-09-17T15:37:42.207161+0000","last_fullsized":"2024-09-17T15:37:42.207161+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20818,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207835+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:37:42.207835+0000","last_peered":"2024-09-17T15:37:42.207835+0000","last_clean":"2024-09-17T15:37:42.207835+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:37:42.207835+0000","last_undegraded":"2024-09-17T15:37:42.207835+0000","last_fullsized":"2024-09-17T15:37:42.207835+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21569,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204561+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:37:42.204561+0000","last_peered":"2024-09-17T15:37:42.204561+0000","last_clean":"2024-09-17T15:37:42.204561+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:37:42.204561+0000","last_undegraded":"2024-09-17T15:37:42.204561+0000","last_fullsized":"2024-09-17T15:37:42.204561+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21112,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204467+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:37:42.204467+0000","last_peered":"2024-09-17T15:37:42.204467+0000","last_clean":"2024-09-17T15:37:42.204467+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:37:42.204467+0000","last_undegraded":"2024-09-17T15:37:42.204467+0000","last_fullsized":"2024-09-17T15:37:42.204467+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21079,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207733+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:37:42.207733+0000","last_peered":"2024-09-17T15:37:42.207733+0000","last_clean":"2024-09-17T15:37:42.207733+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:37:42.207733+0000","last_undegraded":"2024-09-17T15:37:42.207733+0000","last_fullsized":"2024-09-17T15:37:42.207733+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204400+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:37:42.204400+0000","last_peered":"2024-09-17T15:37:42.204400+0000","last_clean":"2024-09-17T15:37:42.204400+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:37:42.204400+0000","last_undegraded":"2024-09-17T15:37:42.204400+0000","last_fullsized":"2024-09-17T15:37:42.204400+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29135,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204670+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:37:42.204670+0000","last_peered":"2024-09-17T15:37:42.204670+0000","last_clean":"2024-09-17T15:37:42.204670+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:37:42.204670+0000","last_undegraded":"2024-09-17T15:37:42.204670+0000","last_fullsized":"2024-09-17T15:37:42.204670+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.206976+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:37:42.206976+0000","last_peered":"2024-09-17T15:37:42.206976+0000","last_clean":"2024-09-17T15:37:42.206976+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:37:42.206976+0000","last_undegraded":"2024-09-17T15:37:42.206976+0000","last_fullsized":"2024-09-17T15:37:42.206976+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20842,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:42.208076+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:37:42.208076+0000","last_peered":"2024-09-17T15:37:42.208076+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:37:42.208076+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":139,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.208056+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:37:42.208056+0000","last_peered":"2024-09-17T15:37:42.208056+0000","last_clean":"2024-09-17T15:37:42.208056+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:37:42.208056+0000","last_undegraded":"2024-09-17T15:37:42.208056+0000","last_fullsized":"2024-09-17T15:37:42.208056+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28839,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.206983+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:37:42.206983+0000","last_peered":"2024-09-17T15:37:42.206983+0000","last_clean":"2024-09-17T15:37:42.206983+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:37:42.206983+0000","last_undegraded":"2024-09-17T15:37:42.206983+0000","last_fullsized":"2024-09-17T15:37:42.206983+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":113,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204822+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:37:42.204822+0000","last_peered":"2024-09-17T15:37:42.204822+0000","last_clean":"2024-09-17T15:37:42.204822+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:37:42.204822+0000","last_undegraded":"2024-09-17T15:37:42.204822+0000","last_fullsized":"2024-09-17T15:37:42.204822+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29177,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.206850+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:37:42.206850+0000","last_peered":"2024-09-17T15:37:42.206850+0000","last_clean":"2024-09-17T15:37:42.206850+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:37:42.206850+0000","last_undegraded":"2024-09-17T15:37:42.206850+0000","last_fullsized":"2024-09-17T15:37:42.206850+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21287,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.283243+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:37:42.283243+0000","last_peered":"2024-09-17T15:37:42.283243+0000","last_clean":"2024-09-17T15:37:42.283243+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:37:42.283243+0000","last_undegraded":"2024-09-17T15:37:42.283243+0000","last_fullsized":"2024-09-17T15:37:42.283243+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204756+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:37:42.204756+0000","last_peered":"2024-09-17T15:37:42.204756+0000","last_clean":"2024-09-17T15:37:42.204756+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:37:42.204756+0000","last_undegraded":"2024-09-17T15:37:42.204756+0000","last_fullsized":"2024-09-17T15:37:42.204756+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28090,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204934+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:37:42.204934+0000","last_peered":"2024-09-17T15:37:42.204934+0000","last_clean":"2024-09-17T15:37:42.204934+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:37:42.204934+0000","last_undegraded":"2024-09-17T15:37:42.204934+0000","last_fullsized":"2024-09-17T15:37:42.204934+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":105,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207240+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:37:42.207240+0000","last_peered":"2024-09-17T15:37:42.207240+0000","last_clean":"2024-09-17T15:37:42.207240+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:37:42.207240+0000","last_undegraded":"2024-09-17T15:37:42.207240+0000","last_fullsized":"2024-09-17T15:37:42.207240+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21219,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:42.207922+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:37:42.207922+0000","last_peered":"2024-09-17T15:37:42.207922+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:37:42.207922+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204900+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:37:42.204900+0000","last_peered":"2024-09-17T15:37:42.204900+0000","last_clean":"2024-09-17T15:37:42.204900+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:37:42.204900+0000","last_undegraded":"2024-09-17T15:37:42.204900+0000","last_fullsized":"2024-09-17T15:37:42.204900+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":108,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.205158+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:37:42.205158+0000","last_peered":"2024-09-17T15:37:42.205158+0000","last_clean":"2024-09-17T15:37:42.205158+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:37:42.205158+0000","last_undegraded":"2024-09-17T15:37:42.205158+0000","last_fullsized":"2024-09-17T15:37:42.205158+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29042,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204673+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:37:42.204673+0000","last_peered":"2024-09-17T15:37:42.204673+0000","last_clean":"2024-09-17T15:37:42.204673+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:37:42.204673+0000","last_undegraded":"2024-09-17T15:37:42.204673+0000","last_fullsized":"2024-09-17T15:37:42.204673+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11374,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38086,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":177815552,"data_stored":177710775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051736,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461248,"kb_used_data":108840,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93910592,"statfs":{"total":96636764160,"available":96164446208,"internally_reserved":0,"allocated":111452160,"data_stored":111097804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724681,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505636,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866204,"statfs":{"total":96636764160,"available":96118992896,"internally_reserved":0,"allocated":540672,"data_stored":171400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":517205675},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247964,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539440,"kb_used_data":66328,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832400,"statfs":{"total":96636764160,"available":96084377600,"internally_reserved":0,"allocated":67919872,"data_stored":67554213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110346240,"data_stored":110317336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:37:47.613 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:37:47.973 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:47.973 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:37:47.988 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":454,"stamp":"2024-09-17T15:37:47.834096+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11374,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38087,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1506324,"kb_used_data":175696,"kb_used_omap":13,"kb_used_meta":1330546,"kb_avail":281609196,"statfs":{"total":289910292480,"available":288367816704,"internally_reserved":0,"allocated":179912704,"data_stored":178823417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1362480127},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-966,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":966,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001423"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":22008,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:44.418451+0000","last_change":"2024-09-17T15:37:41.185578+0000","last_active":"2024-09-17T15:37:44.418451+0000","last_peered":"2024-09-17T15:37:44.418451+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:37:44.418451+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":623,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3447}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21070,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207284+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:37:42.207284+0000","last_peered":"2024-09-17T15:37:42.207284+0000","last_clean":"2024-09-17T15:37:42.207284+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:37:42.207284+0000","last_undegraded":"2024-09-17T15:37:42.207284+0000","last_fullsized":"2024-09-17T15:37:42.207284+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21178,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204749+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:37:42.204749+0000","last_peered":"2024-09-17T15:37:42.204749+0000","last_clean":"2024-09-17T15:37:42.204749+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:37:42.204749+0000","last_undegraded":"2024-09-17T15:37:42.204749+0000","last_fullsized":"2024-09-17T15:37:42.204749+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21075,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207161+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:37:42.207161+0000","last_peered":"2024-09-17T15:37:42.207161+0000","last_clean":"2024-09-17T15:37:42.207161+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:37:42.207161+0000","last_undegraded":"2024-09-17T15:37:42.207161+0000","last_fullsized":"2024-09-17T15:37:42.207161+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20818,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207835+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:37:42.207835+0000","last_peered":"2024-09-17T15:37:42.207835+0000","last_clean":"2024-09-17T15:37:42.207835+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:37:42.207835+0000","last_undegraded":"2024-09-17T15:37:42.207835+0000","last_fullsized":"2024-09-17T15:37:42.207835+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21569,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204561+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:37:42.204561+0000","last_peered":"2024-09-17T15:37:42.204561+0000","last_clean":"2024-09-17T15:37:42.204561+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:37:42.204561+0000","last_undegraded":"2024-09-17T15:37:42.204561+0000","last_fullsized":"2024-09-17T15:37:42.204561+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21112,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204467+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:37:42.204467+0000","last_peered":"2024-09-17T15:37:42.204467+0000","last_clean":"2024-09-17T15:37:42.204467+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:37:42.204467+0000","last_undegraded":"2024-09-17T15:37:42.204467+0000","last_fullsized":"2024-09-17T15:37:42.204467+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21079,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207733+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:37:42.207733+0000","last_peered":"2024-09-17T15:37:42.207733+0000","last_clean":"2024-09-17T15:37:42.207733+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:37:42.207733+0000","last_undegraded":"2024-09-17T15:37:42.207733+0000","last_fullsized":"2024-09-17T15:37:42.207733+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204400+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:37:42.204400+0000","last_peered":"2024-09-17T15:37:42.204400+0000","last_clean":"2024-09-17T15:37:42.204400+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:37:42.204400+0000","last_undegraded":"2024-09-17T15:37:42.204400+0000","last_fullsized":"2024-09-17T15:37:42.204400+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29135,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204670+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:37:42.204670+0000","last_peered":"2024-09-17T15:37:42.204670+0000","last_clean":"2024-09-17T15:37:42.204670+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:37:42.204670+0000","last_undegraded":"2024-09-17T15:37:42.204670+0000","last_fullsized":"2024-09-17T15:37:42.204670+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.206976+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:37:42.206976+0000","last_peered":"2024-09-17T15:37:42.206976+0000","last_clean":"2024-09-17T15:37:42.206976+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:37:42.206976+0000","last_undegraded":"2024-09-17T15:37:42.206976+0000","last_fullsized":"2024-09-17T15:37:42.206976+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20842,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:42.208076+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:37:42.208076+0000","last_peered":"2024-09-17T15:37:42.208076+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:37:42.208076+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":139,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.208056+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:37:42.208056+0000","last_peered":"2024-09-17T15:37:42.208056+0000","last_clean":"2024-09-17T15:37:42.208056+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:37:42.208056+0000","last_undegraded":"2024-09-17T15:37:42.208056+0000","last_fullsized":"2024-09-17T15:37:42.208056+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28839,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.206983+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:37:42.206983+0000","last_peered":"2024-09-17T15:37:42.206983+0000","last_clean":"2024-09-17T15:37:42.206983+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:37:42.206983+0000","last_undegraded":"2024-09-17T15:37:42.206983+0000","last_fullsized":"2024-09-17T15:37:42.206983+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":113,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204822+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:37:42.204822+0000","last_peered":"2024-09-17T15:37:42.204822+0000","last_clean":"2024-09-17T15:37:42.204822+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:37:42.204822+0000","last_undegraded":"2024-09-17T15:37:42.204822+0000","last_fullsized":"2024-09-17T15:37:42.204822+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29177,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.206850+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:37:42.206850+0000","last_peered":"2024-09-17T15:37:42.206850+0000","last_clean":"2024-09-17T15:37:42.206850+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:37:42.206850+0000","last_undegraded":"2024-09-17T15:37:42.206850+0000","last_fullsized":"2024-09-17T15:37:42.206850+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21287,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.283243+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:37:42.283243+0000","last_peered":"2024-09-17T15:37:42.283243+0000","last_clean":"2024-09-17T15:37:42.283243+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:37:42.283243+0000","last_undegraded":"2024-09-17T15:37:42.283243+0000","last_fullsized":"2024-09-17T15:37:42.283243+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204756+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:37:42.204756+0000","last_peered":"2024-09-17T15:37:42.204756+0000","last_clean":"2024-09-17T15:37:42.204756+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:37:42.204756+0000","last_undegraded":"2024-09-17T15:37:42.204756+0000","last_fullsized":"2024-09-17T15:37:42.204756+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28090,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204934+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:37:42.204934+0000","last_peered":"2024-09-17T15:37:42.204934+0000","last_clean":"2024-09-17T15:37:42.204934+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:37:42.204934+0000","last_undegraded":"2024-09-17T15:37:42.204934+0000","last_fullsized":"2024-09-17T15:37:42.204934+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":105,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207240+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:37:42.207240+0000","last_peered":"2024-09-17T15:37:42.207240+0000","last_clean":"2024-09-17T15:37:42.207240+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:37:42.207240+0000","last_undegraded":"2024-09-17T15:37:42.207240+0000","last_fullsized":"2024-09-17T15:37:42.207240+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21219,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:42.207922+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:37:42.207922+0000","last_peered":"2024-09-17T15:37:42.207922+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:37:42.207922+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204900+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:37:42.204900+0000","last_peered":"2024-09-17T15:37:42.204900+0000","last_clean":"2024-09-17T15:37:42.204900+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:37:42.204900+0000","last_undegraded":"2024-09-17T15:37:42.204900+0000","last_fullsized":"2024-09-17T15:37:42.204900+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":108,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.205158+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:37:42.205158+0000","last_peered":"2024-09-17T15:37:42.205158+0000","last_clean":"2024-09-17T15:37:42.205158+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:37:42.205158+0000","last_undegraded":"2024-09-17T15:37:42.205158+0000","last_fullsized":"2024-09-17T15:37:42.205158+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29042,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204673+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:37:42.204673+0000","last_peered":"2024-09-17T15:37:42.204673+0000","last_clean":"2024-09-17T15:37:42.204673+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:37:42.204673+0000","last_undegraded":"2024-09-17T15:37:42.204673+0000","last_fullsized":"2024-09-17T15:37:42.204673+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11374,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38086,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":177815552,"data_stored":177710775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051736,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461248,"kb_used_data":108840,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93910592,"statfs":{"total":96636764160,"available":96164446208,"internally_reserved":0,"allocated":111452160,"data_stored":111097804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724681,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505636,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866204,"statfs":{"total":96636764160,"available":96118992896,"internally_reserved":0,"allocated":540672,"data_stored":171400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":517205675},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247964,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539440,"kb_used_data":66328,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832400,"statfs":{"total":96636764160,"available":96084377600,"internally_reserved":0,"allocated":67919872,"data_stored":67554213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110346240,"data_stored":110317336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:37:47.989 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:37:47.989 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'20648", 'reported_seq': 22008, 'reported_epoch': 59, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:37:44.418451+0000', 'last_change': '2024-09-17T15:37:41.185578+0000', 'last_active': '2024-09-17T15:37:44.418451+0000', 'last_peered': '2024-09-17T15:37:44.418451+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:37:44.418451+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'10600", 'ondisk_log_start': "43'10600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20648, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 623, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3447}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:47.990 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:37:47.990 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20542", 'reported_seq': 20842, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:37:42.208076+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:37:42.208076+0000', 'last_peered': '2024-09-17T15:37:42.208076+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:37:42.208076+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17138, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3943}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:47.990 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:37:47.990 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'20886", 'reported_seq': 21219, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:37:42.207922+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:37:42.207922+0000', 'last_peered': '2024-09-17T15:37:42.207922+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:37:42.207922+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'10800", 'ondisk_log_start': "43'10800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17673, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3980}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:47.990 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:37:48.369 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:48.369 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 11374/24 objects degraded (47391.667%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":805,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1542475776,"bytes_avail":288367816704,"bytes_total":289910292480,"degraded_objects":11374,"degraded_total":24,"degraded_ratio":473.91666666666674,"recovering_objects_per_sec":120,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":966,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:37:48.383 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:37:48.383 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:37:48.698 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:48.698 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:37:48.711 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":454,"stamp":"2024-09-17T15:37:47.834096+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11374,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38087,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1506324,"kb_used_data":175696,"kb_used_omap":13,"kb_used_meta":1330546,"kb_avail":281609196,"statfs":{"total":289910292480,"available":288367816704,"internally_reserved":0,"allocated":179912704,"data_stored":178823417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1362480127},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-966,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":966,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001423"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":22008,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:44.418451+0000","last_change":"2024-09-17T15:37:41.185578+0000","last_active":"2024-09-17T15:37:44.418451+0000","last_peered":"2024-09-17T15:37:44.418451+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:37:44.418451+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":623,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3447}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21070,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207284+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:37:42.207284+0000","last_peered":"2024-09-17T15:37:42.207284+0000","last_clean":"2024-09-17T15:37:42.207284+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:37:42.207284+0000","last_undegraded":"2024-09-17T15:37:42.207284+0000","last_fullsized":"2024-09-17T15:37:42.207284+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21178,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204749+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:37:42.204749+0000","last_peered":"2024-09-17T15:37:42.204749+0000","last_clean":"2024-09-17T15:37:42.204749+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:37:42.204749+0000","last_undegraded":"2024-09-17T15:37:42.204749+0000","last_fullsized":"2024-09-17T15:37:42.204749+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21075,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207161+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:37:42.207161+0000","last_peered":"2024-09-17T15:37:42.207161+0000","last_clean":"2024-09-17T15:37:42.207161+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:37:42.207161+0000","last_undegraded":"2024-09-17T15:37:42.207161+0000","last_fullsized":"2024-09-17T15:37:42.207161+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20818,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207835+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:37:42.207835+0000","last_peered":"2024-09-17T15:37:42.207835+0000","last_clean":"2024-09-17T15:37:42.207835+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:37:42.207835+0000","last_undegraded":"2024-09-17T15:37:42.207835+0000","last_fullsized":"2024-09-17T15:37:42.207835+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21569,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204561+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:37:42.204561+0000","last_peered":"2024-09-17T15:37:42.204561+0000","last_clean":"2024-09-17T15:37:42.204561+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:37:42.204561+0000","last_undegraded":"2024-09-17T15:37:42.204561+0000","last_fullsized":"2024-09-17T15:37:42.204561+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21112,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204467+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:37:42.204467+0000","last_peered":"2024-09-17T15:37:42.204467+0000","last_clean":"2024-09-17T15:37:42.204467+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:37:42.204467+0000","last_undegraded":"2024-09-17T15:37:42.204467+0000","last_fullsized":"2024-09-17T15:37:42.204467+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21079,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207733+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:37:42.207733+0000","last_peered":"2024-09-17T15:37:42.207733+0000","last_clean":"2024-09-17T15:37:42.207733+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:37:42.207733+0000","last_undegraded":"2024-09-17T15:37:42.207733+0000","last_fullsized":"2024-09-17T15:37:42.207733+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204400+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:37:42.204400+0000","last_peered":"2024-09-17T15:37:42.204400+0000","last_clean":"2024-09-17T15:37:42.204400+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:37:42.204400+0000","last_undegraded":"2024-09-17T15:37:42.204400+0000","last_fullsized":"2024-09-17T15:37:42.204400+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29135,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204670+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:37:42.204670+0000","last_peered":"2024-09-17T15:37:42.204670+0000","last_clean":"2024-09-17T15:37:42.204670+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:37:42.204670+0000","last_undegraded":"2024-09-17T15:37:42.204670+0000","last_fullsized":"2024-09-17T15:37:42.204670+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.206976+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:37:42.206976+0000","last_peered":"2024-09-17T15:37:42.206976+0000","last_clean":"2024-09-17T15:37:42.206976+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:37:42.206976+0000","last_undegraded":"2024-09-17T15:37:42.206976+0000","last_fullsized":"2024-09-17T15:37:42.206976+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20842,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:42.208076+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:37:42.208076+0000","last_peered":"2024-09-17T15:37:42.208076+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:37:42.208076+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":139,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.208056+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:37:42.208056+0000","last_peered":"2024-09-17T15:37:42.208056+0000","last_clean":"2024-09-17T15:37:42.208056+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:37:42.208056+0000","last_undegraded":"2024-09-17T15:37:42.208056+0000","last_fullsized":"2024-09-17T15:37:42.208056+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28839,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.206983+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:37:42.206983+0000","last_peered":"2024-09-17T15:37:42.206983+0000","last_clean":"2024-09-17T15:37:42.206983+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:37:42.206983+0000","last_undegraded":"2024-09-17T15:37:42.206983+0000","last_fullsized":"2024-09-17T15:37:42.206983+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":113,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204822+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:37:42.204822+0000","last_peered":"2024-09-17T15:37:42.204822+0000","last_clean":"2024-09-17T15:37:42.204822+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:37:42.204822+0000","last_undegraded":"2024-09-17T15:37:42.204822+0000","last_fullsized":"2024-09-17T15:37:42.204822+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29177,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.206850+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:37:42.206850+0000","last_peered":"2024-09-17T15:37:42.206850+0000","last_clean":"2024-09-17T15:37:42.206850+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:37:42.206850+0000","last_undegraded":"2024-09-17T15:37:42.206850+0000","last_fullsized":"2024-09-17T15:37:42.206850+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21287,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.283243+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:37:42.283243+0000","last_peered":"2024-09-17T15:37:42.283243+0000","last_clean":"2024-09-17T15:37:42.283243+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:37:42.283243+0000","last_undegraded":"2024-09-17T15:37:42.283243+0000","last_fullsized":"2024-09-17T15:37:42.283243+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204756+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:37:42.204756+0000","last_peered":"2024-09-17T15:37:42.204756+0000","last_clean":"2024-09-17T15:37:42.204756+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:37:42.204756+0000","last_undegraded":"2024-09-17T15:37:42.204756+0000","last_fullsized":"2024-09-17T15:37:42.204756+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28090,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204934+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:37:42.204934+0000","last_peered":"2024-09-17T15:37:42.204934+0000","last_clean":"2024-09-17T15:37:42.204934+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:37:42.204934+0000","last_undegraded":"2024-09-17T15:37:42.204934+0000","last_fullsized":"2024-09-17T15:37:42.204934+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":105,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207240+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:37:42.207240+0000","last_peered":"2024-09-17T15:37:42.207240+0000","last_clean":"2024-09-17T15:37:42.207240+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:37:42.207240+0000","last_undegraded":"2024-09-17T15:37:42.207240+0000","last_fullsized":"2024-09-17T15:37:42.207240+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21219,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:42.207922+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:37:42.207922+0000","last_peered":"2024-09-17T15:37:42.207922+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:37:42.207922+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204900+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:37:42.204900+0000","last_peered":"2024-09-17T15:37:42.204900+0000","last_clean":"2024-09-17T15:37:42.204900+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:37:42.204900+0000","last_undegraded":"2024-09-17T15:37:42.204900+0000","last_fullsized":"2024-09-17T15:37:42.204900+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":108,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.205158+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:37:42.205158+0000","last_peered":"2024-09-17T15:37:42.205158+0000","last_clean":"2024-09-17T15:37:42.205158+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:37:42.205158+0000","last_undegraded":"2024-09-17T15:37:42.205158+0000","last_fullsized":"2024-09-17T15:37:42.205158+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29042,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204673+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:37:42.204673+0000","last_peered":"2024-09-17T15:37:42.204673+0000","last_clean":"2024-09-17T15:37:42.204673+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:37:42.204673+0000","last_undegraded":"2024-09-17T15:37:42.204673+0000","last_fullsized":"2024-09-17T15:37:42.204673+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11374,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38086,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":177815552,"data_stored":177710775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051736,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461248,"kb_used_data":108840,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93910592,"statfs":{"total":96636764160,"available":96164446208,"internally_reserved":0,"allocated":111452160,"data_stored":111097804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724681,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505636,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866204,"statfs":{"total":96636764160,"available":96118992896,"internally_reserved":0,"allocated":540672,"data_stored":171400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":517205675},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247964,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539440,"kb_used_data":66328,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832400,"statfs":{"total":96636764160,"available":96084377600,"internally_reserved":0,"allocated":67919872,"data_stored":67554213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110346240,"data_stored":110317336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:37:51.713 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:37:52.021 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:52.022 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:37:52.034 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":456,"stamp":"2024-09-17T15:37:51.834781+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10731,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38730,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1507472,"kb_used_data":172696,"kb_used_omap":13,"kb_used_meta":1334706,"kb_avail":281608048,"statfs":{"total":289910292480,"available":288366641152,"internally_reserved":0,"allocated":176840704,"data_stored":175755030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1366739967},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1609,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1609,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001766"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":23294,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:49.419753+0000","last_change":"2024-09-17T15:37:41.185578+0000","last_active":"2024-09-17T15:37:49.419753+0000","last_peered":"2024-09-17T15:37:49.419753+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:37:49.419753+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2804,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1266,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2804}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21070,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207284+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:37:42.207284+0000","last_peered":"2024-09-17T15:37:42.207284+0000","last_clean":"2024-09-17T15:37:42.207284+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:37:42.207284+0000","last_undegraded":"2024-09-17T15:37:42.207284+0000","last_fullsized":"2024-09-17T15:37:42.207284+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21178,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204749+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:37:42.204749+0000","last_peered":"2024-09-17T15:37:42.204749+0000","last_clean":"2024-09-17T15:37:42.204749+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:37:42.204749+0000","last_undegraded":"2024-09-17T15:37:42.204749+0000","last_fullsized":"2024-09-17T15:37:42.204749+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21075,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207161+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:37:42.207161+0000","last_peered":"2024-09-17T15:37:42.207161+0000","last_clean":"2024-09-17T15:37:42.207161+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:37:42.207161+0000","last_undegraded":"2024-09-17T15:37:42.207161+0000","last_fullsized":"2024-09-17T15:37:42.207161+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20818,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207835+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:37:42.207835+0000","last_peered":"2024-09-17T15:37:42.207835+0000","last_clean":"2024-09-17T15:37:42.207835+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:37:42.207835+0000","last_undegraded":"2024-09-17T15:37:42.207835+0000","last_fullsized":"2024-09-17T15:37:42.207835+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21569,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204561+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:37:42.204561+0000","last_peered":"2024-09-17T15:37:42.204561+0000","last_clean":"2024-09-17T15:37:42.204561+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:37:42.204561+0000","last_undegraded":"2024-09-17T15:37:42.204561+0000","last_fullsized":"2024-09-17T15:37:42.204561+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21112,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204467+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:37:42.204467+0000","last_peered":"2024-09-17T15:37:42.204467+0000","last_clean":"2024-09-17T15:37:42.204467+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:37:42.204467+0000","last_undegraded":"2024-09-17T15:37:42.204467+0000","last_fullsized":"2024-09-17T15:37:42.204467+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21079,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207733+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:37:42.207733+0000","last_peered":"2024-09-17T15:37:42.207733+0000","last_clean":"2024-09-17T15:37:42.207733+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:37:42.207733+0000","last_undegraded":"2024-09-17T15:37:42.207733+0000","last_fullsized":"2024-09-17T15:37:42.207733+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204400+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:37:42.204400+0000","last_peered":"2024-09-17T15:37:42.204400+0000","last_clean":"2024-09-17T15:37:42.204400+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:37:42.204400+0000","last_undegraded":"2024-09-17T15:37:42.204400+0000","last_fullsized":"2024-09-17T15:37:42.204400+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29135,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204670+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:37:42.204670+0000","last_peered":"2024-09-17T15:37:42.204670+0000","last_clean":"2024-09-17T15:37:42.204670+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:37:42.204670+0000","last_undegraded":"2024-09-17T15:37:42.204670+0000","last_fullsized":"2024-09-17T15:37:42.204670+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.206976+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:37:42.206976+0000","last_peered":"2024-09-17T15:37:42.206976+0000","last_clean":"2024-09-17T15:37:42.206976+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:37:42.206976+0000","last_undegraded":"2024-09-17T15:37:42.206976+0000","last_fullsized":"2024-09-17T15:37:42.206976+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20842,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:42.208076+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:37:42.208076+0000","last_peered":"2024-09-17T15:37:42.208076+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:37:42.208076+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":139,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.208056+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:37:42.208056+0000","last_peered":"2024-09-17T15:37:42.208056+0000","last_clean":"2024-09-17T15:37:42.208056+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:37:42.208056+0000","last_undegraded":"2024-09-17T15:37:42.208056+0000","last_fullsized":"2024-09-17T15:37:42.208056+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28839,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.206983+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:37:42.206983+0000","last_peered":"2024-09-17T15:37:42.206983+0000","last_clean":"2024-09-17T15:37:42.206983+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:37:42.206983+0000","last_undegraded":"2024-09-17T15:37:42.206983+0000","last_fullsized":"2024-09-17T15:37:42.206983+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":113,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204822+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:37:42.204822+0000","last_peered":"2024-09-17T15:37:42.204822+0000","last_clean":"2024-09-17T15:37:42.204822+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:37:42.204822+0000","last_undegraded":"2024-09-17T15:37:42.204822+0000","last_fullsized":"2024-09-17T15:37:42.204822+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29177,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.206850+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:37:42.206850+0000","last_peered":"2024-09-17T15:37:42.206850+0000","last_clean":"2024-09-17T15:37:42.206850+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:37:42.206850+0000","last_undegraded":"2024-09-17T15:37:42.206850+0000","last_fullsized":"2024-09-17T15:37:42.206850+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21287,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.283243+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:37:42.283243+0000","last_peered":"2024-09-17T15:37:42.283243+0000","last_clean":"2024-09-17T15:37:42.283243+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:37:42.283243+0000","last_undegraded":"2024-09-17T15:37:42.283243+0000","last_fullsized":"2024-09-17T15:37:42.283243+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204756+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:37:42.204756+0000","last_peered":"2024-09-17T15:37:42.204756+0000","last_clean":"2024-09-17T15:37:42.204756+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:37:42.204756+0000","last_undegraded":"2024-09-17T15:37:42.204756+0000","last_fullsized":"2024-09-17T15:37:42.204756+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28091,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204968+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:37:42.204968+0000","last_peered":"2024-09-17T15:37:42.204968+0000","last_clean":"2024-09-17T15:37:42.204968+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:37:42.204968+0000","last_undegraded":"2024-09-17T15:37:42.204968+0000","last_fullsized":"2024-09-17T15:37:42.204968+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":105,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207240+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:37:42.207240+0000","last_peered":"2024-09-17T15:37:42.207240+0000","last_clean":"2024-09-17T15:37:42.207240+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:37:42.207240+0000","last_undegraded":"2024-09-17T15:37:42.207240+0000","last_fullsized":"2024-09-17T15:37:42.207240+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21220,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:42.208909+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:37:42.208909+0000","last_peered":"2024-09-17T15:37:42.208909+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:37:42.208909+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204900+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:37:42.204900+0000","last_peered":"2024-09-17T15:37:42.204900+0000","last_clean":"2024-09-17T15:37:42.204900+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:37:42.204900+0000","last_undegraded":"2024-09-17T15:37:42.204900+0000","last_fullsized":"2024-09-17T15:37:42.204900+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":108,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.205158+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:37:42.205158+0000","last_peered":"2024-09-17T15:37:42.205158+0000","last_clean":"2024-09-17T15:37:42.205158+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:37:42.205158+0000","last_undegraded":"2024-09-17T15:37:42.205158+0000","last_fullsized":"2024-09-17T15:37:42.205158+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29042,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204673+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:37:42.204673+0000","last_peered":"2024-09-17T15:37:42.204673+0000","last_clean":"2024-09-17T15:37:42.204673+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:37:42.204673+0000","last_undegraded":"2024-09-17T15:37:42.204673+0000","last_fullsized":"2024-09-17T15:37:42.204673+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10731,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38729,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174743552,"data_stored":174642388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051737,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":458240,"kb_used_data":105840,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93913600,"statfs":{"total":96636764160,"available":96167526400,"internally_reserved":0,"allocated":108380160,"data_stored":108029417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724682,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509792,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862048,"statfs":{"total":96636764160,"available":96114737152,"internally_reserved":0,"allocated":540672,"data_stored":171400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":521465515},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247965,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539440,"kb_used_data":66328,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832400,"statfs":{"total":96636764160,"available":96084377600,"internally_reserved":0,"allocated":67919872,"data_stored":67554213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":107274240,"data_stored":107248949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:37:52.035 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:37:52.356 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:52.356 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:37:52.369 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":456,"stamp":"2024-09-17T15:37:51.834781+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10731,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38730,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1507472,"kb_used_data":172696,"kb_used_omap":13,"kb_used_meta":1334706,"kb_avail":281608048,"statfs":{"total":289910292480,"available":288366641152,"internally_reserved":0,"allocated":176840704,"data_stored":175755030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1366739967},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1609,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1609,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001766"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":23294,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:49.419753+0000","last_change":"2024-09-17T15:37:41.185578+0000","last_active":"2024-09-17T15:37:49.419753+0000","last_peered":"2024-09-17T15:37:49.419753+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:37:49.419753+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2804,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1266,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2804}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21070,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207284+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:37:42.207284+0000","last_peered":"2024-09-17T15:37:42.207284+0000","last_clean":"2024-09-17T15:37:42.207284+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:37:42.207284+0000","last_undegraded":"2024-09-17T15:37:42.207284+0000","last_fullsized":"2024-09-17T15:37:42.207284+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21178,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204749+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:37:42.204749+0000","last_peered":"2024-09-17T15:37:42.204749+0000","last_clean":"2024-09-17T15:37:42.204749+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:37:42.204749+0000","last_undegraded":"2024-09-17T15:37:42.204749+0000","last_fullsized":"2024-09-17T15:37:42.204749+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21075,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207161+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:37:42.207161+0000","last_peered":"2024-09-17T15:37:42.207161+0000","last_clean":"2024-09-17T15:37:42.207161+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:37:42.207161+0000","last_undegraded":"2024-09-17T15:37:42.207161+0000","last_fullsized":"2024-09-17T15:37:42.207161+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20818,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207835+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:37:42.207835+0000","last_peered":"2024-09-17T15:37:42.207835+0000","last_clean":"2024-09-17T15:37:42.207835+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:37:42.207835+0000","last_undegraded":"2024-09-17T15:37:42.207835+0000","last_fullsized":"2024-09-17T15:37:42.207835+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21569,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204561+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:37:42.204561+0000","last_peered":"2024-09-17T15:37:42.204561+0000","last_clean":"2024-09-17T15:37:42.204561+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:37:42.204561+0000","last_undegraded":"2024-09-17T15:37:42.204561+0000","last_fullsized":"2024-09-17T15:37:42.204561+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21112,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204467+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:37:42.204467+0000","last_peered":"2024-09-17T15:37:42.204467+0000","last_clean":"2024-09-17T15:37:42.204467+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:37:42.204467+0000","last_undegraded":"2024-09-17T15:37:42.204467+0000","last_fullsized":"2024-09-17T15:37:42.204467+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21079,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207733+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:37:42.207733+0000","last_peered":"2024-09-17T15:37:42.207733+0000","last_clean":"2024-09-17T15:37:42.207733+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:37:42.207733+0000","last_undegraded":"2024-09-17T15:37:42.207733+0000","last_fullsized":"2024-09-17T15:37:42.207733+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204400+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:37:42.204400+0000","last_peered":"2024-09-17T15:37:42.204400+0000","last_clean":"2024-09-17T15:37:42.204400+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:37:42.204400+0000","last_undegraded":"2024-09-17T15:37:42.204400+0000","last_fullsized":"2024-09-17T15:37:42.204400+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29135,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204670+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:37:42.204670+0000","last_peered":"2024-09-17T15:37:42.204670+0000","last_clean":"2024-09-17T15:37:42.204670+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:37:42.204670+0000","last_undegraded":"2024-09-17T15:37:42.204670+0000","last_fullsized":"2024-09-17T15:37:42.204670+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.206976+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:37:42.206976+0000","last_peered":"2024-09-17T15:37:42.206976+0000","last_clean":"2024-09-17T15:37:42.206976+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:37:42.206976+0000","last_undegraded":"2024-09-17T15:37:42.206976+0000","last_fullsized":"2024-09-17T15:37:42.206976+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20842,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:42.208076+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:37:42.208076+0000","last_peered":"2024-09-17T15:37:42.208076+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:37:42.208076+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":139,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.208056+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:37:42.208056+0000","last_peered":"2024-09-17T15:37:42.208056+0000","last_clean":"2024-09-17T15:37:42.208056+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:37:42.208056+0000","last_undegraded":"2024-09-17T15:37:42.208056+0000","last_fullsized":"2024-09-17T15:37:42.208056+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28839,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.206983+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:37:42.206983+0000","last_peered":"2024-09-17T15:37:42.206983+0000","last_clean":"2024-09-17T15:37:42.206983+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:37:42.206983+0000","last_undegraded":"2024-09-17T15:37:42.206983+0000","last_fullsized":"2024-09-17T15:37:42.206983+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":113,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204822+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:37:42.204822+0000","last_peered":"2024-09-17T15:37:42.204822+0000","last_clean":"2024-09-17T15:37:42.204822+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:37:42.204822+0000","last_undegraded":"2024-09-17T15:37:42.204822+0000","last_fullsized":"2024-09-17T15:37:42.204822+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29177,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.206850+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:37:42.206850+0000","last_peered":"2024-09-17T15:37:42.206850+0000","last_clean":"2024-09-17T15:37:42.206850+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:37:42.206850+0000","last_undegraded":"2024-09-17T15:37:42.206850+0000","last_fullsized":"2024-09-17T15:37:42.206850+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21287,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.283243+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:37:42.283243+0000","last_peered":"2024-09-17T15:37:42.283243+0000","last_clean":"2024-09-17T15:37:42.283243+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:37:42.283243+0000","last_undegraded":"2024-09-17T15:37:42.283243+0000","last_fullsized":"2024-09-17T15:37:42.283243+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204756+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:37:42.204756+0000","last_peered":"2024-09-17T15:37:42.204756+0000","last_clean":"2024-09-17T15:37:42.204756+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:37:42.204756+0000","last_undegraded":"2024-09-17T15:37:42.204756+0000","last_fullsized":"2024-09-17T15:37:42.204756+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28091,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204968+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:37:42.204968+0000","last_peered":"2024-09-17T15:37:42.204968+0000","last_clean":"2024-09-17T15:37:42.204968+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:37:42.204968+0000","last_undegraded":"2024-09-17T15:37:42.204968+0000","last_fullsized":"2024-09-17T15:37:42.204968+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":105,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207240+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:37:42.207240+0000","last_peered":"2024-09-17T15:37:42.207240+0000","last_clean":"2024-09-17T15:37:42.207240+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:37:42.207240+0000","last_undegraded":"2024-09-17T15:37:42.207240+0000","last_fullsized":"2024-09-17T15:37:42.207240+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21220,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:42.208909+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:37:42.208909+0000","last_peered":"2024-09-17T15:37:42.208909+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:37:42.208909+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204900+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:37:42.204900+0000","last_peered":"2024-09-17T15:37:42.204900+0000","last_clean":"2024-09-17T15:37:42.204900+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:37:42.204900+0000","last_undegraded":"2024-09-17T15:37:42.204900+0000","last_fullsized":"2024-09-17T15:37:42.204900+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":108,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.205158+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:37:42.205158+0000","last_peered":"2024-09-17T15:37:42.205158+0000","last_clean":"2024-09-17T15:37:42.205158+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:37:42.205158+0000","last_undegraded":"2024-09-17T15:37:42.205158+0000","last_fullsized":"2024-09-17T15:37:42.205158+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29042,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204673+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:37:42.204673+0000","last_peered":"2024-09-17T15:37:42.204673+0000","last_clean":"2024-09-17T15:37:42.204673+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:37:42.204673+0000","last_undegraded":"2024-09-17T15:37:42.204673+0000","last_fullsized":"2024-09-17T15:37:42.204673+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10731,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38729,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174743552,"data_stored":174642388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051737,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":458240,"kb_used_data":105840,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93913600,"statfs":{"total":96636764160,"available":96167526400,"internally_reserved":0,"allocated":108380160,"data_stored":108029417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724682,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509792,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862048,"statfs":{"total":96636764160,"available":96114737152,"internally_reserved":0,"allocated":540672,"data_stored":171400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":521465515},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247965,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539440,"kb_used_data":66328,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832400,"statfs":{"total":96636764160,"available":96084377600,"internally_reserved":0,"allocated":67919872,"data_stored":67554213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":107274240,"data_stored":107248949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:37:52.370 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:37:52.370 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'20648", 'reported_seq': 23294, 'reported_epoch': 59, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:37:49.419753+0000', 'last_change': '2024-09-17T15:37:41.185578+0000', 'last_active': '2024-09-17T15:37:49.419753+0000', 'last_peered': '2024-09-17T15:37:49.419753+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:37:49.419753+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'10600", 'ondisk_log_start': "43'10600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2804, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20648, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1266, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2804}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:52.370 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:37:52.370 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20542", 'reported_seq': 20842, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:37:42.208076+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:37:42.208076+0000', 'last_peered': '2024-09-17T15:37:42.208076+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:37:42.208076+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17138, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3943}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:52.370 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:37:52.370 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'20886", 'reported_seq': 21220, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:37:42.208909+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:37:42.208909+0000', 'last_peered': '2024-09-17T15:37:42.208909+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:37:42.208909+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'10800", 'ondisk_log_start': "43'10800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17673, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3980}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:52.371 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:37:52.827 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:52.827 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 10731/24 objects degraded (44712.500%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":809,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1543651328,"bytes_avail":288366641152,"bytes_total":289910292480,"degraded_objects":10731,"degraded_total":24,"degraded_ratio":447.125,"recovering_objects_per_sec":160,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1609,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:37:52.840 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:37:52.840 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:37:53.147 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:53.147 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:37:53.159 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":456,"stamp":"2024-09-17T15:37:51.834781+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10731,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38730,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1507472,"kb_used_data":172696,"kb_used_omap":13,"kb_used_meta":1334706,"kb_avail":281608048,"statfs":{"total":289910292480,"available":288366641152,"internally_reserved":0,"allocated":176840704,"data_stored":175755030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1366739967},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1609,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1609,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001766"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":23294,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:49.419753+0000","last_change":"2024-09-17T15:37:41.185578+0000","last_active":"2024-09-17T15:37:49.419753+0000","last_peered":"2024-09-17T15:37:49.419753+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:37:49.419753+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2804,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1266,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2804}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21070,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207284+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:37:42.207284+0000","last_peered":"2024-09-17T15:37:42.207284+0000","last_clean":"2024-09-17T15:37:42.207284+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:37:42.207284+0000","last_undegraded":"2024-09-17T15:37:42.207284+0000","last_fullsized":"2024-09-17T15:37:42.207284+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21178,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204749+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:37:42.204749+0000","last_peered":"2024-09-17T15:37:42.204749+0000","last_clean":"2024-09-17T15:37:42.204749+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:37:42.204749+0000","last_undegraded":"2024-09-17T15:37:42.204749+0000","last_fullsized":"2024-09-17T15:37:42.204749+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21075,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207161+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:37:42.207161+0000","last_peered":"2024-09-17T15:37:42.207161+0000","last_clean":"2024-09-17T15:37:42.207161+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:37:42.207161+0000","last_undegraded":"2024-09-17T15:37:42.207161+0000","last_fullsized":"2024-09-17T15:37:42.207161+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20818,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207835+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:37:42.207835+0000","last_peered":"2024-09-17T15:37:42.207835+0000","last_clean":"2024-09-17T15:37:42.207835+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:37:42.207835+0000","last_undegraded":"2024-09-17T15:37:42.207835+0000","last_fullsized":"2024-09-17T15:37:42.207835+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21569,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204561+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:37:42.204561+0000","last_peered":"2024-09-17T15:37:42.204561+0000","last_clean":"2024-09-17T15:37:42.204561+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:37:42.204561+0000","last_undegraded":"2024-09-17T15:37:42.204561+0000","last_fullsized":"2024-09-17T15:37:42.204561+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21112,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204467+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:37:42.204467+0000","last_peered":"2024-09-17T15:37:42.204467+0000","last_clean":"2024-09-17T15:37:42.204467+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:37:42.204467+0000","last_undegraded":"2024-09-17T15:37:42.204467+0000","last_fullsized":"2024-09-17T15:37:42.204467+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21079,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207733+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:37:42.207733+0000","last_peered":"2024-09-17T15:37:42.207733+0000","last_clean":"2024-09-17T15:37:42.207733+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:37:42.207733+0000","last_undegraded":"2024-09-17T15:37:42.207733+0000","last_fullsized":"2024-09-17T15:37:42.207733+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204400+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:37:42.204400+0000","last_peered":"2024-09-17T15:37:42.204400+0000","last_clean":"2024-09-17T15:37:42.204400+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:37:42.204400+0000","last_undegraded":"2024-09-17T15:37:42.204400+0000","last_fullsized":"2024-09-17T15:37:42.204400+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29135,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204670+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:37:42.204670+0000","last_peered":"2024-09-17T15:37:42.204670+0000","last_clean":"2024-09-17T15:37:42.204670+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:37:42.204670+0000","last_undegraded":"2024-09-17T15:37:42.204670+0000","last_fullsized":"2024-09-17T15:37:42.204670+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.206976+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:37:42.206976+0000","last_peered":"2024-09-17T15:37:42.206976+0000","last_clean":"2024-09-17T15:37:42.206976+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:37:42.206976+0000","last_undegraded":"2024-09-17T15:37:42.206976+0000","last_fullsized":"2024-09-17T15:37:42.206976+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20842,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:42.208076+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:37:42.208076+0000","last_peered":"2024-09-17T15:37:42.208076+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:37:42.208076+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":139,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.208056+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:37:42.208056+0000","last_peered":"2024-09-17T15:37:42.208056+0000","last_clean":"2024-09-17T15:37:42.208056+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:37:42.208056+0000","last_undegraded":"2024-09-17T15:37:42.208056+0000","last_fullsized":"2024-09-17T15:37:42.208056+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28839,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.206983+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:37:42.206983+0000","last_peered":"2024-09-17T15:37:42.206983+0000","last_clean":"2024-09-17T15:37:42.206983+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:37:42.206983+0000","last_undegraded":"2024-09-17T15:37:42.206983+0000","last_fullsized":"2024-09-17T15:37:42.206983+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":113,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204822+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:37:42.204822+0000","last_peered":"2024-09-17T15:37:42.204822+0000","last_clean":"2024-09-17T15:37:42.204822+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:37:42.204822+0000","last_undegraded":"2024-09-17T15:37:42.204822+0000","last_fullsized":"2024-09-17T15:37:42.204822+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29177,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.206850+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:37:42.206850+0000","last_peered":"2024-09-17T15:37:42.206850+0000","last_clean":"2024-09-17T15:37:42.206850+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:37:42.206850+0000","last_undegraded":"2024-09-17T15:37:42.206850+0000","last_fullsized":"2024-09-17T15:37:42.206850+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21287,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.283243+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:37:42.283243+0000","last_peered":"2024-09-17T15:37:42.283243+0000","last_clean":"2024-09-17T15:37:42.283243+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:37:42.283243+0000","last_undegraded":"2024-09-17T15:37:42.283243+0000","last_fullsized":"2024-09-17T15:37:42.283243+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204756+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:37:42.204756+0000","last_peered":"2024-09-17T15:37:42.204756+0000","last_clean":"2024-09-17T15:37:42.204756+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:37:42.204756+0000","last_undegraded":"2024-09-17T15:37:42.204756+0000","last_fullsized":"2024-09-17T15:37:42.204756+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28091,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204968+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:37:42.204968+0000","last_peered":"2024-09-17T15:37:42.204968+0000","last_clean":"2024-09-17T15:37:42.204968+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:37:42.204968+0000","last_undegraded":"2024-09-17T15:37:42.204968+0000","last_fullsized":"2024-09-17T15:37:42.204968+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":105,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207240+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:37:42.207240+0000","last_peered":"2024-09-17T15:37:42.207240+0000","last_clean":"2024-09-17T15:37:42.207240+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:37:42.207240+0000","last_undegraded":"2024-09-17T15:37:42.207240+0000","last_fullsized":"2024-09-17T15:37:42.207240+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21220,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:42.208909+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:37:42.208909+0000","last_peered":"2024-09-17T15:37:42.208909+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:37:42.208909+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204900+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:37:42.204900+0000","last_peered":"2024-09-17T15:37:42.204900+0000","last_clean":"2024-09-17T15:37:42.204900+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:37:42.204900+0000","last_undegraded":"2024-09-17T15:37:42.204900+0000","last_fullsized":"2024-09-17T15:37:42.204900+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":108,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.205158+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:37:42.205158+0000","last_peered":"2024-09-17T15:37:42.205158+0000","last_clean":"2024-09-17T15:37:42.205158+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:37:42.205158+0000","last_undegraded":"2024-09-17T15:37:42.205158+0000","last_fullsized":"2024-09-17T15:37:42.205158+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29042,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204673+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:37:42.204673+0000","last_peered":"2024-09-17T15:37:42.204673+0000","last_clean":"2024-09-17T15:37:42.204673+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:37:42.204673+0000","last_undegraded":"2024-09-17T15:37:42.204673+0000","last_fullsized":"2024-09-17T15:37:42.204673+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10731,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38729,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174743552,"data_stored":174642388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051737,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":458240,"kb_used_data":105840,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93913600,"statfs":{"total":96636764160,"available":96167526400,"internally_reserved":0,"allocated":108380160,"data_stored":108029417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724682,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509792,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862048,"statfs":{"total":96636764160,"available":96114737152,"internally_reserved":0,"allocated":540672,"data_stored":171400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":521465515},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247965,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539440,"kb_used_data":66328,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832400,"statfs":{"total":96636764160,"available":96084377600,"internally_reserved":0,"allocated":67919872,"data_stored":67554213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":107274240,"data_stored":107248949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:37:56.162 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:37:56.483 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:56.483 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:37:56.497 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":458,"stamp":"2024-09-17T15:37:55.835449+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10060,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39401,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1496152,"kb_used_data":169696,"kb_used_omap":13,"kb_used_meta":1326386,"kb_avail":281619368,"statfs":{"total":289910292480,"available":288378232832,"internally_reserved":0,"allocated":173768704,"data_stored":172683030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1358220287},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2280,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2280,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002123"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":24636,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:54.422597+0000","last_change":"2024-09-17T15:37:41.185578+0000","last_active":"2024-09-17T15:37:54.422597+0000","last_peered":"2024-09-17T15:37:54.422597+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:37:54.422597+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2133,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1937,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2133}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21070,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207284+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:37:42.207284+0000","last_peered":"2024-09-17T15:37:42.207284+0000","last_clean":"2024-09-17T15:37:42.207284+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:37:42.207284+0000","last_undegraded":"2024-09-17T15:37:42.207284+0000","last_fullsized":"2024-09-17T15:37:42.207284+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21178,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204749+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:37:42.204749+0000","last_peered":"2024-09-17T15:37:42.204749+0000","last_clean":"2024-09-17T15:37:42.204749+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:37:42.204749+0000","last_undegraded":"2024-09-17T15:37:42.204749+0000","last_fullsized":"2024-09-17T15:37:42.204749+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21075,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207161+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:37:42.207161+0000","last_peered":"2024-09-17T15:37:42.207161+0000","last_clean":"2024-09-17T15:37:42.207161+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:37:42.207161+0000","last_undegraded":"2024-09-17T15:37:42.207161+0000","last_fullsized":"2024-09-17T15:37:42.207161+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20818,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207835+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:37:42.207835+0000","last_peered":"2024-09-17T15:37:42.207835+0000","last_clean":"2024-09-17T15:37:42.207835+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:37:42.207835+0000","last_undegraded":"2024-09-17T15:37:42.207835+0000","last_fullsized":"2024-09-17T15:37:42.207835+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21569,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204561+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:37:42.204561+0000","last_peered":"2024-09-17T15:37:42.204561+0000","last_clean":"2024-09-17T15:37:42.204561+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:37:42.204561+0000","last_undegraded":"2024-09-17T15:37:42.204561+0000","last_fullsized":"2024-09-17T15:37:42.204561+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21112,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204467+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:37:42.204467+0000","last_peered":"2024-09-17T15:37:42.204467+0000","last_clean":"2024-09-17T15:37:42.204467+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:37:42.204467+0000","last_undegraded":"2024-09-17T15:37:42.204467+0000","last_fullsized":"2024-09-17T15:37:42.204467+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21079,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207733+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:37:42.207733+0000","last_peered":"2024-09-17T15:37:42.207733+0000","last_clean":"2024-09-17T15:37:42.207733+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:37:42.207733+0000","last_undegraded":"2024-09-17T15:37:42.207733+0000","last_fullsized":"2024-09-17T15:37:42.207733+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204400+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:37:42.204400+0000","last_peered":"2024-09-17T15:37:42.204400+0000","last_clean":"2024-09-17T15:37:42.204400+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:37:42.204400+0000","last_undegraded":"2024-09-17T15:37:42.204400+0000","last_fullsized":"2024-09-17T15:37:42.204400+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29135,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204670+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:37:42.204670+0000","last_peered":"2024-09-17T15:37:42.204670+0000","last_clean":"2024-09-17T15:37:42.204670+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:37:42.204670+0000","last_undegraded":"2024-09-17T15:37:42.204670+0000","last_fullsized":"2024-09-17T15:37:42.204670+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207019+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:37:42.207019+0000","last_peered":"2024-09-17T15:37:42.207019+0000","last_clean":"2024-09-17T15:37:42.207019+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:37:42.207019+0000","last_undegraded":"2024-09-17T15:37:42.207019+0000","last_fullsized":"2024-09-17T15:37:42.207019+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20842,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:42.208076+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:37:42.208076+0000","last_peered":"2024-09-17T15:37:42.208076+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:37:42.208076+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":139,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.208056+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:37:42.208056+0000","last_peered":"2024-09-17T15:37:42.208056+0000","last_clean":"2024-09-17T15:37:42.208056+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:37:42.208056+0000","last_undegraded":"2024-09-17T15:37:42.208056+0000","last_fullsized":"2024-09-17T15:37:42.208056+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28840,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207024+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:37:42.207024+0000","last_peered":"2024-09-17T15:37:42.207024+0000","last_clean":"2024-09-17T15:37:42.207024+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:37:42.207024+0000","last_undegraded":"2024-09-17T15:37:42.207024+0000","last_fullsized":"2024-09-17T15:37:42.207024+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":113,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204822+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:37:42.204822+0000","last_peered":"2024-09-17T15:37:42.204822+0000","last_clean":"2024-09-17T15:37:42.204822+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:37:42.204822+0000","last_undegraded":"2024-09-17T15:37:42.204822+0000","last_fullsized":"2024-09-17T15:37:42.204822+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29177,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.206850+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:37:42.206850+0000","last_peered":"2024-09-17T15:37:42.206850+0000","last_clean":"2024-09-17T15:37:42.206850+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:37:42.206850+0000","last_undegraded":"2024-09-17T15:37:42.206850+0000","last_fullsized":"2024-09-17T15:37:42.206850+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21287,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.283243+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:37:42.283243+0000","last_peered":"2024-09-17T15:37:42.283243+0000","last_clean":"2024-09-17T15:37:42.283243+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:37:42.283243+0000","last_undegraded":"2024-09-17T15:37:42.283243+0000","last_fullsized":"2024-09-17T15:37:42.283243+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204756+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:37:42.204756+0000","last_peered":"2024-09-17T15:37:42.204756+0000","last_clean":"2024-09-17T15:37:42.204756+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:37:42.204756+0000","last_undegraded":"2024-09-17T15:37:42.204756+0000","last_fullsized":"2024-09-17T15:37:42.204756+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28091,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204968+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:37:42.204968+0000","last_peered":"2024-09-17T15:37:42.204968+0000","last_clean":"2024-09-17T15:37:42.204968+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:37:42.204968+0000","last_undegraded":"2024-09-17T15:37:42.204968+0000","last_fullsized":"2024-09-17T15:37:42.204968+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207280+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:37:42.207280+0000","last_peered":"2024-09-17T15:37:42.207280+0000","last_clean":"2024-09-17T15:37:42.207280+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:37:42.207280+0000","last_undegraded":"2024-09-17T15:37:42.207280+0000","last_fullsized":"2024-09-17T15:37:42.207280+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21220,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:42.208909+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:37:42.208909+0000","last_peered":"2024-09-17T15:37:42.208909+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:37:42.208909+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204900+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:37:42.204900+0000","last_peered":"2024-09-17T15:37:42.204900+0000","last_clean":"2024-09-17T15:37:42.204900+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:37:42.204900+0000","last_undegraded":"2024-09-17T15:37:42.204900+0000","last_fullsized":"2024-09-17T15:37:42.204900+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":108,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.205158+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:37:42.205158+0000","last_peered":"2024-09-17T15:37:42.205158+0000","last_clean":"2024-09-17T15:37:42.205158+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:37:42.205158+0000","last_undegraded":"2024-09-17T15:37:42.205158+0000","last_fullsized":"2024-09-17T15:37:42.205158+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29042,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204673+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:37:42.204673+0000","last_peered":"2024-09-17T15:37:42.204673+0000","last_clean":"2024-09-17T15:37:42.204673+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:37:42.204673+0000","last_undegraded":"2024-09-17T15:37:42.204673+0000","last_fullsized":"2024-09-17T15:37:42.204673+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10060,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39400,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":171671552,"data_stored":171570388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051738,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":455240,"kb_used_data":102840,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93916600,"statfs":{"total":96636764160,"available":96170598400,"internally_reserved":0,"allocated":105308160,"data_stored":104957417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724683,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501472,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870368,"statfs":{"total":96636764160,"available":96123256832,"internally_reserved":0,"allocated":540672,"data_stored":171400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":512945835},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247966,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539440,"kb_used_data":66328,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832400,"statfs":{"total":96636764160,"available":96084377600,"internally_reserved":0,"allocated":67919872,"data_stored":67554213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":104202240,"data_stored":104176949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:37:56.499 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:37:56.835 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:56.836 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:37:56.848 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":458,"stamp":"2024-09-17T15:37:55.835449+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10060,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39401,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1496152,"kb_used_data":169696,"kb_used_omap":13,"kb_used_meta":1326386,"kb_avail":281619368,"statfs":{"total":289910292480,"available":288378232832,"internally_reserved":0,"allocated":173768704,"data_stored":172683030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1358220287},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2280,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2280,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002123"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":24636,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:54.422597+0000","last_change":"2024-09-17T15:37:41.185578+0000","last_active":"2024-09-17T15:37:54.422597+0000","last_peered":"2024-09-17T15:37:54.422597+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:37:54.422597+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2133,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1937,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2133}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21070,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207284+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:37:42.207284+0000","last_peered":"2024-09-17T15:37:42.207284+0000","last_clean":"2024-09-17T15:37:42.207284+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:37:42.207284+0000","last_undegraded":"2024-09-17T15:37:42.207284+0000","last_fullsized":"2024-09-17T15:37:42.207284+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21178,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204749+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:37:42.204749+0000","last_peered":"2024-09-17T15:37:42.204749+0000","last_clean":"2024-09-17T15:37:42.204749+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:37:42.204749+0000","last_undegraded":"2024-09-17T15:37:42.204749+0000","last_fullsized":"2024-09-17T15:37:42.204749+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21075,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207161+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:37:42.207161+0000","last_peered":"2024-09-17T15:37:42.207161+0000","last_clean":"2024-09-17T15:37:42.207161+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:37:42.207161+0000","last_undegraded":"2024-09-17T15:37:42.207161+0000","last_fullsized":"2024-09-17T15:37:42.207161+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20818,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207835+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:37:42.207835+0000","last_peered":"2024-09-17T15:37:42.207835+0000","last_clean":"2024-09-17T15:37:42.207835+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:37:42.207835+0000","last_undegraded":"2024-09-17T15:37:42.207835+0000","last_fullsized":"2024-09-17T15:37:42.207835+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21569,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204561+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:37:42.204561+0000","last_peered":"2024-09-17T15:37:42.204561+0000","last_clean":"2024-09-17T15:37:42.204561+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:37:42.204561+0000","last_undegraded":"2024-09-17T15:37:42.204561+0000","last_fullsized":"2024-09-17T15:37:42.204561+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21112,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204467+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:37:42.204467+0000","last_peered":"2024-09-17T15:37:42.204467+0000","last_clean":"2024-09-17T15:37:42.204467+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:37:42.204467+0000","last_undegraded":"2024-09-17T15:37:42.204467+0000","last_fullsized":"2024-09-17T15:37:42.204467+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21079,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207733+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:37:42.207733+0000","last_peered":"2024-09-17T15:37:42.207733+0000","last_clean":"2024-09-17T15:37:42.207733+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:37:42.207733+0000","last_undegraded":"2024-09-17T15:37:42.207733+0000","last_fullsized":"2024-09-17T15:37:42.207733+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204400+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:37:42.204400+0000","last_peered":"2024-09-17T15:37:42.204400+0000","last_clean":"2024-09-17T15:37:42.204400+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:37:42.204400+0000","last_undegraded":"2024-09-17T15:37:42.204400+0000","last_fullsized":"2024-09-17T15:37:42.204400+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29135,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204670+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:37:42.204670+0000","last_peered":"2024-09-17T15:37:42.204670+0000","last_clean":"2024-09-17T15:37:42.204670+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:37:42.204670+0000","last_undegraded":"2024-09-17T15:37:42.204670+0000","last_fullsized":"2024-09-17T15:37:42.204670+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207019+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:37:42.207019+0000","last_peered":"2024-09-17T15:37:42.207019+0000","last_clean":"2024-09-17T15:37:42.207019+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:37:42.207019+0000","last_undegraded":"2024-09-17T15:37:42.207019+0000","last_fullsized":"2024-09-17T15:37:42.207019+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20842,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:42.208076+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:37:42.208076+0000","last_peered":"2024-09-17T15:37:42.208076+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:37:42.208076+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":139,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.208056+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:37:42.208056+0000","last_peered":"2024-09-17T15:37:42.208056+0000","last_clean":"2024-09-17T15:37:42.208056+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:37:42.208056+0000","last_undegraded":"2024-09-17T15:37:42.208056+0000","last_fullsized":"2024-09-17T15:37:42.208056+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28840,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207024+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:37:42.207024+0000","last_peered":"2024-09-17T15:37:42.207024+0000","last_clean":"2024-09-17T15:37:42.207024+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:37:42.207024+0000","last_undegraded":"2024-09-17T15:37:42.207024+0000","last_fullsized":"2024-09-17T15:37:42.207024+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":113,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204822+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:37:42.204822+0000","last_peered":"2024-09-17T15:37:42.204822+0000","last_clean":"2024-09-17T15:37:42.204822+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:37:42.204822+0000","last_undegraded":"2024-09-17T15:37:42.204822+0000","last_fullsized":"2024-09-17T15:37:42.204822+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29177,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.206850+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:37:42.206850+0000","last_peered":"2024-09-17T15:37:42.206850+0000","last_clean":"2024-09-17T15:37:42.206850+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:37:42.206850+0000","last_undegraded":"2024-09-17T15:37:42.206850+0000","last_fullsized":"2024-09-17T15:37:42.206850+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21287,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.283243+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:37:42.283243+0000","last_peered":"2024-09-17T15:37:42.283243+0000","last_clean":"2024-09-17T15:37:42.283243+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:37:42.283243+0000","last_undegraded":"2024-09-17T15:37:42.283243+0000","last_fullsized":"2024-09-17T15:37:42.283243+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204756+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:37:42.204756+0000","last_peered":"2024-09-17T15:37:42.204756+0000","last_clean":"2024-09-17T15:37:42.204756+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:37:42.204756+0000","last_undegraded":"2024-09-17T15:37:42.204756+0000","last_fullsized":"2024-09-17T15:37:42.204756+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28091,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204968+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:37:42.204968+0000","last_peered":"2024-09-17T15:37:42.204968+0000","last_clean":"2024-09-17T15:37:42.204968+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:37:42.204968+0000","last_undegraded":"2024-09-17T15:37:42.204968+0000","last_fullsized":"2024-09-17T15:37:42.204968+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207280+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:37:42.207280+0000","last_peered":"2024-09-17T15:37:42.207280+0000","last_clean":"2024-09-17T15:37:42.207280+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:37:42.207280+0000","last_undegraded":"2024-09-17T15:37:42.207280+0000","last_fullsized":"2024-09-17T15:37:42.207280+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21220,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:42.208909+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:37:42.208909+0000","last_peered":"2024-09-17T15:37:42.208909+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:37:42.208909+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204900+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:37:42.204900+0000","last_peered":"2024-09-17T15:37:42.204900+0000","last_clean":"2024-09-17T15:37:42.204900+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:37:42.204900+0000","last_undegraded":"2024-09-17T15:37:42.204900+0000","last_fullsized":"2024-09-17T15:37:42.204900+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":108,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.205158+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:37:42.205158+0000","last_peered":"2024-09-17T15:37:42.205158+0000","last_clean":"2024-09-17T15:37:42.205158+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:37:42.205158+0000","last_undegraded":"2024-09-17T15:37:42.205158+0000","last_fullsized":"2024-09-17T15:37:42.205158+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29042,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204673+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:37:42.204673+0000","last_peered":"2024-09-17T15:37:42.204673+0000","last_clean":"2024-09-17T15:37:42.204673+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:37:42.204673+0000","last_undegraded":"2024-09-17T15:37:42.204673+0000","last_fullsized":"2024-09-17T15:37:42.204673+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10060,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39400,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":171671552,"data_stored":171570388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051738,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":455240,"kb_used_data":102840,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93916600,"statfs":{"total":96636764160,"available":96170598400,"internally_reserved":0,"allocated":105308160,"data_stored":104957417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724683,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501472,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870368,"statfs":{"total":96636764160,"available":96123256832,"internally_reserved":0,"allocated":540672,"data_stored":171400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":512945835},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247966,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539440,"kb_used_data":66328,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832400,"statfs":{"total":96636764160,"available":96084377600,"internally_reserved":0,"allocated":67919872,"data_stored":67554213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":104202240,"data_stored":104176949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:37:56.849 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:37:56.850 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'20648", 'reported_seq': 24636, 'reported_epoch': 59, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:37:54.422597+0000', 'last_change': '2024-09-17T15:37:41.185578+0000', 'last_active': '2024-09-17T15:37:54.422597+0000', 'last_peered': '2024-09-17T15:37:54.422597+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:37:54.422597+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'10600", 'ondisk_log_start': "43'10600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2133, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20648, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1937, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2133}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:56.850 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:37:56.850 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20542", 'reported_seq': 20842, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:37:42.208076+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:37:42.208076+0000', 'last_peered': '2024-09-17T15:37:42.208076+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:37:42.208076+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17138, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3943}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:56.850 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:37:56.850 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'20886", 'reported_seq': 21220, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:37:42.208909+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:37:42.208909+0000', 'last_peered': '2024-09-17T15:37:42.208909+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:37:42.208909+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'10800", 'ondisk_log_start': "43'10800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17673, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3980}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:37:56.850 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:37:57.224 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:57.224 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 10060/24 objects degraded (41916.667%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":814,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1532059648,"bytes_avail":288378232832,"bytes_total":289910292480,"degraded_objects":10060,"degraded_total":24,"degraded_ratio":419.16666666666674,"recovering_objects_per_sec":189,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2280,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:37:57.237 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:37:57.237 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:37:57.543 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:37:57.544 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:37:57.557 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":458,"stamp":"2024-09-17T15:37:55.835449+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10060,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39401,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1496152,"kb_used_data":169696,"kb_used_omap":13,"kb_used_meta":1326386,"kb_avail":281619368,"statfs":{"total":289910292480,"available":288378232832,"internally_reserved":0,"allocated":173768704,"data_stored":172683030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1358220287},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2280,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2280,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002123"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":24636,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:54.422597+0000","last_change":"2024-09-17T15:37:41.185578+0000","last_active":"2024-09-17T15:37:54.422597+0000","last_peered":"2024-09-17T15:37:54.422597+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:37:54.422597+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2133,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1937,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2133}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21070,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207284+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:37:42.207284+0000","last_peered":"2024-09-17T15:37:42.207284+0000","last_clean":"2024-09-17T15:37:42.207284+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:37:42.207284+0000","last_undegraded":"2024-09-17T15:37:42.207284+0000","last_fullsized":"2024-09-17T15:37:42.207284+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21178,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204749+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:37:42.204749+0000","last_peered":"2024-09-17T15:37:42.204749+0000","last_clean":"2024-09-17T15:37:42.204749+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:37:42.204749+0000","last_undegraded":"2024-09-17T15:37:42.204749+0000","last_fullsized":"2024-09-17T15:37:42.204749+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21075,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207161+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:37:42.207161+0000","last_peered":"2024-09-17T15:37:42.207161+0000","last_clean":"2024-09-17T15:37:42.207161+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:37:42.207161+0000","last_undegraded":"2024-09-17T15:37:42.207161+0000","last_fullsized":"2024-09-17T15:37:42.207161+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20818,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207835+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:37:42.207835+0000","last_peered":"2024-09-17T15:37:42.207835+0000","last_clean":"2024-09-17T15:37:42.207835+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:37:42.207835+0000","last_undegraded":"2024-09-17T15:37:42.207835+0000","last_fullsized":"2024-09-17T15:37:42.207835+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21569,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204561+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:37:42.204561+0000","last_peered":"2024-09-17T15:37:42.204561+0000","last_clean":"2024-09-17T15:37:42.204561+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:37:42.204561+0000","last_undegraded":"2024-09-17T15:37:42.204561+0000","last_fullsized":"2024-09-17T15:37:42.204561+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21112,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204467+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:37:42.204467+0000","last_peered":"2024-09-17T15:37:42.204467+0000","last_clean":"2024-09-17T15:37:42.204467+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:37:42.204467+0000","last_undegraded":"2024-09-17T15:37:42.204467+0000","last_fullsized":"2024-09-17T15:37:42.204467+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21079,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207733+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:37:42.207733+0000","last_peered":"2024-09-17T15:37:42.207733+0000","last_clean":"2024-09-17T15:37:42.207733+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:37:42.207733+0000","last_undegraded":"2024-09-17T15:37:42.207733+0000","last_fullsized":"2024-09-17T15:37:42.207733+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204400+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:37:42.204400+0000","last_peered":"2024-09-17T15:37:42.204400+0000","last_clean":"2024-09-17T15:37:42.204400+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:37:42.204400+0000","last_undegraded":"2024-09-17T15:37:42.204400+0000","last_fullsized":"2024-09-17T15:37:42.204400+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29135,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204670+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:37:42.204670+0000","last_peered":"2024-09-17T15:37:42.204670+0000","last_clean":"2024-09-17T15:37:42.204670+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:37:42.204670+0000","last_undegraded":"2024-09-17T15:37:42.204670+0000","last_fullsized":"2024-09-17T15:37:42.204670+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207019+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:37:42.207019+0000","last_peered":"2024-09-17T15:37:42.207019+0000","last_clean":"2024-09-17T15:37:42.207019+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:37:42.207019+0000","last_undegraded":"2024-09-17T15:37:42.207019+0000","last_fullsized":"2024-09-17T15:37:42.207019+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20842,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:42.208076+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:37:42.208076+0000","last_peered":"2024-09-17T15:37:42.208076+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:37:42.208076+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":139,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.208056+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:37:42.208056+0000","last_peered":"2024-09-17T15:37:42.208056+0000","last_clean":"2024-09-17T15:37:42.208056+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:37:42.208056+0000","last_undegraded":"2024-09-17T15:37:42.208056+0000","last_fullsized":"2024-09-17T15:37:42.208056+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28840,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207024+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:37:42.207024+0000","last_peered":"2024-09-17T15:37:42.207024+0000","last_clean":"2024-09-17T15:37:42.207024+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:37:42.207024+0000","last_undegraded":"2024-09-17T15:37:42.207024+0000","last_fullsized":"2024-09-17T15:37:42.207024+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":113,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204822+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:37:42.204822+0000","last_peered":"2024-09-17T15:37:42.204822+0000","last_clean":"2024-09-17T15:37:42.204822+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:37:42.204822+0000","last_undegraded":"2024-09-17T15:37:42.204822+0000","last_fullsized":"2024-09-17T15:37:42.204822+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29177,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.206850+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:37:42.206850+0000","last_peered":"2024-09-17T15:37:42.206850+0000","last_clean":"2024-09-17T15:37:42.206850+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:37:42.206850+0000","last_undegraded":"2024-09-17T15:37:42.206850+0000","last_fullsized":"2024-09-17T15:37:42.206850+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21287,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.283243+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:37:42.283243+0000","last_peered":"2024-09-17T15:37:42.283243+0000","last_clean":"2024-09-17T15:37:42.283243+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:37:42.283243+0000","last_undegraded":"2024-09-17T15:37:42.283243+0000","last_fullsized":"2024-09-17T15:37:42.283243+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204756+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:37:42.204756+0000","last_peered":"2024-09-17T15:37:42.204756+0000","last_clean":"2024-09-17T15:37:42.204756+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:37:42.204756+0000","last_undegraded":"2024-09-17T15:37:42.204756+0000","last_fullsized":"2024-09-17T15:37:42.204756+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28091,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204968+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:37:42.204968+0000","last_peered":"2024-09-17T15:37:42.204968+0000","last_clean":"2024-09-17T15:37:42.204968+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:37:42.204968+0000","last_undegraded":"2024-09-17T15:37:42.204968+0000","last_fullsized":"2024-09-17T15:37:42.204968+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207280+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:37:42.207280+0000","last_peered":"2024-09-17T15:37:42.207280+0000","last_clean":"2024-09-17T15:37:42.207280+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:37:42.207280+0000","last_undegraded":"2024-09-17T15:37:42.207280+0000","last_fullsized":"2024-09-17T15:37:42.207280+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21220,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:42.208909+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:37:42.208909+0000","last_peered":"2024-09-17T15:37:42.208909+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:37:42.208909+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204900+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:37:42.204900+0000","last_peered":"2024-09-17T15:37:42.204900+0000","last_clean":"2024-09-17T15:37:42.204900+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:37:42.204900+0000","last_undegraded":"2024-09-17T15:37:42.204900+0000","last_fullsized":"2024-09-17T15:37:42.204900+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":108,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.205158+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:37:42.205158+0000","last_peered":"2024-09-17T15:37:42.205158+0000","last_clean":"2024-09-17T15:37:42.205158+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:37:42.205158+0000","last_undegraded":"2024-09-17T15:37:42.205158+0000","last_fullsized":"2024-09-17T15:37:42.205158+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29042,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204673+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:37:42.204673+0000","last_peered":"2024-09-17T15:37:42.204673+0000","last_clean":"2024-09-17T15:37:42.204673+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:37:42.204673+0000","last_undegraded":"2024-09-17T15:37:42.204673+0000","last_fullsized":"2024-09-17T15:37:42.204673+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10060,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39400,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":171671552,"data_stored":171570388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051738,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":455240,"kb_used_data":102840,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93916600,"statfs":{"total":96636764160,"available":96170598400,"internally_reserved":0,"allocated":105308160,"data_stored":104957417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724683,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501472,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870368,"statfs":{"total":96636764160,"available":96123256832,"internally_reserved":0,"allocated":540672,"data_stored":171400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":512945835},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247966,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539440,"kb_used_data":66328,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832400,"statfs":{"total":96636764160,"available":96084377600,"internally_reserved":0,"allocated":67919872,"data_stored":67554213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":104202240,"data_stored":104176949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:38:00.559 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:38:00.908 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:00.909 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:38:00.921 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":460,"stamp":"2024-09-17T15:37:59.836082+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9256,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40205,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1497312,"kb_used_data":166696,"kb_used_omap":13,"kb_used_meta":1330546,"kb_avail":281618208,"statfs":{"total":289910292480,"available":288377044992,"internally_reserved":0,"allocated":170696704,"data_stored":169614643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1362480127},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2118,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2118,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001985"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":26244,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:59.420063+0000","last_change":"2024-09-17T15:37:41.185578+0000","last_active":"2024-09-17T15:37:59.420063+0000","last_peered":"2024-09-17T15:37:59.420063+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:37:59.420063+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1329,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2741,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1329}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21070,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207284+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:37:42.207284+0000","last_peered":"2024-09-17T15:37:42.207284+0000","last_clean":"2024-09-17T15:37:42.207284+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:37:42.207284+0000","last_undegraded":"2024-09-17T15:37:42.207284+0000","last_fullsized":"2024-09-17T15:37:42.207284+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21178,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204749+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:37:42.204749+0000","last_peered":"2024-09-17T15:37:42.204749+0000","last_clean":"2024-09-17T15:37:42.204749+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:37:42.204749+0000","last_undegraded":"2024-09-17T15:37:42.204749+0000","last_fullsized":"2024-09-17T15:37:42.204749+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21075,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207161+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:37:42.207161+0000","last_peered":"2024-09-17T15:37:42.207161+0000","last_clean":"2024-09-17T15:37:42.207161+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:37:42.207161+0000","last_undegraded":"2024-09-17T15:37:42.207161+0000","last_fullsized":"2024-09-17T15:37:42.207161+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20818,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207835+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:37:42.207835+0000","last_peered":"2024-09-17T15:37:42.207835+0000","last_clean":"2024-09-17T15:37:42.207835+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:37:42.207835+0000","last_undegraded":"2024-09-17T15:37:42.207835+0000","last_fullsized":"2024-09-17T15:37:42.207835+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21569,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204561+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:37:42.204561+0000","last_peered":"2024-09-17T15:37:42.204561+0000","last_clean":"2024-09-17T15:37:42.204561+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:37:42.204561+0000","last_undegraded":"2024-09-17T15:37:42.204561+0000","last_fullsized":"2024-09-17T15:37:42.204561+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21112,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204467+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:37:42.204467+0000","last_peered":"2024-09-17T15:37:42.204467+0000","last_clean":"2024-09-17T15:37:42.204467+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:37:42.204467+0000","last_undegraded":"2024-09-17T15:37:42.204467+0000","last_fullsized":"2024-09-17T15:37:42.204467+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21079,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207733+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:37:42.207733+0000","last_peered":"2024-09-17T15:37:42.207733+0000","last_clean":"2024-09-17T15:37:42.207733+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:37:42.207733+0000","last_undegraded":"2024-09-17T15:37:42.207733+0000","last_fullsized":"2024-09-17T15:37:42.207733+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204400+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:37:42.204400+0000","last_peered":"2024-09-17T15:37:42.204400+0000","last_clean":"2024-09-17T15:37:42.204400+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:37:42.204400+0000","last_undegraded":"2024-09-17T15:37:42.204400+0000","last_fullsized":"2024-09-17T15:37:42.204400+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29135,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204670+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:37:42.204670+0000","last_peered":"2024-09-17T15:37:42.204670+0000","last_clean":"2024-09-17T15:37:42.204670+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:37:42.204670+0000","last_undegraded":"2024-09-17T15:37:42.204670+0000","last_fullsized":"2024-09-17T15:37:42.204670+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207019+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:37:42.207019+0000","last_peered":"2024-09-17T15:37:42.207019+0000","last_clean":"2024-09-17T15:37:42.207019+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:37:42.207019+0000","last_undegraded":"2024-09-17T15:37:42.207019+0000","last_fullsized":"2024-09-17T15:37:42.207019+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20842,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:42.208076+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:37:42.208076+0000","last_peered":"2024-09-17T15:37:42.208076+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:37:42.208076+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":139,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.208056+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:37:42.208056+0000","last_peered":"2024-09-17T15:37:42.208056+0000","last_clean":"2024-09-17T15:37:42.208056+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:37:42.208056+0000","last_undegraded":"2024-09-17T15:37:42.208056+0000","last_fullsized":"2024-09-17T15:37:42.208056+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28840,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207024+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:37:42.207024+0000","last_peered":"2024-09-17T15:37:42.207024+0000","last_clean":"2024-09-17T15:37:42.207024+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:37:42.207024+0000","last_undegraded":"2024-09-17T15:37:42.207024+0000","last_fullsized":"2024-09-17T15:37:42.207024+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":113,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204822+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:37:42.204822+0000","last_peered":"2024-09-17T15:37:42.204822+0000","last_clean":"2024-09-17T15:37:42.204822+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:37:42.204822+0000","last_undegraded":"2024-09-17T15:37:42.204822+0000","last_fullsized":"2024-09-17T15:37:42.204822+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29177,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.206850+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:37:42.206850+0000","last_peered":"2024-09-17T15:37:42.206850+0000","last_clean":"2024-09-17T15:37:42.206850+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:37:42.206850+0000","last_undegraded":"2024-09-17T15:37:42.206850+0000","last_fullsized":"2024-09-17T15:37:42.206850+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21287,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.283243+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:37:42.283243+0000","last_peered":"2024-09-17T15:37:42.283243+0000","last_clean":"2024-09-17T15:37:42.283243+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:37:42.283243+0000","last_undegraded":"2024-09-17T15:37:42.283243+0000","last_fullsized":"2024-09-17T15:37:42.283243+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204756+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:37:42.204756+0000","last_peered":"2024-09-17T15:37:42.204756+0000","last_clean":"2024-09-17T15:37:42.204756+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:37:42.204756+0000","last_undegraded":"2024-09-17T15:37:42.204756+0000","last_fullsized":"2024-09-17T15:37:42.204756+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28091,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204968+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:37:42.204968+0000","last_peered":"2024-09-17T15:37:42.204968+0000","last_clean":"2024-09-17T15:37:42.204968+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:37:42.204968+0000","last_undegraded":"2024-09-17T15:37:42.204968+0000","last_fullsized":"2024-09-17T15:37:42.204968+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207280+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:37:42.207280+0000","last_peered":"2024-09-17T15:37:42.207280+0000","last_clean":"2024-09-17T15:37:42.207280+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:37:42.207280+0000","last_undegraded":"2024-09-17T15:37:42.207280+0000","last_fullsized":"2024-09-17T15:37:42.207280+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21220,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:42.208909+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:37:42.208909+0000","last_peered":"2024-09-17T15:37:42.208909+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:37:42.208909+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204900+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:37:42.204900+0000","last_peered":"2024-09-17T15:37:42.204900+0000","last_clean":"2024-09-17T15:37:42.204900+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:37:42.204900+0000","last_undegraded":"2024-09-17T15:37:42.204900+0000","last_fullsized":"2024-09-17T15:37:42.204900+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":108,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.205158+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:37:42.205158+0000","last_peered":"2024-09-17T15:37:42.205158+0000","last_clean":"2024-09-17T15:37:42.205158+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:37:42.205158+0000","last_undegraded":"2024-09-17T15:37:42.205158+0000","last_fullsized":"2024-09-17T15:37:42.205158+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29042,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204673+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:37:42.204673+0000","last_peered":"2024-09-17T15:37:42.204673+0000","last_clean":"2024-09-17T15:37:42.204673+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:37:42.204673+0000","last_undegraded":"2024-09-17T15:37:42.204673+0000","last_fullsized":"2024-09-17T15:37:42.204673+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9256,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40204,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":168599552,"data_stored":168502001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051739,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":452240,"kb_used_data":99840,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93919600,"statfs":{"total":96636764160,"available":96173670400,"internally_reserved":0,"allocated":102236160,"data_stored":101889030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724684,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505632,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866208,"statfs":{"total":96636764160,"available":96118996992,"internally_reserved":0,"allocated":540672,"data_stored":171400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":517205675},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247967,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539440,"kb_used_data":66328,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832400,"statfs":{"total":96636764160,"available":96084377600,"internally_reserved":0,"allocated":67919872,"data_stored":67554213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":101130240,"data_stored":101108562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:38:00.923 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:38:01.238 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:01.238 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:38:01.254 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":460,"stamp":"2024-09-17T15:37:59.836082+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9256,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40205,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1497312,"kb_used_data":166696,"kb_used_omap":13,"kb_used_meta":1330546,"kb_avail":281618208,"statfs":{"total":289910292480,"available":288377044992,"internally_reserved":0,"allocated":170696704,"data_stored":169614643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1362480127},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2118,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2118,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001985"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":26244,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:59.420063+0000","last_change":"2024-09-17T15:37:41.185578+0000","last_active":"2024-09-17T15:37:59.420063+0000","last_peered":"2024-09-17T15:37:59.420063+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:37:59.420063+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1329,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2741,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1329}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21070,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207284+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:37:42.207284+0000","last_peered":"2024-09-17T15:37:42.207284+0000","last_clean":"2024-09-17T15:37:42.207284+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:37:42.207284+0000","last_undegraded":"2024-09-17T15:37:42.207284+0000","last_fullsized":"2024-09-17T15:37:42.207284+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21178,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204749+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:37:42.204749+0000","last_peered":"2024-09-17T15:37:42.204749+0000","last_clean":"2024-09-17T15:37:42.204749+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:37:42.204749+0000","last_undegraded":"2024-09-17T15:37:42.204749+0000","last_fullsized":"2024-09-17T15:37:42.204749+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21075,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207161+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:37:42.207161+0000","last_peered":"2024-09-17T15:37:42.207161+0000","last_clean":"2024-09-17T15:37:42.207161+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:37:42.207161+0000","last_undegraded":"2024-09-17T15:37:42.207161+0000","last_fullsized":"2024-09-17T15:37:42.207161+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20818,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207835+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:37:42.207835+0000","last_peered":"2024-09-17T15:37:42.207835+0000","last_clean":"2024-09-17T15:37:42.207835+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:37:42.207835+0000","last_undegraded":"2024-09-17T15:37:42.207835+0000","last_fullsized":"2024-09-17T15:37:42.207835+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21569,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204561+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:37:42.204561+0000","last_peered":"2024-09-17T15:37:42.204561+0000","last_clean":"2024-09-17T15:37:42.204561+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:37:42.204561+0000","last_undegraded":"2024-09-17T15:37:42.204561+0000","last_fullsized":"2024-09-17T15:37:42.204561+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21112,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204467+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:37:42.204467+0000","last_peered":"2024-09-17T15:37:42.204467+0000","last_clean":"2024-09-17T15:37:42.204467+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:37:42.204467+0000","last_undegraded":"2024-09-17T15:37:42.204467+0000","last_fullsized":"2024-09-17T15:37:42.204467+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21079,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207733+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:37:42.207733+0000","last_peered":"2024-09-17T15:37:42.207733+0000","last_clean":"2024-09-17T15:37:42.207733+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:37:42.207733+0000","last_undegraded":"2024-09-17T15:37:42.207733+0000","last_fullsized":"2024-09-17T15:37:42.207733+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204400+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:37:42.204400+0000","last_peered":"2024-09-17T15:37:42.204400+0000","last_clean":"2024-09-17T15:37:42.204400+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:37:42.204400+0000","last_undegraded":"2024-09-17T15:37:42.204400+0000","last_fullsized":"2024-09-17T15:37:42.204400+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29135,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204670+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:37:42.204670+0000","last_peered":"2024-09-17T15:37:42.204670+0000","last_clean":"2024-09-17T15:37:42.204670+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:37:42.204670+0000","last_undegraded":"2024-09-17T15:37:42.204670+0000","last_fullsized":"2024-09-17T15:37:42.204670+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207019+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:37:42.207019+0000","last_peered":"2024-09-17T15:37:42.207019+0000","last_clean":"2024-09-17T15:37:42.207019+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:37:42.207019+0000","last_undegraded":"2024-09-17T15:37:42.207019+0000","last_fullsized":"2024-09-17T15:37:42.207019+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20842,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:42.208076+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:37:42.208076+0000","last_peered":"2024-09-17T15:37:42.208076+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:37:42.208076+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":139,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.208056+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:37:42.208056+0000","last_peered":"2024-09-17T15:37:42.208056+0000","last_clean":"2024-09-17T15:37:42.208056+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:37:42.208056+0000","last_undegraded":"2024-09-17T15:37:42.208056+0000","last_fullsized":"2024-09-17T15:37:42.208056+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28840,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207024+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:37:42.207024+0000","last_peered":"2024-09-17T15:37:42.207024+0000","last_clean":"2024-09-17T15:37:42.207024+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:37:42.207024+0000","last_undegraded":"2024-09-17T15:37:42.207024+0000","last_fullsized":"2024-09-17T15:37:42.207024+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":113,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204822+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:37:42.204822+0000","last_peered":"2024-09-17T15:37:42.204822+0000","last_clean":"2024-09-17T15:37:42.204822+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:37:42.204822+0000","last_undegraded":"2024-09-17T15:37:42.204822+0000","last_fullsized":"2024-09-17T15:37:42.204822+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29177,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.206850+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:37:42.206850+0000","last_peered":"2024-09-17T15:37:42.206850+0000","last_clean":"2024-09-17T15:37:42.206850+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:37:42.206850+0000","last_undegraded":"2024-09-17T15:37:42.206850+0000","last_fullsized":"2024-09-17T15:37:42.206850+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21287,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.283243+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:37:42.283243+0000","last_peered":"2024-09-17T15:37:42.283243+0000","last_clean":"2024-09-17T15:37:42.283243+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:37:42.283243+0000","last_undegraded":"2024-09-17T15:37:42.283243+0000","last_fullsized":"2024-09-17T15:37:42.283243+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204756+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:37:42.204756+0000","last_peered":"2024-09-17T15:37:42.204756+0000","last_clean":"2024-09-17T15:37:42.204756+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:37:42.204756+0000","last_undegraded":"2024-09-17T15:37:42.204756+0000","last_fullsized":"2024-09-17T15:37:42.204756+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28091,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204968+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:37:42.204968+0000","last_peered":"2024-09-17T15:37:42.204968+0000","last_clean":"2024-09-17T15:37:42.204968+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:37:42.204968+0000","last_undegraded":"2024-09-17T15:37:42.204968+0000","last_fullsized":"2024-09-17T15:37:42.204968+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207280+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:37:42.207280+0000","last_peered":"2024-09-17T15:37:42.207280+0000","last_clean":"2024-09-17T15:37:42.207280+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:37:42.207280+0000","last_undegraded":"2024-09-17T15:37:42.207280+0000","last_fullsized":"2024-09-17T15:37:42.207280+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21220,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:42.208909+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:37:42.208909+0000","last_peered":"2024-09-17T15:37:42.208909+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:37:42.208909+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204900+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:37:42.204900+0000","last_peered":"2024-09-17T15:37:42.204900+0000","last_clean":"2024-09-17T15:37:42.204900+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:37:42.204900+0000","last_undegraded":"2024-09-17T15:37:42.204900+0000","last_fullsized":"2024-09-17T15:37:42.204900+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":108,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.205158+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:37:42.205158+0000","last_peered":"2024-09-17T15:37:42.205158+0000","last_clean":"2024-09-17T15:37:42.205158+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:37:42.205158+0000","last_undegraded":"2024-09-17T15:37:42.205158+0000","last_fullsized":"2024-09-17T15:37:42.205158+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29042,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204673+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:37:42.204673+0000","last_peered":"2024-09-17T15:37:42.204673+0000","last_clean":"2024-09-17T15:37:42.204673+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:37:42.204673+0000","last_undegraded":"2024-09-17T15:37:42.204673+0000","last_fullsized":"2024-09-17T15:37:42.204673+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9256,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40204,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":168599552,"data_stored":168502001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051739,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":452240,"kb_used_data":99840,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93919600,"statfs":{"total":96636764160,"available":96173670400,"internally_reserved":0,"allocated":102236160,"data_stored":101889030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724684,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505632,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866208,"statfs":{"total":96636764160,"available":96118996992,"internally_reserved":0,"allocated":540672,"data_stored":171400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":517205675},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247967,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539440,"kb_used_data":66328,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832400,"statfs":{"total":96636764160,"available":96084377600,"internally_reserved":0,"allocated":67919872,"data_stored":67554213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":101130240,"data_stored":101108562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:38:01.256 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:38:01.256 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'20648", 'reported_seq': 26244, 'reported_epoch': 59, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:37:59.420063+0000', 'last_change': '2024-09-17T15:37:41.185578+0000', 'last_active': '2024-09-17T15:37:59.420063+0000', 'last_peered': '2024-09-17T15:37:59.420063+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:37:59.420063+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'10600", 'ondisk_log_start': "43'10600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1329, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20648, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2741, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1329}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:38:01.256 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:38:01.256 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20542", 'reported_seq': 20842, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:37:42.208076+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:37:42.208076+0000', 'last_peered': '2024-09-17T15:37:42.208076+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:37:42.208076+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17138, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3943}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:38:01.256 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:38:01.256 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'20886", 'reported_seq': 21220, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:37:42.208909+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:37:42.208909+0000', 'last_peered': '2024-09-17T15:37:42.208909+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:37:42.208909+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'10800", 'ondisk_log_start': "43'10800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17673, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3980}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:38:01.257 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:38:01.650 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:01.650 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 9256/24 objects degraded (38566.667%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":818,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1533247488,"bytes_avail":288377044992,"bytes_total":289910292480,"degraded_objects":9256,"degraded_total":24,"degraded_ratio":385.66666666666674,"recovering_objects_per_sec":176,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2118,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:38:01.664 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:38:01.664 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:38:01.971 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:01.972 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:38:01.984 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":461,"stamp":"2024-09-17T15:38:01.836418+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9256,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40205,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1497312,"kb_used_data":166696,"kb_used_omap":13,"kb_used_meta":1330546,"kb_avail":281618208,"statfs":{"total":289910292480,"available":288377044992,"internally_reserved":0,"allocated":170696704,"data_stored":169614643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1362480127},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1475,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1475,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001994"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":26244,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:59.420063+0000","last_change":"2024-09-17T15:37:41.185578+0000","last_active":"2024-09-17T15:37:59.420063+0000","last_peered":"2024-09-17T15:37:59.420063+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:37:59.420063+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1329,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2741,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1329}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21070,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207284+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:37:42.207284+0000","last_peered":"2024-09-17T15:37:42.207284+0000","last_clean":"2024-09-17T15:37:42.207284+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:37:42.207284+0000","last_undegraded":"2024-09-17T15:37:42.207284+0000","last_fullsized":"2024-09-17T15:37:42.207284+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21178,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204749+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:37:42.204749+0000","last_peered":"2024-09-17T15:37:42.204749+0000","last_clean":"2024-09-17T15:37:42.204749+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:37:42.204749+0000","last_undegraded":"2024-09-17T15:37:42.204749+0000","last_fullsized":"2024-09-17T15:37:42.204749+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21075,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207161+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:37:42.207161+0000","last_peered":"2024-09-17T15:37:42.207161+0000","last_clean":"2024-09-17T15:37:42.207161+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:37:42.207161+0000","last_undegraded":"2024-09-17T15:37:42.207161+0000","last_fullsized":"2024-09-17T15:37:42.207161+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20818,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207835+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:37:42.207835+0000","last_peered":"2024-09-17T15:37:42.207835+0000","last_clean":"2024-09-17T15:37:42.207835+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:37:42.207835+0000","last_undegraded":"2024-09-17T15:37:42.207835+0000","last_fullsized":"2024-09-17T15:37:42.207835+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21569,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204561+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:37:42.204561+0000","last_peered":"2024-09-17T15:37:42.204561+0000","last_clean":"2024-09-17T15:37:42.204561+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:37:42.204561+0000","last_undegraded":"2024-09-17T15:37:42.204561+0000","last_fullsized":"2024-09-17T15:37:42.204561+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21112,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204467+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:37:42.204467+0000","last_peered":"2024-09-17T15:37:42.204467+0000","last_clean":"2024-09-17T15:37:42.204467+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:37:42.204467+0000","last_undegraded":"2024-09-17T15:37:42.204467+0000","last_fullsized":"2024-09-17T15:37:42.204467+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21079,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207733+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:37:42.207733+0000","last_peered":"2024-09-17T15:37:42.207733+0000","last_clean":"2024-09-17T15:37:42.207733+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:37:42.207733+0000","last_undegraded":"2024-09-17T15:37:42.207733+0000","last_fullsized":"2024-09-17T15:37:42.207733+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204400+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:37:42.204400+0000","last_peered":"2024-09-17T15:37:42.204400+0000","last_clean":"2024-09-17T15:37:42.204400+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:37:42.204400+0000","last_undegraded":"2024-09-17T15:37:42.204400+0000","last_fullsized":"2024-09-17T15:37:42.204400+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29135,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204670+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:37:42.204670+0000","last_peered":"2024-09-17T15:37:42.204670+0000","last_clean":"2024-09-17T15:37:42.204670+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:37:42.204670+0000","last_undegraded":"2024-09-17T15:37:42.204670+0000","last_fullsized":"2024-09-17T15:37:42.204670+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207019+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:37:42.207019+0000","last_peered":"2024-09-17T15:37:42.207019+0000","last_clean":"2024-09-17T15:37:42.207019+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:37:42.207019+0000","last_undegraded":"2024-09-17T15:37:42.207019+0000","last_fullsized":"2024-09-17T15:37:42.207019+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20842,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:42.208076+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:37:42.208076+0000","last_peered":"2024-09-17T15:37:42.208076+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:37:42.208076+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":139,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.208056+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:37:42.208056+0000","last_peered":"2024-09-17T15:37:42.208056+0000","last_clean":"2024-09-17T15:37:42.208056+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:37:42.208056+0000","last_undegraded":"2024-09-17T15:37:42.208056+0000","last_fullsized":"2024-09-17T15:37:42.208056+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28840,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207024+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:37:42.207024+0000","last_peered":"2024-09-17T15:37:42.207024+0000","last_clean":"2024-09-17T15:37:42.207024+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:37:42.207024+0000","last_undegraded":"2024-09-17T15:37:42.207024+0000","last_fullsized":"2024-09-17T15:37:42.207024+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":113,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204822+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:37:42.204822+0000","last_peered":"2024-09-17T15:37:42.204822+0000","last_clean":"2024-09-17T15:37:42.204822+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:37:42.204822+0000","last_undegraded":"2024-09-17T15:37:42.204822+0000","last_fullsized":"2024-09-17T15:37:42.204822+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29177,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.206850+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:37:42.206850+0000","last_peered":"2024-09-17T15:37:42.206850+0000","last_clean":"2024-09-17T15:37:42.206850+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:37:42.206850+0000","last_undegraded":"2024-09-17T15:37:42.206850+0000","last_fullsized":"2024-09-17T15:37:42.206850+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21287,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.283243+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:37:42.283243+0000","last_peered":"2024-09-17T15:37:42.283243+0000","last_clean":"2024-09-17T15:37:42.283243+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:37:42.283243+0000","last_undegraded":"2024-09-17T15:37:42.283243+0000","last_fullsized":"2024-09-17T15:37:42.283243+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204756+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:37:42.204756+0000","last_peered":"2024-09-17T15:37:42.204756+0000","last_clean":"2024-09-17T15:37:42.204756+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:37:42.204756+0000","last_undegraded":"2024-09-17T15:37:42.204756+0000","last_fullsized":"2024-09-17T15:37:42.204756+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28091,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204968+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:37:42.204968+0000","last_peered":"2024-09-17T15:37:42.204968+0000","last_clean":"2024-09-17T15:37:42.204968+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:37:42.204968+0000","last_undegraded":"2024-09-17T15:37:42.204968+0000","last_fullsized":"2024-09-17T15:37:42.204968+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207280+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:37:42.207280+0000","last_peered":"2024-09-17T15:37:42.207280+0000","last_clean":"2024-09-17T15:37:42.207280+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:37:42.207280+0000","last_undegraded":"2024-09-17T15:37:42.207280+0000","last_fullsized":"2024-09-17T15:37:42.207280+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21220,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:42.208909+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:37:42.208909+0000","last_peered":"2024-09-17T15:37:42.208909+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:37:42.208909+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204900+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:37:42.204900+0000","last_peered":"2024-09-17T15:37:42.204900+0000","last_clean":"2024-09-17T15:37:42.204900+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:37:42.204900+0000","last_undegraded":"2024-09-17T15:37:42.204900+0000","last_fullsized":"2024-09-17T15:37:42.204900+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":108,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.205158+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:37:42.205158+0000","last_peered":"2024-09-17T15:37:42.205158+0000","last_clean":"2024-09-17T15:37:42.205158+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:37:42.205158+0000","last_undegraded":"2024-09-17T15:37:42.205158+0000","last_fullsized":"2024-09-17T15:37:42.205158+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29042,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204673+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:37:42.204673+0000","last_peered":"2024-09-17T15:37:42.204673+0000","last_clean":"2024-09-17T15:37:42.204673+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:37:42.204673+0000","last_undegraded":"2024-09-17T15:37:42.204673+0000","last_fullsized":"2024-09-17T15:37:42.204673+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9256,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40204,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":168599552,"data_stored":168502001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051739,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":452240,"kb_used_data":99840,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93919600,"statfs":{"total":96636764160,"available":96173670400,"internally_reserved":0,"allocated":102236160,"data_stored":101889030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724684,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505632,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866208,"statfs":{"total":96636764160,"available":96118996992,"internally_reserved":0,"allocated":540672,"data_stored":171400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":517205675},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247967,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539440,"kb_used_data":66328,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832400,"statfs":{"total":96636764160,"available":96084377600,"internally_reserved":0,"allocated":67919872,"data_stored":67554213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":101130240,"data_stored":101108562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:38:04.987 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:38:05.324 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:05.324 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:38:05.338 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":463,"stamp":"2024-09-17T15:38:05.253875+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41312,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1485992,"kb_used_data":163696,"kb_used_omap":13,"kb_used_meta":1322226,"kb_avail":281629528,"statfs":{"total":289910292480,"available":288388636672,"internally_reserved":0,"allocated":167624704,"data_stored":166542643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1353960447},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2582,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2582,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.418792"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":28458,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:04.424194+0000","last_change":"2024-09-17T15:37:41.185578+0000","last_active":"2024-09-17T15:38:04.424194+0000","last_peered":"2024-09-17T15:38:04.424194+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:38:04.424194+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":222,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3848,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-222}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21070,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207284+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:37:42.207284+0000","last_peered":"2024-09-17T15:37:42.207284+0000","last_clean":"2024-09-17T15:37:42.207284+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:37:42.207284+0000","last_undegraded":"2024-09-17T15:37:42.207284+0000","last_fullsized":"2024-09-17T15:37:42.207284+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21178,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204749+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:37:42.204749+0000","last_peered":"2024-09-17T15:37:42.204749+0000","last_clean":"2024-09-17T15:37:42.204749+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:37:42.204749+0000","last_undegraded":"2024-09-17T15:37:42.204749+0000","last_fullsized":"2024-09-17T15:37:42.204749+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21075,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207161+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:37:42.207161+0000","last_peered":"2024-09-17T15:37:42.207161+0000","last_clean":"2024-09-17T15:37:42.207161+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:37:42.207161+0000","last_undegraded":"2024-09-17T15:37:42.207161+0000","last_fullsized":"2024-09-17T15:37:42.207161+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20818,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207835+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:37:42.207835+0000","last_peered":"2024-09-17T15:37:42.207835+0000","last_clean":"2024-09-17T15:37:42.207835+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:37:42.207835+0000","last_undegraded":"2024-09-17T15:37:42.207835+0000","last_fullsized":"2024-09-17T15:37:42.207835+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21569,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204561+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:37:42.204561+0000","last_peered":"2024-09-17T15:37:42.204561+0000","last_clean":"2024-09-17T15:37:42.204561+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:37:42.204561+0000","last_undegraded":"2024-09-17T15:37:42.204561+0000","last_fullsized":"2024-09-17T15:37:42.204561+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21112,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204467+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:37:42.204467+0000","last_peered":"2024-09-17T15:37:42.204467+0000","last_clean":"2024-09-17T15:37:42.204467+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:37:42.204467+0000","last_undegraded":"2024-09-17T15:37:42.204467+0000","last_fullsized":"2024-09-17T15:37:42.204467+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21079,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207733+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:37:42.207733+0000","last_peered":"2024-09-17T15:37:42.207733+0000","last_clean":"2024-09-17T15:37:42.207733+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:37:42.207733+0000","last_undegraded":"2024-09-17T15:37:42.207733+0000","last_fullsized":"2024-09-17T15:37:42.207733+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204400+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:37:42.204400+0000","last_peered":"2024-09-17T15:37:42.204400+0000","last_clean":"2024-09-17T15:37:42.204400+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:37:42.204400+0000","last_undegraded":"2024-09-17T15:37:42.204400+0000","last_fullsized":"2024-09-17T15:37:42.204400+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29135,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204670+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:37:42.204670+0000","last_peered":"2024-09-17T15:37:42.204670+0000","last_clean":"2024-09-17T15:37:42.204670+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:37:42.204670+0000","last_undegraded":"2024-09-17T15:37:42.204670+0000","last_fullsized":"2024-09-17T15:37:42.204670+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207019+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:37:42.207019+0000","last_peered":"2024-09-17T15:37:42.207019+0000","last_clean":"2024-09-17T15:37:42.207019+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:37:42.207019+0000","last_undegraded":"2024-09-17T15:37:42.207019+0000","last_fullsized":"2024-09-17T15:37:42.207019+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20842,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:42.208076+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:37:42.208076+0000","last_peered":"2024-09-17T15:37:42.208076+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:37:42.208076+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":139,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.208056+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:37:42.208056+0000","last_peered":"2024-09-17T15:37:42.208056+0000","last_clean":"2024-09-17T15:37:42.208056+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:37:42.208056+0000","last_undegraded":"2024-09-17T15:37:42.208056+0000","last_fullsized":"2024-09-17T15:37:42.208056+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28840,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207024+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:37:42.207024+0000","last_peered":"2024-09-17T15:37:42.207024+0000","last_clean":"2024-09-17T15:37:42.207024+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:37:42.207024+0000","last_undegraded":"2024-09-17T15:37:42.207024+0000","last_fullsized":"2024-09-17T15:37:42.207024+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":113,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204822+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:37:42.204822+0000","last_peered":"2024-09-17T15:37:42.204822+0000","last_clean":"2024-09-17T15:37:42.204822+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:37:42.204822+0000","last_undegraded":"2024-09-17T15:37:42.204822+0000","last_fullsized":"2024-09-17T15:37:42.204822+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29177,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.206850+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:37:42.206850+0000","last_peered":"2024-09-17T15:37:42.206850+0000","last_clean":"2024-09-17T15:37:42.206850+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:37:42.206850+0000","last_undegraded":"2024-09-17T15:37:42.206850+0000","last_fullsized":"2024-09-17T15:37:42.206850+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21287,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.283243+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:37:42.283243+0000","last_peered":"2024-09-17T15:37:42.283243+0000","last_clean":"2024-09-17T15:37:42.283243+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:37:42.283243+0000","last_undegraded":"2024-09-17T15:37:42.283243+0000","last_fullsized":"2024-09-17T15:37:42.283243+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204756+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:37:42.204756+0000","last_peered":"2024-09-17T15:37:42.204756+0000","last_clean":"2024-09-17T15:37:42.204756+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:37:42.204756+0000","last_undegraded":"2024-09-17T15:37:42.204756+0000","last_fullsized":"2024-09-17T15:37:42.204756+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28091,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204968+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:37:42.204968+0000","last_peered":"2024-09-17T15:37:42.204968+0000","last_clean":"2024-09-17T15:37:42.204968+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:37:42.204968+0000","last_undegraded":"2024-09-17T15:37:42.204968+0000","last_fullsized":"2024-09-17T15:37:42.204968+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207280+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:37:42.207280+0000","last_peered":"2024-09-17T15:37:42.207280+0000","last_clean":"2024-09-17T15:37:42.207280+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:37:42.207280+0000","last_undegraded":"2024-09-17T15:37:42.207280+0000","last_fullsized":"2024-09-17T15:37:42.207280+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21220,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:42.208909+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:37:42.208909+0000","last_peered":"2024-09-17T15:37:42.208909+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:37:42.208909+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204900+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:37:42.204900+0000","last_peered":"2024-09-17T15:37:42.204900+0000","last_clean":"2024-09-17T15:37:42.204900+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:37:42.204900+0000","last_undegraded":"2024-09-17T15:37:42.204900+0000","last_fullsized":"2024-09-17T15:37:42.204900+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":108,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.205158+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:37:42.205158+0000","last_peered":"2024-09-17T15:37:42.205158+0000","last_clean":"2024-09-17T15:37:42.205158+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:37:42.205158+0000","last_undegraded":"2024-09-17T15:37:42.205158+0000","last_fullsized":"2024-09-17T15:37:42.205158+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29042,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204673+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:37:42.204673+0000","last_peered":"2024-09-17T15:37:42.204673+0000","last_clean":"2024-09-17T15:37:42.204673+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:37:42.204673+0000","last_undegraded":"2024-09-17T15:37:42.204673+0000","last_fullsized":"2024-09-17T15:37:42.204673+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41311,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":165527552,"data_stored":165430001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051740,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":449240,"kb_used_data":96840,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93922600,"statfs":{"total":96636764160,"available":96176742400,"internally_reserved":0,"allocated":99164160,"data_stored":98817030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724685,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497312,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874528,"statfs":{"total":96636764160,"available":96127516672,"internally_reserved":0,"allocated":540672,"data_stored":171400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":508685995},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247968,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539440,"kb_used_data":66328,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832400,"statfs":{"total":96636764160,"available":96084377600,"internally_reserved":0,"allocated":67919872,"data_stored":67554213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98058240,"data_stored":98036562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:38:05.340 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:38:05.679 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:05.679 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:38:05.695 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":463,"stamp":"2024-09-17T15:38:05.253875+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41312,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1485992,"kb_used_data":163696,"kb_used_omap":13,"kb_used_meta":1322226,"kb_avail":281629528,"statfs":{"total":289910292480,"available":288388636672,"internally_reserved":0,"allocated":167624704,"data_stored":166542643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1353960447},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2582,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2582,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.418792"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":28458,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:04.424194+0000","last_change":"2024-09-17T15:37:41.185578+0000","last_active":"2024-09-17T15:38:04.424194+0000","last_peered":"2024-09-17T15:38:04.424194+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:38:04.424194+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":222,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3848,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-222}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21070,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207284+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:37:42.207284+0000","last_peered":"2024-09-17T15:37:42.207284+0000","last_clean":"2024-09-17T15:37:42.207284+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:37:42.207284+0000","last_undegraded":"2024-09-17T15:37:42.207284+0000","last_fullsized":"2024-09-17T15:37:42.207284+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21178,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204749+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:37:42.204749+0000","last_peered":"2024-09-17T15:37:42.204749+0000","last_clean":"2024-09-17T15:37:42.204749+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:37:42.204749+0000","last_undegraded":"2024-09-17T15:37:42.204749+0000","last_fullsized":"2024-09-17T15:37:42.204749+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21075,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207161+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:37:42.207161+0000","last_peered":"2024-09-17T15:37:42.207161+0000","last_clean":"2024-09-17T15:37:42.207161+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:37:42.207161+0000","last_undegraded":"2024-09-17T15:37:42.207161+0000","last_fullsized":"2024-09-17T15:37:42.207161+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20818,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207835+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:37:42.207835+0000","last_peered":"2024-09-17T15:37:42.207835+0000","last_clean":"2024-09-17T15:37:42.207835+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:37:42.207835+0000","last_undegraded":"2024-09-17T15:37:42.207835+0000","last_fullsized":"2024-09-17T15:37:42.207835+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21569,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204561+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:37:42.204561+0000","last_peered":"2024-09-17T15:37:42.204561+0000","last_clean":"2024-09-17T15:37:42.204561+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:37:42.204561+0000","last_undegraded":"2024-09-17T15:37:42.204561+0000","last_fullsized":"2024-09-17T15:37:42.204561+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21112,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204467+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:37:42.204467+0000","last_peered":"2024-09-17T15:37:42.204467+0000","last_clean":"2024-09-17T15:37:42.204467+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:37:42.204467+0000","last_undegraded":"2024-09-17T15:37:42.204467+0000","last_fullsized":"2024-09-17T15:37:42.204467+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21079,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207733+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:37:42.207733+0000","last_peered":"2024-09-17T15:37:42.207733+0000","last_clean":"2024-09-17T15:37:42.207733+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:37:42.207733+0000","last_undegraded":"2024-09-17T15:37:42.207733+0000","last_fullsized":"2024-09-17T15:37:42.207733+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204400+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:37:42.204400+0000","last_peered":"2024-09-17T15:37:42.204400+0000","last_clean":"2024-09-17T15:37:42.204400+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:37:42.204400+0000","last_undegraded":"2024-09-17T15:37:42.204400+0000","last_fullsized":"2024-09-17T15:37:42.204400+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29135,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204670+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:37:42.204670+0000","last_peered":"2024-09-17T15:37:42.204670+0000","last_clean":"2024-09-17T15:37:42.204670+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:37:42.204670+0000","last_undegraded":"2024-09-17T15:37:42.204670+0000","last_fullsized":"2024-09-17T15:37:42.204670+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207019+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:37:42.207019+0000","last_peered":"2024-09-17T15:37:42.207019+0000","last_clean":"2024-09-17T15:37:42.207019+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:37:42.207019+0000","last_undegraded":"2024-09-17T15:37:42.207019+0000","last_fullsized":"2024-09-17T15:37:42.207019+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20842,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:42.208076+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:37:42.208076+0000","last_peered":"2024-09-17T15:37:42.208076+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:37:42.208076+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":139,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.208056+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:37:42.208056+0000","last_peered":"2024-09-17T15:37:42.208056+0000","last_clean":"2024-09-17T15:37:42.208056+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:37:42.208056+0000","last_undegraded":"2024-09-17T15:37:42.208056+0000","last_fullsized":"2024-09-17T15:37:42.208056+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28840,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207024+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:37:42.207024+0000","last_peered":"2024-09-17T15:37:42.207024+0000","last_clean":"2024-09-17T15:37:42.207024+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:37:42.207024+0000","last_undegraded":"2024-09-17T15:37:42.207024+0000","last_fullsized":"2024-09-17T15:37:42.207024+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":113,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204822+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:37:42.204822+0000","last_peered":"2024-09-17T15:37:42.204822+0000","last_clean":"2024-09-17T15:37:42.204822+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:37:42.204822+0000","last_undegraded":"2024-09-17T15:37:42.204822+0000","last_fullsized":"2024-09-17T15:37:42.204822+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29177,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.206850+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:37:42.206850+0000","last_peered":"2024-09-17T15:37:42.206850+0000","last_clean":"2024-09-17T15:37:42.206850+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:37:42.206850+0000","last_undegraded":"2024-09-17T15:37:42.206850+0000","last_fullsized":"2024-09-17T15:37:42.206850+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21287,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.283243+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:37:42.283243+0000","last_peered":"2024-09-17T15:37:42.283243+0000","last_clean":"2024-09-17T15:37:42.283243+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:37:42.283243+0000","last_undegraded":"2024-09-17T15:37:42.283243+0000","last_fullsized":"2024-09-17T15:37:42.283243+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204756+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:37:42.204756+0000","last_peered":"2024-09-17T15:37:42.204756+0000","last_clean":"2024-09-17T15:37:42.204756+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:37:42.204756+0000","last_undegraded":"2024-09-17T15:37:42.204756+0000","last_fullsized":"2024-09-17T15:37:42.204756+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28091,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204968+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:37:42.204968+0000","last_peered":"2024-09-17T15:37:42.204968+0000","last_clean":"2024-09-17T15:37:42.204968+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:37:42.204968+0000","last_undegraded":"2024-09-17T15:37:42.204968+0000","last_fullsized":"2024-09-17T15:37:42.204968+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207280+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:37:42.207280+0000","last_peered":"2024-09-17T15:37:42.207280+0000","last_clean":"2024-09-17T15:37:42.207280+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:37:42.207280+0000","last_undegraded":"2024-09-17T15:37:42.207280+0000","last_fullsized":"2024-09-17T15:37:42.207280+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21220,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:42.208909+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:37:42.208909+0000","last_peered":"2024-09-17T15:37:42.208909+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:37:42.208909+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204900+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:37:42.204900+0000","last_peered":"2024-09-17T15:37:42.204900+0000","last_clean":"2024-09-17T15:37:42.204900+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:37:42.204900+0000","last_undegraded":"2024-09-17T15:37:42.204900+0000","last_fullsized":"2024-09-17T15:37:42.204900+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":108,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.205158+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:37:42.205158+0000","last_peered":"2024-09-17T15:37:42.205158+0000","last_clean":"2024-09-17T15:37:42.205158+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:37:42.205158+0000","last_undegraded":"2024-09-17T15:37:42.205158+0000","last_fullsized":"2024-09-17T15:37:42.205158+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29042,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204673+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:37:42.204673+0000","last_peered":"2024-09-17T15:37:42.204673+0000","last_clean":"2024-09-17T15:37:42.204673+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:37:42.204673+0000","last_undegraded":"2024-09-17T15:37:42.204673+0000","last_fullsized":"2024-09-17T15:37:42.204673+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41311,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":165527552,"data_stored":165430001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051740,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":449240,"kb_used_data":96840,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93922600,"statfs":{"total":96636764160,"available":96176742400,"internally_reserved":0,"allocated":99164160,"data_stored":98817030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724685,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497312,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874528,"statfs":{"total":96636764160,"available":96127516672,"internally_reserved":0,"allocated":540672,"data_stored":171400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":508685995},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247968,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539440,"kb_used_data":66328,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832400,"statfs":{"total":96636764160,"available":96084377600,"internally_reserved":0,"allocated":67919872,"data_stored":67554213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98058240,"data_stored":98036562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:38:05.696 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-17T15:38:05.696 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'20648", 'reported_seq': 28458, 'reported_epoch': 59, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:38:04.424194+0000', 'last_change': '2024-09-17T15:37:41.185578+0000', 'last_active': '2024-09-17T15:38:04.424194+0000', 'last_peered': '2024-09-17T15:38:04.424194+0000', 'last_clean': '2024-09-17T15:26:20.176124+0000', 'last_became_active': '2024-09-17T15:26:22.740530+0000', 'last_became_peered': '2024-09-17T15:26:22.740530+0000', 'last_unstale': '2024-09-17T15:38:04.424194+0000', 'last_undegraded': '2024-09-17T15:26:22.268637+0000', 'last_fullsized': '2024-09-17T15:26:22.194188+0000', 'mapping_epoch': 40, 'log_start': "43'10600", 'ondisk_log_start': "43'10600", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'objects_scrubbed': 0, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 222, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20648, 'num_write_kb': 41296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3848, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -222}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:38:05.697 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:38:05.697 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20542", 'reported_seq': 20842, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:37:42.208076+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:37:42.208076+0000', 'last_peered': '2024-09-17T15:37:42.208076+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:37:42.208076+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17138, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3943}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:38:05.697 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:38:05.697 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'20886", 'reported_seq': 21220, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:37:42.208909+0000', 'last_change': '2024-09-17T15:26:22.750947+0000', 'last_active': '2024-09-17T15:37:42.208909+0000', 'last_peered': '2024-09-17T15:37:42.208909+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:37:42.208909+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'10800", 'ondisk_log_start': "43'10800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17673, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3980}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:38:05.697 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:38:06.178 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:06.178 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 9256/24 objects degraded (38566.667%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":823,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":60,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1530175488,"bytes_avail":288380116992,"bytes_total":289910292480,"degraded_objects":9256,"degraded_total":24,"degraded_ratio":385.66666666666674,"recovering_objects_per_sec":122,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1475,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:38:06.193 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:38:06.194 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:38:06.543 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:06.543 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:38:06.558 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":465,"stamp":"2024-09-17T15:38:06.274018+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41312,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1485992,"kb_used_data":163696,"kb_used_omap":13,"kb_used_meta":1322226,"kb_avail":281629528,"statfs":{"total":289910292480,"available":288388636672,"internally_reserved":0,"allocated":167624704,"data_stored":166542643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1353960447},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1911,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1911,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.438229"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":28458,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:04.424194+0000","last_change":"2024-09-17T15:37:41.185578+0000","last_active":"2024-09-17T15:38:04.424194+0000","last_peered":"2024-09-17T15:38:04.424194+0000","last_clean":"2024-09-17T15:26:20.176124+0000","last_became_active":"2024-09-17T15:26:22.740530+0000","last_became_peered":"2024-09-17T15:26:22.740530+0000","last_unstale":"2024-09-17T15:38:04.424194+0000","last_undegraded":"2024-09-17T15:26:22.268637+0000","last_fullsized":"2024-09-17T15:26:22.194188+0000","mapping_epoch":40,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":222,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3848,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-222}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21070,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207284+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:37:42.207284+0000","last_peered":"2024-09-17T15:37:42.207284+0000","last_clean":"2024-09-17T15:37:42.207284+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:37:42.207284+0000","last_undegraded":"2024-09-17T15:37:42.207284+0000","last_fullsized":"2024-09-17T15:37:42.207284+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21178,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204749+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:37:42.204749+0000","last_peered":"2024-09-17T15:37:42.204749+0000","last_clean":"2024-09-17T15:37:42.204749+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:37:42.204749+0000","last_undegraded":"2024-09-17T15:37:42.204749+0000","last_fullsized":"2024-09-17T15:37:42.204749+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21075,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207161+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:37:42.207161+0000","last_peered":"2024-09-17T15:37:42.207161+0000","last_clean":"2024-09-17T15:37:42.207161+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:37:42.207161+0000","last_undegraded":"2024-09-17T15:37:42.207161+0000","last_fullsized":"2024-09-17T15:37:42.207161+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20818,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207835+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:37:42.207835+0000","last_peered":"2024-09-17T15:37:42.207835+0000","last_clean":"2024-09-17T15:37:42.207835+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:37:42.207835+0000","last_undegraded":"2024-09-17T15:37:42.207835+0000","last_fullsized":"2024-09-17T15:37:42.207835+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21569,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204561+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:37:42.204561+0000","last_peered":"2024-09-17T15:37:42.204561+0000","last_clean":"2024-09-17T15:37:42.204561+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:37:42.204561+0000","last_undegraded":"2024-09-17T15:37:42.204561+0000","last_fullsized":"2024-09-17T15:37:42.204561+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21112,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204467+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:37:42.204467+0000","last_peered":"2024-09-17T15:37:42.204467+0000","last_clean":"2024-09-17T15:37:42.204467+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:37:42.204467+0000","last_undegraded":"2024-09-17T15:37:42.204467+0000","last_fullsized":"2024-09-17T15:37:42.204467+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21079,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207733+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:37:42.207733+0000","last_peered":"2024-09-17T15:37:42.207733+0000","last_clean":"2024-09-17T15:37:42.207733+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:37:42.207733+0000","last_undegraded":"2024-09-17T15:37:42.207733+0000","last_fullsized":"2024-09-17T15:37:42.207733+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204400+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:37:42.204400+0000","last_peered":"2024-09-17T15:37:42.204400+0000","last_clean":"2024-09-17T15:37:42.204400+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:37:42.204400+0000","last_undegraded":"2024-09-17T15:37:42.204400+0000","last_fullsized":"2024-09-17T15:37:42.204400+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29135,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204670+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:37:42.204670+0000","last_peered":"2024-09-17T15:37:42.204670+0000","last_clean":"2024-09-17T15:37:42.204670+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:37:42.204670+0000","last_undegraded":"2024-09-17T15:37:42.204670+0000","last_fullsized":"2024-09-17T15:37:42.204670+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207019+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:37:42.207019+0000","last_peered":"2024-09-17T15:37:42.207019+0000","last_clean":"2024-09-17T15:37:42.207019+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:37:42.207019+0000","last_undegraded":"2024-09-17T15:37:42.207019+0000","last_fullsized":"2024-09-17T15:37:42.207019+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20842,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:42.208076+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:37:42.208076+0000","last_peered":"2024-09-17T15:37:42.208076+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:37:42.208076+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":139,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.208056+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:37:42.208056+0000","last_peered":"2024-09-17T15:37:42.208056+0000","last_clean":"2024-09-17T15:37:42.208056+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:37:42.208056+0000","last_undegraded":"2024-09-17T15:37:42.208056+0000","last_fullsized":"2024-09-17T15:37:42.208056+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28840,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207024+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:37:42.207024+0000","last_peered":"2024-09-17T15:37:42.207024+0000","last_clean":"2024-09-17T15:37:42.207024+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:37:42.207024+0000","last_undegraded":"2024-09-17T15:37:42.207024+0000","last_fullsized":"2024-09-17T15:37:42.207024+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":113,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204822+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:37:42.204822+0000","last_peered":"2024-09-17T15:37:42.204822+0000","last_clean":"2024-09-17T15:37:42.204822+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:37:42.204822+0000","last_undegraded":"2024-09-17T15:37:42.204822+0000","last_fullsized":"2024-09-17T15:37:42.204822+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29177,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.206850+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:37:42.206850+0000","last_peered":"2024-09-17T15:37:42.206850+0000","last_clean":"2024-09-17T15:37:42.206850+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:37:42.206850+0000","last_undegraded":"2024-09-17T15:37:42.206850+0000","last_fullsized":"2024-09-17T15:37:42.206850+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21287,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.283243+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:37:42.283243+0000","last_peered":"2024-09-17T15:37:42.283243+0000","last_clean":"2024-09-17T15:37:42.283243+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:37:42.283243+0000","last_undegraded":"2024-09-17T15:37:42.283243+0000","last_fullsized":"2024-09-17T15:37:42.283243+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204756+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:37:42.204756+0000","last_peered":"2024-09-17T15:37:42.204756+0000","last_clean":"2024-09-17T15:37:42.204756+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:37:42.204756+0000","last_undegraded":"2024-09-17T15:37:42.204756+0000","last_fullsized":"2024-09-17T15:37:42.204756+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28091,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204968+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:37:42.204968+0000","last_peered":"2024-09-17T15:37:42.204968+0000","last_clean":"2024-09-17T15:37:42.204968+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:37:42.204968+0000","last_undegraded":"2024-09-17T15:37:42.204968+0000","last_fullsized":"2024-09-17T15:37:42.204968+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.207280+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:37:42.207280+0000","last_peered":"2024-09-17T15:37:42.207280+0000","last_clean":"2024-09-17T15:37:42.207280+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:37:42.207280+0000","last_undegraded":"2024-09-17T15:37:42.207280+0000","last_fullsized":"2024-09-17T15:37:42.207280+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21220,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:37:42.208909+0000","last_change":"2024-09-17T15:26:22.750947+0000","last_active":"2024-09-17T15:37:42.208909+0000","last_peered":"2024-09-17T15:37:42.208909+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:37:42.208909+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204900+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:37:42.204900+0000","last_peered":"2024-09-17T15:37:42.204900+0000","last_clean":"2024-09-17T15:37:42.204900+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:37:42.204900+0000","last_undegraded":"2024-09-17T15:37:42.204900+0000","last_fullsized":"2024-09-17T15:37:42.204900+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":108,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.205158+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:37:42.205158+0000","last_peered":"2024-09-17T15:37:42.205158+0000","last_clean":"2024-09-17T15:37:42.205158+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:37:42.205158+0000","last_undegraded":"2024-09-17T15:37:42.205158+0000","last_fullsized":"2024-09-17T15:37:42.205158+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29042,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T15:37:42.204673+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:37:42.204673+0000","last_peered":"2024-09-17T15:37:42.204673+0000","last_clean":"2024-09-17T15:37:42.204673+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:37:42.204673+0000","last_undegraded":"2024-09-17T15:37:42.204673+0000","last_fullsized":"2024-09-17T15:37:42.204673+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41311,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":165527552,"data_stored":165430001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051740,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":449240,"kb_used_data":96840,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93922600,"statfs":{"total":96636764160,"available":96176742400,"internally_reserved":0,"allocated":99164160,"data_stored":98817030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724685,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497312,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874528,"statfs":{"total":96636764160,"available":96127516672,"internally_reserved":0,"allocated":540672,"data_stored":171400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":508685995},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247968,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539440,"kb_used_data":66328,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832400,"statfs":{"total":96636764160,"available":96084377600,"internally_reserved":0,"allocated":67919872,"data_stored":67554213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98058240,"data_stored":98036562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:38:09.560 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:38:09.876 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:09.877 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:38:09.891 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":467,"stamp":"2024-09-17T15:38:09.837717+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42253,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1491116,"kb_used_data":160500,"kb_used_omap":13,"kb_used_meta":1330546,"kb_avail":281624404,"statfs":{"total":289910292480,"available":288383389696,"internally_reserved":0,"allocated":164352000,"data_stored":163244612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1362480127},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2048,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001298"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":28913,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.362719+0000","last_change":"2024-09-17T15:38:06.362595+0000","last_active":"2024-09-17T15:38:06.362719+0000","last_peered":"2024-09-17T15:38:06.362719+0000","last_clean":"2024-09-17T15:38:06.362719+0000","last_became_active":"2024-09-17T15:38:06.284139+0000","last_became_peered":"2024-09-17T15:38:06.284139+0000","last_unstale":"2024-09-17T15:38:06.362719+0000","last_undegraded":"2024-09-17T15:38:06.362719+0000","last_fullsized":"2024-09-17T15:38:06.362719+0000","mapping_epoch":60,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:42:51.418256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4070,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21074,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276467+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:38:06.276467+0000","last_peered":"2024-09-17T15:38:06.276467+0000","last_clean":"2024-09-17T15:38:06.276467+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:38:06.276467+0000","last_undegraded":"2024-09-17T15:38:06.276467+0000","last_fullsized":"2024-09-17T15:38:06.276467+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21182,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277370+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:38:06.277370+0000","last_peered":"2024-09-17T15:38:06.277370+0000","last_clean":"2024-09-17T15:38:06.277370+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:38:06.277370+0000","last_undegraded":"2024-09-17T15:38:06.277370+0000","last_fullsized":"2024-09-17T15:38:06.277370+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21079,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276509+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:38:06.276509+0000","last_peered":"2024-09-17T15:38:06.276509+0000","last_clean":"2024-09-17T15:38:06.276509+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:38:06.276509+0000","last_undegraded":"2024-09-17T15:38:06.276509+0000","last_fullsized":"2024-09-17T15:38:06.276509+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20822,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.283149+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:38:06.283149+0000","last_peered":"2024-09-17T15:38:06.283149+0000","last_clean":"2024-09-17T15:38:06.283149+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:38:06.283149+0000","last_undegraded":"2024-09-17T15:38:06.283149+0000","last_fullsized":"2024-09-17T15:38:06.283149+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21573,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276969+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:38:06.276969+0000","last_peered":"2024-09-17T15:38:06.276969+0000","last_clean":"2024-09-17T15:38:06.276969+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:38:06.276969+0000","last_undegraded":"2024-09-17T15:38:06.276969+0000","last_fullsized":"2024-09-17T15:38:06.276969+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21116,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276860+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:38:06.276860+0000","last_peered":"2024-09-17T15:38:06.276860+0000","last_clean":"2024-09-17T15:38:06.276860+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:38:06.276860+0000","last_undegraded":"2024-09-17T15:38:06.276860+0000","last_fullsized":"2024-09-17T15:38:06.276860+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21084,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.281124+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:38:06.281124+0000","last_peered":"2024-09-17T15:38:06.281124+0000","last_clean":"2024-09-17T15:38:06.281124+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:38:06.281124+0000","last_undegraded":"2024-09-17T15:38:06.281124+0000","last_fullsized":"2024-09-17T15:38:06.281124+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":107,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276773+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:38:06.276773+0000","last_peered":"2024-09-17T15:38:06.276773+0000","last_clean":"2024-09-17T15:38:06.276773+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:38:06.276773+0000","last_undegraded":"2024-09-17T15:38:06.276773+0000","last_fullsized":"2024-09-17T15:38:06.276773+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29139,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277107+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:38:06.277107+0000","last_peered":"2024-09-17T15:38:06.277107+0000","last_clean":"2024-09-17T15:38:06.277107+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:38:06.277107+0000","last_undegraded":"2024-09-17T15:38:06.277107+0000","last_fullsized":"2024-09-17T15:38:06.277107+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.275989+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:38:06.275989+0000","last_peered":"2024-09-17T15:38:06.275989+0000","last_clean":"2024-09-17T15:38:06.275989+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:38:06.275989+0000","last_undegraded":"2024-09-17T15:38:06.275989+0000","last_fullsized":"2024-09-17T15:38:06.275989+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20846,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:06.281521+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:38:06.281521+0000","last_peered":"2024-09-17T15:38:06.281521+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:38:06.281521+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":143,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.281361+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:38:06.281361+0000","last_peered":"2024-09-17T15:38:06.281361+0000","last_clean":"2024-09-17T15:38:06.281361+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:38:06.281361+0000","last_undegraded":"2024-09-17T15:38:06.281361+0000","last_fullsized":"2024-09-17T15:38:06.281361+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28844,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276074+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:38:06.276074+0000","last_peered":"2024-09-17T15:38:06.276074+0000","last_clean":"2024-09-17T15:38:06.276074+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:38:06.276074+0000","last_undegraded":"2024-09-17T15:38:06.276074+0000","last_fullsized":"2024-09-17T15:38:06.276074+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":117,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277066+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:38:06.277066+0000","last_peered":"2024-09-17T15:38:06.277066+0000","last_clean":"2024-09-17T15:38:06.277066+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:38:06.277066+0000","last_undegraded":"2024-09-17T15:38:06.277066+0000","last_fullsized":"2024-09-17T15:38:06.277066+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29181,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.275684+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:38:06.275684+0000","last_peered":"2024-09-17T15:38:06.275684+0000","last_clean":"2024-09-17T15:38:06.275684+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:38:06.275684+0000","last_undegraded":"2024-09-17T15:38:06.275684+0000","last_fullsized":"2024-09-17T15:38:06.275684+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21291,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277656+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:38:06.277656+0000","last_peered":"2024-09-17T15:38:06.277656+0000","last_clean":"2024-09-17T15:38:06.277656+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:38:06.277656+0000","last_undegraded":"2024-09-17T15:38:06.277656+0000","last_fullsized":"2024-09-17T15:38:06.277656+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":107,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277209+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:38:06.277209+0000","last_peered":"2024-09-17T15:38:06.277209+0000","last_clean":"2024-09-17T15:38:06.277209+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:38:06.277209+0000","last_undegraded":"2024-09-17T15:38:06.277209+0000","last_fullsized":"2024-09-17T15:38:06.277209+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28095,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277618+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:38:06.277618+0000","last_peered":"2024-09-17T15:38:06.277618+0000","last_clean":"2024-09-17T15:38:06.277618+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:38:06.277618+0000","last_undegraded":"2024-09-17T15:38:06.277618+0000","last_fullsized":"2024-09-17T15:38:06.277618+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276036+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:38:06.276036+0000","last_peered":"2024-09-17T15:38:06.276036+0000","last_clean":"2024-09-17T15:38:06.276036+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:38:06.276036+0000","last_undegraded":"2024-09-17T15:38:06.276036+0000","last_fullsized":"2024-09-17T15:38:06.276036+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":22663,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:09.430915+0000","last_change":"2024-09-17T15:38:05.220032+0000","last_active":"2024-09-17T15:38:09.430915+0000","last_peered":"2024-09-17T15:38:09.430915+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:38:09.430915+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3263,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":802,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3263}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":107,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277568+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:38:06.277568+0000","last_peered":"2024-09-17T15:38:06.277568+0000","last_clean":"2024-09-17T15:38:06.277568+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:38:06.277568+0000","last_undegraded":"2024-09-17T15:38:06.277568+0000","last_fullsized":"2024-09-17T15:38:06.277568+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":112,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277607+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:38:06.277607+0000","last_peered":"2024-09-17T15:38:06.277607+0000","last_clean":"2024-09-17T15:38:06.277607+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:38:06.277607+0000","last_undegraded":"2024-09-17T15:38:06.277607+0000","last_fullsized":"2024-09-17T15:38:06.277607+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29046,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277163+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:38:06.277163+0000","last_peered":"2024-09-17T15:38:06.277163+0000","last_clean":"2024-09-17T15:38:06.277163+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:38:06.277163+0000","last_undegraded":"2024-09-17T15:38:06.277163+0000","last_fullsized":"2024-09-17T15:38:06.277163+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42252,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":162082816,"data_stored":161992491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051741,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":446020,"kb_used_data":93620,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93925820,"statfs":{"total":96636764160,"available":96180039680,"internally_reserved":0,"allocated":95866880,"data_stored":95508899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724686,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505640,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866200,"statfs":{"total":96636764160,"available":96118988800,"internally_reserved":0,"allocated":548864,"data_stored":175967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":517205675},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247969,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539456,"kb_used_data":66344,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832384,"statfs":{"total":96636764160,"available":96084361216,"internally_reserved":0,"allocated":67936256,"data_stored":67559746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94621696,"data_stored":94600018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:38:09.893 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:38:10.259 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:10.259 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:38:10.271 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":467,"stamp":"2024-09-17T15:38:09.837717+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42253,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1491116,"kb_used_data":160500,"kb_used_omap":13,"kb_used_meta":1330546,"kb_avail":281624404,"statfs":{"total":289910292480,"available":288383389696,"internally_reserved":0,"allocated":164352000,"data_stored":163244612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1362480127},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2048,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001298"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":28913,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.362719+0000","last_change":"2024-09-17T15:38:06.362595+0000","last_active":"2024-09-17T15:38:06.362719+0000","last_peered":"2024-09-17T15:38:06.362719+0000","last_clean":"2024-09-17T15:38:06.362719+0000","last_became_active":"2024-09-17T15:38:06.284139+0000","last_became_peered":"2024-09-17T15:38:06.284139+0000","last_unstale":"2024-09-17T15:38:06.362719+0000","last_undegraded":"2024-09-17T15:38:06.362719+0000","last_fullsized":"2024-09-17T15:38:06.362719+0000","mapping_epoch":60,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:42:51.418256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4070,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21074,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276467+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:38:06.276467+0000","last_peered":"2024-09-17T15:38:06.276467+0000","last_clean":"2024-09-17T15:38:06.276467+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:38:06.276467+0000","last_undegraded":"2024-09-17T15:38:06.276467+0000","last_fullsized":"2024-09-17T15:38:06.276467+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21182,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277370+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:38:06.277370+0000","last_peered":"2024-09-17T15:38:06.277370+0000","last_clean":"2024-09-17T15:38:06.277370+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:38:06.277370+0000","last_undegraded":"2024-09-17T15:38:06.277370+0000","last_fullsized":"2024-09-17T15:38:06.277370+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21079,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276509+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:38:06.276509+0000","last_peered":"2024-09-17T15:38:06.276509+0000","last_clean":"2024-09-17T15:38:06.276509+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:38:06.276509+0000","last_undegraded":"2024-09-17T15:38:06.276509+0000","last_fullsized":"2024-09-17T15:38:06.276509+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20822,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.283149+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:38:06.283149+0000","last_peered":"2024-09-17T15:38:06.283149+0000","last_clean":"2024-09-17T15:38:06.283149+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:38:06.283149+0000","last_undegraded":"2024-09-17T15:38:06.283149+0000","last_fullsized":"2024-09-17T15:38:06.283149+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21573,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276969+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:38:06.276969+0000","last_peered":"2024-09-17T15:38:06.276969+0000","last_clean":"2024-09-17T15:38:06.276969+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:38:06.276969+0000","last_undegraded":"2024-09-17T15:38:06.276969+0000","last_fullsized":"2024-09-17T15:38:06.276969+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21116,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276860+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:38:06.276860+0000","last_peered":"2024-09-17T15:38:06.276860+0000","last_clean":"2024-09-17T15:38:06.276860+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:38:06.276860+0000","last_undegraded":"2024-09-17T15:38:06.276860+0000","last_fullsized":"2024-09-17T15:38:06.276860+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21084,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.281124+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:38:06.281124+0000","last_peered":"2024-09-17T15:38:06.281124+0000","last_clean":"2024-09-17T15:38:06.281124+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:38:06.281124+0000","last_undegraded":"2024-09-17T15:38:06.281124+0000","last_fullsized":"2024-09-17T15:38:06.281124+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":107,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276773+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:38:06.276773+0000","last_peered":"2024-09-17T15:38:06.276773+0000","last_clean":"2024-09-17T15:38:06.276773+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:38:06.276773+0000","last_undegraded":"2024-09-17T15:38:06.276773+0000","last_fullsized":"2024-09-17T15:38:06.276773+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29139,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277107+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:38:06.277107+0000","last_peered":"2024-09-17T15:38:06.277107+0000","last_clean":"2024-09-17T15:38:06.277107+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:38:06.277107+0000","last_undegraded":"2024-09-17T15:38:06.277107+0000","last_fullsized":"2024-09-17T15:38:06.277107+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.275989+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:38:06.275989+0000","last_peered":"2024-09-17T15:38:06.275989+0000","last_clean":"2024-09-17T15:38:06.275989+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:38:06.275989+0000","last_undegraded":"2024-09-17T15:38:06.275989+0000","last_fullsized":"2024-09-17T15:38:06.275989+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20846,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:06.281521+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:38:06.281521+0000","last_peered":"2024-09-17T15:38:06.281521+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:38:06.281521+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":143,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.281361+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:38:06.281361+0000","last_peered":"2024-09-17T15:38:06.281361+0000","last_clean":"2024-09-17T15:38:06.281361+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:38:06.281361+0000","last_undegraded":"2024-09-17T15:38:06.281361+0000","last_fullsized":"2024-09-17T15:38:06.281361+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28844,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276074+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:38:06.276074+0000","last_peered":"2024-09-17T15:38:06.276074+0000","last_clean":"2024-09-17T15:38:06.276074+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:38:06.276074+0000","last_undegraded":"2024-09-17T15:38:06.276074+0000","last_fullsized":"2024-09-17T15:38:06.276074+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":117,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277066+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:38:06.277066+0000","last_peered":"2024-09-17T15:38:06.277066+0000","last_clean":"2024-09-17T15:38:06.277066+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:38:06.277066+0000","last_undegraded":"2024-09-17T15:38:06.277066+0000","last_fullsized":"2024-09-17T15:38:06.277066+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29181,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.275684+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:38:06.275684+0000","last_peered":"2024-09-17T15:38:06.275684+0000","last_clean":"2024-09-17T15:38:06.275684+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:38:06.275684+0000","last_undegraded":"2024-09-17T15:38:06.275684+0000","last_fullsized":"2024-09-17T15:38:06.275684+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21291,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277656+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:38:06.277656+0000","last_peered":"2024-09-17T15:38:06.277656+0000","last_clean":"2024-09-17T15:38:06.277656+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:38:06.277656+0000","last_undegraded":"2024-09-17T15:38:06.277656+0000","last_fullsized":"2024-09-17T15:38:06.277656+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":107,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277209+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:38:06.277209+0000","last_peered":"2024-09-17T15:38:06.277209+0000","last_clean":"2024-09-17T15:38:06.277209+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:38:06.277209+0000","last_undegraded":"2024-09-17T15:38:06.277209+0000","last_fullsized":"2024-09-17T15:38:06.277209+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28095,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277618+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:38:06.277618+0000","last_peered":"2024-09-17T15:38:06.277618+0000","last_clean":"2024-09-17T15:38:06.277618+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:38:06.277618+0000","last_undegraded":"2024-09-17T15:38:06.277618+0000","last_fullsized":"2024-09-17T15:38:06.277618+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276036+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:38:06.276036+0000","last_peered":"2024-09-17T15:38:06.276036+0000","last_clean":"2024-09-17T15:38:06.276036+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:38:06.276036+0000","last_undegraded":"2024-09-17T15:38:06.276036+0000","last_fullsized":"2024-09-17T15:38:06.276036+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":22663,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:09.430915+0000","last_change":"2024-09-17T15:38:05.220032+0000","last_active":"2024-09-17T15:38:09.430915+0000","last_peered":"2024-09-17T15:38:09.430915+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:38:09.430915+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3263,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":802,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3263}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":107,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277568+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:38:06.277568+0000","last_peered":"2024-09-17T15:38:06.277568+0000","last_clean":"2024-09-17T15:38:06.277568+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:38:06.277568+0000","last_undegraded":"2024-09-17T15:38:06.277568+0000","last_fullsized":"2024-09-17T15:38:06.277568+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":112,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277607+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:38:06.277607+0000","last_peered":"2024-09-17T15:38:06.277607+0000","last_clean":"2024-09-17T15:38:06.277607+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:38:06.277607+0000","last_undegraded":"2024-09-17T15:38:06.277607+0000","last_fullsized":"2024-09-17T15:38:06.277607+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29046,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277163+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:38:06.277163+0000","last_peered":"2024-09-17T15:38:06.277163+0000","last_clean":"2024-09-17T15:38:06.277163+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:38:06.277163+0000","last_undegraded":"2024-09-17T15:38:06.277163+0000","last_fullsized":"2024-09-17T15:38:06.277163+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42252,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":162082816,"data_stored":161992491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051741,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":446020,"kb_used_data":93620,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93925820,"statfs":{"total":96636764160,"available":96180039680,"internally_reserved":0,"allocated":95866880,"data_stored":95508899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724686,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505640,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866200,"statfs":{"total":96636764160,"available":96118988800,"internally_reserved":0,"allocated":548864,"data_stored":175967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":517205675},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247969,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539456,"kb_used_data":66344,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832384,"statfs":{"total":96636764160,"available":96084361216,"internally_reserved":0,"allocated":67936256,"data_stored":67559746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94621696,"data_stored":94600018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:38:10.272 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:38:10.273 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20542", 'reported_seq': 20846, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:38:06.281521+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:38:06.281521+0000', 'last_peered': '2024-09-17T15:38:06.281521+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:38:06.281521+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17138, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3943}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:38:10.273 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:38:10.273 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'20886", 'reported_seq': 22663, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:38:09.430915+0000', 'last_change': '2024-09-17T15:38:05.220032+0000', 'last_active': '2024-09-17T15:38:09.430915+0000', 'last_peered': '2024-09-17T15:38:09.430915+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:38:09.430915+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'10800", 'ondisk_log_start': "43'10800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3263, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17673, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 802, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3263}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:38:10.273 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:38:10.687 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:10.687 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 7208/24 objects degraded (30033.333%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":827,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1526902784,"bytes_avail":288383389696,"bytes_total":289910292480,"degraded_objects":7208,"degraded_total":24,"degraded_ratio":300.33333333333331,"recovering_objects_per_sec":255,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2048,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:38:10.701 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:38:10.701 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:38:11.060 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:11.060 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:38:11.072 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":467,"stamp":"2024-09-17T15:38:09.837717+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42253,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1491116,"kb_used_data":160500,"kb_used_omap":13,"kb_used_meta":1330546,"kb_avail":281624404,"statfs":{"total":289910292480,"available":288383389696,"internally_reserved":0,"allocated":164352000,"data_stored":163244612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1362480127},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2048,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001298"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":28913,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.362719+0000","last_change":"2024-09-17T15:38:06.362595+0000","last_active":"2024-09-17T15:38:06.362719+0000","last_peered":"2024-09-17T15:38:06.362719+0000","last_clean":"2024-09-17T15:38:06.362719+0000","last_became_active":"2024-09-17T15:38:06.284139+0000","last_became_peered":"2024-09-17T15:38:06.284139+0000","last_unstale":"2024-09-17T15:38:06.362719+0000","last_undegraded":"2024-09-17T15:38:06.362719+0000","last_fullsized":"2024-09-17T15:38:06.362719+0000","mapping_epoch":60,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:42:51.418256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4070,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21074,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276467+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:38:06.276467+0000","last_peered":"2024-09-17T15:38:06.276467+0000","last_clean":"2024-09-17T15:38:06.276467+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:38:06.276467+0000","last_undegraded":"2024-09-17T15:38:06.276467+0000","last_fullsized":"2024-09-17T15:38:06.276467+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21182,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277370+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:38:06.277370+0000","last_peered":"2024-09-17T15:38:06.277370+0000","last_clean":"2024-09-17T15:38:06.277370+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:38:06.277370+0000","last_undegraded":"2024-09-17T15:38:06.277370+0000","last_fullsized":"2024-09-17T15:38:06.277370+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21079,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276509+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:38:06.276509+0000","last_peered":"2024-09-17T15:38:06.276509+0000","last_clean":"2024-09-17T15:38:06.276509+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:38:06.276509+0000","last_undegraded":"2024-09-17T15:38:06.276509+0000","last_fullsized":"2024-09-17T15:38:06.276509+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20822,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.283149+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:38:06.283149+0000","last_peered":"2024-09-17T15:38:06.283149+0000","last_clean":"2024-09-17T15:38:06.283149+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:38:06.283149+0000","last_undegraded":"2024-09-17T15:38:06.283149+0000","last_fullsized":"2024-09-17T15:38:06.283149+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21573,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276969+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:38:06.276969+0000","last_peered":"2024-09-17T15:38:06.276969+0000","last_clean":"2024-09-17T15:38:06.276969+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:38:06.276969+0000","last_undegraded":"2024-09-17T15:38:06.276969+0000","last_fullsized":"2024-09-17T15:38:06.276969+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21116,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276860+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:38:06.276860+0000","last_peered":"2024-09-17T15:38:06.276860+0000","last_clean":"2024-09-17T15:38:06.276860+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:38:06.276860+0000","last_undegraded":"2024-09-17T15:38:06.276860+0000","last_fullsized":"2024-09-17T15:38:06.276860+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21084,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.281124+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:38:06.281124+0000","last_peered":"2024-09-17T15:38:06.281124+0000","last_clean":"2024-09-17T15:38:06.281124+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:38:06.281124+0000","last_undegraded":"2024-09-17T15:38:06.281124+0000","last_fullsized":"2024-09-17T15:38:06.281124+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":107,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276773+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:38:06.276773+0000","last_peered":"2024-09-17T15:38:06.276773+0000","last_clean":"2024-09-17T15:38:06.276773+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:38:06.276773+0000","last_undegraded":"2024-09-17T15:38:06.276773+0000","last_fullsized":"2024-09-17T15:38:06.276773+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29139,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277107+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:38:06.277107+0000","last_peered":"2024-09-17T15:38:06.277107+0000","last_clean":"2024-09-17T15:38:06.277107+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:38:06.277107+0000","last_undegraded":"2024-09-17T15:38:06.277107+0000","last_fullsized":"2024-09-17T15:38:06.277107+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.275989+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:38:06.275989+0000","last_peered":"2024-09-17T15:38:06.275989+0000","last_clean":"2024-09-17T15:38:06.275989+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:38:06.275989+0000","last_undegraded":"2024-09-17T15:38:06.275989+0000","last_fullsized":"2024-09-17T15:38:06.275989+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20846,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:06.281521+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:38:06.281521+0000","last_peered":"2024-09-17T15:38:06.281521+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:38:06.281521+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":143,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.281361+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:38:06.281361+0000","last_peered":"2024-09-17T15:38:06.281361+0000","last_clean":"2024-09-17T15:38:06.281361+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:38:06.281361+0000","last_undegraded":"2024-09-17T15:38:06.281361+0000","last_fullsized":"2024-09-17T15:38:06.281361+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28844,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276074+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:38:06.276074+0000","last_peered":"2024-09-17T15:38:06.276074+0000","last_clean":"2024-09-17T15:38:06.276074+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:38:06.276074+0000","last_undegraded":"2024-09-17T15:38:06.276074+0000","last_fullsized":"2024-09-17T15:38:06.276074+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":117,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277066+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:38:06.277066+0000","last_peered":"2024-09-17T15:38:06.277066+0000","last_clean":"2024-09-17T15:38:06.277066+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:38:06.277066+0000","last_undegraded":"2024-09-17T15:38:06.277066+0000","last_fullsized":"2024-09-17T15:38:06.277066+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29181,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.275684+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:38:06.275684+0000","last_peered":"2024-09-17T15:38:06.275684+0000","last_clean":"2024-09-17T15:38:06.275684+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:38:06.275684+0000","last_undegraded":"2024-09-17T15:38:06.275684+0000","last_fullsized":"2024-09-17T15:38:06.275684+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21291,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277656+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:38:06.277656+0000","last_peered":"2024-09-17T15:38:06.277656+0000","last_clean":"2024-09-17T15:38:06.277656+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:38:06.277656+0000","last_undegraded":"2024-09-17T15:38:06.277656+0000","last_fullsized":"2024-09-17T15:38:06.277656+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":107,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277209+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:38:06.277209+0000","last_peered":"2024-09-17T15:38:06.277209+0000","last_clean":"2024-09-17T15:38:06.277209+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:38:06.277209+0000","last_undegraded":"2024-09-17T15:38:06.277209+0000","last_fullsized":"2024-09-17T15:38:06.277209+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28095,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277618+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:38:06.277618+0000","last_peered":"2024-09-17T15:38:06.277618+0000","last_clean":"2024-09-17T15:38:06.277618+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:38:06.277618+0000","last_undegraded":"2024-09-17T15:38:06.277618+0000","last_fullsized":"2024-09-17T15:38:06.277618+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276036+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:38:06.276036+0000","last_peered":"2024-09-17T15:38:06.276036+0000","last_clean":"2024-09-17T15:38:06.276036+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:38:06.276036+0000","last_undegraded":"2024-09-17T15:38:06.276036+0000","last_fullsized":"2024-09-17T15:38:06.276036+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":22663,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:09.430915+0000","last_change":"2024-09-17T15:38:05.220032+0000","last_active":"2024-09-17T15:38:09.430915+0000","last_peered":"2024-09-17T15:38:09.430915+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:38:09.430915+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3263,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":802,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3263}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":107,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277568+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:38:06.277568+0000","last_peered":"2024-09-17T15:38:06.277568+0000","last_clean":"2024-09-17T15:38:06.277568+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:38:06.277568+0000","last_undegraded":"2024-09-17T15:38:06.277568+0000","last_fullsized":"2024-09-17T15:38:06.277568+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":112,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277607+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:38:06.277607+0000","last_peered":"2024-09-17T15:38:06.277607+0000","last_clean":"2024-09-17T15:38:06.277607+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:38:06.277607+0000","last_undegraded":"2024-09-17T15:38:06.277607+0000","last_fullsized":"2024-09-17T15:38:06.277607+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29046,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277163+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:38:06.277163+0000","last_peered":"2024-09-17T15:38:06.277163+0000","last_clean":"2024-09-17T15:38:06.277163+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:38:06.277163+0000","last_undegraded":"2024-09-17T15:38:06.277163+0000","last_fullsized":"2024-09-17T15:38:06.277163+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42252,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":162082816,"data_stored":161992491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051741,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":446020,"kb_used_data":93620,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93925820,"statfs":{"total":96636764160,"available":96180039680,"internally_reserved":0,"allocated":95866880,"data_stored":95508899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724686,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505640,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866200,"statfs":{"total":96636764160,"available":96118988800,"internally_reserved":0,"allocated":548864,"data_stored":175967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":517205675},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247969,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539456,"kb_used_data":66344,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832384,"statfs":{"total":96636764160,"available":96084361216,"internally_reserved":0,"allocated":67936256,"data_stored":67559746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94621696,"data_stored":94600018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:38:14.075 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:38:14.409 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:14.409 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:38:14.423 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":469,"stamp":"2024-09-17T15:38:13.838389+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42253,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1487156,"kb_used_data":156540,"kb_used_omap":13,"kb_used_meta":1330546,"kb_avail":281628364,"statfs":{"total":289910292480,"available":288387444736,"internally_reserved":0,"allocated":160296960,"data_stored":159193185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1362480127},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-941,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":941,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.584513"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":28913,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.362719+0000","last_change":"2024-09-17T15:38:06.362595+0000","last_active":"2024-09-17T15:38:06.362719+0000","last_peered":"2024-09-17T15:38:06.362719+0000","last_clean":"2024-09-17T15:38:06.362719+0000","last_became_active":"2024-09-17T15:38:06.284139+0000","last_became_peered":"2024-09-17T15:38:06.284139+0000","last_unstale":"2024-09-17T15:38:06.362719+0000","last_undegraded":"2024-09-17T15:38:06.362719+0000","last_fullsized":"2024-09-17T15:38:06.362719+0000","mapping_epoch":60,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:42:51.418256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4070,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21074,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276467+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:38:06.276467+0000","last_peered":"2024-09-17T15:38:06.276467+0000","last_clean":"2024-09-17T15:38:06.276467+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:38:06.276467+0000","last_undegraded":"2024-09-17T15:38:06.276467+0000","last_fullsized":"2024-09-17T15:38:06.276467+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21182,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277370+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:38:06.277370+0000","last_peered":"2024-09-17T15:38:06.277370+0000","last_clean":"2024-09-17T15:38:06.277370+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:38:06.277370+0000","last_undegraded":"2024-09-17T15:38:06.277370+0000","last_fullsized":"2024-09-17T15:38:06.277370+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21079,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276509+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:38:06.276509+0000","last_peered":"2024-09-17T15:38:06.276509+0000","last_clean":"2024-09-17T15:38:06.276509+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:38:06.276509+0000","last_undegraded":"2024-09-17T15:38:06.276509+0000","last_fullsized":"2024-09-17T15:38:06.276509+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20822,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.283149+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:38:06.283149+0000","last_peered":"2024-09-17T15:38:06.283149+0000","last_clean":"2024-09-17T15:38:06.283149+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:38:06.283149+0000","last_undegraded":"2024-09-17T15:38:06.283149+0000","last_fullsized":"2024-09-17T15:38:06.283149+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21573,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276969+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:38:06.276969+0000","last_peered":"2024-09-17T15:38:06.276969+0000","last_clean":"2024-09-17T15:38:06.276969+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:38:06.276969+0000","last_undegraded":"2024-09-17T15:38:06.276969+0000","last_fullsized":"2024-09-17T15:38:06.276969+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21116,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276860+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:38:06.276860+0000","last_peered":"2024-09-17T15:38:06.276860+0000","last_clean":"2024-09-17T15:38:06.276860+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:38:06.276860+0000","last_undegraded":"2024-09-17T15:38:06.276860+0000","last_fullsized":"2024-09-17T15:38:06.276860+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21084,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.281124+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:38:06.281124+0000","last_peered":"2024-09-17T15:38:06.281124+0000","last_clean":"2024-09-17T15:38:06.281124+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:38:06.281124+0000","last_undegraded":"2024-09-17T15:38:06.281124+0000","last_fullsized":"2024-09-17T15:38:06.281124+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":107,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276773+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:38:06.276773+0000","last_peered":"2024-09-17T15:38:06.276773+0000","last_clean":"2024-09-17T15:38:06.276773+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:38:06.276773+0000","last_undegraded":"2024-09-17T15:38:06.276773+0000","last_fullsized":"2024-09-17T15:38:06.276773+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29139,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277107+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:38:06.277107+0000","last_peered":"2024-09-17T15:38:06.277107+0000","last_clean":"2024-09-17T15:38:06.277107+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:38:06.277107+0000","last_undegraded":"2024-09-17T15:38:06.277107+0000","last_fullsized":"2024-09-17T15:38:06.277107+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.275989+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:38:06.275989+0000","last_peered":"2024-09-17T15:38:06.275989+0000","last_clean":"2024-09-17T15:38:06.275989+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:38:06.275989+0000","last_undegraded":"2024-09-17T15:38:06.275989+0000","last_fullsized":"2024-09-17T15:38:06.275989+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20846,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:06.281521+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:38:06.281521+0000","last_peered":"2024-09-17T15:38:06.281521+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:38:06.281521+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":143,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.281361+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:38:06.281361+0000","last_peered":"2024-09-17T15:38:06.281361+0000","last_clean":"2024-09-17T15:38:06.281361+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:38:06.281361+0000","last_undegraded":"2024-09-17T15:38:06.281361+0000","last_fullsized":"2024-09-17T15:38:06.281361+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28844,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276074+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:38:06.276074+0000","last_peered":"2024-09-17T15:38:06.276074+0000","last_clean":"2024-09-17T15:38:06.276074+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:38:06.276074+0000","last_undegraded":"2024-09-17T15:38:06.276074+0000","last_fullsized":"2024-09-17T15:38:06.276074+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":117,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277066+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:38:06.277066+0000","last_peered":"2024-09-17T15:38:06.277066+0000","last_clean":"2024-09-17T15:38:06.277066+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:38:06.277066+0000","last_undegraded":"2024-09-17T15:38:06.277066+0000","last_fullsized":"2024-09-17T15:38:06.277066+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29181,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.275684+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:38:06.275684+0000","last_peered":"2024-09-17T15:38:06.275684+0000","last_clean":"2024-09-17T15:38:06.275684+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:38:06.275684+0000","last_undegraded":"2024-09-17T15:38:06.275684+0000","last_fullsized":"2024-09-17T15:38:06.275684+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21291,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277656+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:38:06.277656+0000","last_peered":"2024-09-17T15:38:06.277656+0000","last_clean":"2024-09-17T15:38:06.277656+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:38:06.277656+0000","last_undegraded":"2024-09-17T15:38:06.277656+0000","last_fullsized":"2024-09-17T15:38:06.277656+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":107,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277209+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:38:06.277209+0000","last_peered":"2024-09-17T15:38:06.277209+0000","last_clean":"2024-09-17T15:38:06.277209+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:38:06.277209+0000","last_undegraded":"2024-09-17T15:38:06.277209+0000","last_fullsized":"2024-09-17T15:38:06.277209+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28095,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277618+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:38:06.277618+0000","last_peered":"2024-09-17T15:38:06.277618+0000","last_clean":"2024-09-17T15:38:06.277618+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:38:06.277618+0000","last_undegraded":"2024-09-17T15:38:06.277618+0000","last_fullsized":"2024-09-17T15:38:06.277618+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276036+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:38:06.276036+0000","last_peered":"2024-09-17T15:38:06.276036+0000","last_clean":"2024-09-17T15:38:06.276036+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:38:06.276036+0000","last_undegraded":"2024-09-17T15:38:06.276036+0000","last_fullsized":"2024-09-17T15:38:06.276036+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":22663,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:09.430915+0000","last_change":"2024-09-17T15:38:05.220032+0000","last_active":"2024-09-17T15:38:09.430915+0000","last_peered":"2024-09-17T15:38:09.430915+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:38:09.430915+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3263,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":802,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3263}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":107,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277568+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:38:06.277568+0000","last_peered":"2024-09-17T15:38:06.277568+0000","last_clean":"2024-09-17T15:38:06.277568+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:38:06.277568+0000","last_undegraded":"2024-09-17T15:38:06.277568+0000","last_fullsized":"2024-09-17T15:38:06.277568+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":112,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277607+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:38:06.277607+0000","last_peered":"2024-09-17T15:38:06.277607+0000","last_clean":"2024-09-17T15:38:06.277607+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:38:06.277607+0000","last_undegraded":"2024-09-17T15:38:06.277607+0000","last_fullsized":"2024-09-17T15:38:06.277607+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29046,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277163+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:38:06.277163+0000","last_peered":"2024-09-17T15:38:06.277163+0000","last_clean":"2024-09-17T15:38:06.277163+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:38:06.277163+0000","last_undegraded":"2024-09-17T15:38:06.277163+0000","last_fullsized":"2024-09-17T15:38:06.277163+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42252,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":158396416,"data_stored":158309704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051742,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442060,"kb_used_data":89660,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93929780,"statfs":{"total":96636764160,"available":96184094720,"internally_reserved":0,"allocated":91811840,"data_stored":91457472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724686,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505640,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866200,"statfs":{"total":96636764160,"available":96118988800,"internally_reserved":0,"allocated":548864,"data_stored":175967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":517205675},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247969,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539456,"kb_used_data":66344,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832384,"statfs":{"total":96636764160,"available":96084361216,"internally_reserved":0,"allocated":67936256,"data_stored":67559746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90935296,"data_stored":90917231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:38:14.425 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:38:14.766 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:14.766 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:38:14.780 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":469,"stamp":"2024-09-17T15:38:13.838389+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42253,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1487156,"kb_used_data":156540,"kb_used_omap":13,"kb_used_meta":1330546,"kb_avail":281628364,"statfs":{"total":289910292480,"available":288387444736,"internally_reserved":0,"allocated":160296960,"data_stored":159193185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1362480127},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-941,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":941,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.584513"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":28913,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.362719+0000","last_change":"2024-09-17T15:38:06.362595+0000","last_active":"2024-09-17T15:38:06.362719+0000","last_peered":"2024-09-17T15:38:06.362719+0000","last_clean":"2024-09-17T15:38:06.362719+0000","last_became_active":"2024-09-17T15:38:06.284139+0000","last_became_peered":"2024-09-17T15:38:06.284139+0000","last_unstale":"2024-09-17T15:38:06.362719+0000","last_undegraded":"2024-09-17T15:38:06.362719+0000","last_fullsized":"2024-09-17T15:38:06.362719+0000","mapping_epoch":60,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:42:51.418256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4070,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21074,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276467+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:38:06.276467+0000","last_peered":"2024-09-17T15:38:06.276467+0000","last_clean":"2024-09-17T15:38:06.276467+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:38:06.276467+0000","last_undegraded":"2024-09-17T15:38:06.276467+0000","last_fullsized":"2024-09-17T15:38:06.276467+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21182,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277370+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:38:06.277370+0000","last_peered":"2024-09-17T15:38:06.277370+0000","last_clean":"2024-09-17T15:38:06.277370+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:38:06.277370+0000","last_undegraded":"2024-09-17T15:38:06.277370+0000","last_fullsized":"2024-09-17T15:38:06.277370+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21079,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276509+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:38:06.276509+0000","last_peered":"2024-09-17T15:38:06.276509+0000","last_clean":"2024-09-17T15:38:06.276509+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:38:06.276509+0000","last_undegraded":"2024-09-17T15:38:06.276509+0000","last_fullsized":"2024-09-17T15:38:06.276509+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20822,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.283149+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:38:06.283149+0000","last_peered":"2024-09-17T15:38:06.283149+0000","last_clean":"2024-09-17T15:38:06.283149+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:38:06.283149+0000","last_undegraded":"2024-09-17T15:38:06.283149+0000","last_fullsized":"2024-09-17T15:38:06.283149+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21573,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276969+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:38:06.276969+0000","last_peered":"2024-09-17T15:38:06.276969+0000","last_clean":"2024-09-17T15:38:06.276969+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:38:06.276969+0000","last_undegraded":"2024-09-17T15:38:06.276969+0000","last_fullsized":"2024-09-17T15:38:06.276969+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21116,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276860+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:38:06.276860+0000","last_peered":"2024-09-17T15:38:06.276860+0000","last_clean":"2024-09-17T15:38:06.276860+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:38:06.276860+0000","last_undegraded":"2024-09-17T15:38:06.276860+0000","last_fullsized":"2024-09-17T15:38:06.276860+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21084,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.281124+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:38:06.281124+0000","last_peered":"2024-09-17T15:38:06.281124+0000","last_clean":"2024-09-17T15:38:06.281124+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:38:06.281124+0000","last_undegraded":"2024-09-17T15:38:06.281124+0000","last_fullsized":"2024-09-17T15:38:06.281124+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":107,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276773+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:38:06.276773+0000","last_peered":"2024-09-17T15:38:06.276773+0000","last_clean":"2024-09-17T15:38:06.276773+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:38:06.276773+0000","last_undegraded":"2024-09-17T15:38:06.276773+0000","last_fullsized":"2024-09-17T15:38:06.276773+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29139,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277107+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:38:06.277107+0000","last_peered":"2024-09-17T15:38:06.277107+0000","last_clean":"2024-09-17T15:38:06.277107+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:38:06.277107+0000","last_undegraded":"2024-09-17T15:38:06.277107+0000","last_fullsized":"2024-09-17T15:38:06.277107+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.275989+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:38:06.275989+0000","last_peered":"2024-09-17T15:38:06.275989+0000","last_clean":"2024-09-17T15:38:06.275989+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:38:06.275989+0000","last_undegraded":"2024-09-17T15:38:06.275989+0000","last_fullsized":"2024-09-17T15:38:06.275989+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20846,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:06.281521+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:38:06.281521+0000","last_peered":"2024-09-17T15:38:06.281521+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:38:06.281521+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":143,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.281361+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:38:06.281361+0000","last_peered":"2024-09-17T15:38:06.281361+0000","last_clean":"2024-09-17T15:38:06.281361+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:38:06.281361+0000","last_undegraded":"2024-09-17T15:38:06.281361+0000","last_fullsized":"2024-09-17T15:38:06.281361+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28844,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276074+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:38:06.276074+0000","last_peered":"2024-09-17T15:38:06.276074+0000","last_clean":"2024-09-17T15:38:06.276074+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:38:06.276074+0000","last_undegraded":"2024-09-17T15:38:06.276074+0000","last_fullsized":"2024-09-17T15:38:06.276074+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":117,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277066+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:38:06.277066+0000","last_peered":"2024-09-17T15:38:06.277066+0000","last_clean":"2024-09-17T15:38:06.277066+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:38:06.277066+0000","last_undegraded":"2024-09-17T15:38:06.277066+0000","last_fullsized":"2024-09-17T15:38:06.277066+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29181,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.275684+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:38:06.275684+0000","last_peered":"2024-09-17T15:38:06.275684+0000","last_clean":"2024-09-17T15:38:06.275684+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:38:06.275684+0000","last_undegraded":"2024-09-17T15:38:06.275684+0000","last_fullsized":"2024-09-17T15:38:06.275684+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21291,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277656+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:38:06.277656+0000","last_peered":"2024-09-17T15:38:06.277656+0000","last_clean":"2024-09-17T15:38:06.277656+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:38:06.277656+0000","last_undegraded":"2024-09-17T15:38:06.277656+0000","last_fullsized":"2024-09-17T15:38:06.277656+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":107,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277209+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:38:06.277209+0000","last_peered":"2024-09-17T15:38:06.277209+0000","last_clean":"2024-09-17T15:38:06.277209+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:38:06.277209+0000","last_undegraded":"2024-09-17T15:38:06.277209+0000","last_fullsized":"2024-09-17T15:38:06.277209+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28095,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277618+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:38:06.277618+0000","last_peered":"2024-09-17T15:38:06.277618+0000","last_clean":"2024-09-17T15:38:06.277618+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:38:06.277618+0000","last_undegraded":"2024-09-17T15:38:06.277618+0000","last_fullsized":"2024-09-17T15:38:06.277618+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276036+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:38:06.276036+0000","last_peered":"2024-09-17T15:38:06.276036+0000","last_clean":"2024-09-17T15:38:06.276036+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:38:06.276036+0000","last_undegraded":"2024-09-17T15:38:06.276036+0000","last_fullsized":"2024-09-17T15:38:06.276036+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":22663,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:09.430915+0000","last_change":"2024-09-17T15:38:05.220032+0000","last_active":"2024-09-17T15:38:09.430915+0000","last_peered":"2024-09-17T15:38:09.430915+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:38:09.430915+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3263,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":802,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3263}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":107,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277568+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:38:06.277568+0000","last_peered":"2024-09-17T15:38:06.277568+0000","last_clean":"2024-09-17T15:38:06.277568+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:38:06.277568+0000","last_undegraded":"2024-09-17T15:38:06.277568+0000","last_fullsized":"2024-09-17T15:38:06.277568+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":112,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277607+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:38:06.277607+0000","last_peered":"2024-09-17T15:38:06.277607+0000","last_clean":"2024-09-17T15:38:06.277607+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:38:06.277607+0000","last_undegraded":"2024-09-17T15:38:06.277607+0000","last_fullsized":"2024-09-17T15:38:06.277607+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29046,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277163+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:38:06.277163+0000","last_peered":"2024-09-17T15:38:06.277163+0000","last_clean":"2024-09-17T15:38:06.277163+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:38:06.277163+0000","last_undegraded":"2024-09-17T15:38:06.277163+0000","last_fullsized":"2024-09-17T15:38:06.277163+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42252,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":158396416,"data_stored":158309704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051742,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442060,"kb_used_data":89660,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93929780,"statfs":{"total":96636764160,"available":96184094720,"internally_reserved":0,"allocated":91811840,"data_stored":91457472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724686,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505640,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866200,"statfs":{"total":96636764160,"available":96118988800,"internally_reserved":0,"allocated":548864,"data_stored":175967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":517205675},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247969,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539456,"kb_used_data":66344,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832384,"statfs":{"total":96636764160,"available":96084361216,"internally_reserved":0,"allocated":67936256,"data_stored":67559746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90935296,"data_stored":90917231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:38:14.781 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:38:14.781 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20542", 'reported_seq': 20846, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:38:06.281521+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:38:06.281521+0000', 'last_peered': '2024-09-17T15:38:06.281521+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:38:06.281521+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17138, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3943}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:38:14.781 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:38:14.782 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'20886", 'reported_seq': 22663, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:38:09.430915+0000', 'last_change': '2024-09-17T15:38:05.220032+0000', 'last_active': '2024-09-17T15:38:09.430915+0000', 'last_peered': '2024-09-17T15:38:09.430915+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:38:09.430915+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'10800", 'ondisk_log_start': "43'10800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3263, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17673, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 802, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3263}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:38:14.782 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:38:15.198 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:15.198 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 7208/24 objects degraded (30033.333%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":832,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1522847744,"bytes_avail":288387444736,"bytes_total":289910292480,"degraded_objects":7208,"degraded_total":24,"degraded_ratio":300.33333333333331,"recovering_objects_per_sec":109,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":941,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:38:15.212 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:38:15.212 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:38:15.535 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:15.536 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:38:15.548 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":469,"stamp":"2024-09-17T15:38:13.838389+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42253,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1487156,"kb_used_data":156540,"kb_used_omap":13,"kb_used_meta":1330546,"kb_avail":281628364,"statfs":{"total":289910292480,"available":288387444736,"internally_reserved":0,"allocated":160296960,"data_stored":159193185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1362480127},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-941,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":941,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.584513"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":28913,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.362719+0000","last_change":"2024-09-17T15:38:06.362595+0000","last_active":"2024-09-17T15:38:06.362719+0000","last_peered":"2024-09-17T15:38:06.362719+0000","last_clean":"2024-09-17T15:38:06.362719+0000","last_became_active":"2024-09-17T15:38:06.284139+0000","last_became_peered":"2024-09-17T15:38:06.284139+0000","last_unstale":"2024-09-17T15:38:06.362719+0000","last_undegraded":"2024-09-17T15:38:06.362719+0000","last_fullsized":"2024-09-17T15:38:06.362719+0000","mapping_epoch":60,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:42:51.418256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4070,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21074,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276467+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:38:06.276467+0000","last_peered":"2024-09-17T15:38:06.276467+0000","last_clean":"2024-09-17T15:38:06.276467+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:38:06.276467+0000","last_undegraded":"2024-09-17T15:38:06.276467+0000","last_fullsized":"2024-09-17T15:38:06.276467+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21182,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277370+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:38:06.277370+0000","last_peered":"2024-09-17T15:38:06.277370+0000","last_clean":"2024-09-17T15:38:06.277370+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:38:06.277370+0000","last_undegraded":"2024-09-17T15:38:06.277370+0000","last_fullsized":"2024-09-17T15:38:06.277370+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21079,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276509+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:38:06.276509+0000","last_peered":"2024-09-17T15:38:06.276509+0000","last_clean":"2024-09-17T15:38:06.276509+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:38:06.276509+0000","last_undegraded":"2024-09-17T15:38:06.276509+0000","last_fullsized":"2024-09-17T15:38:06.276509+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20822,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.283149+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:38:06.283149+0000","last_peered":"2024-09-17T15:38:06.283149+0000","last_clean":"2024-09-17T15:38:06.283149+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:38:06.283149+0000","last_undegraded":"2024-09-17T15:38:06.283149+0000","last_fullsized":"2024-09-17T15:38:06.283149+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21573,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276969+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:38:06.276969+0000","last_peered":"2024-09-17T15:38:06.276969+0000","last_clean":"2024-09-17T15:38:06.276969+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:38:06.276969+0000","last_undegraded":"2024-09-17T15:38:06.276969+0000","last_fullsized":"2024-09-17T15:38:06.276969+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21116,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276860+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:38:06.276860+0000","last_peered":"2024-09-17T15:38:06.276860+0000","last_clean":"2024-09-17T15:38:06.276860+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:38:06.276860+0000","last_undegraded":"2024-09-17T15:38:06.276860+0000","last_fullsized":"2024-09-17T15:38:06.276860+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21084,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.281124+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:38:06.281124+0000","last_peered":"2024-09-17T15:38:06.281124+0000","last_clean":"2024-09-17T15:38:06.281124+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:38:06.281124+0000","last_undegraded":"2024-09-17T15:38:06.281124+0000","last_fullsized":"2024-09-17T15:38:06.281124+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":107,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276773+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:38:06.276773+0000","last_peered":"2024-09-17T15:38:06.276773+0000","last_clean":"2024-09-17T15:38:06.276773+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:38:06.276773+0000","last_undegraded":"2024-09-17T15:38:06.276773+0000","last_fullsized":"2024-09-17T15:38:06.276773+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29139,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277107+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:38:06.277107+0000","last_peered":"2024-09-17T15:38:06.277107+0000","last_clean":"2024-09-17T15:38:06.277107+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:38:06.277107+0000","last_undegraded":"2024-09-17T15:38:06.277107+0000","last_fullsized":"2024-09-17T15:38:06.277107+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.275989+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:38:06.275989+0000","last_peered":"2024-09-17T15:38:06.275989+0000","last_clean":"2024-09-17T15:38:06.275989+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:38:06.275989+0000","last_undegraded":"2024-09-17T15:38:06.275989+0000","last_fullsized":"2024-09-17T15:38:06.275989+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20846,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:06.281521+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:38:06.281521+0000","last_peered":"2024-09-17T15:38:06.281521+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:38:06.281521+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":143,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.281361+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:38:06.281361+0000","last_peered":"2024-09-17T15:38:06.281361+0000","last_clean":"2024-09-17T15:38:06.281361+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:38:06.281361+0000","last_undegraded":"2024-09-17T15:38:06.281361+0000","last_fullsized":"2024-09-17T15:38:06.281361+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28844,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276074+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:38:06.276074+0000","last_peered":"2024-09-17T15:38:06.276074+0000","last_clean":"2024-09-17T15:38:06.276074+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:38:06.276074+0000","last_undegraded":"2024-09-17T15:38:06.276074+0000","last_fullsized":"2024-09-17T15:38:06.276074+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":117,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277066+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:38:06.277066+0000","last_peered":"2024-09-17T15:38:06.277066+0000","last_clean":"2024-09-17T15:38:06.277066+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:38:06.277066+0000","last_undegraded":"2024-09-17T15:38:06.277066+0000","last_fullsized":"2024-09-17T15:38:06.277066+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29181,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.275684+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:38:06.275684+0000","last_peered":"2024-09-17T15:38:06.275684+0000","last_clean":"2024-09-17T15:38:06.275684+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:38:06.275684+0000","last_undegraded":"2024-09-17T15:38:06.275684+0000","last_fullsized":"2024-09-17T15:38:06.275684+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21291,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277656+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:38:06.277656+0000","last_peered":"2024-09-17T15:38:06.277656+0000","last_clean":"2024-09-17T15:38:06.277656+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:38:06.277656+0000","last_undegraded":"2024-09-17T15:38:06.277656+0000","last_fullsized":"2024-09-17T15:38:06.277656+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":107,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277209+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:38:06.277209+0000","last_peered":"2024-09-17T15:38:06.277209+0000","last_clean":"2024-09-17T15:38:06.277209+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:38:06.277209+0000","last_undegraded":"2024-09-17T15:38:06.277209+0000","last_fullsized":"2024-09-17T15:38:06.277209+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28095,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277618+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:38:06.277618+0000","last_peered":"2024-09-17T15:38:06.277618+0000","last_clean":"2024-09-17T15:38:06.277618+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:38:06.277618+0000","last_undegraded":"2024-09-17T15:38:06.277618+0000","last_fullsized":"2024-09-17T15:38:06.277618+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276036+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:38:06.276036+0000","last_peered":"2024-09-17T15:38:06.276036+0000","last_clean":"2024-09-17T15:38:06.276036+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:38:06.276036+0000","last_undegraded":"2024-09-17T15:38:06.276036+0000","last_fullsized":"2024-09-17T15:38:06.276036+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":22663,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:09.430915+0000","last_change":"2024-09-17T15:38:05.220032+0000","last_active":"2024-09-17T15:38:09.430915+0000","last_peered":"2024-09-17T15:38:09.430915+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:38:09.430915+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3263,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":802,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3263}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":107,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277568+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:38:06.277568+0000","last_peered":"2024-09-17T15:38:06.277568+0000","last_clean":"2024-09-17T15:38:06.277568+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:38:06.277568+0000","last_undegraded":"2024-09-17T15:38:06.277568+0000","last_fullsized":"2024-09-17T15:38:06.277568+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":112,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277607+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:38:06.277607+0000","last_peered":"2024-09-17T15:38:06.277607+0000","last_clean":"2024-09-17T15:38:06.277607+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:38:06.277607+0000","last_undegraded":"2024-09-17T15:38:06.277607+0000","last_fullsized":"2024-09-17T15:38:06.277607+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29046,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277163+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:38:06.277163+0000","last_peered":"2024-09-17T15:38:06.277163+0000","last_clean":"2024-09-17T15:38:06.277163+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:38:06.277163+0000","last_undegraded":"2024-09-17T15:38:06.277163+0000","last_fullsized":"2024-09-17T15:38:06.277163+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42252,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":158396416,"data_stored":158309704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051742,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442060,"kb_used_data":89660,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93929780,"statfs":{"total":96636764160,"available":96184094720,"internally_reserved":0,"allocated":91811840,"data_stored":91457472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724686,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505640,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866200,"statfs":{"total":96636764160,"available":96118988800,"internally_reserved":0,"allocated":548864,"data_stored":175967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":517205675},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247969,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539456,"kb_used_data":66344,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832384,"statfs":{"total":96636764160,"available":96084361216,"internally_reserved":0,"allocated":67936256,"data_stored":67559746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90935296,"data_stored":90917231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:38:18.550 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:38:18.849 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:18.850 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:38:18.864 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":471,"stamp":"2024-09-17T15:38:17.838996+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6182,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43279,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1478836,"kb_used_data":156540,"kb_used_omap":13,"kb_used_meta":1322226,"kb_avail":281636684,"statfs":{"total":289910292480,"available":288395964416,"internally_reserved":0,"allocated":160296960,"data_stored":159193185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1353960447},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1967,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1967,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.564977"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":28913,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.362719+0000","last_change":"2024-09-17T15:38:06.362595+0000","last_active":"2024-09-17T15:38:06.362719+0000","last_peered":"2024-09-17T15:38:06.362719+0000","last_clean":"2024-09-17T15:38:06.362719+0000","last_became_active":"2024-09-17T15:38:06.284139+0000","last_became_peered":"2024-09-17T15:38:06.284139+0000","last_unstale":"2024-09-17T15:38:06.362719+0000","last_undegraded":"2024-09-17T15:38:06.362719+0000","last_fullsized":"2024-09-17T15:38:06.362719+0000","mapping_epoch":60,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:42:51.418256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4070,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21074,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276467+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:38:06.276467+0000","last_peered":"2024-09-17T15:38:06.276467+0000","last_clean":"2024-09-17T15:38:06.276467+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:38:06.276467+0000","last_undegraded":"2024-09-17T15:38:06.276467+0000","last_fullsized":"2024-09-17T15:38:06.276467+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21182,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277370+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:38:06.277370+0000","last_peered":"2024-09-17T15:38:06.277370+0000","last_clean":"2024-09-17T15:38:06.277370+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:38:06.277370+0000","last_undegraded":"2024-09-17T15:38:06.277370+0000","last_fullsized":"2024-09-17T15:38:06.277370+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21079,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276509+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:38:06.276509+0000","last_peered":"2024-09-17T15:38:06.276509+0000","last_clean":"2024-09-17T15:38:06.276509+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:38:06.276509+0000","last_undegraded":"2024-09-17T15:38:06.276509+0000","last_fullsized":"2024-09-17T15:38:06.276509+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20822,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.283149+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:38:06.283149+0000","last_peered":"2024-09-17T15:38:06.283149+0000","last_clean":"2024-09-17T15:38:06.283149+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:38:06.283149+0000","last_undegraded":"2024-09-17T15:38:06.283149+0000","last_fullsized":"2024-09-17T15:38:06.283149+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21573,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276969+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:38:06.276969+0000","last_peered":"2024-09-17T15:38:06.276969+0000","last_clean":"2024-09-17T15:38:06.276969+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:38:06.276969+0000","last_undegraded":"2024-09-17T15:38:06.276969+0000","last_fullsized":"2024-09-17T15:38:06.276969+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21116,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276860+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:38:06.276860+0000","last_peered":"2024-09-17T15:38:06.276860+0000","last_clean":"2024-09-17T15:38:06.276860+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:38:06.276860+0000","last_undegraded":"2024-09-17T15:38:06.276860+0000","last_fullsized":"2024-09-17T15:38:06.276860+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21084,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.281124+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:38:06.281124+0000","last_peered":"2024-09-17T15:38:06.281124+0000","last_clean":"2024-09-17T15:38:06.281124+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:38:06.281124+0000","last_undegraded":"2024-09-17T15:38:06.281124+0000","last_fullsized":"2024-09-17T15:38:06.281124+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":107,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276773+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:38:06.276773+0000","last_peered":"2024-09-17T15:38:06.276773+0000","last_clean":"2024-09-17T15:38:06.276773+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:38:06.276773+0000","last_undegraded":"2024-09-17T15:38:06.276773+0000","last_fullsized":"2024-09-17T15:38:06.276773+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29139,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277107+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:38:06.277107+0000","last_peered":"2024-09-17T15:38:06.277107+0000","last_clean":"2024-09-17T15:38:06.277107+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:38:06.277107+0000","last_undegraded":"2024-09-17T15:38:06.277107+0000","last_fullsized":"2024-09-17T15:38:06.277107+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.275989+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:38:06.275989+0000","last_peered":"2024-09-17T15:38:06.275989+0000","last_clean":"2024-09-17T15:38:06.275989+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:38:06.275989+0000","last_undegraded":"2024-09-17T15:38:06.275989+0000","last_fullsized":"2024-09-17T15:38:06.275989+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20846,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:06.281521+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:38:06.281521+0000","last_peered":"2024-09-17T15:38:06.281521+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:38:06.281521+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":143,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.281361+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:38:06.281361+0000","last_peered":"2024-09-17T15:38:06.281361+0000","last_clean":"2024-09-17T15:38:06.281361+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:38:06.281361+0000","last_undegraded":"2024-09-17T15:38:06.281361+0000","last_fullsized":"2024-09-17T15:38:06.281361+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28844,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276074+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:38:06.276074+0000","last_peered":"2024-09-17T15:38:06.276074+0000","last_clean":"2024-09-17T15:38:06.276074+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:38:06.276074+0000","last_undegraded":"2024-09-17T15:38:06.276074+0000","last_fullsized":"2024-09-17T15:38:06.276074+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":117,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277066+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:38:06.277066+0000","last_peered":"2024-09-17T15:38:06.277066+0000","last_clean":"2024-09-17T15:38:06.277066+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:38:06.277066+0000","last_undegraded":"2024-09-17T15:38:06.277066+0000","last_fullsized":"2024-09-17T15:38:06.277066+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29181,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.275684+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:38:06.275684+0000","last_peered":"2024-09-17T15:38:06.275684+0000","last_clean":"2024-09-17T15:38:06.275684+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:38:06.275684+0000","last_undegraded":"2024-09-17T15:38:06.275684+0000","last_fullsized":"2024-09-17T15:38:06.275684+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21291,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277656+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:38:06.277656+0000","last_peered":"2024-09-17T15:38:06.277656+0000","last_clean":"2024-09-17T15:38:06.277656+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:38:06.277656+0000","last_undegraded":"2024-09-17T15:38:06.277656+0000","last_fullsized":"2024-09-17T15:38:06.277656+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":107,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277209+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:38:06.277209+0000","last_peered":"2024-09-17T15:38:06.277209+0000","last_clean":"2024-09-17T15:38:06.277209+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:38:06.277209+0000","last_undegraded":"2024-09-17T15:38:06.277209+0000","last_fullsized":"2024-09-17T15:38:06.277209+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28095,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277618+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:38:06.277618+0000","last_peered":"2024-09-17T15:38:06.277618+0000","last_clean":"2024-09-17T15:38:06.277618+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:38:06.277618+0000","last_undegraded":"2024-09-17T15:38:06.277618+0000","last_fullsized":"2024-09-17T15:38:06.277618+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276036+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:38:06.276036+0000","last_peered":"2024-09-17T15:38:06.276036+0000","last_clean":"2024-09-17T15:38:06.276036+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:38:06.276036+0000","last_undegraded":"2024-09-17T15:38:06.276036+0000","last_fullsized":"2024-09-17T15:38:06.276036+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":24715,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:14.431063+0000","last_change":"2024-09-17T15:38:05.220032+0000","last_active":"2024-09-17T15:38:14.431063+0000","last_peered":"2024-09-17T15:38:14.431063+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:38:14.431063+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1828,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":107,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277568+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:38:06.277568+0000","last_peered":"2024-09-17T15:38:06.277568+0000","last_clean":"2024-09-17T15:38:06.277568+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:38:06.277568+0000","last_undegraded":"2024-09-17T15:38:06.277568+0000","last_fullsized":"2024-09-17T15:38:06.277568+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":112,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277607+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:38:06.277607+0000","last_peered":"2024-09-17T15:38:06.277607+0000","last_clean":"2024-09-17T15:38:06.277607+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:38:06.277607+0000","last_undegraded":"2024-09-17T15:38:06.277607+0000","last_fullsized":"2024-09-17T15:38:06.277607+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29046,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277163+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:38:06.277163+0000","last_peered":"2024-09-17T15:38:06.277163+0000","last_clean":"2024-09-17T15:38:06.277163+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:38:06.277163+0000","last_undegraded":"2024-09-17T15:38:06.277163+0000","last_fullsized":"2024-09-17T15:38:06.277163+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6182,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43278,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":158396416,"data_stored":158309704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051742,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442060,"kb_used_data":89660,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93929780,"statfs":{"total":96636764160,"available":96184094720,"internally_reserved":0,"allocated":91811840,"data_stored":91457472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724687,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497320,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874520,"statfs":{"total":96636764160,"available":96127508480,"internally_reserved":0,"allocated":548864,"data_stored":175967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":508685995},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247970,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539456,"kb_used_data":66344,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832384,"statfs":{"total":96636764160,"available":96084361216,"internally_reserved":0,"allocated":67936256,"data_stored":67559746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90935296,"data_stored":90917231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:38:18.865 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:38:19.200 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:19.200 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:38:19.213 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":471,"stamp":"2024-09-17T15:38:17.838996+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6182,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43279,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1478836,"kb_used_data":156540,"kb_used_omap":13,"kb_used_meta":1322226,"kb_avail":281636684,"statfs":{"total":289910292480,"available":288395964416,"internally_reserved":0,"allocated":160296960,"data_stored":159193185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1353960447},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1967,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1967,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.564977"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":28913,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.362719+0000","last_change":"2024-09-17T15:38:06.362595+0000","last_active":"2024-09-17T15:38:06.362719+0000","last_peered":"2024-09-17T15:38:06.362719+0000","last_clean":"2024-09-17T15:38:06.362719+0000","last_became_active":"2024-09-17T15:38:06.284139+0000","last_became_peered":"2024-09-17T15:38:06.284139+0000","last_unstale":"2024-09-17T15:38:06.362719+0000","last_undegraded":"2024-09-17T15:38:06.362719+0000","last_fullsized":"2024-09-17T15:38:06.362719+0000","mapping_epoch":60,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:42:51.418256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4070,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21074,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276467+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:38:06.276467+0000","last_peered":"2024-09-17T15:38:06.276467+0000","last_clean":"2024-09-17T15:38:06.276467+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:38:06.276467+0000","last_undegraded":"2024-09-17T15:38:06.276467+0000","last_fullsized":"2024-09-17T15:38:06.276467+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21182,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277370+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:38:06.277370+0000","last_peered":"2024-09-17T15:38:06.277370+0000","last_clean":"2024-09-17T15:38:06.277370+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:38:06.277370+0000","last_undegraded":"2024-09-17T15:38:06.277370+0000","last_fullsized":"2024-09-17T15:38:06.277370+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21079,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276509+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:38:06.276509+0000","last_peered":"2024-09-17T15:38:06.276509+0000","last_clean":"2024-09-17T15:38:06.276509+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:38:06.276509+0000","last_undegraded":"2024-09-17T15:38:06.276509+0000","last_fullsized":"2024-09-17T15:38:06.276509+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20822,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.283149+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:38:06.283149+0000","last_peered":"2024-09-17T15:38:06.283149+0000","last_clean":"2024-09-17T15:38:06.283149+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:38:06.283149+0000","last_undegraded":"2024-09-17T15:38:06.283149+0000","last_fullsized":"2024-09-17T15:38:06.283149+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21573,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276969+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:38:06.276969+0000","last_peered":"2024-09-17T15:38:06.276969+0000","last_clean":"2024-09-17T15:38:06.276969+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:38:06.276969+0000","last_undegraded":"2024-09-17T15:38:06.276969+0000","last_fullsized":"2024-09-17T15:38:06.276969+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21116,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276860+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:38:06.276860+0000","last_peered":"2024-09-17T15:38:06.276860+0000","last_clean":"2024-09-17T15:38:06.276860+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:38:06.276860+0000","last_undegraded":"2024-09-17T15:38:06.276860+0000","last_fullsized":"2024-09-17T15:38:06.276860+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21084,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.281124+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:38:06.281124+0000","last_peered":"2024-09-17T15:38:06.281124+0000","last_clean":"2024-09-17T15:38:06.281124+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:38:06.281124+0000","last_undegraded":"2024-09-17T15:38:06.281124+0000","last_fullsized":"2024-09-17T15:38:06.281124+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":107,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276773+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:38:06.276773+0000","last_peered":"2024-09-17T15:38:06.276773+0000","last_clean":"2024-09-17T15:38:06.276773+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:38:06.276773+0000","last_undegraded":"2024-09-17T15:38:06.276773+0000","last_fullsized":"2024-09-17T15:38:06.276773+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29139,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277107+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:38:06.277107+0000","last_peered":"2024-09-17T15:38:06.277107+0000","last_clean":"2024-09-17T15:38:06.277107+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:38:06.277107+0000","last_undegraded":"2024-09-17T15:38:06.277107+0000","last_fullsized":"2024-09-17T15:38:06.277107+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.275989+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:38:06.275989+0000","last_peered":"2024-09-17T15:38:06.275989+0000","last_clean":"2024-09-17T15:38:06.275989+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:38:06.275989+0000","last_undegraded":"2024-09-17T15:38:06.275989+0000","last_fullsized":"2024-09-17T15:38:06.275989+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20846,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:06.281521+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:38:06.281521+0000","last_peered":"2024-09-17T15:38:06.281521+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:38:06.281521+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":143,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.281361+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:38:06.281361+0000","last_peered":"2024-09-17T15:38:06.281361+0000","last_clean":"2024-09-17T15:38:06.281361+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:38:06.281361+0000","last_undegraded":"2024-09-17T15:38:06.281361+0000","last_fullsized":"2024-09-17T15:38:06.281361+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28844,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276074+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:38:06.276074+0000","last_peered":"2024-09-17T15:38:06.276074+0000","last_clean":"2024-09-17T15:38:06.276074+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:38:06.276074+0000","last_undegraded":"2024-09-17T15:38:06.276074+0000","last_fullsized":"2024-09-17T15:38:06.276074+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":117,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277066+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:38:06.277066+0000","last_peered":"2024-09-17T15:38:06.277066+0000","last_clean":"2024-09-17T15:38:06.277066+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:38:06.277066+0000","last_undegraded":"2024-09-17T15:38:06.277066+0000","last_fullsized":"2024-09-17T15:38:06.277066+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29181,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.275684+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:38:06.275684+0000","last_peered":"2024-09-17T15:38:06.275684+0000","last_clean":"2024-09-17T15:38:06.275684+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:38:06.275684+0000","last_undegraded":"2024-09-17T15:38:06.275684+0000","last_fullsized":"2024-09-17T15:38:06.275684+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21291,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277656+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:38:06.277656+0000","last_peered":"2024-09-17T15:38:06.277656+0000","last_clean":"2024-09-17T15:38:06.277656+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:38:06.277656+0000","last_undegraded":"2024-09-17T15:38:06.277656+0000","last_fullsized":"2024-09-17T15:38:06.277656+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":107,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277209+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:38:06.277209+0000","last_peered":"2024-09-17T15:38:06.277209+0000","last_clean":"2024-09-17T15:38:06.277209+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:38:06.277209+0000","last_undegraded":"2024-09-17T15:38:06.277209+0000","last_fullsized":"2024-09-17T15:38:06.277209+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28095,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277618+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:38:06.277618+0000","last_peered":"2024-09-17T15:38:06.277618+0000","last_clean":"2024-09-17T15:38:06.277618+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:38:06.277618+0000","last_undegraded":"2024-09-17T15:38:06.277618+0000","last_fullsized":"2024-09-17T15:38:06.277618+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276036+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:38:06.276036+0000","last_peered":"2024-09-17T15:38:06.276036+0000","last_clean":"2024-09-17T15:38:06.276036+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:38:06.276036+0000","last_undegraded":"2024-09-17T15:38:06.276036+0000","last_fullsized":"2024-09-17T15:38:06.276036+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":24715,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:14.431063+0000","last_change":"2024-09-17T15:38:05.220032+0000","last_active":"2024-09-17T15:38:14.431063+0000","last_peered":"2024-09-17T15:38:14.431063+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:38:14.431063+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1828,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":107,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277568+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:38:06.277568+0000","last_peered":"2024-09-17T15:38:06.277568+0000","last_clean":"2024-09-17T15:38:06.277568+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:38:06.277568+0000","last_undegraded":"2024-09-17T15:38:06.277568+0000","last_fullsized":"2024-09-17T15:38:06.277568+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":112,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277607+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:38:06.277607+0000","last_peered":"2024-09-17T15:38:06.277607+0000","last_clean":"2024-09-17T15:38:06.277607+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:38:06.277607+0000","last_undegraded":"2024-09-17T15:38:06.277607+0000","last_fullsized":"2024-09-17T15:38:06.277607+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29046,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277163+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:38:06.277163+0000","last_peered":"2024-09-17T15:38:06.277163+0000","last_clean":"2024-09-17T15:38:06.277163+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:38:06.277163+0000","last_undegraded":"2024-09-17T15:38:06.277163+0000","last_fullsized":"2024-09-17T15:38:06.277163+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6182,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43278,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":158396416,"data_stored":158309704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051742,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442060,"kb_used_data":89660,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93929780,"statfs":{"total":96636764160,"available":96184094720,"internally_reserved":0,"allocated":91811840,"data_stored":91457472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724687,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497320,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874520,"statfs":{"total":96636764160,"available":96127508480,"internally_reserved":0,"allocated":548864,"data_stored":175967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":508685995},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247970,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539456,"kb_used_data":66344,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832384,"statfs":{"total":96636764160,"available":96084361216,"internally_reserved":0,"allocated":67936256,"data_stored":67559746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90935296,"data_stored":90917231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:38:19.215 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:38:19.215 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20542", 'reported_seq': 20846, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:38:06.281521+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:38:06.281521+0000', 'last_peered': '2024-09-17T15:38:06.281521+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:38:06.281521+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17138, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3943}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:38:19.215 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:38:19.215 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'20886", 'reported_seq': 24715, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:38:14.431063+0000', 'last_change': '2024-09-17T15:38:05.220032+0000', 'last_active': '2024-09-17T15:38:14.431063+0000', 'last_peered': '2024-09-17T15:38:14.431063+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:38:14.431063+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'10800", 'ondisk_log_start': "43'10800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17673, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1828, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2237}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:38:19.216 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:38:19.587 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:19.587 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 6182/24 objects degraded (25758.333%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":836,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1514328064,"bytes_avail":288395964416,"bytes_total":289910292480,"degraded_objects":6182,"degraded_total":24,"degraded_ratio":257.58333333333331,"recovering_objects_per_sec":170,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1967,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:38:19.600 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:38:19.600 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:38:19.902 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:19.903 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:38:19.917 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":472,"stamp":"2024-09-17T15:38:19.839284+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5324,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44137,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1483556,"kb_used_data":152940,"kb_used_omap":13,"kb_used_meta":1330546,"kb_avail":281631964,"statfs":{"total":289910292480,"available":288391131136,"internally_reserved":0,"allocated":156610560,"data_stored":155506785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1362480127},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2825,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2825,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001919"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":28913,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.362719+0000","last_change":"2024-09-17T15:38:06.362595+0000","last_active":"2024-09-17T15:38:06.362719+0000","last_peered":"2024-09-17T15:38:06.362719+0000","last_clean":"2024-09-17T15:38:06.362719+0000","last_became_active":"2024-09-17T15:38:06.284139+0000","last_became_peered":"2024-09-17T15:38:06.284139+0000","last_unstale":"2024-09-17T15:38:06.362719+0000","last_undegraded":"2024-09-17T15:38:06.362719+0000","last_fullsized":"2024-09-17T15:38:06.362719+0000","mapping_epoch":60,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:42:51.418256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4070,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21074,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276467+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:38:06.276467+0000","last_peered":"2024-09-17T15:38:06.276467+0000","last_clean":"2024-09-17T15:38:06.276467+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:38:06.276467+0000","last_undegraded":"2024-09-17T15:38:06.276467+0000","last_fullsized":"2024-09-17T15:38:06.276467+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21182,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277370+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:38:06.277370+0000","last_peered":"2024-09-17T15:38:06.277370+0000","last_clean":"2024-09-17T15:38:06.277370+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:38:06.277370+0000","last_undegraded":"2024-09-17T15:38:06.277370+0000","last_fullsized":"2024-09-17T15:38:06.277370+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21079,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276509+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:38:06.276509+0000","last_peered":"2024-09-17T15:38:06.276509+0000","last_clean":"2024-09-17T15:38:06.276509+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:38:06.276509+0000","last_undegraded":"2024-09-17T15:38:06.276509+0000","last_fullsized":"2024-09-17T15:38:06.276509+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20822,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.283149+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:38:06.283149+0000","last_peered":"2024-09-17T15:38:06.283149+0000","last_clean":"2024-09-17T15:38:06.283149+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:38:06.283149+0000","last_undegraded":"2024-09-17T15:38:06.283149+0000","last_fullsized":"2024-09-17T15:38:06.283149+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21573,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276969+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:38:06.276969+0000","last_peered":"2024-09-17T15:38:06.276969+0000","last_clean":"2024-09-17T15:38:06.276969+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:38:06.276969+0000","last_undegraded":"2024-09-17T15:38:06.276969+0000","last_fullsized":"2024-09-17T15:38:06.276969+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21116,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276860+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:38:06.276860+0000","last_peered":"2024-09-17T15:38:06.276860+0000","last_clean":"2024-09-17T15:38:06.276860+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:38:06.276860+0000","last_undegraded":"2024-09-17T15:38:06.276860+0000","last_fullsized":"2024-09-17T15:38:06.276860+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21084,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.281124+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:38:06.281124+0000","last_peered":"2024-09-17T15:38:06.281124+0000","last_clean":"2024-09-17T15:38:06.281124+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:38:06.281124+0000","last_undegraded":"2024-09-17T15:38:06.281124+0000","last_fullsized":"2024-09-17T15:38:06.281124+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":107,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276773+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:38:06.276773+0000","last_peered":"2024-09-17T15:38:06.276773+0000","last_clean":"2024-09-17T15:38:06.276773+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:38:06.276773+0000","last_undegraded":"2024-09-17T15:38:06.276773+0000","last_fullsized":"2024-09-17T15:38:06.276773+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29139,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277107+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:38:06.277107+0000","last_peered":"2024-09-17T15:38:06.277107+0000","last_clean":"2024-09-17T15:38:06.277107+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:38:06.277107+0000","last_undegraded":"2024-09-17T15:38:06.277107+0000","last_fullsized":"2024-09-17T15:38:06.277107+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.275989+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:38:06.275989+0000","last_peered":"2024-09-17T15:38:06.275989+0000","last_clean":"2024-09-17T15:38:06.275989+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:38:06.275989+0000","last_undegraded":"2024-09-17T15:38:06.275989+0000","last_fullsized":"2024-09-17T15:38:06.275989+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20846,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:06.281521+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:38:06.281521+0000","last_peered":"2024-09-17T15:38:06.281521+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:38:06.281521+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":143,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.281361+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:38:06.281361+0000","last_peered":"2024-09-17T15:38:06.281361+0000","last_clean":"2024-09-17T15:38:06.281361+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:38:06.281361+0000","last_undegraded":"2024-09-17T15:38:06.281361+0000","last_fullsized":"2024-09-17T15:38:06.281361+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28844,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276074+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:38:06.276074+0000","last_peered":"2024-09-17T15:38:06.276074+0000","last_clean":"2024-09-17T15:38:06.276074+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:38:06.276074+0000","last_undegraded":"2024-09-17T15:38:06.276074+0000","last_fullsized":"2024-09-17T15:38:06.276074+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":117,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277066+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:38:06.277066+0000","last_peered":"2024-09-17T15:38:06.277066+0000","last_clean":"2024-09-17T15:38:06.277066+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:38:06.277066+0000","last_undegraded":"2024-09-17T15:38:06.277066+0000","last_fullsized":"2024-09-17T15:38:06.277066+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29181,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.275684+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:38:06.275684+0000","last_peered":"2024-09-17T15:38:06.275684+0000","last_clean":"2024-09-17T15:38:06.275684+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:38:06.275684+0000","last_undegraded":"2024-09-17T15:38:06.275684+0000","last_fullsized":"2024-09-17T15:38:06.275684+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21291,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277656+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:38:06.277656+0000","last_peered":"2024-09-17T15:38:06.277656+0000","last_clean":"2024-09-17T15:38:06.277656+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:38:06.277656+0000","last_undegraded":"2024-09-17T15:38:06.277656+0000","last_fullsized":"2024-09-17T15:38:06.277656+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":107,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277209+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:38:06.277209+0000","last_peered":"2024-09-17T15:38:06.277209+0000","last_clean":"2024-09-17T15:38:06.277209+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:38:06.277209+0000","last_undegraded":"2024-09-17T15:38:06.277209+0000","last_fullsized":"2024-09-17T15:38:06.277209+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28095,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277618+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:38:06.277618+0000","last_peered":"2024-09-17T15:38:06.277618+0000","last_clean":"2024-09-17T15:38:06.277618+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:38:06.277618+0000","last_undegraded":"2024-09-17T15:38:06.277618+0000","last_fullsized":"2024-09-17T15:38:06.277618+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276036+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:38:06.276036+0000","last_peered":"2024-09-17T15:38:06.276036+0000","last_clean":"2024-09-17T15:38:06.276036+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:38:06.276036+0000","last_undegraded":"2024-09-17T15:38:06.276036+0000","last_fullsized":"2024-09-17T15:38:06.276036+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":26431,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:19.433206+0000","last_change":"2024-09-17T15:38:05.220032+0000","last_active":"2024-09-17T15:38:19.433206+0000","last_peered":"2024-09-17T15:38:19.433206+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:38:19.433206+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1379,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2686,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":107,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277568+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:38:06.277568+0000","last_peered":"2024-09-17T15:38:06.277568+0000","last_clean":"2024-09-17T15:38:06.277568+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:38:06.277568+0000","last_undegraded":"2024-09-17T15:38:06.277568+0000","last_fullsized":"2024-09-17T15:38:06.277568+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":112,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277607+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:38:06.277607+0000","last_peered":"2024-09-17T15:38:06.277607+0000","last_clean":"2024-09-17T15:38:06.277607+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:38:06.277607+0000","last_undegraded":"2024-09-17T15:38:06.277607+0000","last_fullsized":"2024-09-17T15:38:06.277607+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29046,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277163+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:38:06.277163+0000","last_peered":"2024-09-17T15:38:06.277163+0000","last_clean":"2024-09-17T15:38:06.277163+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:38:06.277163+0000","last_undegraded":"2024-09-17T15:38:06.277163+0000","last_fullsized":"2024-09-17T15:38:06.277163+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5324,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44136,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":154710016,"data_stored":154623304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051743,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":438460,"kb_used_data":86060,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93933380,"statfs":{"total":96636764160,"available":96187781120,"internally_reserved":0,"allocated":88125440,"data_stored":87771072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724688,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505640,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866200,"statfs":{"total":96636764160,"available":96118988800,"internally_reserved":0,"allocated":548864,"data_stored":175967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":517205675},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247971,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539456,"kb_used_data":66344,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832384,"statfs":{"total":96636764160,"available":96084361216,"internally_reserved":0,"allocated":67936256,"data_stored":67559746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":87248896,"data_stored":87230831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:38:22.920 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:38:23.222 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:23.223 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:38:23.236 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":473,"stamp":"2024-09-17T15:38:21.839728+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5324,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44137,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1483556,"kb_used_data":152940,"kb_used_omap":13,"kb_used_meta":1330546,"kb_avail":281631964,"statfs":{"total":289910292480,"available":288391131136,"internally_reserved":0,"allocated":156610560,"data_stored":155506785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1362480127},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1884,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1884,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002011"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":28913,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.362719+0000","last_change":"2024-09-17T15:38:06.362595+0000","last_active":"2024-09-17T15:38:06.362719+0000","last_peered":"2024-09-17T15:38:06.362719+0000","last_clean":"2024-09-17T15:38:06.362719+0000","last_became_active":"2024-09-17T15:38:06.284139+0000","last_became_peered":"2024-09-17T15:38:06.284139+0000","last_unstale":"2024-09-17T15:38:06.362719+0000","last_undegraded":"2024-09-17T15:38:06.362719+0000","last_fullsized":"2024-09-17T15:38:06.362719+0000","mapping_epoch":60,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:42:51.418256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4070,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21074,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276467+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:38:06.276467+0000","last_peered":"2024-09-17T15:38:06.276467+0000","last_clean":"2024-09-17T15:38:06.276467+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:38:06.276467+0000","last_undegraded":"2024-09-17T15:38:06.276467+0000","last_fullsized":"2024-09-17T15:38:06.276467+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21182,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277370+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:38:06.277370+0000","last_peered":"2024-09-17T15:38:06.277370+0000","last_clean":"2024-09-17T15:38:06.277370+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:38:06.277370+0000","last_undegraded":"2024-09-17T15:38:06.277370+0000","last_fullsized":"2024-09-17T15:38:06.277370+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21079,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276509+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:38:06.276509+0000","last_peered":"2024-09-17T15:38:06.276509+0000","last_clean":"2024-09-17T15:38:06.276509+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:38:06.276509+0000","last_undegraded":"2024-09-17T15:38:06.276509+0000","last_fullsized":"2024-09-17T15:38:06.276509+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20822,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.283149+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:38:06.283149+0000","last_peered":"2024-09-17T15:38:06.283149+0000","last_clean":"2024-09-17T15:38:06.283149+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:38:06.283149+0000","last_undegraded":"2024-09-17T15:38:06.283149+0000","last_fullsized":"2024-09-17T15:38:06.283149+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21573,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276969+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:38:06.276969+0000","last_peered":"2024-09-17T15:38:06.276969+0000","last_clean":"2024-09-17T15:38:06.276969+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:38:06.276969+0000","last_undegraded":"2024-09-17T15:38:06.276969+0000","last_fullsized":"2024-09-17T15:38:06.276969+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21116,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276860+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:38:06.276860+0000","last_peered":"2024-09-17T15:38:06.276860+0000","last_clean":"2024-09-17T15:38:06.276860+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:38:06.276860+0000","last_undegraded":"2024-09-17T15:38:06.276860+0000","last_fullsized":"2024-09-17T15:38:06.276860+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21084,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.281124+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:38:06.281124+0000","last_peered":"2024-09-17T15:38:06.281124+0000","last_clean":"2024-09-17T15:38:06.281124+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:38:06.281124+0000","last_undegraded":"2024-09-17T15:38:06.281124+0000","last_fullsized":"2024-09-17T15:38:06.281124+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":107,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276773+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:38:06.276773+0000","last_peered":"2024-09-17T15:38:06.276773+0000","last_clean":"2024-09-17T15:38:06.276773+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:38:06.276773+0000","last_undegraded":"2024-09-17T15:38:06.276773+0000","last_fullsized":"2024-09-17T15:38:06.276773+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29139,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277107+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:38:06.277107+0000","last_peered":"2024-09-17T15:38:06.277107+0000","last_clean":"2024-09-17T15:38:06.277107+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:38:06.277107+0000","last_undegraded":"2024-09-17T15:38:06.277107+0000","last_fullsized":"2024-09-17T15:38:06.277107+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.275989+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:38:06.275989+0000","last_peered":"2024-09-17T15:38:06.275989+0000","last_clean":"2024-09-17T15:38:06.275989+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:38:06.275989+0000","last_undegraded":"2024-09-17T15:38:06.275989+0000","last_fullsized":"2024-09-17T15:38:06.275989+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20846,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:06.281521+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:38:06.281521+0000","last_peered":"2024-09-17T15:38:06.281521+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:38:06.281521+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":143,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.281361+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:38:06.281361+0000","last_peered":"2024-09-17T15:38:06.281361+0000","last_clean":"2024-09-17T15:38:06.281361+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:38:06.281361+0000","last_undegraded":"2024-09-17T15:38:06.281361+0000","last_fullsized":"2024-09-17T15:38:06.281361+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28844,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276074+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:38:06.276074+0000","last_peered":"2024-09-17T15:38:06.276074+0000","last_clean":"2024-09-17T15:38:06.276074+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:38:06.276074+0000","last_undegraded":"2024-09-17T15:38:06.276074+0000","last_fullsized":"2024-09-17T15:38:06.276074+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":117,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277066+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:38:06.277066+0000","last_peered":"2024-09-17T15:38:06.277066+0000","last_clean":"2024-09-17T15:38:06.277066+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:38:06.277066+0000","last_undegraded":"2024-09-17T15:38:06.277066+0000","last_fullsized":"2024-09-17T15:38:06.277066+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29181,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.275684+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:38:06.275684+0000","last_peered":"2024-09-17T15:38:06.275684+0000","last_clean":"2024-09-17T15:38:06.275684+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:38:06.275684+0000","last_undegraded":"2024-09-17T15:38:06.275684+0000","last_fullsized":"2024-09-17T15:38:06.275684+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21291,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277656+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:38:06.277656+0000","last_peered":"2024-09-17T15:38:06.277656+0000","last_clean":"2024-09-17T15:38:06.277656+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:38:06.277656+0000","last_undegraded":"2024-09-17T15:38:06.277656+0000","last_fullsized":"2024-09-17T15:38:06.277656+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":107,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277209+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:38:06.277209+0000","last_peered":"2024-09-17T15:38:06.277209+0000","last_clean":"2024-09-17T15:38:06.277209+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:38:06.277209+0000","last_undegraded":"2024-09-17T15:38:06.277209+0000","last_fullsized":"2024-09-17T15:38:06.277209+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28095,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277618+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:38:06.277618+0000","last_peered":"2024-09-17T15:38:06.277618+0000","last_clean":"2024-09-17T15:38:06.277618+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:38:06.277618+0000","last_undegraded":"2024-09-17T15:38:06.277618+0000","last_fullsized":"2024-09-17T15:38:06.277618+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276036+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:38:06.276036+0000","last_peered":"2024-09-17T15:38:06.276036+0000","last_clean":"2024-09-17T15:38:06.276036+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:38:06.276036+0000","last_undegraded":"2024-09-17T15:38:06.276036+0000","last_fullsized":"2024-09-17T15:38:06.276036+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":26431,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:19.433206+0000","last_change":"2024-09-17T15:38:05.220032+0000","last_active":"2024-09-17T15:38:19.433206+0000","last_peered":"2024-09-17T15:38:19.433206+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:38:19.433206+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1379,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2686,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":107,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277568+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:38:06.277568+0000","last_peered":"2024-09-17T15:38:06.277568+0000","last_clean":"2024-09-17T15:38:06.277568+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:38:06.277568+0000","last_undegraded":"2024-09-17T15:38:06.277568+0000","last_fullsized":"2024-09-17T15:38:06.277568+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":112,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277607+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:38:06.277607+0000","last_peered":"2024-09-17T15:38:06.277607+0000","last_clean":"2024-09-17T15:38:06.277607+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:38:06.277607+0000","last_undegraded":"2024-09-17T15:38:06.277607+0000","last_fullsized":"2024-09-17T15:38:06.277607+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29046,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277163+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:38:06.277163+0000","last_peered":"2024-09-17T15:38:06.277163+0000","last_clean":"2024-09-17T15:38:06.277163+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:38:06.277163+0000","last_undegraded":"2024-09-17T15:38:06.277163+0000","last_fullsized":"2024-09-17T15:38:06.277163+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5324,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44136,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":154710016,"data_stored":154623304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051743,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":438460,"kb_used_data":86060,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93933380,"statfs":{"total":96636764160,"available":96187781120,"internally_reserved":0,"allocated":88125440,"data_stored":87771072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724688,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505640,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866200,"statfs":{"total":96636764160,"available":96118988800,"internally_reserved":0,"allocated":548864,"data_stored":175967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":517205675},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247971,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539456,"kb_used_data":66344,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832384,"statfs":{"total":96636764160,"available":96084361216,"internally_reserved":0,"allocated":67936256,"data_stored":67559746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":87248896,"data_stored":87230831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:38:23.237 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:38:23.541 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:23.541 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:38:23.554 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":473,"stamp":"2024-09-17T15:38:21.839728+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5324,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44137,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1483556,"kb_used_data":152940,"kb_used_omap":13,"kb_used_meta":1330546,"kb_avail":281631964,"statfs":{"total":289910292480,"available":288391131136,"internally_reserved":0,"allocated":156610560,"data_stored":155506785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1362480127},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1884,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1884,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002011"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":28913,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.362719+0000","last_change":"2024-09-17T15:38:06.362595+0000","last_active":"2024-09-17T15:38:06.362719+0000","last_peered":"2024-09-17T15:38:06.362719+0000","last_clean":"2024-09-17T15:38:06.362719+0000","last_became_active":"2024-09-17T15:38:06.284139+0000","last_became_peered":"2024-09-17T15:38:06.284139+0000","last_unstale":"2024-09-17T15:38:06.362719+0000","last_undegraded":"2024-09-17T15:38:06.362719+0000","last_fullsized":"2024-09-17T15:38:06.362719+0000","mapping_epoch":60,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:42:51.418256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4070,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21074,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276467+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:38:06.276467+0000","last_peered":"2024-09-17T15:38:06.276467+0000","last_clean":"2024-09-17T15:38:06.276467+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:38:06.276467+0000","last_undegraded":"2024-09-17T15:38:06.276467+0000","last_fullsized":"2024-09-17T15:38:06.276467+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21182,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277370+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:38:06.277370+0000","last_peered":"2024-09-17T15:38:06.277370+0000","last_clean":"2024-09-17T15:38:06.277370+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:38:06.277370+0000","last_undegraded":"2024-09-17T15:38:06.277370+0000","last_fullsized":"2024-09-17T15:38:06.277370+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21079,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276509+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:38:06.276509+0000","last_peered":"2024-09-17T15:38:06.276509+0000","last_clean":"2024-09-17T15:38:06.276509+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:38:06.276509+0000","last_undegraded":"2024-09-17T15:38:06.276509+0000","last_fullsized":"2024-09-17T15:38:06.276509+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20822,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.283149+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:38:06.283149+0000","last_peered":"2024-09-17T15:38:06.283149+0000","last_clean":"2024-09-17T15:38:06.283149+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:38:06.283149+0000","last_undegraded":"2024-09-17T15:38:06.283149+0000","last_fullsized":"2024-09-17T15:38:06.283149+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21573,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276969+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:38:06.276969+0000","last_peered":"2024-09-17T15:38:06.276969+0000","last_clean":"2024-09-17T15:38:06.276969+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:38:06.276969+0000","last_undegraded":"2024-09-17T15:38:06.276969+0000","last_fullsized":"2024-09-17T15:38:06.276969+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21116,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276860+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:38:06.276860+0000","last_peered":"2024-09-17T15:38:06.276860+0000","last_clean":"2024-09-17T15:38:06.276860+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:38:06.276860+0000","last_undegraded":"2024-09-17T15:38:06.276860+0000","last_fullsized":"2024-09-17T15:38:06.276860+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21084,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.281124+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:38:06.281124+0000","last_peered":"2024-09-17T15:38:06.281124+0000","last_clean":"2024-09-17T15:38:06.281124+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:38:06.281124+0000","last_undegraded":"2024-09-17T15:38:06.281124+0000","last_fullsized":"2024-09-17T15:38:06.281124+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":107,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276773+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:38:06.276773+0000","last_peered":"2024-09-17T15:38:06.276773+0000","last_clean":"2024-09-17T15:38:06.276773+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:38:06.276773+0000","last_undegraded":"2024-09-17T15:38:06.276773+0000","last_fullsized":"2024-09-17T15:38:06.276773+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29139,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277107+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:38:06.277107+0000","last_peered":"2024-09-17T15:38:06.277107+0000","last_clean":"2024-09-17T15:38:06.277107+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:38:06.277107+0000","last_undegraded":"2024-09-17T15:38:06.277107+0000","last_fullsized":"2024-09-17T15:38:06.277107+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.275989+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:38:06.275989+0000","last_peered":"2024-09-17T15:38:06.275989+0000","last_clean":"2024-09-17T15:38:06.275989+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:38:06.275989+0000","last_undegraded":"2024-09-17T15:38:06.275989+0000","last_fullsized":"2024-09-17T15:38:06.275989+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20846,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:06.281521+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:38:06.281521+0000","last_peered":"2024-09-17T15:38:06.281521+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:38:06.281521+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":143,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.281361+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:38:06.281361+0000","last_peered":"2024-09-17T15:38:06.281361+0000","last_clean":"2024-09-17T15:38:06.281361+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:38:06.281361+0000","last_undegraded":"2024-09-17T15:38:06.281361+0000","last_fullsized":"2024-09-17T15:38:06.281361+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28844,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276074+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:38:06.276074+0000","last_peered":"2024-09-17T15:38:06.276074+0000","last_clean":"2024-09-17T15:38:06.276074+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:38:06.276074+0000","last_undegraded":"2024-09-17T15:38:06.276074+0000","last_fullsized":"2024-09-17T15:38:06.276074+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":117,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277066+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:38:06.277066+0000","last_peered":"2024-09-17T15:38:06.277066+0000","last_clean":"2024-09-17T15:38:06.277066+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:38:06.277066+0000","last_undegraded":"2024-09-17T15:38:06.277066+0000","last_fullsized":"2024-09-17T15:38:06.277066+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29181,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.275684+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:38:06.275684+0000","last_peered":"2024-09-17T15:38:06.275684+0000","last_clean":"2024-09-17T15:38:06.275684+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:38:06.275684+0000","last_undegraded":"2024-09-17T15:38:06.275684+0000","last_fullsized":"2024-09-17T15:38:06.275684+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21291,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277656+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:38:06.277656+0000","last_peered":"2024-09-17T15:38:06.277656+0000","last_clean":"2024-09-17T15:38:06.277656+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:38:06.277656+0000","last_undegraded":"2024-09-17T15:38:06.277656+0000","last_fullsized":"2024-09-17T15:38:06.277656+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":107,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277209+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:38:06.277209+0000","last_peered":"2024-09-17T15:38:06.277209+0000","last_clean":"2024-09-17T15:38:06.277209+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:38:06.277209+0000","last_undegraded":"2024-09-17T15:38:06.277209+0000","last_fullsized":"2024-09-17T15:38:06.277209+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28095,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277618+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:38:06.277618+0000","last_peered":"2024-09-17T15:38:06.277618+0000","last_clean":"2024-09-17T15:38:06.277618+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:38:06.277618+0000","last_undegraded":"2024-09-17T15:38:06.277618+0000","last_fullsized":"2024-09-17T15:38:06.277618+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276036+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:38:06.276036+0000","last_peered":"2024-09-17T15:38:06.276036+0000","last_clean":"2024-09-17T15:38:06.276036+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:38:06.276036+0000","last_undegraded":"2024-09-17T15:38:06.276036+0000","last_fullsized":"2024-09-17T15:38:06.276036+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":26431,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:19.433206+0000","last_change":"2024-09-17T15:38:05.220032+0000","last_active":"2024-09-17T15:38:19.433206+0000","last_peered":"2024-09-17T15:38:19.433206+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:38:19.433206+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1379,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2686,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":107,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277568+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:38:06.277568+0000","last_peered":"2024-09-17T15:38:06.277568+0000","last_clean":"2024-09-17T15:38:06.277568+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:38:06.277568+0000","last_undegraded":"2024-09-17T15:38:06.277568+0000","last_fullsized":"2024-09-17T15:38:06.277568+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":112,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277607+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:38:06.277607+0000","last_peered":"2024-09-17T15:38:06.277607+0000","last_clean":"2024-09-17T15:38:06.277607+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:38:06.277607+0000","last_undegraded":"2024-09-17T15:38:06.277607+0000","last_fullsized":"2024-09-17T15:38:06.277607+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29046,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277163+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:38:06.277163+0000","last_peered":"2024-09-17T15:38:06.277163+0000","last_clean":"2024-09-17T15:38:06.277163+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:38:06.277163+0000","last_undegraded":"2024-09-17T15:38:06.277163+0000","last_fullsized":"2024-09-17T15:38:06.277163+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5324,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44136,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":154710016,"data_stored":154623304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051743,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":438460,"kb_used_data":86060,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93933380,"statfs":{"total":96636764160,"available":96187781120,"internally_reserved":0,"allocated":88125440,"data_stored":87771072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724688,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505640,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866200,"statfs":{"total":96636764160,"available":96118988800,"internally_reserved":0,"allocated":548864,"data_stored":175967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":517205675},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247971,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539456,"kb_used_data":66344,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832384,"statfs":{"total":96636764160,"available":96084361216,"internally_reserved":0,"allocated":67936256,"data_stored":67559746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":87248896,"data_stored":87230831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:38:23.555 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:38:23.555 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20542", 'reported_seq': 20846, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:38:06.281521+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:38:06.281521+0000', 'last_peered': '2024-09-17T15:38:06.281521+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:38:06.281521+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17138, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3943}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:38:23.556 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:38:23.556 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'20886", 'reported_seq': 26431, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:38:19.433206+0000', 'last_change': '2024-09-17T15:38:05.220032+0000', 'last_active': '2024-09-17T15:38:19.433206+0000', 'last_peered': '2024-09-17T15:38:19.433206+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:38:19.433206+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'10800", 'ondisk_log_start': "43'10800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1379, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17673, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2686, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1379}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:38:23.556 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:38:23.949 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:23.950 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 5324/24 objects degraded (22183.333%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":840,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1519161344,"bytes_avail":288391131136,"bytes_total":289910292480,"degraded_objects":5324,"degraded_total":24,"degraded_ratio":221.83333333333337,"recovering_objects_per_sec":156,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1884,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:38:23.966 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:38:23.966 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:38:24.313 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:24.314 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:38:24.327 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":474,"stamp":"2024-09-17T15:38:23.840209+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5324,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44137,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1479956,"kb_used_data":149340,"kb_used_omap":13,"kb_used_meta":1330546,"kb_avail":281635564,"statfs":{"total":289910292480,"available":288394817536,"internally_reserved":0,"allocated":152924160,"data_stored":151820385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1362480127},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1884,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1884,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002145"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":28913,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.362719+0000","last_change":"2024-09-17T15:38:06.362595+0000","last_active":"2024-09-17T15:38:06.362719+0000","last_peered":"2024-09-17T15:38:06.362719+0000","last_clean":"2024-09-17T15:38:06.362719+0000","last_became_active":"2024-09-17T15:38:06.284139+0000","last_became_peered":"2024-09-17T15:38:06.284139+0000","last_unstale":"2024-09-17T15:38:06.362719+0000","last_undegraded":"2024-09-17T15:38:06.362719+0000","last_fullsized":"2024-09-17T15:38:06.362719+0000","mapping_epoch":60,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:42:51.418256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4070,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21074,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276467+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:38:06.276467+0000","last_peered":"2024-09-17T15:38:06.276467+0000","last_clean":"2024-09-17T15:38:06.276467+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:38:06.276467+0000","last_undegraded":"2024-09-17T15:38:06.276467+0000","last_fullsized":"2024-09-17T15:38:06.276467+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21182,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277370+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:38:06.277370+0000","last_peered":"2024-09-17T15:38:06.277370+0000","last_clean":"2024-09-17T15:38:06.277370+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:38:06.277370+0000","last_undegraded":"2024-09-17T15:38:06.277370+0000","last_fullsized":"2024-09-17T15:38:06.277370+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21079,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276509+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:38:06.276509+0000","last_peered":"2024-09-17T15:38:06.276509+0000","last_clean":"2024-09-17T15:38:06.276509+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:38:06.276509+0000","last_undegraded":"2024-09-17T15:38:06.276509+0000","last_fullsized":"2024-09-17T15:38:06.276509+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20822,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.283149+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:38:06.283149+0000","last_peered":"2024-09-17T15:38:06.283149+0000","last_clean":"2024-09-17T15:38:06.283149+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:38:06.283149+0000","last_undegraded":"2024-09-17T15:38:06.283149+0000","last_fullsized":"2024-09-17T15:38:06.283149+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21573,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276969+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:38:06.276969+0000","last_peered":"2024-09-17T15:38:06.276969+0000","last_clean":"2024-09-17T15:38:06.276969+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:38:06.276969+0000","last_undegraded":"2024-09-17T15:38:06.276969+0000","last_fullsized":"2024-09-17T15:38:06.276969+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21116,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276860+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:38:06.276860+0000","last_peered":"2024-09-17T15:38:06.276860+0000","last_clean":"2024-09-17T15:38:06.276860+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:38:06.276860+0000","last_undegraded":"2024-09-17T15:38:06.276860+0000","last_fullsized":"2024-09-17T15:38:06.276860+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21084,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.281124+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:38:06.281124+0000","last_peered":"2024-09-17T15:38:06.281124+0000","last_clean":"2024-09-17T15:38:06.281124+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:38:06.281124+0000","last_undegraded":"2024-09-17T15:38:06.281124+0000","last_fullsized":"2024-09-17T15:38:06.281124+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":107,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276773+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:38:06.276773+0000","last_peered":"2024-09-17T15:38:06.276773+0000","last_clean":"2024-09-17T15:38:06.276773+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:38:06.276773+0000","last_undegraded":"2024-09-17T15:38:06.276773+0000","last_fullsized":"2024-09-17T15:38:06.276773+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29139,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277107+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:38:06.277107+0000","last_peered":"2024-09-17T15:38:06.277107+0000","last_clean":"2024-09-17T15:38:06.277107+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:38:06.277107+0000","last_undegraded":"2024-09-17T15:38:06.277107+0000","last_fullsized":"2024-09-17T15:38:06.277107+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.275989+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:38:06.275989+0000","last_peered":"2024-09-17T15:38:06.275989+0000","last_clean":"2024-09-17T15:38:06.275989+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:38:06.275989+0000","last_undegraded":"2024-09-17T15:38:06.275989+0000","last_fullsized":"2024-09-17T15:38:06.275989+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20846,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:06.281521+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:38:06.281521+0000","last_peered":"2024-09-17T15:38:06.281521+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:38:06.281521+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":143,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.281361+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:38:06.281361+0000","last_peered":"2024-09-17T15:38:06.281361+0000","last_clean":"2024-09-17T15:38:06.281361+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:38:06.281361+0000","last_undegraded":"2024-09-17T15:38:06.281361+0000","last_fullsized":"2024-09-17T15:38:06.281361+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28844,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276074+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:38:06.276074+0000","last_peered":"2024-09-17T15:38:06.276074+0000","last_clean":"2024-09-17T15:38:06.276074+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:38:06.276074+0000","last_undegraded":"2024-09-17T15:38:06.276074+0000","last_fullsized":"2024-09-17T15:38:06.276074+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":117,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277066+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:38:06.277066+0000","last_peered":"2024-09-17T15:38:06.277066+0000","last_clean":"2024-09-17T15:38:06.277066+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:38:06.277066+0000","last_undegraded":"2024-09-17T15:38:06.277066+0000","last_fullsized":"2024-09-17T15:38:06.277066+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29181,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.275684+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:38:06.275684+0000","last_peered":"2024-09-17T15:38:06.275684+0000","last_clean":"2024-09-17T15:38:06.275684+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:38:06.275684+0000","last_undegraded":"2024-09-17T15:38:06.275684+0000","last_fullsized":"2024-09-17T15:38:06.275684+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21291,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277656+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:38:06.277656+0000","last_peered":"2024-09-17T15:38:06.277656+0000","last_clean":"2024-09-17T15:38:06.277656+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:38:06.277656+0000","last_undegraded":"2024-09-17T15:38:06.277656+0000","last_fullsized":"2024-09-17T15:38:06.277656+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":107,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277209+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:38:06.277209+0000","last_peered":"2024-09-17T15:38:06.277209+0000","last_clean":"2024-09-17T15:38:06.277209+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:38:06.277209+0000","last_undegraded":"2024-09-17T15:38:06.277209+0000","last_fullsized":"2024-09-17T15:38:06.277209+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28095,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277618+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:38:06.277618+0000","last_peered":"2024-09-17T15:38:06.277618+0000","last_clean":"2024-09-17T15:38:06.277618+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:38:06.277618+0000","last_undegraded":"2024-09-17T15:38:06.277618+0000","last_fullsized":"2024-09-17T15:38:06.277618+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276036+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:38:06.276036+0000","last_peered":"2024-09-17T15:38:06.276036+0000","last_clean":"2024-09-17T15:38:06.276036+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:38:06.276036+0000","last_undegraded":"2024-09-17T15:38:06.276036+0000","last_fullsized":"2024-09-17T15:38:06.276036+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":26431,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:19.433206+0000","last_change":"2024-09-17T15:38:05.220032+0000","last_active":"2024-09-17T15:38:19.433206+0000","last_peered":"2024-09-17T15:38:19.433206+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:38:19.433206+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1379,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2686,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":107,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277568+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:38:06.277568+0000","last_peered":"2024-09-17T15:38:06.277568+0000","last_clean":"2024-09-17T15:38:06.277568+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:38:06.277568+0000","last_undegraded":"2024-09-17T15:38:06.277568+0000","last_fullsized":"2024-09-17T15:38:06.277568+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":112,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277607+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:38:06.277607+0000","last_peered":"2024-09-17T15:38:06.277607+0000","last_clean":"2024-09-17T15:38:06.277607+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:38:06.277607+0000","last_undegraded":"2024-09-17T15:38:06.277607+0000","last_fullsized":"2024-09-17T15:38:06.277607+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29046,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277163+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:38:06.277163+0000","last_peered":"2024-09-17T15:38:06.277163+0000","last_clean":"2024-09-17T15:38:06.277163+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:38:06.277163+0000","last_undegraded":"2024-09-17T15:38:06.277163+0000","last_fullsized":"2024-09-17T15:38:06.277163+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5324,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44136,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":151023616,"data_stored":150936904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051744,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":434860,"kb_used_data":82460,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93936980,"statfs":{"total":96636764160,"available":96191467520,"internally_reserved":0,"allocated":84439040,"data_stored":84084672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724688,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505640,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866200,"statfs":{"total":96636764160,"available":96118988800,"internally_reserved":0,"allocated":548864,"data_stored":175967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":517205675},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247971,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539456,"kb_used_data":66344,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832384,"statfs":{"total":96636764160,"available":96084361216,"internally_reserved":0,"allocated":67936256,"data_stored":67559746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":83562496,"data_stored":83544431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:38:27.329 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:38:27.700 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:27.700 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:38:27.714 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":477,"stamp":"2024-09-17T15:38:27.335791+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4312,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45149,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1471636,"kb_used_data":149340,"kb_used_omap":13,"kb_used_meta":1322226,"kb_avail":281643884,"statfs":{"total":289910292480,"available":288403337216,"internally_reserved":0,"allocated":152924160,"data_stored":151820385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1353960447},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1870,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.496795"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":28913,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.362719+0000","last_change":"2024-09-17T15:38:06.362595+0000","last_active":"2024-09-17T15:38:06.362719+0000","last_peered":"2024-09-17T15:38:06.362719+0000","last_clean":"2024-09-17T15:38:06.362719+0000","last_became_active":"2024-09-17T15:38:06.284139+0000","last_became_peered":"2024-09-17T15:38:06.284139+0000","last_unstale":"2024-09-17T15:38:06.362719+0000","last_undegraded":"2024-09-17T15:38:06.362719+0000","last_fullsized":"2024-09-17T15:38:06.362719+0000","mapping_epoch":60,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:42:51.418256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4070,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21074,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276467+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:38:06.276467+0000","last_peered":"2024-09-17T15:38:06.276467+0000","last_clean":"2024-09-17T15:38:06.276467+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:38:06.276467+0000","last_undegraded":"2024-09-17T15:38:06.276467+0000","last_fullsized":"2024-09-17T15:38:06.276467+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21182,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277370+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:38:06.277370+0000","last_peered":"2024-09-17T15:38:06.277370+0000","last_clean":"2024-09-17T15:38:06.277370+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:38:06.277370+0000","last_undegraded":"2024-09-17T15:38:06.277370+0000","last_fullsized":"2024-09-17T15:38:06.277370+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21079,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276509+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:38:06.276509+0000","last_peered":"2024-09-17T15:38:06.276509+0000","last_clean":"2024-09-17T15:38:06.276509+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:38:06.276509+0000","last_undegraded":"2024-09-17T15:38:06.276509+0000","last_fullsized":"2024-09-17T15:38:06.276509+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20822,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.283149+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:38:06.283149+0000","last_peered":"2024-09-17T15:38:06.283149+0000","last_clean":"2024-09-17T15:38:06.283149+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:38:06.283149+0000","last_undegraded":"2024-09-17T15:38:06.283149+0000","last_fullsized":"2024-09-17T15:38:06.283149+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21573,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276969+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:38:06.276969+0000","last_peered":"2024-09-17T15:38:06.276969+0000","last_clean":"2024-09-17T15:38:06.276969+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:38:06.276969+0000","last_undegraded":"2024-09-17T15:38:06.276969+0000","last_fullsized":"2024-09-17T15:38:06.276969+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21116,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276860+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:38:06.276860+0000","last_peered":"2024-09-17T15:38:06.276860+0000","last_clean":"2024-09-17T15:38:06.276860+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:38:06.276860+0000","last_undegraded":"2024-09-17T15:38:06.276860+0000","last_fullsized":"2024-09-17T15:38:06.276860+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21084,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.281124+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:38:06.281124+0000","last_peered":"2024-09-17T15:38:06.281124+0000","last_clean":"2024-09-17T15:38:06.281124+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:38:06.281124+0000","last_undegraded":"2024-09-17T15:38:06.281124+0000","last_fullsized":"2024-09-17T15:38:06.281124+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":108,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276842+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:38:06.276842+0000","last_peered":"2024-09-17T15:38:06.276842+0000","last_clean":"2024-09-17T15:38:06.276842+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:38:06.276842+0000","last_undegraded":"2024-09-17T15:38:06.276842+0000","last_fullsized":"2024-09-17T15:38:06.276842+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29139,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277107+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:38:06.277107+0000","last_peered":"2024-09-17T15:38:06.277107+0000","last_clean":"2024-09-17T15:38:06.277107+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:38:06.277107+0000","last_undegraded":"2024-09-17T15:38:06.277107+0000","last_fullsized":"2024-09-17T15:38:06.277107+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.275989+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:38:06.275989+0000","last_peered":"2024-09-17T15:38:06.275989+0000","last_clean":"2024-09-17T15:38:06.275989+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:38:06.275989+0000","last_undegraded":"2024-09-17T15:38:06.275989+0000","last_fullsized":"2024-09-17T15:38:06.275989+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20846,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:06.281521+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:38:06.281521+0000","last_peered":"2024-09-17T15:38:06.281521+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:38:06.281521+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":143,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.281361+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:38:06.281361+0000","last_peered":"2024-09-17T15:38:06.281361+0000","last_clean":"2024-09-17T15:38:06.281361+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:38:06.281361+0000","last_undegraded":"2024-09-17T15:38:06.281361+0000","last_fullsized":"2024-09-17T15:38:06.281361+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28844,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276074+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:38:06.276074+0000","last_peered":"2024-09-17T15:38:06.276074+0000","last_clean":"2024-09-17T15:38:06.276074+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:38:06.276074+0000","last_undegraded":"2024-09-17T15:38:06.276074+0000","last_fullsized":"2024-09-17T15:38:06.276074+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":117,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277066+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:38:06.277066+0000","last_peered":"2024-09-17T15:38:06.277066+0000","last_clean":"2024-09-17T15:38:06.277066+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:38:06.277066+0000","last_undegraded":"2024-09-17T15:38:06.277066+0000","last_fullsized":"2024-09-17T15:38:06.277066+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29181,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.275684+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:38:06.275684+0000","last_peered":"2024-09-17T15:38:06.275684+0000","last_clean":"2024-09-17T15:38:06.275684+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:38:06.275684+0000","last_undegraded":"2024-09-17T15:38:06.275684+0000","last_fullsized":"2024-09-17T15:38:06.275684+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21291,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277656+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:38:06.277656+0000","last_peered":"2024-09-17T15:38:06.277656+0000","last_clean":"2024-09-17T15:38:06.277656+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:38:06.277656+0000","last_undegraded":"2024-09-17T15:38:06.277656+0000","last_fullsized":"2024-09-17T15:38:06.277656+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":107,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277209+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:38:06.277209+0000","last_peered":"2024-09-17T15:38:06.277209+0000","last_clean":"2024-09-17T15:38:06.277209+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:38:06.277209+0000","last_undegraded":"2024-09-17T15:38:06.277209+0000","last_fullsized":"2024-09-17T15:38:06.277209+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28095,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277618+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:38:06.277618+0000","last_peered":"2024-09-17T15:38:06.277618+0000","last_clean":"2024-09-17T15:38:06.277618+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:38:06.277618+0000","last_undegraded":"2024-09-17T15:38:06.277618+0000","last_fullsized":"2024-09-17T15:38:06.277618+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276036+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:38:06.276036+0000","last_peered":"2024-09-17T15:38:06.276036+0000","last_clean":"2024-09-17T15:38:06.276036+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:38:06.276036+0000","last_undegraded":"2024-09-17T15:38:06.276036+0000","last_fullsized":"2024-09-17T15:38:06.276036+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":28455,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:24.435236+0000","last_change":"2024-09-17T15:38:05.220032+0000","last_active":"2024-09-17T15:38:24.435236+0000","last_peered":"2024-09-17T15:38:24.435236+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:38:24.435236+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3698,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-367}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":107,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277568+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:38:06.277568+0000","last_peered":"2024-09-17T15:38:06.277568+0000","last_clean":"2024-09-17T15:38:06.277568+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:38:06.277568+0000","last_undegraded":"2024-09-17T15:38:06.277568+0000","last_fullsized":"2024-09-17T15:38:06.277568+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":112,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277607+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:38:06.277607+0000","last_peered":"2024-09-17T15:38:06.277607+0000","last_clean":"2024-09-17T15:38:06.277607+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:38:06.277607+0000","last_undegraded":"2024-09-17T15:38:06.277607+0000","last_fullsized":"2024-09-17T15:38:06.277607+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29046,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277163+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:38:06.277163+0000","last_peered":"2024-09-17T15:38:06.277163+0000","last_clean":"2024-09-17T15:38:06.277163+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:38:06.277163+0000","last_undegraded":"2024-09-17T15:38:06.277163+0000","last_fullsized":"2024-09-17T15:38:06.277163+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4312,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45148,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":151023616,"data_stored":150936904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051744,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":434860,"kb_used_data":82460,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93936980,"statfs":{"total":96636764160,"available":96191467520,"internally_reserved":0,"allocated":84439040,"data_stored":84084672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724689,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497320,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874520,"statfs":{"total":96636764160,"available":96127508480,"internally_reserved":0,"allocated":548864,"data_stored":175967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":508685995},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247972,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539456,"kb_used_data":66344,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832384,"statfs":{"total":96636764160,"available":96084361216,"internally_reserved":0,"allocated":67936256,"data_stored":67559746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":83562496,"data_stored":83544431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:38:27.716 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:38:28.070 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:28.070 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:38:28.083 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":478,"stamp":"2024-09-17T15:38:27.840874+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4312,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45149,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1471636,"kb_used_data":149340,"kb_used_omap":13,"kb_used_meta":1322226,"kb_avail":281643884,"statfs":{"total":289910292480,"available":288403337216,"internally_reserved":0,"allocated":152924160,"data_stored":151820385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1353960447},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1012,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001590"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":28913,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.362719+0000","last_change":"2024-09-17T15:38:06.362595+0000","last_active":"2024-09-17T15:38:06.362719+0000","last_peered":"2024-09-17T15:38:06.362719+0000","last_clean":"2024-09-17T15:38:06.362719+0000","last_became_active":"2024-09-17T15:38:06.284139+0000","last_became_peered":"2024-09-17T15:38:06.284139+0000","last_unstale":"2024-09-17T15:38:06.362719+0000","last_undegraded":"2024-09-17T15:38:06.362719+0000","last_fullsized":"2024-09-17T15:38:06.362719+0000","mapping_epoch":60,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:42:51.418256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4070,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21074,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276467+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:38:06.276467+0000","last_peered":"2024-09-17T15:38:06.276467+0000","last_clean":"2024-09-17T15:38:06.276467+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:38:06.276467+0000","last_undegraded":"2024-09-17T15:38:06.276467+0000","last_fullsized":"2024-09-17T15:38:06.276467+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21182,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277370+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:38:06.277370+0000","last_peered":"2024-09-17T15:38:06.277370+0000","last_clean":"2024-09-17T15:38:06.277370+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:38:06.277370+0000","last_undegraded":"2024-09-17T15:38:06.277370+0000","last_fullsized":"2024-09-17T15:38:06.277370+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21079,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276509+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:38:06.276509+0000","last_peered":"2024-09-17T15:38:06.276509+0000","last_clean":"2024-09-17T15:38:06.276509+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:38:06.276509+0000","last_undegraded":"2024-09-17T15:38:06.276509+0000","last_fullsized":"2024-09-17T15:38:06.276509+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20822,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.283149+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:38:06.283149+0000","last_peered":"2024-09-17T15:38:06.283149+0000","last_clean":"2024-09-17T15:38:06.283149+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:38:06.283149+0000","last_undegraded":"2024-09-17T15:38:06.283149+0000","last_fullsized":"2024-09-17T15:38:06.283149+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21573,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276969+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:38:06.276969+0000","last_peered":"2024-09-17T15:38:06.276969+0000","last_clean":"2024-09-17T15:38:06.276969+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:38:06.276969+0000","last_undegraded":"2024-09-17T15:38:06.276969+0000","last_fullsized":"2024-09-17T15:38:06.276969+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21116,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276860+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:38:06.276860+0000","last_peered":"2024-09-17T15:38:06.276860+0000","last_clean":"2024-09-17T15:38:06.276860+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:38:06.276860+0000","last_undegraded":"2024-09-17T15:38:06.276860+0000","last_fullsized":"2024-09-17T15:38:06.276860+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21084,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.281124+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:38:06.281124+0000","last_peered":"2024-09-17T15:38:06.281124+0000","last_clean":"2024-09-17T15:38:06.281124+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:38:06.281124+0000","last_undegraded":"2024-09-17T15:38:06.281124+0000","last_fullsized":"2024-09-17T15:38:06.281124+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":108,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276842+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:38:06.276842+0000","last_peered":"2024-09-17T15:38:06.276842+0000","last_clean":"2024-09-17T15:38:06.276842+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:38:06.276842+0000","last_undegraded":"2024-09-17T15:38:06.276842+0000","last_fullsized":"2024-09-17T15:38:06.276842+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29139,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277107+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:38:06.277107+0000","last_peered":"2024-09-17T15:38:06.277107+0000","last_clean":"2024-09-17T15:38:06.277107+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:38:06.277107+0000","last_undegraded":"2024-09-17T15:38:06.277107+0000","last_fullsized":"2024-09-17T15:38:06.277107+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.275989+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:38:06.275989+0000","last_peered":"2024-09-17T15:38:06.275989+0000","last_clean":"2024-09-17T15:38:06.275989+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:38:06.275989+0000","last_undegraded":"2024-09-17T15:38:06.275989+0000","last_fullsized":"2024-09-17T15:38:06.275989+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20846,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:06.281521+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:38:06.281521+0000","last_peered":"2024-09-17T15:38:06.281521+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:38:06.281521+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":143,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.281361+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:38:06.281361+0000","last_peered":"2024-09-17T15:38:06.281361+0000","last_clean":"2024-09-17T15:38:06.281361+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:38:06.281361+0000","last_undegraded":"2024-09-17T15:38:06.281361+0000","last_fullsized":"2024-09-17T15:38:06.281361+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28844,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276074+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:38:06.276074+0000","last_peered":"2024-09-17T15:38:06.276074+0000","last_clean":"2024-09-17T15:38:06.276074+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:38:06.276074+0000","last_undegraded":"2024-09-17T15:38:06.276074+0000","last_fullsized":"2024-09-17T15:38:06.276074+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":117,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277066+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:38:06.277066+0000","last_peered":"2024-09-17T15:38:06.277066+0000","last_clean":"2024-09-17T15:38:06.277066+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:38:06.277066+0000","last_undegraded":"2024-09-17T15:38:06.277066+0000","last_fullsized":"2024-09-17T15:38:06.277066+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29181,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.275684+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:38:06.275684+0000","last_peered":"2024-09-17T15:38:06.275684+0000","last_clean":"2024-09-17T15:38:06.275684+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:38:06.275684+0000","last_undegraded":"2024-09-17T15:38:06.275684+0000","last_fullsized":"2024-09-17T15:38:06.275684+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21291,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277656+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:38:06.277656+0000","last_peered":"2024-09-17T15:38:06.277656+0000","last_clean":"2024-09-17T15:38:06.277656+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:38:06.277656+0000","last_undegraded":"2024-09-17T15:38:06.277656+0000","last_fullsized":"2024-09-17T15:38:06.277656+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":107,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277209+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:38:06.277209+0000","last_peered":"2024-09-17T15:38:06.277209+0000","last_clean":"2024-09-17T15:38:06.277209+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:38:06.277209+0000","last_undegraded":"2024-09-17T15:38:06.277209+0000","last_fullsized":"2024-09-17T15:38:06.277209+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28095,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277618+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:38:06.277618+0000","last_peered":"2024-09-17T15:38:06.277618+0000","last_clean":"2024-09-17T15:38:06.277618+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:38:06.277618+0000","last_undegraded":"2024-09-17T15:38:06.277618+0000","last_fullsized":"2024-09-17T15:38:06.277618+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276036+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:38:06.276036+0000","last_peered":"2024-09-17T15:38:06.276036+0000","last_clean":"2024-09-17T15:38:06.276036+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:38:06.276036+0000","last_undegraded":"2024-09-17T15:38:06.276036+0000","last_fullsized":"2024-09-17T15:38:06.276036+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":28455,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:24.435236+0000","last_change":"2024-09-17T15:38:05.220032+0000","last_active":"2024-09-17T15:38:24.435236+0000","last_peered":"2024-09-17T15:38:24.435236+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:38:24.435236+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3698,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-367}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":107,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277568+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:38:06.277568+0000","last_peered":"2024-09-17T15:38:06.277568+0000","last_clean":"2024-09-17T15:38:06.277568+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:38:06.277568+0000","last_undegraded":"2024-09-17T15:38:06.277568+0000","last_fullsized":"2024-09-17T15:38:06.277568+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":112,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277607+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:38:06.277607+0000","last_peered":"2024-09-17T15:38:06.277607+0000","last_clean":"2024-09-17T15:38:06.277607+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:38:06.277607+0000","last_undegraded":"2024-09-17T15:38:06.277607+0000","last_fullsized":"2024-09-17T15:38:06.277607+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29046,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277163+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:38:06.277163+0000","last_peered":"2024-09-17T15:38:06.277163+0000","last_clean":"2024-09-17T15:38:06.277163+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:38:06.277163+0000","last_undegraded":"2024-09-17T15:38:06.277163+0000","last_fullsized":"2024-09-17T15:38:06.277163+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4312,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45148,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":151023616,"data_stored":150936904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051744,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":434860,"kb_used_data":82460,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93936980,"statfs":{"total":96636764160,"available":96191467520,"internally_reserved":0,"allocated":84439040,"data_stored":84084672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724689,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497320,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874520,"statfs":{"total":96636764160,"available":96127508480,"internally_reserved":0,"allocated":548864,"data_stored":175967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":508685995},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247972,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539456,"kb_used_data":66344,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832384,"statfs":{"total":96636764160,"available":96084361216,"internally_reserved":0,"allocated":67936256,"data_stored":67559746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":83562496,"data_stored":83544431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:38:28.084 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:38:28.084 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20542", 'reported_seq': 20846, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:38:06.281521+0000', 'last_change': '2024-09-17T15:26:22.753615+0000', 'last_active': '2024-09-17T15:38:06.281521+0000', 'last_peered': '2024-09-17T15:38:06.281521+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:38:06.281521+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17138, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3943}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:38:28.084 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-17T15:38:28.084 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "47'20886", 'reported_seq': 28455, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:38:24.435236+0000', 'last_change': '2024-09-17T15:38:05.220032+0000', 'last_active': '2024-09-17T15:38:24.435236+0000', 'last_peered': '2024-09-17T15:38:24.435236+0000', 'last_clean': '2024-09-17T15:26:20.148104+0000', 'last_became_active': '2024-09-17T15:26:22.740386+0000', 'last_became_peered': '2024-09-17T15:26:22.740386+0000', 'last_unstale': '2024-09-17T15:38:24.435236+0000', 'last_undegraded': '2024-09-17T15:26:22.268192+0000', 'last_fullsized': '2024-09-17T15:26:22.190864+0000', 'mapping_epoch': 40, 'log_start': "43'10800", 'ondisk_log_start': "43'10800", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3221", 'last_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:07.177603+0000', 'objects_scrubbed': 3209, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 528, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17673, 'num_write_kb': 28928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3698, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -367}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:38:28.085 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:38:28.545 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:28.546 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 4312/24 objects degraded (17966.667%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":845,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1506955264,"bytes_avail":288403337216,"bytes_total":289910292480,"degraded_objects":4312,"degraded_total":24,"degraded_ratio":179.66666666666663,"recovering_objects_per_sec":126,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1012,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:38:28.559 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:38:28.559 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:38:28.897 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:28.898 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:38:28.910 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":478,"stamp":"2024-09-17T15:38:27.840874+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4312,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45149,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1471636,"kb_used_data":149340,"kb_used_omap":13,"kb_used_meta":1322226,"kb_avail":281643884,"statfs":{"total":289910292480,"available":288403337216,"internally_reserved":0,"allocated":152924160,"data_stored":151820385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1353960447},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1012,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001590"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":28913,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.362719+0000","last_change":"2024-09-17T15:38:06.362595+0000","last_active":"2024-09-17T15:38:06.362719+0000","last_peered":"2024-09-17T15:38:06.362719+0000","last_clean":"2024-09-17T15:38:06.362719+0000","last_became_active":"2024-09-17T15:38:06.284139+0000","last_became_peered":"2024-09-17T15:38:06.284139+0000","last_unstale":"2024-09-17T15:38:06.362719+0000","last_undegraded":"2024-09-17T15:38:06.362719+0000","last_fullsized":"2024-09-17T15:38:06.362719+0000","mapping_epoch":60,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:42:51.418256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4070,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21074,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276467+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:38:06.276467+0000","last_peered":"2024-09-17T15:38:06.276467+0000","last_clean":"2024-09-17T15:38:06.276467+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:38:06.276467+0000","last_undegraded":"2024-09-17T15:38:06.276467+0000","last_fullsized":"2024-09-17T15:38:06.276467+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21182,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277370+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:38:06.277370+0000","last_peered":"2024-09-17T15:38:06.277370+0000","last_clean":"2024-09-17T15:38:06.277370+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:38:06.277370+0000","last_undegraded":"2024-09-17T15:38:06.277370+0000","last_fullsized":"2024-09-17T15:38:06.277370+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21079,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276509+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:38:06.276509+0000","last_peered":"2024-09-17T15:38:06.276509+0000","last_clean":"2024-09-17T15:38:06.276509+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:38:06.276509+0000","last_undegraded":"2024-09-17T15:38:06.276509+0000","last_fullsized":"2024-09-17T15:38:06.276509+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20822,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.283149+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:38:06.283149+0000","last_peered":"2024-09-17T15:38:06.283149+0000","last_clean":"2024-09-17T15:38:06.283149+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:38:06.283149+0000","last_undegraded":"2024-09-17T15:38:06.283149+0000","last_fullsized":"2024-09-17T15:38:06.283149+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21573,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276969+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:38:06.276969+0000","last_peered":"2024-09-17T15:38:06.276969+0000","last_clean":"2024-09-17T15:38:06.276969+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:38:06.276969+0000","last_undegraded":"2024-09-17T15:38:06.276969+0000","last_fullsized":"2024-09-17T15:38:06.276969+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21116,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276860+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:38:06.276860+0000","last_peered":"2024-09-17T15:38:06.276860+0000","last_clean":"2024-09-17T15:38:06.276860+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:38:06.276860+0000","last_undegraded":"2024-09-17T15:38:06.276860+0000","last_fullsized":"2024-09-17T15:38:06.276860+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21084,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.281124+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:38:06.281124+0000","last_peered":"2024-09-17T15:38:06.281124+0000","last_clean":"2024-09-17T15:38:06.281124+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:38:06.281124+0000","last_undegraded":"2024-09-17T15:38:06.281124+0000","last_fullsized":"2024-09-17T15:38:06.281124+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":108,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276842+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:38:06.276842+0000","last_peered":"2024-09-17T15:38:06.276842+0000","last_clean":"2024-09-17T15:38:06.276842+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:38:06.276842+0000","last_undegraded":"2024-09-17T15:38:06.276842+0000","last_fullsized":"2024-09-17T15:38:06.276842+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29139,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277107+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:38:06.277107+0000","last_peered":"2024-09-17T15:38:06.277107+0000","last_clean":"2024-09-17T15:38:06.277107+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:38:06.277107+0000","last_undegraded":"2024-09-17T15:38:06.277107+0000","last_fullsized":"2024-09-17T15:38:06.277107+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.275989+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:38:06.275989+0000","last_peered":"2024-09-17T15:38:06.275989+0000","last_clean":"2024-09-17T15:38:06.275989+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:38:06.275989+0000","last_undegraded":"2024-09-17T15:38:06.275989+0000","last_fullsized":"2024-09-17T15:38:06.275989+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":20846,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:06.281521+0000","last_change":"2024-09-17T15:26:22.753615+0000","last_active":"2024-09-17T15:38:06.281521+0000","last_peered":"2024-09-17T15:38:06.281521+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:38:06.281521+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":143,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.281361+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:38:06.281361+0000","last_peered":"2024-09-17T15:38:06.281361+0000","last_clean":"2024-09-17T15:38:06.281361+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:38:06.281361+0000","last_undegraded":"2024-09-17T15:38:06.281361+0000","last_fullsized":"2024-09-17T15:38:06.281361+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28844,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276074+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:38:06.276074+0000","last_peered":"2024-09-17T15:38:06.276074+0000","last_clean":"2024-09-17T15:38:06.276074+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:38:06.276074+0000","last_undegraded":"2024-09-17T15:38:06.276074+0000","last_fullsized":"2024-09-17T15:38:06.276074+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":117,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277066+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:38:06.277066+0000","last_peered":"2024-09-17T15:38:06.277066+0000","last_clean":"2024-09-17T15:38:06.277066+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:38:06.277066+0000","last_undegraded":"2024-09-17T15:38:06.277066+0000","last_fullsized":"2024-09-17T15:38:06.277066+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29181,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.275684+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:38:06.275684+0000","last_peered":"2024-09-17T15:38:06.275684+0000","last_clean":"2024-09-17T15:38:06.275684+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:38:06.275684+0000","last_undegraded":"2024-09-17T15:38:06.275684+0000","last_fullsized":"2024-09-17T15:38:06.275684+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21291,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277656+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:38:06.277656+0000","last_peered":"2024-09-17T15:38:06.277656+0000","last_clean":"2024-09-17T15:38:06.277656+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:38:06.277656+0000","last_undegraded":"2024-09-17T15:38:06.277656+0000","last_fullsized":"2024-09-17T15:38:06.277656+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":107,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277209+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:38:06.277209+0000","last_peered":"2024-09-17T15:38:06.277209+0000","last_clean":"2024-09-17T15:38:06.277209+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:38:06.277209+0000","last_undegraded":"2024-09-17T15:38:06.277209+0000","last_fullsized":"2024-09-17T15:38:06.277209+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28095,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277618+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:38:06.277618+0000","last_peered":"2024-09-17T15:38:06.277618+0000","last_clean":"2024-09-17T15:38:06.277618+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:38:06.277618+0000","last_undegraded":"2024-09-17T15:38:06.277618+0000","last_fullsized":"2024-09-17T15:38:06.277618+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.276036+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:38:06.276036+0000","last_peered":"2024-09-17T15:38:06.276036+0000","last_clean":"2024-09-17T15:38:06.276036+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:38:06.276036+0000","last_undegraded":"2024-09-17T15:38:06.276036+0000","last_fullsized":"2024-09-17T15:38:06.276036+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":28455,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:24.435236+0000","last_change":"2024-09-17T15:38:05.220032+0000","last_active":"2024-09-17T15:38:24.435236+0000","last_peered":"2024-09-17T15:38:24.435236+0000","last_clean":"2024-09-17T15:26:20.148104+0000","last_became_active":"2024-09-17T15:26:22.740386+0000","last_became_peered":"2024-09-17T15:26:22.740386+0000","last_unstale":"2024-09-17T15:38:24.435236+0000","last_undegraded":"2024-09-17T15:26:22.268192+0000","last_fullsized":"2024-09-17T15:26:22.190864+0000","mapping_epoch":40,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3698,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-367}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":107,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277568+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:38:06.277568+0000","last_peered":"2024-09-17T15:38:06.277568+0000","last_clean":"2024-09-17T15:38:06.277568+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:38:06.277568+0000","last_undegraded":"2024-09-17T15:38:06.277568+0000","last_fullsized":"2024-09-17T15:38:06.277568+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":112,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277607+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:38:06.277607+0000","last_peered":"2024-09-17T15:38:06.277607+0000","last_clean":"2024-09-17T15:38:06.277607+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:38:06.277607+0000","last_undegraded":"2024-09-17T15:38:06.277607+0000","last_fullsized":"2024-09-17T15:38:06.277607+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29046,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-17T15:38:06.277163+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:38:06.277163+0000","last_peered":"2024-09-17T15:38:06.277163+0000","last_clean":"2024-09-17T15:38:06.277163+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:38:06.277163+0000","last_undegraded":"2024-09-17T15:38:06.277163+0000","last_fullsized":"2024-09-17T15:38:06.277163+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4312,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45148,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":151023616,"data_stored":150936904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051744,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":434860,"kb_used_data":82460,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93936980,"statfs":{"total":96636764160,"available":96191467520,"internally_reserved":0,"allocated":84439040,"data_stored":84084672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724689,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497320,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874520,"statfs":{"total":96636764160,"available":96127508480,"internally_reserved":0,"allocated":548864,"data_stored":175967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":508685995},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247972,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539456,"kb_used_data":66344,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832384,"statfs":{"total":96636764160,"available":96084361216,"internally_reserved":0,"allocated":67936256,"data_stored":67559746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":83562496,"data_stored":83544431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:38:31.912 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:38:32.215 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:32.216 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:38:32.228 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":480,"stamp":"2024-09-17T15:38:31.841471+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3462,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45999,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1494340,"kb_used_data":145664,"kb_used_omap":13,"kb_used_meta":1348594,"kb_avail":281621180,"statfs":{"total":289910292480,"available":288380088320,"internally_reserved":0,"allocated":149159936,"data_stored":148030684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1380961279},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1862,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1862,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001262"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":28917,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.343011+0000","last_change":"2024-09-17T15:38:06.362595+0000","last_active":"2024-09-17T15:38:27.343011+0000","last_peered":"2024-09-17T15:38:27.343011+0000","last_clean":"2024-09-17T15:38:27.343011+0000","last_became_active":"2024-09-17T15:38:06.284139+0000","last_became_peered":"2024-09-17T15:38:06.284139+0000","last_unstale":"2024-09-17T15:38:27.343011+0000","last_undegraded":"2024-09-17T15:38:27.343011+0000","last_fullsized":"2024-09-17T15:38:27.343011+0000","mapping_epoch":60,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:42:51.418256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4070,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21078,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.339083+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:38:27.339083+0000","last_peered":"2024-09-17T15:38:27.339083+0000","last_clean":"2024-09-17T15:38:27.339083+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:38:27.339083+0000","last_undegraded":"2024-09-17T15:38:27.339083+0000","last_fullsized":"2024-09-17T15:38:27.339083+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21186,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337203+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:38:27.337203+0000","last_peered":"2024-09-17T15:38:27.337203+0000","last_clean":"2024-09-17T15:38:27.337203+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:38:27.337203+0000","last_undegraded":"2024-09-17T15:38:27.337203+0000","last_fullsized":"2024-09-17T15:38:27.337203+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21083,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338986+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:38:27.338986+0000","last_peered":"2024-09-17T15:38:27.338986+0000","last_clean":"2024-09-17T15:38:27.338986+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:38:27.338986+0000","last_undegraded":"2024-09-17T15:38:27.338986+0000","last_fullsized":"2024-09-17T15:38:27.338986+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20826,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342773+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:38:27.342773+0000","last_peered":"2024-09-17T15:38:27.342773+0000","last_clean":"2024-09-17T15:38:27.342773+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:38:27.342773+0000","last_undegraded":"2024-09-17T15:38:27.342773+0000","last_fullsized":"2024-09-17T15:38:27.342773+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21577,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337752+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:38:27.337752+0000","last_peered":"2024-09-17T15:38:27.337752+0000","last_clean":"2024-09-17T15:38:27.337752+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:38:27.337752+0000","last_undegraded":"2024-09-17T15:38:27.337752+0000","last_fullsized":"2024-09-17T15:38:27.337752+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337115+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:38:27.337115+0000","last_peered":"2024-09-17T15:38:27.337115+0000","last_clean":"2024-09-17T15:38:27.337115+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:38:27.337115+0000","last_undegraded":"2024-09-17T15:38:27.337115+0000","last_fullsized":"2024-09-17T15:38:27.337115+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21088,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342372+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:38:27.342372+0000","last_peered":"2024-09-17T15:38:27.342372+0000","last_clean":"2024-09-17T15:38:27.342372+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:38:27.342372+0000","last_undegraded":"2024-09-17T15:38:27.342372+0000","last_fullsized":"2024-09-17T15:38:27.342372+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337064+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:38:27.337064+0000","last_peered":"2024-09-17T15:38:27.337064+0000","last_clean":"2024-09-17T15:38:27.337064+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:38:27.337064+0000","last_undegraded":"2024-09-17T15:38:27.337064+0000","last_fullsized":"2024-09-17T15:38:27.337064+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29143,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337570+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:38:27.337570+0000","last_peered":"2024-09-17T15:38:27.337570+0000","last_clean":"2024-09-17T15:38:27.337570+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:38:27.337570+0000","last_undegraded":"2024-09-17T15:38:27.337570+0000","last_fullsized":"2024-09-17T15:38:27.337570+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338510+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:38:27.338510+0000","last_peered":"2024-09-17T15:38:27.338510+0000","last_clean":"2024-09-17T15:38:27.338510+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:38:27.338510+0000","last_undegraded":"2024-09-17T15:38:27.338510+0000","last_fullsized":"2024-09-17T15:38:27.338510+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":21817,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:29.435741+0000","last_change":"2024-09-17T15:38:25.720175+0000","last_active":"2024-09-17T15:38:29.435741+0000","last_peered":"2024-09-17T15:38:29.435741+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:38:29.435741+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3462,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":550,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3462}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":147,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342526+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:38:27.342526+0000","last_peered":"2024-09-17T15:38:27.342526+0000","last_clean":"2024-09-17T15:38:27.342526+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:38:27.342526+0000","last_undegraded":"2024-09-17T15:38:27.342526+0000","last_fullsized":"2024-09-17T15:38:27.342526+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28848,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338595+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:38:27.338595+0000","last_peered":"2024-09-17T15:38:27.338595+0000","last_clean":"2024-09-17T15:38:27.338595+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:38:27.338595+0000","last_undegraded":"2024-09-17T15:38:27.338595+0000","last_fullsized":"2024-09-17T15:38:27.338595+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":121,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337473+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:38:27.337473+0000","last_peered":"2024-09-17T15:38:27.337473+0000","last_clean":"2024-09-17T15:38:27.337473+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:38:27.337473+0000","last_undegraded":"2024-09-17T15:38:27.337473+0000","last_fullsized":"2024-09-17T15:38:27.337473+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29185,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338248+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:38:27.338248+0000","last_peered":"2024-09-17T15:38:27.338248+0000","last_clean":"2024-09-17T15:38:27.338248+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:38:27.338248+0000","last_undegraded":"2024-09-17T15:38:27.338248+0000","last_fullsized":"2024-09-17T15:38:27.338248+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21295,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337742+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:38:27.337742+0000","last_peered":"2024-09-17T15:38:27.337742+0000","last_clean":"2024-09-17T15:38:27.337742+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:38:27.337742+0000","last_undegraded":"2024-09-17T15:38:27.337742+0000","last_fullsized":"2024-09-17T15:38:27.337742+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":111,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338212+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:38:27.338212+0000","last_peered":"2024-09-17T15:38:27.338212+0000","last_clean":"2024-09-17T15:38:27.338212+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:38:27.338212+0000","last_undegraded":"2024-09-17T15:38:27.338212+0000","last_fullsized":"2024-09-17T15:38:27.338212+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28099,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338109+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:38:27.338109+0000","last_peered":"2024-09-17T15:38:27.338109+0000","last_clean":"2024-09-17T15:38:27.338109+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:38:27.338109+0000","last_undegraded":"2024-09-17T15:38:27.338109+0000","last_fullsized":"2024-09-17T15:38:27.338109+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338779+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:38:27.338779+0000","last_peered":"2024-09-17T15:38:27.338779+0000","last_clean":"2024-09-17T15:38:27.338779+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:38:27.338779+0000","last_undegraded":"2024-09-17T15:38:27.338779+0000","last_fullsized":"2024-09-17T15:38:27.338779+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21204,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.430121+0000","last_change":"2024-09-17T15:38:27.429539+0000","last_active":"2024-09-17T15:38:27.430121+0000","last_peered":"2024-09-17T15:38:27.430121+0000","last_clean":"2024-09-17T15:38:27.430121+0000","last_became_active":"2024-09-17T15:38:27.345429+0000","last_became_peered":"2024-09-17T15:38:27.345429+0000","last_unstale":"2024-09-17T15:38:27.430121+0000","last_undegraded":"2024-09-17T15:38:27.430121+0000","last_fullsized":"2024-09-17T15:38:27.430121+0000","mapping_epoch":62,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:49:28.575667+0000","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4065,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":111,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338015+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:38:27.338015+0000","last_peered":"2024-09-17T15:38:27.338015+0000","last_clean":"2024-09-17T15:38:27.338015+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:38:27.338015+0000","last_undegraded":"2024-09-17T15:38:27.338015+0000","last_fullsized":"2024-09-17T15:38:27.338015+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":116,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337667+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:38:27.337667+0000","last_peered":"2024-09-17T15:38:27.337667+0000","last_clean":"2024-09-17T15:38:27.337667+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:38:27.337667+0000","last_undegraded":"2024-09-17T15:38:27.337667+0000","last_fullsized":"2024-09-17T15:38:27.337667+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29050,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338223+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:38:27.338223+0000","last_peered":"2024-09-17T15:38:27.338223+0000","last_clean":"2024-09-17T15:38:27.338223+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:38:27.338223+0000","last_undegraded":"2024-09-17T15:38:27.338223+0000","last_fullsized":"2024-09-17T15:38:27.338223+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3462,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45998,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147087360,"data_stored":147007874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051745,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":431168,"kb_used_data":78760,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93940672,"statfs":{"total":96636764160,"available":96195248128,"internally_reserved":0,"allocated":80650240,"data_stored":80284971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724690,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523700,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":523131,"kb_avail":93848140,"statfs":{"total":96636764160,"available":96100495360,"internally_reserved":0,"allocated":557056,"data_stored":180484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":535686827},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247973,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539472,"kb_used_data":66360,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832368,"statfs":{"total":96636764160,"available":96084344832,"internally_reserved":0,"allocated":67952640,"data_stored":67565229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":79634432,"data_stored":79616367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:38:32.229 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:38:32.566 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:32.566 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:38:32.581 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":480,"stamp":"2024-09-17T15:38:31.841471+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3462,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45999,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1494340,"kb_used_data":145664,"kb_used_omap":13,"kb_used_meta":1348594,"kb_avail":281621180,"statfs":{"total":289910292480,"available":288380088320,"internally_reserved":0,"allocated":149159936,"data_stored":148030684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1380961279},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1862,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1862,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001262"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":28917,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.343011+0000","last_change":"2024-09-17T15:38:06.362595+0000","last_active":"2024-09-17T15:38:27.343011+0000","last_peered":"2024-09-17T15:38:27.343011+0000","last_clean":"2024-09-17T15:38:27.343011+0000","last_became_active":"2024-09-17T15:38:06.284139+0000","last_became_peered":"2024-09-17T15:38:06.284139+0000","last_unstale":"2024-09-17T15:38:27.343011+0000","last_undegraded":"2024-09-17T15:38:27.343011+0000","last_fullsized":"2024-09-17T15:38:27.343011+0000","mapping_epoch":60,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:42:51.418256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4070,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21078,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.339083+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:38:27.339083+0000","last_peered":"2024-09-17T15:38:27.339083+0000","last_clean":"2024-09-17T15:38:27.339083+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:38:27.339083+0000","last_undegraded":"2024-09-17T15:38:27.339083+0000","last_fullsized":"2024-09-17T15:38:27.339083+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21186,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337203+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:38:27.337203+0000","last_peered":"2024-09-17T15:38:27.337203+0000","last_clean":"2024-09-17T15:38:27.337203+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:38:27.337203+0000","last_undegraded":"2024-09-17T15:38:27.337203+0000","last_fullsized":"2024-09-17T15:38:27.337203+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21083,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338986+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:38:27.338986+0000","last_peered":"2024-09-17T15:38:27.338986+0000","last_clean":"2024-09-17T15:38:27.338986+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:38:27.338986+0000","last_undegraded":"2024-09-17T15:38:27.338986+0000","last_fullsized":"2024-09-17T15:38:27.338986+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20826,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342773+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:38:27.342773+0000","last_peered":"2024-09-17T15:38:27.342773+0000","last_clean":"2024-09-17T15:38:27.342773+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:38:27.342773+0000","last_undegraded":"2024-09-17T15:38:27.342773+0000","last_fullsized":"2024-09-17T15:38:27.342773+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21577,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337752+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:38:27.337752+0000","last_peered":"2024-09-17T15:38:27.337752+0000","last_clean":"2024-09-17T15:38:27.337752+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:38:27.337752+0000","last_undegraded":"2024-09-17T15:38:27.337752+0000","last_fullsized":"2024-09-17T15:38:27.337752+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337115+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:38:27.337115+0000","last_peered":"2024-09-17T15:38:27.337115+0000","last_clean":"2024-09-17T15:38:27.337115+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:38:27.337115+0000","last_undegraded":"2024-09-17T15:38:27.337115+0000","last_fullsized":"2024-09-17T15:38:27.337115+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21088,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342372+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:38:27.342372+0000","last_peered":"2024-09-17T15:38:27.342372+0000","last_clean":"2024-09-17T15:38:27.342372+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:38:27.342372+0000","last_undegraded":"2024-09-17T15:38:27.342372+0000","last_fullsized":"2024-09-17T15:38:27.342372+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337064+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:38:27.337064+0000","last_peered":"2024-09-17T15:38:27.337064+0000","last_clean":"2024-09-17T15:38:27.337064+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:38:27.337064+0000","last_undegraded":"2024-09-17T15:38:27.337064+0000","last_fullsized":"2024-09-17T15:38:27.337064+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29143,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337570+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:38:27.337570+0000","last_peered":"2024-09-17T15:38:27.337570+0000","last_clean":"2024-09-17T15:38:27.337570+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:38:27.337570+0000","last_undegraded":"2024-09-17T15:38:27.337570+0000","last_fullsized":"2024-09-17T15:38:27.337570+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338510+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:38:27.338510+0000","last_peered":"2024-09-17T15:38:27.338510+0000","last_clean":"2024-09-17T15:38:27.338510+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:38:27.338510+0000","last_undegraded":"2024-09-17T15:38:27.338510+0000","last_fullsized":"2024-09-17T15:38:27.338510+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":21817,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:29.435741+0000","last_change":"2024-09-17T15:38:25.720175+0000","last_active":"2024-09-17T15:38:29.435741+0000","last_peered":"2024-09-17T15:38:29.435741+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:38:29.435741+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3462,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":550,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3462}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":147,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342526+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:38:27.342526+0000","last_peered":"2024-09-17T15:38:27.342526+0000","last_clean":"2024-09-17T15:38:27.342526+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:38:27.342526+0000","last_undegraded":"2024-09-17T15:38:27.342526+0000","last_fullsized":"2024-09-17T15:38:27.342526+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28848,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338595+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:38:27.338595+0000","last_peered":"2024-09-17T15:38:27.338595+0000","last_clean":"2024-09-17T15:38:27.338595+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:38:27.338595+0000","last_undegraded":"2024-09-17T15:38:27.338595+0000","last_fullsized":"2024-09-17T15:38:27.338595+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":121,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337473+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:38:27.337473+0000","last_peered":"2024-09-17T15:38:27.337473+0000","last_clean":"2024-09-17T15:38:27.337473+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:38:27.337473+0000","last_undegraded":"2024-09-17T15:38:27.337473+0000","last_fullsized":"2024-09-17T15:38:27.337473+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29185,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338248+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:38:27.338248+0000","last_peered":"2024-09-17T15:38:27.338248+0000","last_clean":"2024-09-17T15:38:27.338248+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:38:27.338248+0000","last_undegraded":"2024-09-17T15:38:27.338248+0000","last_fullsized":"2024-09-17T15:38:27.338248+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21295,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337742+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:38:27.337742+0000","last_peered":"2024-09-17T15:38:27.337742+0000","last_clean":"2024-09-17T15:38:27.337742+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:38:27.337742+0000","last_undegraded":"2024-09-17T15:38:27.337742+0000","last_fullsized":"2024-09-17T15:38:27.337742+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":111,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338212+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:38:27.338212+0000","last_peered":"2024-09-17T15:38:27.338212+0000","last_clean":"2024-09-17T15:38:27.338212+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:38:27.338212+0000","last_undegraded":"2024-09-17T15:38:27.338212+0000","last_fullsized":"2024-09-17T15:38:27.338212+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28099,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338109+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:38:27.338109+0000","last_peered":"2024-09-17T15:38:27.338109+0000","last_clean":"2024-09-17T15:38:27.338109+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:38:27.338109+0000","last_undegraded":"2024-09-17T15:38:27.338109+0000","last_fullsized":"2024-09-17T15:38:27.338109+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338779+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:38:27.338779+0000","last_peered":"2024-09-17T15:38:27.338779+0000","last_clean":"2024-09-17T15:38:27.338779+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:38:27.338779+0000","last_undegraded":"2024-09-17T15:38:27.338779+0000","last_fullsized":"2024-09-17T15:38:27.338779+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21204,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.430121+0000","last_change":"2024-09-17T15:38:27.429539+0000","last_active":"2024-09-17T15:38:27.430121+0000","last_peered":"2024-09-17T15:38:27.430121+0000","last_clean":"2024-09-17T15:38:27.430121+0000","last_became_active":"2024-09-17T15:38:27.345429+0000","last_became_peered":"2024-09-17T15:38:27.345429+0000","last_unstale":"2024-09-17T15:38:27.430121+0000","last_undegraded":"2024-09-17T15:38:27.430121+0000","last_fullsized":"2024-09-17T15:38:27.430121+0000","mapping_epoch":62,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:49:28.575667+0000","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4065,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":111,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338015+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:38:27.338015+0000","last_peered":"2024-09-17T15:38:27.338015+0000","last_clean":"2024-09-17T15:38:27.338015+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:38:27.338015+0000","last_undegraded":"2024-09-17T15:38:27.338015+0000","last_fullsized":"2024-09-17T15:38:27.338015+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":116,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337667+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:38:27.337667+0000","last_peered":"2024-09-17T15:38:27.337667+0000","last_clean":"2024-09-17T15:38:27.337667+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:38:27.337667+0000","last_undegraded":"2024-09-17T15:38:27.337667+0000","last_fullsized":"2024-09-17T15:38:27.337667+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29050,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338223+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:38:27.338223+0000","last_peered":"2024-09-17T15:38:27.338223+0000","last_clean":"2024-09-17T15:38:27.338223+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:38:27.338223+0000","last_undegraded":"2024-09-17T15:38:27.338223+0000","last_fullsized":"2024-09-17T15:38:27.338223+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3462,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45998,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147087360,"data_stored":147007874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051745,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":431168,"kb_used_data":78760,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93940672,"statfs":{"total":96636764160,"available":96195248128,"internally_reserved":0,"allocated":80650240,"data_stored":80284971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724690,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523700,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":523131,"kb_avail":93848140,"statfs":{"total":96636764160,"available":96100495360,"internally_reserved":0,"allocated":557056,"data_stored":180484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":535686827},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247973,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539472,"kb_used_data":66360,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832368,"statfs":{"total":96636764160,"available":96084344832,"internally_reserved":0,"allocated":67952640,"data_stored":67565229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":79634432,"data_stored":79616367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:38:32.582 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:38:32.583 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20542", 'reported_seq': 21817, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:38:29.435741+0000', 'last_change': '2024-09-17T15:38:25.720175+0000', 'last_active': '2024-09-17T15:38:29.435741+0000', 'last_peered': '2024-09-17T15:38:29.435741+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:38:29.435741+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3462, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17138, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 550, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3462}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:38:32.583 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:38:32.955 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:32.955 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 3462/24 objects degraded (14425.000%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":849,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1530204160,"bytes_avail":288380088320,"bytes_total":289910292480,"degraded_objects":3462,"degraded_total":24,"degraded_ratio":144.25,"recovering_objects_per_sec":232,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1862,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:38:32.969 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:38:32.970 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:38:33.277 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:33.278 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:38:33.293 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":480,"stamp":"2024-09-17T15:38:31.841471+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3462,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45999,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1494340,"kb_used_data":145664,"kb_used_omap":13,"kb_used_meta":1348594,"kb_avail":281621180,"statfs":{"total":289910292480,"available":288380088320,"internally_reserved":0,"allocated":149159936,"data_stored":148030684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1380961279},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1862,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1862,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001262"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":28917,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.343011+0000","last_change":"2024-09-17T15:38:06.362595+0000","last_active":"2024-09-17T15:38:27.343011+0000","last_peered":"2024-09-17T15:38:27.343011+0000","last_clean":"2024-09-17T15:38:27.343011+0000","last_became_active":"2024-09-17T15:38:06.284139+0000","last_became_peered":"2024-09-17T15:38:06.284139+0000","last_unstale":"2024-09-17T15:38:27.343011+0000","last_undegraded":"2024-09-17T15:38:27.343011+0000","last_fullsized":"2024-09-17T15:38:27.343011+0000","mapping_epoch":60,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:42:51.418256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4070,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21078,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.339083+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:38:27.339083+0000","last_peered":"2024-09-17T15:38:27.339083+0000","last_clean":"2024-09-17T15:38:27.339083+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:38:27.339083+0000","last_undegraded":"2024-09-17T15:38:27.339083+0000","last_fullsized":"2024-09-17T15:38:27.339083+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21186,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337203+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:38:27.337203+0000","last_peered":"2024-09-17T15:38:27.337203+0000","last_clean":"2024-09-17T15:38:27.337203+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:38:27.337203+0000","last_undegraded":"2024-09-17T15:38:27.337203+0000","last_fullsized":"2024-09-17T15:38:27.337203+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21083,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338986+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:38:27.338986+0000","last_peered":"2024-09-17T15:38:27.338986+0000","last_clean":"2024-09-17T15:38:27.338986+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:38:27.338986+0000","last_undegraded":"2024-09-17T15:38:27.338986+0000","last_fullsized":"2024-09-17T15:38:27.338986+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20826,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342773+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:38:27.342773+0000","last_peered":"2024-09-17T15:38:27.342773+0000","last_clean":"2024-09-17T15:38:27.342773+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:38:27.342773+0000","last_undegraded":"2024-09-17T15:38:27.342773+0000","last_fullsized":"2024-09-17T15:38:27.342773+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21577,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337752+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:38:27.337752+0000","last_peered":"2024-09-17T15:38:27.337752+0000","last_clean":"2024-09-17T15:38:27.337752+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:38:27.337752+0000","last_undegraded":"2024-09-17T15:38:27.337752+0000","last_fullsized":"2024-09-17T15:38:27.337752+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337115+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:38:27.337115+0000","last_peered":"2024-09-17T15:38:27.337115+0000","last_clean":"2024-09-17T15:38:27.337115+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:38:27.337115+0000","last_undegraded":"2024-09-17T15:38:27.337115+0000","last_fullsized":"2024-09-17T15:38:27.337115+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21088,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342372+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:38:27.342372+0000","last_peered":"2024-09-17T15:38:27.342372+0000","last_clean":"2024-09-17T15:38:27.342372+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:38:27.342372+0000","last_undegraded":"2024-09-17T15:38:27.342372+0000","last_fullsized":"2024-09-17T15:38:27.342372+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337064+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:38:27.337064+0000","last_peered":"2024-09-17T15:38:27.337064+0000","last_clean":"2024-09-17T15:38:27.337064+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:38:27.337064+0000","last_undegraded":"2024-09-17T15:38:27.337064+0000","last_fullsized":"2024-09-17T15:38:27.337064+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29143,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337570+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:38:27.337570+0000","last_peered":"2024-09-17T15:38:27.337570+0000","last_clean":"2024-09-17T15:38:27.337570+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:38:27.337570+0000","last_undegraded":"2024-09-17T15:38:27.337570+0000","last_fullsized":"2024-09-17T15:38:27.337570+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338510+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:38:27.338510+0000","last_peered":"2024-09-17T15:38:27.338510+0000","last_clean":"2024-09-17T15:38:27.338510+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:38:27.338510+0000","last_undegraded":"2024-09-17T15:38:27.338510+0000","last_fullsized":"2024-09-17T15:38:27.338510+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":21817,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:29.435741+0000","last_change":"2024-09-17T15:38:25.720175+0000","last_active":"2024-09-17T15:38:29.435741+0000","last_peered":"2024-09-17T15:38:29.435741+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:38:29.435741+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3462,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":550,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3462}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":147,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342526+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:38:27.342526+0000","last_peered":"2024-09-17T15:38:27.342526+0000","last_clean":"2024-09-17T15:38:27.342526+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:38:27.342526+0000","last_undegraded":"2024-09-17T15:38:27.342526+0000","last_fullsized":"2024-09-17T15:38:27.342526+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28848,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338595+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:38:27.338595+0000","last_peered":"2024-09-17T15:38:27.338595+0000","last_clean":"2024-09-17T15:38:27.338595+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:38:27.338595+0000","last_undegraded":"2024-09-17T15:38:27.338595+0000","last_fullsized":"2024-09-17T15:38:27.338595+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":121,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337473+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:38:27.337473+0000","last_peered":"2024-09-17T15:38:27.337473+0000","last_clean":"2024-09-17T15:38:27.337473+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:38:27.337473+0000","last_undegraded":"2024-09-17T15:38:27.337473+0000","last_fullsized":"2024-09-17T15:38:27.337473+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29185,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338248+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:38:27.338248+0000","last_peered":"2024-09-17T15:38:27.338248+0000","last_clean":"2024-09-17T15:38:27.338248+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:38:27.338248+0000","last_undegraded":"2024-09-17T15:38:27.338248+0000","last_fullsized":"2024-09-17T15:38:27.338248+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21295,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337742+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:38:27.337742+0000","last_peered":"2024-09-17T15:38:27.337742+0000","last_clean":"2024-09-17T15:38:27.337742+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:38:27.337742+0000","last_undegraded":"2024-09-17T15:38:27.337742+0000","last_fullsized":"2024-09-17T15:38:27.337742+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":111,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338212+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:38:27.338212+0000","last_peered":"2024-09-17T15:38:27.338212+0000","last_clean":"2024-09-17T15:38:27.338212+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:38:27.338212+0000","last_undegraded":"2024-09-17T15:38:27.338212+0000","last_fullsized":"2024-09-17T15:38:27.338212+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28099,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338109+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:38:27.338109+0000","last_peered":"2024-09-17T15:38:27.338109+0000","last_clean":"2024-09-17T15:38:27.338109+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:38:27.338109+0000","last_undegraded":"2024-09-17T15:38:27.338109+0000","last_fullsized":"2024-09-17T15:38:27.338109+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338779+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:38:27.338779+0000","last_peered":"2024-09-17T15:38:27.338779+0000","last_clean":"2024-09-17T15:38:27.338779+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:38:27.338779+0000","last_undegraded":"2024-09-17T15:38:27.338779+0000","last_fullsized":"2024-09-17T15:38:27.338779+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21204,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.430121+0000","last_change":"2024-09-17T15:38:27.429539+0000","last_active":"2024-09-17T15:38:27.430121+0000","last_peered":"2024-09-17T15:38:27.430121+0000","last_clean":"2024-09-17T15:38:27.430121+0000","last_became_active":"2024-09-17T15:38:27.345429+0000","last_became_peered":"2024-09-17T15:38:27.345429+0000","last_unstale":"2024-09-17T15:38:27.430121+0000","last_undegraded":"2024-09-17T15:38:27.430121+0000","last_fullsized":"2024-09-17T15:38:27.430121+0000","mapping_epoch":62,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:49:28.575667+0000","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4065,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":111,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338015+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:38:27.338015+0000","last_peered":"2024-09-17T15:38:27.338015+0000","last_clean":"2024-09-17T15:38:27.338015+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:38:27.338015+0000","last_undegraded":"2024-09-17T15:38:27.338015+0000","last_fullsized":"2024-09-17T15:38:27.338015+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":116,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337667+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:38:27.337667+0000","last_peered":"2024-09-17T15:38:27.337667+0000","last_clean":"2024-09-17T15:38:27.337667+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:38:27.337667+0000","last_undegraded":"2024-09-17T15:38:27.337667+0000","last_fullsized":"2024-09-17T15:38:27.337667+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29050,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338223+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:38:27.338223+0000","last_peered":"2024-09-17T15:38:27.338223+0000","last_clean":"2024-09-17T15:38:27.338223+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:38:27.338223+0000","last_undegraded":"2024-09-17T15:38:27.338223+0000","last_fullsized":"2024-09-17T15:38:27.338223+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3462,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45998,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147087360,"data_stored":147007874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051745,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":431168,"kb_used_data":78760,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93940672,"statfs":{"total":96636764160,"available":96195248128,"internally_reserved":0,"allocated":80650240,"data_stored":80284971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724690,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523700,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":523131,"kb_avail":93848140,"statfs":{"total":96636764160,"available":96100495360,"internally_reserved":0,"allocated":557056,"data_stored":180484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":535686827},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247973,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539472,"kb_used_data":66360,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832368,"statfs":{"total":96636764160,"available":96084344832,"internally_reserved":0,"allocated":67952640,"data_stored":67565229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":79634432,"data_stored":79616367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:38:36.296 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:38:36.632 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:36.632 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:38:36.646 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":482,"stamp":"2024-09-17T15:38:35.842065+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2889,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46572,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1494288,"kb_used_data":141464,"kb_used_omap":13,"kb_used_meta":1352754,"kb_avail":281621232,"statfs":{"total":289910292480,"available":288380141568,"internally_reserved":0,"allocated":144859136,"data_stored":143729884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1385221119},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1423,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1423,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.509815"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":28917,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.343011+0000","last_change":"2024-09-17T15:38:06.362595+0000","last_active":"2024-09-17T15:38:27.343011+0000","last_peered":"2024-09-17T15:38:27.343011+0000","last_clean":"2024-09-17T15:38:27.343011+0000","last_became_active":"2024-09-17T15:38:06.284139+0000","last_became_peered":"2024-09-17T15:38:06.284139+0000","last_unstale":"2024-09-17T15:38:27.343011+0000","last_undegraded":"2024-09-17T15:38:27.343011+0000","last_fullsized":"2024-09-17T15:38:27.343011+0000","mapping_epoch":60,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:42:51.418256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4070,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21078,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.339083+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:38:27.339083+0000","last_peered":"2024-09-17T15:38:27.339083+0000","last_clean":"2024-09-17T15:38:27.339083+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:38:27.339083+0000","last_undegraded":"2024-09-17T15:38:27.339083+0000","last_fullsized":"2024-09-17T15:38:27.339083+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21186,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337203+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:38:27.337203+0000","last_peered":"2024-09-17T15:38:27.337203+0000","last_clean":"2024-09-17T15:38:27.337203+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:38:27.337203+0000","last_undegraded":"2024-09-17T15:38:27.337203+0000","last_fullsized":"2024-09-17T15:38:27.337203+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21083,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338986+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:38:27.338986+0000","last_peered":"2024-09-17T15:38:27.338986+0000","last_clean":"2024-09-17T15:38:27.338986+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:38:27.338986+0000","last_undegraded":"2024-09-17T15:38:27.338986+0000","last_fullsized":"2024-09-17T15:38:27.338986+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20826,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342773+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:38:27.342773+0000","last_peered":"2024-09-17T15:38:27.342773+0000","last_clean":"2024-09-17T15:38:27.342773+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:38:27.342773+0000","last_undegraded":"2024-09-17T15:38:27.342773+0000","last_fullsized":"2024-09-17T15:38:27.342773+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21577,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337752+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:38:27.337752+0000","last_peered":"2024-09-17T15:38:27.337752+0000","last_clean":"2024-09-17T15:38:27.337752+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:38:27.337752+0000","last_undegraded":"2024-09-17T15:38:27.337752+0000","last_fullsized":"2024-09-17T15:38:27.337752+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21121,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337152+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:38:27.337152+0000","last_peered":"2024-09-17T15:38:27.337152+0000","last_clean":"2024-09-17T15:38:27.337152+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:38:27.337152+0000","last_undegraded":"2024-09-17T15:38:27.337152+0000","last_fullsized":"2024-09-17T15:38:27.337152+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21088,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342372+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:38:27.342372+0000","last_peered":"2024-09-17T15:38:27.342372+0000","last_clean":"2024-09-17T15:38:27.342372+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:38:27.342372+0000","last_undegraded":"2024-09-17T15:38:27.342372+0000","last_fullsized":"2024-09-17T15:38:27.342372+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337064+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:38:27.337064+0000","last_peered":"2024-09-17T15:38:27.337064+0000","last_clean":"2024-09-17T15:38:27.337064+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:38:27.337064+0000","last_undegraded":"2024-09-17T15:38:27.337064+0000","last_fullsized":"2024-09-17T15:38:27.337064+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29143,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337570+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:38:27.337570+0000","last_peered":"2024-09-17T15:38:27.337570+0000","last_clean":"2024-09-17T15:38:27.337570+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:38:27.337570+0000","last_undegraded":"2024-09-17T15:38:27.337570+0000","last_fullsized":"2024-09-17T15:38:27.337570+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338510+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:38:27.338510+0000","last_peered":"2024-09-17T15:38:27.338510+0000","last_clean":"2024-09-17T15:38:27.338510+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:38:27.338510+0000","last_undegraded":"2024-09-17T15:38:27.338510+0000","last_fullsized":"2024-09-17T15:38:27.338510+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":22963,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:34.440389+0000","last_change":"2024-09-17T15:38:25.720175+0000","last_active":"2024-09-17T15:38:34.440389+0000","last_peered":"2024-09-17T15:38:34.440389+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:38:34.440389+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2889,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2889}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":147,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342526+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:38:27.342526+0000","last_peered":"2024-09-17T15:38:27.342526+0000","last_clean":"2024-09-17T15:38:27.342526+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:38:27.342526+0000","last_undegraded":"2024-09-17T15:38:27.342526+0000","last_fullsized":"2024-09-17T15:38:27.342526+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28848,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338595+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:38:27.338595+0000","last_peered":"2024-09-17T15:38:27.338595+0000","last_clean":"2024-09-17T15:38:27.338595+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:38:27.338595+0000","last_undegraded":"2024-09-17T15:38:27.338595+0000","last_fullsized":"2024-09-17T15:38:27.338595+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":121,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337473+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:38:27.337473+0000","last_peered":"2024-09-17T15:38:27.337473+0000","last_clean":"2024-09-17T15:38:27.337473+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:38:27.337473+0000","last_undegraded":"2024-09-17T15:38:27.337473+0000","last_fullsized":"2024-09-17T15:38:27.337473+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29185,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338248+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:38:27.338248+0000","last_peered":"2024-09-17T15:38:27.338248+0000","last_clean":"2024-09-17T15:38:27.338248+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:38:27.338248+0000","last_undegraded":"2024-09-17T15:38:27.338248+0000","last_fullsized":"2024-09-17T15:38:27.338248+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21295,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337742+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:38:27.337742+0000","last_peered":"2024-09-17T15:38:27.337742+0000","last_clean":"2024-09-17T15:38:27.337742+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:38:27.337742+0000","last_undegraded":"2024-09-17T15:38:27.337742+0000","last_fullsized":"2024-09-17T15:38:27.337742+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":111,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338212+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:38:27.338212+0000","last_peered":"2024-09-17T15:38:27.338212+0000","last_clean":"2024-09-17T15:38:27.338212+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:38:27.338212+0000","last_undegraded":"2024-09-17T15:38:27.338212+0000","last_fullsized":"2024-09-17T15:38:27.338212+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28099,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338109+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:38:27.338109+0000","last_peered":"2024-09-17T15:38:27.338109+0000","last_clean":"2024-09-17T15:38:27.338109+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:38:27.338109+0000","last_undegraded":"2024-09-17T15:38:27.338109+0000","last_fullsized":"2024-09-17T15:38:27.338109+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338779+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:38:27.338779+0000","last_peered":"2024-09-17T15:38:27.338779+0000","last_clean":"2024-09-17T15:38:27.338779+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:38:27.338779+0000","last_undegraded":"2024-09-17T15:38:27.338779+0000","last_fullsized":"2024-09-17T15:38:27.338779+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21204,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.430121+0000","last_change":"2024-09-17T15:38:27.429539+0000","last_active":"2024-09-17T15:38:27.430121+0000","last_peered":"2024-09-17T15:38:27.430121+0000","last_clean":"2024-09-17T15:38:27.430121+0000","last_became_active":"2024-09-17T15:38:27.345429+0000","last_became_peered":"2024-09-17T15:38:27.345429+0000","last_unstale":"2024-09-17T15:38:27.430121+0000","last_undegraded":"2024-09-17T15:38:27.430121+0000","last_fullsized":"2024-09-17T15:38:27.430121+0000","mapping_epoch":62,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:49:28.575667+0000","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4065,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":111,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338015+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:38:27.338015+0000","last_peered":"2024-09-17T15:38:27.338015+0000","last_clean":"2024-09-17T15:38:27.338015+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:38:27.338015+0000","last_undegraded":"2024-09-17T15:38:27.338015+0000","last_fullsized":"2024-09-17T15:38:27.338015+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":116,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337667+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:38:27.337667+0000","last_peered":"2024-09-17T15:38:27.337667+0000","last_clean":"2024-09-17T15:38:27.337667+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:38:27.337667+0000","last_undegraded":"2024-09-17T15:38:27.337667+0000","last_fullsized":"2024-09-17T15:38:27.337667+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29050,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338223+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:38:27.338223+0000","last_peered":"2024-09-17T15:38:27.338223+0000","last_clean":"2024-09-17T15:38:27.338223+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:38:27.338223+0000","last_undegraded":"2024-09-17T15:38:27.338223+0000","last_fullsized":"2024-09-17T15:38:27.338223+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2889,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46571,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":142909440,"data_stored":142829954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051746,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":426960,"kb_used_data":74560,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93944880,"statfs":{"total":96636764160,"available":96199557120,"internally_reserved":0,"allocated":76349440,"data_stored":75984171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724691,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527856,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93843984,"statfs":{"total":96636764160,"available":96096239616,"internally_reserved":0,"allocated":557056,"data_stored":180484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":539946667},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247974,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539472,"kb_used_data":66360,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832368,"statfs":{"total":96636764160,"available":96084344832,"internally_reserved":0,"allocated":67952640,"data_stored":67565229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":75456512,"data_stored":75438447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:38:36.647 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:38:36.953 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:36.953 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:38:36.963 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":482,"stamp":"2024-09-17T15:38:35.842065+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2889,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46572,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1494288,"kb_used_data":141464,"kb_used_omap":13,"kb_used_meta":1352754,"kb_avail":281621232,"statfs":{"total":289910292480,"available":288380141568,"internally_reserved":0,"allocated":144859136,"data_stored":143729884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1385221119},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1423,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1423,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.509815"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":28917,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.343011+0000","last_change":"2024-09-17T15:38:06.362595+0000","last_active":"2024-09-17T15:38:27.343011+0000","last_peered":"2024-09-17T15:38:27.343011+0000","last_clean":"2024-09-17T15:38:27.343011+0000","last_became_active":"2024-09-17T15:38:06.284139+0000","last_became_peered":"2024-09-17T15:38:06.284139+0000","last_unstale":"2024-09-17T15:38:27.343011+0000","last_undegraded":"2024-09-17T15:38:27.343011+0000","last_fullsized":"2024-09-17T15:38:27.343011+0000","mapping_epoch":60,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:42:51.418256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4070,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21078,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.339083+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:38:27.339083+0000","last_peered":"2024-09-17T15:38:27.339083+0000","last_clean":"2024-09-17T15:38:27.339083+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:38:27.339083+0000","last_undegraded":"2024-09-17T15:38:27.339083+0000","last_fullsized":"2024-09-17T15:38:27.339083+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21186,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337203+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:38:27.337203+0000","last_peered":"2024-09-17T15:38:27.337203+0000","last_clean":"2024-09-17T15:38:27.337203+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:38:27.337203+0000","last_undegraded":"2024-09-17T15:38:27.337203+0000","last_fullsized":"2024-09-17T15:38:27.337203+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21083,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338986+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:38:27.338986+0000","last_peered":"2024-09-17T15:38:27.338986+0000","last_clean":"2024-09-17T15:38:27.338986+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:38:27.338986+0000","last_undegraded":"2024-09-17T15:38:27.338986+0000","last_fullsized":"2024-09-17T15:38:27.338986+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20826,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342773+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:38:27.342773+0000","last_peered":"2024-09-17T15:38:27.342773+0000","last_clean":"2024-09-17T15:38:27.342773+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:38:27.342773+0000","last_undegraded":"2024-09-17T15:38:27.342773+0000","last_fullsized":"2024-09-17T15:38:27.342773+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21577,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337752+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:38:27.337752+0000","last_peered":"2024-09-17T15:38:27.337752+0000","last_clean":"2024-09-17T15:38:27.337752+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:38:27.337752+0000","last_undegraded":"2024-09-17T15:38:27.337752+0000","last_fullsized":"2024-09-17T15:38:27.337752+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21121,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337152+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:38:27.337152+0000","last_peered":"2024-09-17T15:38:27.337152+0000","last_clean":"2024-09-17T15:38:27.337152+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:38:27.337152+0000","last_undegraded":"2024-09-17T15:38:27.337152+0000","last_fullsized":"2024-09-17T15:38:27.337152+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21088,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342372+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:38:27.342372+0000","last_peered":"2024-09-17T15:38:27.342372+0000","last_clean":"2024-09-17T15:38:27.342372+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:38:27.342372+0000","last_undegraded":"2024-09-17T15:38:27.342372+0000","last_fullsized":"2024-09-17T15:38:27.342372+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337064+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:38:27.337064+0000","last_peered":"2024-09-17T15:38:27.337064+0000","last_clean":"2024-09-17T15:38:27.337064+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:38:27.337064+0000","last_undegraded":"2024-09-17T15:38:27.337064+0000","last_fullsized":"2024-09-17T15:38:27.337064+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29143,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337570+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:38:27.337570+0000","last_peered":"2024-09-17T15:38:27.337570+0000","last_clean":"2024-09-17T15:38:27.337570+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:38:27.337570+0000","last_undegraded":"2024-09-17T15:38:27.337570+0000","last_fullsized":"2024-09-17T15:38:27.337570+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338510+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:38:27.338510+0000","last_peered":"2024-09-17T15:38:27.338510+0000","last_clean":"2024-09-17T15:38:27.338510+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:38:27.338510+0000","last_undegraded":"2024-09-17T15:38:27.338510+0000","last_fullsized":"2024-09-17T15:38:27.338510+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":22963,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:34.440389+0000","last_change":"2024-09-17T15:38:25.720175+0000","last_active":"2024-09-17T15:38:34.440389+0000","last_peered":"2024-09-17T15:38:34.440389+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:38:34.440389+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2889,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2889}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":147,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342526+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:38:27.342526+0000","last_peered":"2024-09-17T15:38:27.342526+0000","last_clean":"2024-09-17T15:38:27.342526+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:38:27.342526+0000","last_undegraded":"2024-09-17T15:38:27.342526+0000","last_fullsized":"2024-09-17T15:38:27.342526+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28848,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338595+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:38:27.338595+0000","last_peered":"2024-09-17T15:38:27.338595+0000","last_clean":"2024-09-17T15:38:27.338595+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:38:27.338595+0000","last_undegraded":"2024-09-17T15:38:27.338595+0000","last_fullsized":"2024-09-17T15:38:27.338595+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":121,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337473+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:38:27.337473+0000","last_peered":"2024-09-17T15:38:27.337473+0000","last_clean":"2024-09-17T15:38:27.337473+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:38:27.337473+0000","last_undegraded":"2024-09-17T15:38:27.337473+0000","last_fullsized":"2024-09-17T15:38:27.337473+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29185,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338248+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:38:27.338248+0000","last_peered":"2024-09-17T15:38:27.338248+0000","last_clean":"2024-09-17T15:38:27.338248+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:38:27.338248+0000","last_undegraded":"2024-09-17T15:38:27.338248+0000","last_fullsized":"2024-09-17T15:38:27.338248+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21295,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337742+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:38:27.337742+0000","last_peered":"2024-09-17T15:38:27.337742+0000","last_clean":"2024-09-17T15:38:27.337742+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:38:27.337742+0000","last_undegraded":"2024-09-17T15:38:27.337742+0000","last_fullsized":"2024-09-17T15:38:27.337742+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":111,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338212+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:38:27.338212+0000","last_peered":"2024-09-17T15:38:27.338212+0000","last_clean":"2024-09-17T15:38:27.338212+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:38:27.338212+0000","last_undegraded":"2024-09-17T15:38:27.338212+0000","last_fullsized":"2024-09-17T15:38:27.338212+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28099,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338109+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:38:27.338109+0000","last_peered":"2024-09-17T15:38:27.338109+0000","last_clean":"2024-09-17T15:38:27.338109+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:38:27.338109+0000","last_undegraded":"2024-09-17T15:38:27.338109+0000","last_fullsized":"2024-09-17T15:38:27.338109+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338779+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:38:27.338779+0000","last_peered":"2024-09-17T15:38:27.338779+0000","last_clean":"2024-09-17T15:38:27.338779+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:38:27.338779+0000","last_undegraded":"2024-09-17T15:38:27.338779+0000","last_fullsized":"2024-09-17T15:38:27.338779+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21204,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.430121+0000","last_change":"2024-09-17T15:38:27.429539+0000","last_active":"2024-09-17T15:38:27.430121+0000","last_peered":"2024-09-17T15:38:27.430121+0000","last_clean":"2024-09-17T15:38:27.430121+0000","last_became_active":"2024-09-17T15:38:27.345429+0000","last_became_peered":"2024-09-17T15:38:27.345429+0000","last_unstale":"2024-09-17T15:38:27.430121+0000","last_undegraded":"2024-09-17T15:38:27.430121+0000","last_fullsized":"2024-09-17T15:38:27.430121+0000","mapping_epoch":62,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:49:28.575667+0000","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4065,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":111,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338015+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:38:27.338015+0000","last_peered":"2024-09-17T15:38:27.338015+0000","last_clean":"2024-09-17T15:38:27.338015+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:38:27.338015+0000","last_undegraded":"2024-09-17T15:38:27.338015+0000","last_fullsized":"2024-09-17T15:38:27.338015+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":116,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337667+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:38:27.337667+0000","last_peered":"2024-09-17T15:38:27.337667+0000","last_clean":"2024-09-17T15:38:27.337667+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:38:27.337667+0000","last_undegraded":"2024-09-17T15:38:27.337667+0000","last_fullsized":"2024-09-17T15:38:27.337667+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29050,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338223+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:38:27.338223+0000","last_peered":"2024-09-17T15:38:27.338223+0000","last_clean":"2024-09-17T15:38:27.338223+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:38:27.338223+0000","last_undegraded":"2024-09-17T15:38:27.338223+0000","last_fullsized":"2024-09-17T15:38:27.338223+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2889,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46571,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":142909440,"data_stored":142829954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051746,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":426960,"kb_used_data":74560,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93944880,"statfs":{"total":96636764160,"available":96199557120,"internally_reserved":0,"allocated":76349440,"data_stored":75984171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724691,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527856,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93843984,"statfs":{"total":96636764160,"available":96096239616,"internally_reserved":0,"allocated":557056,"data_stored":180484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":539946667},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247974,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539472,"kb_used_data":66360,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832368,"statfs":{"total":96636764160,"available":96084344832,"internally_reserved":0,"allocated":67952640,"data_stored":67565229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":75456512,"data_stored":75438447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:38:36.965 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:38:36.965 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20542", 'reported_seq': 22963, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:38:34.440389+0000', 'last_change': '2024-09-17T15:38:25.720175+0000', 'last_active': '2024-09-17T15:38:34.440389+0000', 'last_peered': '2024-09-17T15:38:34.440389+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:38:34.440389+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2889, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17138, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1123, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2889}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:38:36.965 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:38:37.361 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:37.361 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2889/24 objects degraded (12037.500%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":854,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1530150912,"bytes_avail":288380141568,"bytes_total":289910292480,"degraded_objects":2889,"degraded_total":24,"degraded_ratio":120.375,"recovering_objects_per_sec":149,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1423,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:38:37.378 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:38:37.378 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:38:37.682 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:37.682 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:38:37.695 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":482,"stamp":"2024-09-17T15:38:35.842065+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2889,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46572,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1494288,"kb_used_data":141464,"kb_used_omap":13,"kb_used_meta":1352754,"kb_avail":281621232,"statfs":{"total":289910292480,"available":288380141568,"internally_reserved":0,"allocated":144859136,"data_stored":143729884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1385221119},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1423,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1423,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.509815"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":28917,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.343011+0000","last_change":"2024-09-17T15:38:06.362595+0000","last_active":"2024-09-17T15:38:27.343011+0000","last_peered":"2024-09-17T15:38:27.343011+0000","last_clean":"2024-09-17T15:38:27.343011+0000","last_became_active":"2024-09-17T15:38:06.284139+0000","last_became_peered":"2024-09-17T15:38:06.284139+0000","last_unstale":"2024-09-17T15:38:27.343011+0000","last_undegraded":"2024-09-17T15:38:27.343011+0000","last_fullsized":"2024-09-17T15:38:27.343011+0000","mapping_epoch":60,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:42:51.418256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4070,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21078,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.339083+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:38:27.339083+0000","last_peered":"2024-09-17T15:38:27.339083+0000","last_clean":"2024-09-17T15:38:27.339083+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:38:27.339083+0000","last_undegraded":"2024-09-17T15:38:27.339083+0000","last_fullsized":"2024-09-17T15:38:27.339083+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21186,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337203+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:38:27.337203+0000","last_peered":"2024-09-17T15:38:27.337203+0000","last_clean":"2024-09-17T15:38:27.337203+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:38:27.337203+0000","last_undegraded":"2024-09-17T15:38:27.337203+0000","last_fullsized":"2024-09-17T15:38:27.337203+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21083,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338986+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:38:27.338986+0000","last_peered":"2024-09-17T15:38:27.338986+0000","last_clean":"2024-09-17T15:38:27.338986+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:38:27.338986+0000","last_undegraded":"2024-09-17T15:38:27.338986+0000","last_fullsized":"2024-09-17T15:38:27.338986+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20826,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342773+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:38:27.342773+0000","last_peered":"2024-09-17T15:38:27.342773+0000","last_clean":"2024-09-17T15:38:27.342773+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:38:27.342773+0000","last_undegraded":"2024-09-17T15:38:27.342773+0000","last_fullsized":"2024-09-17T15:38:27.342773+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21577,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337752+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:38:27.337752+0000","last_peered":"2024-09-17T15:38:27.337752+0000","last_clean":"2024-09-17T15:38:27.337752+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:38:27.337752+0000","last_undegraded":"2024-09-17T15:38:27.337752+0000","last_fullsized":"2024-09-17T15:38:27.337752+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21121,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337152+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:38:27.337152+0000","last_peered":"2024-09-17T15:38:27.337152+0000","last_clean":"2024-09-17T15:38:27.337152+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:38:27.337152+0000","last_undegraded":"2024-09-17T15:38:27.337152+0000","last_fullsized":"2024-09-17T15:38:27.337152+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21088,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342372+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:38:27.342372+0000","last_peered":"2024-09-17T15:38:27.342372+0000","last_clean":"2024-09-17T15:38:27.342372+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:38:27.342372+0000","last_undegraded":"2024-09-17T15:38:27.342372+0000","last_fullsized":"2024-09-17T15:38:27.342372+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337064+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:38:27.337064+0000","last_peered":"2024-09-17T15:38:27.337064+0000","last_clean":"2024-09-17T15:38:27.337064+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:38:27.337064+0000","last_undegraded":"2024-09-17T15:38:27.337064+0000","last_fullsized":"2024-09-17T15:38:27.337064+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29143,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337570+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:38:27.337570+0000","last_peered":"2024-09-17T15:38:27.337570+0000","last_clean":"2024-09-17T15:38:27.337570+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:38:27.337570+0000","last_undegraded":"2024-09-17T15:38:27.337570+0000","last_fullsized":"2024-09-17T15:38:27.337570+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338510+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:38:27.338510+0000","last_peered":"2024-09-17T15:38:27.338510+0000","last_clean":"2024-09-17T15:38:27.338510+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:38:27.338510+0000","last_undegraded":"2024-09-17T15:38:27.338510+0000","last_fullsized":"2024-09-17T15:38:27.338510+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":22963,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:34.440389+0000","last_change":"2024-09-17T15:38:25.720175+0000","last_active":"2024-09-17T15:38:34.440389+0000","last_peered":"2024-09-17T15:38:34.440389+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:38:34.440389+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2889,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2889}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":147,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342526+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:38:27.342526+0000","last_peered":"2024-09-17T15:38:27.342526+0000","last_clean":"2024-09-17T15:38:27.342526+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:38:27.342526+0000","last_undegraded":"2024-09-17T15:38:27.342526+0000","last_fullsized":"2024-09-17T15:38:27.342526+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28848,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338595+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:38:27.338595+0000","last_peered":"2024-09-17T15:38:27.338595+0000","last_clean":"2024-09-17T15:38:27.338595+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:38:27.338595+0000","last_undegraded":"2024-09-17T15:38:27.338595+0000","last_fullsized":"2024-09-17T15:38:27.338595+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":121,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337473+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:38:27.337473+0000","last_peered":"2024-09-17T15:38:27.337473+0000","last_clean":"2024-09-17T15:38:27.337473+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:38:27.337473+0000","last_undegraded":"2024-09-17T15:38:27.337473+0000","last_fullsized":"2024-09-17T15:38:27.337473+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29185,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338248+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:38:27.338248+0000","last_peered":"2024-09-17T15:38:27.338248+0000","last_clean":"2024-09-17T15:38:27.338248+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:38:27.338248+0000","last_undegraded":"2024-09-17T15:38:27.338248+0000","last_fullsized":"2024-09-17T15:38:27.338248+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21295,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337742+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:38:27.337742+0000","last_peered":"2024-09-17T15:38:27.337742+0000","last_clean":"2024-09-17T15:38:27.337742+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:38:27.337742+0000","last_undegraded":"2024-09-17T15:38:27.337742+0000","last_fullsized":"2024-09-17T15:38:27.337742+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":111,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338212+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:38:27.338212+0000","last_peered":"2024-09-17T15:38:27.338212+0000","last_clean":"2024-09-17T15:38:27.338212+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:38:27.338212+0000","last_undegraded":"2024-09-17T15:38:27.338212+0000","last_fullsized":"2024-09-17T15:38:27.338212+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28099,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338109+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:38:27.338109+0000","last_peered":"2024-09-17T15:38:27.338109+0000","last_clean":"2024-09-17T15:38:27.338109+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:38:27.338109+0000","last_undegraded":"2024-09-17T15:38:27.338109+0000","last_fullsized":"2024-09-17T15:38:27.338109+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338779+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:38:27.338779+0000","last_peered":"2024-09-17T15:38:27.338779+0000","last_clean":"2024-09-17T15:38:27.338779+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:38:27.338779+0000","last_undegraded":"2024-09-17T15:38:27.338779+0000","last_fullsized":"2024-09-17T15:38:27.338779+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21204,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.430121+0000","last_change":"2024-09-17T15:38:27.429539+0000","last_active":"2024-09-17T15:38:27.430121+0000","last_peered":"2024-09-17T15:38:27.430121+0000","last_clean":"2024-09-17T15:38:27.430121+0000","last_became_active":"2024-09-17T15:38:27.345429+0000","last_became_peered":"2024-09-17T15:38:27.345429+0000","last_unstale":"2024-09-17T15:38:27.430121+0000","last_undegraded":"2024-09-17T15:38:27.430121+0000","last_fullsized":"2024-09-17T15:38:27.430121+0000","mapping_epoch":62,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:49:28.575667+0000","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4065,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":111,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338015+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:38:27.338015+0000","last_peered":"2024-09-17T15:38:27.338015+0000","last_clean":"2024-09-17T15:38:27.338015+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:38:27.338015+0000","last_undegraded":"2024-09-17T15:38:27.338015+0000","last_fullsized":"2024-09-17T15:38:27.338015+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":116,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337667+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:38:27.337667+0000","last_peered":"2024-09-17T15:38:27.337667+0000","last_clean":"2024-09-17T15:38:27.337667+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:38:27.337667+0000","last_undegraded":"2024-09-17T15:38:27.337667+0000","last_fullsized":"2024-09-17T15:38:27.337667+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29050,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338223+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:38:27.338223+0000","last_peered":"2024-09-17T15:38:27.338223+0000","last_clean":"2024-09-17T15:38:27.338223+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:38:27.338223+0000","last_undegraded":"2024-09-17T15:38:27.338223+0000","last_fullsized":"2024-09-17T15:38:27.338223+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2889,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46571,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":142909440,"data_stored":142829954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051746,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":426960,"kb_used_data":74560,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93944880,"statfs":{"total":96636764160,"available":96199557120,"internally_reserved":0,"allocated":76349440,"data_stored":75984171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724691,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527856,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93843984,"statfs":{"total":96636764160,"available":96096239616,"internally_reserved":0,"allocated":557056,"data_stored":180484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":539946667},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247974,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539472,"kb_used_data":66360,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832368,"statfs":{"total":96636764160,"available":96084344832,"internally_reserved":0,"allocated":67952640,"data_stored":67565229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":75456512,"data_stored":75438447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:38:40.697 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:38:41.036 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:41.037 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:38:41.048 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":484,"stamp":"2024-09-17T15:38:39.842698+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2250,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47211,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1477608,"kb_used_data":137264,"kb_used_omap":13,"kb_used_meta":1340274,"kb_avail":281637912,"statfs":{"total":289910292480,"available":288397221888,"internally_reserved":0,"allocated":140558336,"data_stored":139429084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1372441599},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2062,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2062,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001824"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":28918,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.343038+0000","last_change":"2024-09-17T15:38:06.362595+0000","last_active":"2024-09-17T15:38:27.343038+0000","last_peered":"2024-09-17T15:38:27.343038+0000","last_clean":"2024-09-17T15:38:27.343038+0000","last_became_active":"2024-09-17T15:38:06.284139+0000","last_became_peered":"2024-09-17T15:38:06.284139+0000","last_unstale":"2024-09-17T15:38:27.343038+0000","last_undegraded":"2024-09-17T15:38:27.343038+0000","last_fullsized":"2024-09-17T15:38:27.343038+0000","mapping_epoch":60,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:42:51.418256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4070,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21078,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.339083+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:38:27.339083+0000","last_peered":"2024-09-17T15:38:27.339083+0000","last_clean":"2024-09-17T15:38:27.339083+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:38:27.339083+0000","last_undegraded":"2024-09-17T15:38:27.339083+0000","last_fullsized":"2024-09-17T15:38:27.339083+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21186,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337203+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:38:27.337203+0000","last_peered":"2024-09-17T15:38:27.337203+0000","last_clean":"2024-09-17T15:38:27.337203+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:38:27.337203+0000","last_undegraded":"2024-09-17T15:38:27.337203+0000","last_fullsized":"2024-09-17T15:38:27.337203+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21083,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338986+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:38:27.338986+0000","last_peered":"2024-09-17T15:38:27.338986+0000","last_clean":"2024-09-17T15:38:27.338986+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:38:27.338986+0000","last_undegraded":"2024-09-17T15:38:27.338986+0000","last_fullsized":"2024-09-17T15:38:27.338986+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20826,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342773+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:38:27.342773+0000","last_peered":"2024-09-17T15:38:27.342773+0000","last_clean":"2024-09-17T15:38:27.342773+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:38:27.342773+0000","last_undegraded":"2024-09-17T15:38:27.342773+0000","last_fullsized":"2024-09-17T15:38:27.342773+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21577,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337752+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:38:27.337752+0000","last_peered":"2024-09-17T15:38:27.337752+0000","last_clean":"2024-09-17T15:38:27.337752+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:38:27.337752+0000","last_undegraded":"2024-09-17T15:38:27.337752+0000","last_fullsized":"2024-09-17T15:38:27.337752+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21121,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337152+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:38:27.337152+0000","last_peered":"2024-09-17T15:38:27.337152+0000","last_clean":"2024-09-17T15:38:27.337152+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:38:27.337152+0000","last_undegraded":"2024-09-17T15:38:27.337152+0000","last_fullsized":"2024-09-17T15:38:27.337152+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21088,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342372+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:38:27.342372+0000","last_peered":"2024-09-17T15:38:27.342372+0000","last_clean":"2024-09-17T15:38:27.342372+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:38:27.342372+0000","last_undegraded":"2024-09-17T15:38:27.342372+0000","last_fullsized":"2024-09-17T15:38:27.342372+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337064+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:38:27.337064+0000","last_peered":"2024-09-17T15:38:27.337064+0000","last_clean":"2024-09-17T15:38:27.337064+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:38:27.337064+0000","last_undegraded":"2024-09-17T15:38:27.337064+0000","last_fullsized":"2024-09-17T15:38:27.337064+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29143,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337570+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:38:27.337570+0000","last_peered":"2024-09-17T15:38:27.337570+0000","last_clean":"2024-09-17T15:38:27.337570+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:38:27.337570+0000","last_undegraded":"2024-09-17T15:38:27.337570+0000","last_fullsized":"2024-09-17T15:38:27.337570+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338510+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:38:27.338510+0000","last_peered":"2024-09-17T15:38:27.338510+0000","last_clean":"2024-09-17T15:38:27.338510+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:38:27.338510+0000","last_undegraded":"2024-09-17T15:38:27.338510+0000","last_fullsized":"2024-09-17T15:38:27.338510+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":24241,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:39.442217+0000","last_change":"2024-09-17T15:38:25.720175+0000","last_active":"2024-09-17T15:38:39.442217+0000","last_peered":"2024-09-17T15:38:39.442217+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:38:39.442217+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2250,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1762,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2250}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":147,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342526+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:38:27.342526+0000","last_peered":"2024-09-17T15:38:27.342526+0000","last_clean":"2024-09-17T15:38:27.342526+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:38:27.342526+0000","last_undegraded":"2024-09-17T15:38:27.342526+0000","last_fullsized":"2024-09-17T15:38:27.342526+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28848,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338595+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:38:27.338595+0000","last_peered":"2024-09-17T15:38:27.338595+0000","last_clean":"2024-09-17T15:38:27.338595+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:38:27.338595+0000","last_undegraded":"2024-09-17T15:38:27.338595+0000","last_fullsized":"2024-09-17T15:38:27.338595+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":121,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337473+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:38:27.337473+0000","last_peered":"2024-09-17T15:38:27.337473+0000","last_clean":"2024-09-17T15:38:27.337473+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:38:27.337473+0000","last_undegraded":"2024-09-17T15:38:27.337473+0000","last_fullsized":"2024-09-17T15:38:27.337473+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29185,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338248+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:38:27.338248+0000","last_peered":"2024-09-17T15:38:27.338248+0000","last_clean":"2024-09-17T15:38:27.338248+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:38:27.338248+0000","last_undegraded":"2024-09-17T15:38:27.338248+0000","last_fullsized":"2024-09-17T15:38:27.338248+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21295,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337742+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:38:27.337742+0000","last_peered":"2024-09-17T15:38:27.337742+0000","last_clean":"2024-09-17T15:38:27.337742+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:38:27.337742+0000","last_undegraded":"2024-09-17T15:38:27.337742+0000","last_fullsized":"2024-09-17T15:38:27.337742+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":111,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338212+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:38:27.338212+0000","last_peered":"2024-09-17T15:38:27.338212+0000","last_clean":"2024-09-17T15:38:27.338212+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:38:27.338212+0000","last_undegraded":"2024-09-17T15:38:27.338212+0000","last_fullsized":"2024-09-17T15:38:27.338212+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28099,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338109+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:38:27.338109+0000","last_peered":"2024-09-17T15:38:27.338109+0000","last_clean":"2024-09-17T15:38:27.338109+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:38:27.338109+0000","last_undegraded":"2024-09-17T15:38:27.338109+0000","last_fullsized":"2024-09-17T15:38:27.338109+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338779+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:38:27.338779+0000","last_peered":"2024-09-17T15:38:27.338779+0000","last_clean":"2024-09-17T15:38:27.338779+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:38:27.338779+0000","last_undegraded":"2024-09-17T15:38:27.338779+0000","last_fullsized":"2024-09-17T15:38:27.338779+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21204,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.430121+0000","last_change":"2024-09-17T15:38:27.429539+0000","last_active":"2024-09-17T15:38:27.430121+0000","last_peered":"2024-09-17T15:38:27.430121+0000","last_clean":"2024-09-17T15:38:27.430121+0000","last_became_active":"2024-09-17T15:38:27.345429+0000","last_became_peered":"2024-09-17T15:38:27.345429+0000","last_unstale":"2024-09-17T15:38:27.430121+0000","last_undegraded":"2024-09-17T15:38:27.430121+0000","last_fullsized":"2024-09-17T15:38:27.430121+0000","mapping_epoch":62,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:49:28.575667+0000","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4065,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":111,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338015+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:38:27.338015+0000","last_peered":"2024-09-17T15:38:27.338015+0000","last_clean":"2024-09-17T15:38:27.338015+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:38:27.338015+0000","last_undegraded":"2024-09-17T15:38:27.338015+0000","last_fullsized":"2024-09-17T15:38:27.338015+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":116,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337667+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:38:27.337667+0000","last_peered":"2024-09-17T15:38:27.337667+0000","last_clean":"2024-09-17T15:38:27.337667+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:38:27.337667+0000","last_undegraded":"2024-09-17T15:38:27.337667+0000","last_fullsized":"2024-09-17T15:38:27.337667+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29050,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338223+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:38:27.338223+0000","last_peered":"2024-09-17T15:38:27.338223+0000","last_clean":"2024-09-17T15:38:27.338223+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:38:27.338223+0000","last_undegraded":"2024-09-17T15:38:27.338223+0000","last_fullsized":"2024-09-17T15:38:27.338223+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2250,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47210,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":138608640,"data_stored":138529154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051747,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":422760,"kb_used_data":70360,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93949080,"statfs":{"total":96636764160,"available":96203857920,"internally_reserved":0,"allocated":72048640,"data_stored":71683371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724692,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515376,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":514811,"kb_avail":93856464,"statfs":{"total":96636764160,"available":96109019136,"internally_reserved":0,"allocated":557056,"data_stored":180484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":527167147},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247975,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539472,"kb_used_data":66360,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832368,"statfs":{"total":96636764160,"available":96084344832,"internally_reserved":0,"allocated":67952640,"data_stored":67565229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":71155712,"data_stored":71137647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:38:41.049 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:38:41.352 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:41.353 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:38:41.365 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":484,"stamp":"2024-09-17T15:38:39.842698+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2250,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47211,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1477608,"kb_used_data":137264,"kb_used_omap":13,"kb_used_meta":1340274,"kb_avail":281637912,"statfs":{"total":289910292480,"available":288397221888,"internally_reserved":0,"allocated":140558336,"data_stored":139429084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1372441599},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2062,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2062,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001824"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":28918,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.343038+0000","last_change":"2024-09-17T15:38:06.362595+0000","last_active":"2024-09-17T15:38:27.343038+0000","last_peered":"2024-09-17T15:38:27.343038+0000","last_clean":"2024-09-17T15:38:27.343038+0000","last_became_active":"2024-09-17T15:38:06.284139+0000","last_became_peered":"2024-09-17T15:38:06.284139+0000","last_unstale":"2024-09-17T15:38:27.343038+0000","last_undegraded":"2024-09-17T15:38:27.343038+0000","last_fullsized":"2024-09-17T15:38:27.343038+0000","mapping_epoch":60,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:42:51.418256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4070,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21078,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.339083+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:38:27.339083+0000","last_peered":"2024-09-17T15:38:27.339083+0000","last_clean":"2024-09-17T15:38:27.339083+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:38:27.339083+0000","last_undegraded":"2024-09-17T15:38:27.339083+0000","last_fullsized":"2024-09-17T15:38:27.339083+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21186,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337203+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:38:27.337203+0000","last_peered":"2024-09-17T15:38:27.337203+0000","last_clean":"2024-09-17T15:38:27.337203+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:38:27.337203+0000","last_undegraded":"2024-09-17T15:38:27.337203+0000","last_fullsized":"2024-09-17T15:38:27.337203+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21083,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338986+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:38:27.338986+0000","last_peered":"2024-09-17T15:38:27.338986+0000","last_clean":"2024-09-17T15:38:27.338986+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:38:27.338986+0000","last_undegraded":"2024-09-17T15:38:27.338986+0000","last_fullsized":"2024-09-17T15:38:27.338986+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20826,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342773+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:38:27.342773+0000","last_peered":"2024-09-17T15:38:27.342773+0000","last_clean":"2024-09-17T15:38:27.342773+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:38:27.342773+0000","last_undegraded":"2024-09-17T15:38:27.342773+0000","last_fullsized":"2024-09-17T15:38:27.342773+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21577,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337752+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:38:27.337752+0000","last_peered":"2024-09-17T15:38:27.337752+0000","last_clean":"2024-09-17T15:38:27.337752+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:38:27.337752+0000","last_undegraded":"2024-09-17T15:38:27.337752+0000","last_fullsized":"2024-09-17T15:38:27.337752+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21121,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337152+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:38:27.337152+0000","last_peered":"2024-09-17T15:38:27.337152+0000","last_clean":"2024-09-17T15:38:27.337152+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:38:27.337152+0000","last_undegraded":"2024-09-17T15:38:27.337152+0000","last_fullsized":"2024-09-17T15:38:27.337152+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21088,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342372+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:38:27.342372+0000","last_peered":"2024-09-17T15:38:27.342372+0000","last_clean":"2024-09-17T15:38:27.342372+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:38:27.342372+0000","last_undegraded":"2024-09-17T15:38:27.342372+0000","last_fullsized":"2024-09-17T15:38:27.342372+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337064+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:38:27.337064+0000","last_peered":"2024-09-17T15:38:27.337064+0000","last_clean":"2024-09-17T15:38:27.337064+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:38:27.337064+0000","last_undegraded":"2024-09-17T15:38:27.337064+0000","last_fullsized":"2024-09-17T15:38:27.337064+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29143,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337570+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:38:27.337570+0000","last_peered":"2024-09-17T15:38:27.337570+0000","last_clean":"2024-09-17T15:38:27.337570+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:38:27.337570+0000","last_undegraded":"2024-09-17T15:38:27.337570+0000","last_fullsized":"2024-09-17T15:38:27.337570+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338510+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:38:27.338510+0000","last_peered":"2024-09-17T15:38:27.338510+0000","last_clean":"2024-09-17T15:38:27.338510+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:38:27.338510+0000","last_undegraded":"2024-09-17T15:38:27.338510+0000","last_fullsized":"2024-09-17T15:38:27.338510+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":24241,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:39.442217+0000","last_change":"2024-09-17T15:38:25.720175+0000","last_active":"2024-09-17T15:38:39.442217+0000","last_peered":"2024-09-17T15:38:39.442217+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:38:39.442217+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2250,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1762,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2250}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":147,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342526+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:38:27.342526+0000","last_peered":"2024-09-17T15:38:27.342526+0000","last_clean":"2024-09-17T15:38:27.342526+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:38:27.342526+0000","last_undegraded":"2024-09-17T15:38:27.342526+0000","last_fullsized":"2024-09-17T15:38:27.342526+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28848,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338595+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:38:27.338595+0000","last_peered":"2024-09-17T15:38:27.338595+0000","last_clean":"2024-09-17T15:38:27.338595+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:38:27.338595+0000","last_undegraded":"2024-09-17T15:38:27.338595+0000","last_fullsized":"2024-09-17T15:38:27.338595+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":121,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337473+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:38:27.337473+0000","last_peered":"2024-09-17T15:38:27.337473+0000","last_clean":"2024-09-17T15:38:27.337473+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:38:27.337473+0000","last_undegraded":"2024-09-17T15:38:27.337473+0000","last_fullsized":"2024-09-17T15:38:27.337473+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29185,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338248+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:38:27.338248+0000","last_peered":"2024-09-17T15:38:27.338248+0000","last_clean":"2024-09-17T15:38:27.338248+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:38:27.338248+0000","last_undegraded":"2024-09-17T15:38:27.338248+0000","last_fullsized":"2024-09-17T15:38:27.338248+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21295,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337742+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:38:27.337742+0000","last_peered":"2024-09-17T15:38:27.337742+0000","last_clean":"2024-09-17T15:38:27.337742+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:38:27.337742+0000","last_undegraded":"2024-09-17T15:38:27.337742+0000","last_fullsized":"2024-09-17T15:38:27.337742+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":111,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338212+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:38:27.338212+0000","last_peered":"2024-09-17T15:38:27.338212+0000","last_clean":"2024-09-17T15:38:27.338212+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:38:27.338212+0000","last_undegraded":"2024-09-17T15:38:27.338212+0000","last_fullsized":"2024-09-17T15:38:27.338212+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28099,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338109+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:38:27.338109+0000","last_peered":"2024-09-17T15:38:27.338109+0000","last_clean":"2024-09-17T15:38:27.338109+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:38:27.338109+0000","last_undegraded":"2024-09-17T15:38:27.338109+0000","last_fullsized":"2024-09-17T15:38:27.338109+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338779+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:38:27.338779+0000","last_peered":"2024-09-17T15:38:27.338779+0000","last_clean":"2024-09-17T15:38:27.338779+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:38:27.338779+0000","last_undegraded":"2024-09-17T15:38:27.338779+0000","last_fullsized":"2024-09-17T15:38:27.338779+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21204,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.430121+0000","last_change":"2024-09-17T15:38:27.429539+0000","last_active":"2024-09-17T15:38:27.430121+0000","last_peered":"2024-09-17T15:38:27.430121+0000","last_clean":"2024-09-17T15:38:27.430121+0000","last_became_active":"2024-09-17T15:38:27.345429+0000","last_became_peered":"2024-09-17T15:38:27.345429+0000","last_unstale":"2024-09-17T15:38:27.430121+0000","last_undegraded":"2024-09-17T15:38:27.430121+0000","last_fullsized":"2024-09-17T15:38:27.430121+0000","mapping_epoch":62,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:49:28.575667+0000","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4065,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":111,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338015+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:38:27.338015+0000","last_peered":"2024-09-17T15:38:27.338015+0000","last_clean":"2024-09-17T15:38:27.338015+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:38:27.338015+0000","last_undegraded":"2024-09-17T15:38:27.338015+0000","last_fullsized":"2024-09-17T15:38:27.338015+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":116,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337667+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:38:27.337667+0000","last_peered":"2024-09-17T15:38:27.337667+0000","last_clean":"2024-09-17T15:38:27.337667+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:38:27.337667+0000","last_undegraded":"2024-09-17T15:38:27.337667+0000","last_fullsized":"2024-09-17T15:38:27.337667+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29050,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338223+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:38:27.338223+0000","last_peered":"2024-09-17T15:38:27.338223+0000","last_clean":"2024-09-17T15:38:27.338223+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:38:27.338223+0000","last_undegraded":"2024-09-17T15:38:27.338223+0000","last_fullsized":"2024-09-17T15:38:27.338223+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2250,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47210,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":138608640,"data_stored":138529154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051747,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":422760,"kb_used_data":70360,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93949080,"statfs":{"total":96636764160,"available":96203857920,"internally_reserved":0,"allocated":72048640,"data_stored":71683371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724692,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515376,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":514811,"kb_avail":93856464,"statfs":{"total":96636764160,"available":96109019136,"internally_reserved":0,"allocated":557056,"data_stored":180484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":527167147},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247975,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539472,"kb_used_data":66360,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832368,"statfs":{"total":96636764160,"available":96084344832,"internally_reserved":0,"allocated":67952640,"data_stored":67565229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":71155712,"data_stored":71137647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:38:41.367 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:38:41.367 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20542", 'reported_seq': 24241, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:38:39.442217+0000', 'last_change': '2024-09-17T15:38:25.720175+0000', 'last_active': '2024-09-17T15:38:39.442217+0000', 'last_peered': '2024-09-17T15:38:39.442217+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:38:39.442217+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2250, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17138, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1762, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2250}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:38:41.368 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:38:41.753 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:41.753 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2250/24 objects degraded (9375.000%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":858,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1513070592,"bytes_avail":288397221888,"bytes_total":289910292480,"degraded_objects":2250,"degraded_total":24,"degraded_ratio":93.75,"recovering_objects_per_sec":171,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2062,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:38:41.766 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:38:41.767 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:38:42.066 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:42.067 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:38:42.078 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":485,"stamp":"2024-09-17T15:38:41.843015+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2250,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47211,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1477608,"kb_used_data":137264,"kb_used_omap":13,"kb_used_meta":1340274,"kb_avail":281637912,"statfs":{"total":289910292480,"available":288397221888,"internally_reserved":0,"allocated":140558336,"data_stored":139429084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1372441599},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1212,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1212,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001860"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":28918,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.343038+0000","last_change":"2024-09-17T15:38:06.362595+0000","last_active":"2024-09-17T15:38:27.343038+0000","last_peered":"2024-09-17T15:38:27.343038+0000","last_clean":"2024-09-17T15:38:27.343038+0000","last_became_active":"2024-09-17T15:38:06.284139+0000","last_became_peered":"2024-09-17T15:38:06.284139+0000","last_unstale":"2024-09-17T15:38:27.343038+0000","last_undegraded":"2024-09-17T15:38:27.343038+0000","last_fullsized":"2024-09-17T15:38:27.343038+0000","mapping_epoch":60,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:42:51.418256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4070,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21078,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.339083+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:38:27.339083+0000","last_peered":"2024-09-17T15:38:27.339083+0000","last_clean":"2024-09-17T15:38:27.339083+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:38:27.339083+0000","last_undegraded":"2024-09-17T15:38:27.339083+0000","last_fullsized":"2024-09-17T15:38:27.339083+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21186,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337203+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:38:27.337203+0000","last_peered":"2024-09-17T15:38:27.337203+0000","last_clean":"2024-09-17T15:38:27.337203+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:38:27.337203+0000","last_undegraded":"2024-09-17T15:38:27.337203+0000","last_fullsized":"2024-09-17T15:38:27.337203+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21083,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338986+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:38:27.338986+0000","last_peered":"2024-09-17T15:38:27.338986+0000","last_clean":"2024-09-17T15:38:27.338986+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:38:27.338986+0000","last_undegraded":"2024-09-17T15:38:27.338986+0000","last_fullsized":"2024-09-17T15:38:27.338986+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20826,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342773+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:38:27.342773+0000","last_peered":"2024-09-17T15:38:27.342773+0000","last_clean":"2024-09-17T15:38:27.342773+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:38:27.342773+0000","last_undegraded":"2024-09-17T15:38:27.342773+0000","last_fullsized":"2024-09-17T15:38:27.342773+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21577,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337752+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:38:27.337752+0000","last_peered":"2024-09-17T15:38:27.337752+0000","last_clean":"2024-09-17T15:38:27.337752+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:38:27.337752+0000","last_undegraded":"2024-09-17T15:38:27.337752+0000","last_fullsized":"2024-09-17T15:38:27.337752+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21121,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337152+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:38:27.337152+0000","last_peered":"2024-09-17T15:38:27.337152+0000","last_clean":"2024-09-17T15:38:27.337152+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:38:27.337152+0000","last_undegraded":"2024-09-17T15:38:27.337152+0000","last_fullsized":"2024-09-17T15:38:27.337152+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21088,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342372+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:38:27.342372+0000","last_peered":"2024-09-17T15:38:27.342372+0000","last_clean":"2024-09-17T15:38:27.342372+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:38:27.342372+0000","last_undegraded":"2024-09-17T15:38:27.342372+0000","last_fullsized":"2024-09-17T15:38:27.342372+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337064+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:38:27.337064+0000","last_peered":"2024-09-17T15:38:27.337064+0000","last_clean":"2024-09-17T15:38:27.337064+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:38:27.337064+0000","last_undegraded":"2024-09-17T15:38:27.337064+0000","last_fullsized":"2024-09-17T15:38:27.337064+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29143,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337570+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:38:27.337570+0000","last_peered":"2024-09-17T15:38:27.337570+0000","last_clean":"2024-09-17T15:38:27.337570+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:38:27.337570+0000","last_undegraded":"2024-09-17T15:38:27.337570+0000","last_fullsized":"2024-09-17T15:38:27.337570+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338510+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:38:27.338510+0000","last_peered":"2024-09-17T15:38:27.338510+0000","last_clean":"2024-09-17T15:38:27.338510+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:38:27.338510+0000","last_undegraded":"2024-09-17T15:38:27.338510+0000","last_fullsized":"2024-09-17T15:38:27.338510+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":24241,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:39.442217+0000","last_change":"2024-09-17T15:38:25.720175+0000","last_active":"2024-09-17T15:38:39.442217+0000","last_peered":"2024-09-17T15:38:39.442217+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:38:39.442217+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2250,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1762,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2250}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":147,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342526+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:38:27.342526+0000","last_peered":"2024-09-17T15:38:27.342526+0000","last_clean":"2024-09-17T15:38:27.342526+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:38:27.342526+0000","last_undegraded":"2024-09-17T15:38:27.342526+0000","last_fullsized":"2024-09-17T15:38:27.342526+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28848,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338595+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:38:27.338595+0000","last_peered":"2024-09-17T15:38:27.338595+0000","last_clean":"2024-09-17T15:38:27.338595+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:38:27.338595+0000","last_undegraded":"2024-09-17T15:38:27.338595+0000","last_fullsized":"2024-09-17T15:38:27.338595+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":121,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337473+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:38:27.337473+0000","last_peered":"2024-09-17T15:38:27.337473+0000","last_clean":"2024-09-17T15:38:27.337473+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:38:27.337473+0000","last_undegraded":"2024-09-17T15:38:27.337473+0000","last_fullsized":"2024-09-17T15:38:27.337473+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29185,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338248+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:38:27.338248+0000","last_peered":"2024-09-17T15:38:27.338248+0000","last_clean":"2024-09-17T15:38:27.338248+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:38:27.338248+0000","last_undegraded":"2024-09-17T15:38:27.338248+0000","last_fullsized":"2024-09-17T15:38:27.338248+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21295,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337742+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:38:27.337742+0000","last_peered":"2024-09-17T15:38:27.337742+0000","last_clean":"2024-09-17T15:38:27.337742+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:38:27.337742+0000","last_undegraded":"2024-09-17T15:38:27.337742+0000","last_fullsized":"2024-09-17T15:38:27.337742+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":111,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338212+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:38:27.338212+0000","last_peered":"2024-09-17T15:38:27.338212+0000","last_clean":"2024-09-17T15:38:27.338212+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:38:27.338212+0000","last_undegraded":"2024-09-17T15:38:27.338212+0000","last_fullsized":"2024-09-17T15:38:27.338212+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28099,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338109+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:38:27.338109+0000","last_peered":"2024-09-17T15:38:27.338109+0000","last_clean":"2024-09-17T15:38:27.338109+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:38:27.338109+0000","last_undegraded":"2024-09-17T15:38:27.338109+0000","last_fullsized":"2024-09-17T15:38:27.338109+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338779+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:38:27.338779+0000","last_peered":"2024-09-17T15:38:27.338779+0000","last_clean":"2024-09-17T15:38:27.338779+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:38:27.338779+0000","last_undegraded":"2024-09-17T15:38:27.338779+0000","last_fullsized":"2024-09-17T15:38:27.338779+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21204,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.430121+0000","last_change":"2024-09-17T15:38:27.429539+0000","last_active":"2024-09-17T15:38:27.430121+0000","last_peered":"2024-09-17T15:38:27.430121+0000","last_clean":"2024-09-17T15:38:27.430121+0000","last_became_active":"2024-09-17T15:38:27.345429+0000","last_became_peered":"2024-09-17T15:38:27.345429+0000","last_unstale":"2024-09-17T15:38:27.430121+0000","last_undegraded":"2024-09-17T15:38:27.430121+0000","last_fullsized":"2024-09-17T15:38:27.430121+0000","mapping_epoch":62,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:49:28.575667+0000","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4065,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":111,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338015+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:38:27.338015+0000","last_peered":"2024-09-17T15:38:27.338015+0000","last_clean":"2024-09-17T15:38:27.338015+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:38:27.338015+0000","last_undegraded":"2024-09-17T15:38:27.338015+0000","last_fullsized":"2024-09-17T15:38:27.338015+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":116,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337667+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:38:27.337667+0000","last_peered":"2024-09-17T15:38:27.337667+0000","last_clean":"2024-09-17T15:38:27.337667+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:38:27.337667+0000","last_undegraded":"2024-09-17T15:38:27.337667+0000","last_fullsized":"2024-09-17T15:38:27.337667+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29050,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338223+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:38:27.338223+0000","last_peered":"2024-09-17T15:38:27.338223+0000","last_clean":"2024-09-17T15:38:27.338223+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:38:27.338223+0000","last_undegraded":"2024-09-17T15:38:27.338223+0000","last_fullsized":"2024-09-17T15:38:27.338223+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2250,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47210,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":138608640,"data_stored":138529154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051747,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":422760,"kb_used_data":70360,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93949080,"statfs":{"total":96636764160,"available":96203857920,"internally_reserved":0,"allocated":72048640,"data_stored":71683371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724692,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515376,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":514811,"kb_avail":93856464,"statfs":{"total":96636764160,"available":96109019136,"internally_reserved":0,"allocated":557056,"data_stored":180484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":527167147},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247975,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539472,"kb_used_data":66360,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832368,"statfs":{"total":96636764160,"available":96084344832,"internally_reserved":0,"allocated":67952640,"data_stored":67565229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":71155712,"data_stored":71137647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:38:45.082 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:38:45.407 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:45.408 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:38:45.419 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":486,"stamp":"2024-09-17T15:38:43.843333+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2250,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47211,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1473408,"kb_used_data":133064,"kb_used_omap":13,"kb_used_meta":1340274,"kb_avail":281642112,"statfs":{"total":289910292480,"available":288401522688,"internally_reserved":0,"allocated":136257536,"data_stored":135128284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1372441599},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1212,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1212,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001861"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":28918,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.343038+0000","last_change":"2024-09-17T15:38:06.362595+0000","last_active":"2024-09-17T15:38:27.343038+0000","last_peered":"2024-09-17T15:38:27.343038+0000","last_clean":"2024-09-17T15:38:27.343038+0000","last_became_active":"2024-09-17T15:38:06.284139+0000","last_became_peered":"2024-09-17T15:38:06.284139+0000","last_unstale":"2024-09-17T15:38:27.343038+0000","last_undegraded":"2024-09-17T15:38:27.343038+0000","last_fullsized":"2024-09-17T15:38:27.343038+0000","mapping_epoch":60,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:42:51.418256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4070,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21078,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.339083+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:38:27.339083+0000","last_peered":"2024-09-17T15:38:27.339083+0000","last_clean":"2024-09-17T15:38:27.339083+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:38:27.339083+0000","last_undegraded":"2024-09-17T15:38:27.339083+0000","last_fullsized":"2024-09-17T15:38:27.339083+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21186,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337203+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:38:27.337203+0000","last_peered":"2024-09-17T15:38:27.337203+0000","last_clean":"2024-09-17T15:38:27.337203+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:38:27.337203+0000","last_undegraded":"2024-09-17T15:38:27.337203+0000","last_fullsized":"2024-09-17T15:38:27.337203+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21083,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338986+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:38:27.338986+0000","last_peered":"2024-09-17T15:38:27.338986+0000","last_clean":"2024-09-17T15:38:27.338986+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:38:27.338986+0000","last_undegraded":"2024-09-17T15:38:27.338986+0000","last_fullsized":"2024-09-17T15:38:27.338986+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20826,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342773+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:38:27.342773+0000","last_peered":"2024-09-17T15:38:27.342773+0000","last_clean":"2024-09-17T15:38:27.342773+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:38:27.342773+0000","last_undegraded":"2024-09-17T15:38:27.342773+0000","last_fullsized":"2024-09-17T15:38:27.342773+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21577,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337752+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:38:27.337752+0000","last_peered":"2024-09-17T15:38:27.337752+0000","last_clean":"2024-09-17T15:38:27.337752+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:38:27.337752+0000","last_undegraded":"2024-09-17T15:38:27.337752+0000","last_fullsized":"2024-09-17T15:38:27.337752+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21121,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337152+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:38:27.337152+0000","last_peered":"2024-09-17T15:38:27.337152+0000","last_clean":"2024-09-17T15:38:27.337152+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:38:27.337152+0000","last_undegraded":"2024-09-17T15:38:27.337152+0000","last_fullsized":"2024-09-17T15:38:27.337152+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21088,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342372+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:38:27.342372+0000","last_peered":"2024-09-17T15:38:27.342372+0000","last_clean":"2024-09-17T15:38:27.342372+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:38:27.342372+0000","last_undegraded":"2024-09-17T15:38:27.342372+0000","last_fullsized":"2024-09-17T15:38:27.342372+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337064+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:38:27.337064+0000","last_peered":"2024-09-17T15:38:27.337064+0000","last_clean":"2024-09-17T15:38:27.337064+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:38:27.337064+0000","last_undegraded":"2024-09-17T15:38:27.337064+0000","last_fullsized":"2024-09-17T15:38:27.337064+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29143,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337570+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:38:27.337570+0000","last_peered":"2024-09-17T15:38:27.337570+0000","last_clean":"2024-09-17T15:38:27.337570+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:38:27.337570+0000","last_undegraded":"2024-09-17T15:38:27.337570+0000","last_fullsized":"2024-09-17T15:38:27.337570+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338510+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:38:27.338510+0000","last_peered":"2024-09-17T15:38:27.338510+0000","last_clean":"2024-09-17T15:38:27.338510+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:38:27.338510+0000","last_undegraded":"2024-09-17T15:38:27.338510+0000","last_fullsized":"2024-09-17T15:38:27.338510+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":24241,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:39.442217+0000","last_change":"2024-09-17T15:38:25.720175+0000","last_active":"2024-09-17T15:38:39.442217+0000","last_peered":"2024-09-17T15:38:39.442217+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:38:39.442217+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2250,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1762,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2250}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":147,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342526+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:38:27.342526+0000","last_peered":"2024-09-17T15:38:27.342526+0000","last_clean":"2024-09-17T15:38:27.342526+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:38:27.342526+0000","last_undegraded":"2024-09-17T15:38:27.342526+0000","last_fullsized":"2024-09-17T15:38:27.342526+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28848,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338595+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:38:27.338595+0000","last_peered":"2024-09-17T15:38:27.338595+0000","last_clean":"2024-09-17T15:38:27.338595+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:38:27.338595+0000","last_undegraded":"2024-09-17T15:38:27.338595+0000","last_fullsized":"2024-09-17T15:38:27.338595+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":121,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337473+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:38:27.337473+0000","last_peered":"2024-09-17T15:38:27.337473+0000","last_clean":"2024-09-17T15:38:27.337473+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:38:27.337473+0000","last_undegraded":"2024-09-17T15:38:27.337473+0000","last_fullsized":"2024-09-17T15:38:27.337473+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29185,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338248+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:38:27.338248+0000","last_peered":"2024-09-17T15:38:27.338248+0000","last_clean":"2024-09-17T15:38:27.338248+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:38:27.338248+0000","last_undegraded":"2024-09-17T15:38:27.338248+0000","last_fullsized":"2024-09-17T15:38:27.338248+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21295,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337742+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:38:27.337742+0000","last_peered":"2024-09-17T15:38:27.337742+0000","last_clean":"2024-09-17T15:38:27.337742+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:38:27.337742+0000","last_undegraded":"2024-09-17T15:38:27.337742+0000","last_fullsized":"2024-09-17T15:38:27.337742+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":111,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338212+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:38:27.338212+0000","last_peered":"2024-09-17T15:38:27.338212+0000","last_clean":"2024-09-17T15:38:27.338212+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:38:27.338212+0000","last_undegraded":"2024-09-17T15:38:27.338212+0000","last_fullsized":"2024-09-17T15:38:27.338212+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28099,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338109+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:38:27.338109+0000","last_peered":"2024-09-17T15:38:27.338109+0000","last_clean":"2024-09-17T15:38:27.338109+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:38:27.338109+0000","last_undegraded":"2024-09-17T15:38:27.338109+0000","last_fullsized":"2024-09-17T15:38:27.338109+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338779+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:38:27.338779+0000","last_peered":"2024-09-17T15:38:27.338779+0000","last_clean":"2024-09-17T15:38:27.338779+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:38:27.338779+0000","last_undegraded":"2024-09-17T15:38:27.338779+0000","last_fullsized":"2024-09-17T15:38:27.338779+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21204,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.430121+0000","last_change":"2024-09-17T15:38:27.429539+0000","last_active":"2024-09-17T15:38:27.430121+0000","last_peered":"2024-09-17T15:38:27.430121+0000","last_clean":"2024-09-17T15:38:27.430121+0000","last_became_active":"2024-09-17T15:38:27.345429+0000","last_became_peered":"2024-09-17T15:38:27.345429+0000","last_unstale":"2024-09-17T15:38:27.430121+0000","last_undegraded":"2024-09-17T15:38:27.430121+0000","last_fullsized":"2024-09-17T15:38:27.430121+0000","mapping_epoch":62,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:49:28.575667+0000","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4065,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":111,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338015+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:38:27.338015+0000","last_peered":"2024-09-17T15:38:27.338015+0000","last_clean":"2024-09-17T15:38:27.338015+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:38:27.338015+0000","last_undegraded":"2024-09-17T15:38:27.338015+0000","last_fullsized":"2024-09-17T15:38:27.338015+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":116,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337667+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:38:27.337667+0000","last_peered":"2024-09-17T15:38:27.337667+0000","last_clean":"2024-09-17T15:38:27.337667+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:38:27.337667+0000","last_undegraded":"2024-09-17T15:38:27.337667+0000","last_fullsized":"2024-09-17T15:38:27.337667+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29050,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338223+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:38:27.338223+0000","last_peered":"2024-09-17T15:38:27.338223+0000","last_clean":"2024-09-17T15:38:27.338223+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:38:27.338223+0000","last_undegraded":"2024-09-17T15:38:27.338223+0000","last_fullsized":"2024-09-17T15:38:27.338223+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2250,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47210,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":134307840,"data_stored":134228354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051748,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418560,"kb_used_data":66160,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93953280,"statfs":{"total":96636764160,"available":96208158720,"internally_reserved":0,"allocated":67747840,"data_stored":67382571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724692,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515376,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":514811,"kb_avail":93856464,"statfs":{"total":96636764160,"available":96109019136,"internally_reserved":0,"allocated":557056,"data_stored":180484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":527167147},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247975,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539472,"kb_used_data":66360,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832368,"statfs":{"total":96636764160,"available":96084344832,"internally_reserved":0,"allocated":67952640,"data_stored":67565229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":66854912,"data_stored":66836847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:38:45.421 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:38:45.768 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:45.769 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:38:45.781 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":486,"stamp":"2024-09-17T15:38:43.843333+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2250,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47211,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1473408,"kb_used_data":133064,"kb_used_omap":13,"kb_used_meta":1340274,"kb_avail":281642112,"statfs":{"total":289910292480,"available":288401522688,"internally_reserved":0,"allocated":136257536,"data_stored":135128284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1372441599},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1212,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1212,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001861"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":28918,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.343038+0000","last_change":"2024-09-17T15:38:06.362595+0000","last_active":"2024-09-17T15:38:27.343038+0000","last_peered":"2024-09-17T15:38:27.343038+0000","last_clean":"2024-09-17T15:38:27.343038+0000","last_became_active":"2024-09-17T15:38:06.284139+0000","last_became_peered":"2024-09-17T15:38:06.284139+0000","last_unstale":"2024-09-17T15:38:27.343038+0000","last_undegraded":"2024-09-17T15:38:27.343038+0000","last_fullsized":"2024-09-17T15:38:27.343038+0000","mapping_epoch":60,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:42:51.418256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4070,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21078,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.339083+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:38:27.339083+0000","last_peered":"2024-09-17T15:38:27.339083+0000","last_clean":"2024-09-17T15:38:27.339083+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:38:27.339083+0000","last_undegraded":"2024-09-17T15:38:27.339083+0000","last_fullsized":"2024-09-17T15:38:27.339083+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21186,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337203+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:38:27.337203+0000","last_peered":"2024-09-17T15:38:27.337203+0000","last_clean":"2024-09-17T15:38:27.337203+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:38:27.337203+0000","last_undegraded":"2024-09-17T15:38:27.337203+0000","last_fullsized":"2024-09-17T15:38:27.337203+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21083,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338986+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:38:27.338986+0000","last_peered":"2024-09-17T15:38:27.338986+0000","last_clean":"2024-09-17T15:38:27.338986+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:38:27.338986+0000","last_undegraded":"2024-09-17T15:38:27.338986+0000","last_fullsized":"2024-09-17T15:38:27.338986+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20826,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342773+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:38:27.342773+0000","last_peered":"2024-09-17T15:38:27.342773+0000","last_clean":"2024-09-17T15:38:27.342773+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:38:27.342773+0000","last_undegraded":"2024-09-17T15:38:27.342773+0000","last_fullsized":"2024-09-17T15:38:27.342773+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21577,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337752+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:38:27.337752+0000","last_peered":"2024-09-17T15:38:27.337752+0000","last_clean":"2024-09-17T15:38:27.337752+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:38:27.337752+0000","last_undegraded":"2024-09-17T15:38:27.337752+0000","last_fullsized":"2024-09-17T15:38:27.337752+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21121,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337152+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:38:27.337152+0000","last_peered":"2024-09-17T15:38:27.337152+0000","last_clean":"2024-09-17T15:38:27.337152+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:38:27.337152+0000","last_undegraded":"2024-09-17T15:38:27.337152+0000","last_fullsized":"2024-09-17T15:38:27.337152+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21088,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342372+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:38:27.342372+0000","last_peered":"2024-09-17T15:38:27.342372+0000","last_clean":"2024-09-17T15:38:27.342372+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:38:27.342372+0000","last_undegraded":"2024-09-17T15:38:27.342372+0000","last_fullsized":"2024-09-17T15:38:27.342372+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337064+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:38:27.337064+0000","last_peered":"2024-09-17T15:38:27.337064+0000","last_clean":"2024-09-17T15:38:27.337064+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:38:27.337064+0000","last_undegraded":"2024-09-17T15:38:27.337064+0000","last_fullsized":"2024-09-17T15:38:27.337064+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29143,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337570+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:38:27.337570+0000","last_peered":"2024-09-17T15:38:27.337570+0000","last_clean":"2024-09-17T15:38:27.337570+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:38:27.337570+0000","last_undegraded":"2024-09-17T15:38:27.337570+0000","last_fullsized":"2024-09-17T15:38:27.337570+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338510+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:38:27.338510+0000","last_peered":"2024-09-17T15:38:27.338510+0000","last_clean":"2024-09-17T15:38:27.338510+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:38:27.338510+0000","last_undegraded":"2024-09-17T15:38:27.338510+0000","last_fullsized":"2024-09-17T15:38:27.338510+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":24241,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:39.442217+0000","last_change":"2024-09-17T15:38:25.720175+0000","last_active":"2024-09-17T15:38:39.442217+0000","last_peered":"2024-09-17T15:38:39.442217+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:38:39.442217+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2250,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1762,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2250}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":147,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342526+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:38:27.342526+0000","last_peered":"2024-09-17T15:38:27.342526+0000","last_clean":"2024-09-17T15:38:27.342526+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:38:27.342526+0000","last_undegraded":"2024-09-17T15:38:27.342526+0000","last_fullsized":"2024-09-17T15:38:27.342526+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28848,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338595+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:38:27.338595+0000","last_peered":"2024-09-17T15:38:27.338595+0000","last_clean":"2024-09-17T15:38:27.338595+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:38:27.338595+0000","last_undegraded":"2024-09-17T15:38:27.338595+0000","last_fullsized":"2024-09-17T15:38:27.338595+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":121,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337473+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:38:27.337473+0000","last_peered":"2024-09-17T15:38:27.337473+0000","last_clean":"2024-09-17T15:38:27.337473+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:38:27.337473+0000","last_undegraded":"2024-09-17T15:38:27.337473+0000","last_fullsized":"2024-09-17T15:38:27.337473+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29185,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338248+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:38:27.338248+0000","last_peered":"2024-09-17T15:38:27.338248+0000","last_clean":"2024-09-17T15:38:27.338248+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:38:27.338248+0000","last_undegraded":"2024-09-17T15:38:27.338248+0000","last_fullsized":"2024-09-17T15:38:27.338248+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21295,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337742+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:38:27.337742+0000","last_peered":"2024-09-17T15:38:27.337742+0000","last_clean":"2024-09-17T15:38:27.337742+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:38:27.337742+0000","last_undegraded":"2024-09-17T15:38:27.337742+0000","last_fullsized":"2024-09-17T15:38:27.337742+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":111,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338212+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:38:27.338212+0000","last_peered":"2024-09-17T15:38:27.338212+0000","last_clean":"2024-09-17T15:38:27.338212+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:38:27.338212+0000","last_undegraded":"2024-09-17T15:38:27.338212+0000","last_fullsized":"2024-09-17T15:38:27.338212+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28099,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338109+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:38:27.338109+0000","last_peered":"2024-09-17T15:38:27.338109+0000","last_clean":"2024-09-17T15:38:27.338109+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:38:27.338109+0000","last_undegraded":"2024-09-17T15:38:27.338109+0000","last_fullsized":"2024-09-17T15:38:27.338109+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338779+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:38:27.338779+0000","last_peered":"2024-09-17T15:38:27.338779+0000","last_clean":"2024-09-17T15:38:27.338779+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:38:27.338779+0000","last_undegraded":"2024-09-17T15:38:27.338779+0000","last_fullsized":"2024-09-17T15:38:27.338779+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21204,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.430121+0000","last_change":"2024-09-17T15:38:27.429539+0000","last_active":"2024-09-17T15:38:27.430121+0000","last_peered":"2024-09-17T15:38:27.430121+0000","last_clean":"2024-09-17T15:38:27.430121+0000","last_became_active":"2024-09-17T15:38:27.345429+0000","last_became_peered":"2024-09-17T15:38:27.345429+0000","last_unstale":"2024-09-17T15:38:27.430121+0000","last_undegraded":"2024-09-17T15:38:27.430121+0000","last_fullsized":"2024-09-17T15:38:27.430121+0000","mapping_epoch":62,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:49:28.575667+0000","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4065,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":111,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338015+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:38:27.338015+0000","last_peered":"2024-09-17T15:38:27.338015+0000","last_clean":"2024-09-17T15:38:27.338015+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:38:27.338015+0000","last_undegraded":"2024-09-17T15:38:27.338015+0000","last_fullsized":"2024-09-17T15:38:27.338015+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":116,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337667+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:38:27.337667+0000","last_peered":"2024-09-17T15:38:27.337667+0000","last_clean":"2024-09-17T15:38:27.337667+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:38:27.337667+0000","last_undegraded":"2024-09-17T15:38:27.337667+0000","last_fullsized":"2024-09-17T15:38:27.337667+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29050,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338223+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:38:27.338223+0000","last_peered":"2024-09-17T15:38:27.338223+0000","last_clean":"2024-09-17T15:38:27.338223+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:38:27.338223+0000","last_undegraded":"2024-09-17T15:38:27.338223+0000","last_fullsized":"2024-09-17T15:38:27.338223+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2250,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47210,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":134307840,"data_stored":134228354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051748,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418560,"kb_used_data":66160,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93953280,"statfs":{"total":96636764160,"available":96208158720,"internally_reserved":0,"allocated":67747840,"data_stored":67382571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724692,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515376,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":514811,"kb_avail":93856464,"statfs":{"total":96636764160,"available":96109019136,"internally_reserved":0,"allocated":557056,"data_stored":180484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":527167147},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247975,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539472,"kb_used_data":66360,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832368,"statfs":{"total":96636764160,"available":96084344832,"internally_reserved":0,"allocated":67952640,"data_stored":67565229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":66854912,"data_stored":66836847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:38:45.782 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:38:45.782 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20542", 'reported_seq': 24241, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:38:39.442217+0000', 'last_change': '2024-09-17T15:38:25.720175+0000', 'last_active': '2024-09-17T15:38:39.442217+0000', 'last_peered': '2024-09-17T15:38:39.442217+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:38:39.442217+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2250, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17138, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1762, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2250}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:38:45.783 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:38:46.164 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:46.164 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2250/24 objects degraded (9375.000%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":863,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1508769792,"bytes_avail":288401522688,"bytes_total":289910292480,"degraded_objects":2250,"degraded_total":24,"degraded_ratio":93.75,"recovering_objects_per_sec":100,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1212,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:38:46.176 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:38:46.177 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:38:46.485 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:46.485 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:38:46.497 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":487,"stamp":"2024-09-17T15:38:45.843609+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1469,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47992,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1477568,"kb_used_data":133064,"kb_used_omap":13,"kb_used_meta":1344434,"kb_avail":281637952,"statfs":{"total":289910292480,"available":288397262848,"internally_reserved":0,"allocated":136257536,"data_stored":135128284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1376701439},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1993,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1993,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001849"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":28918,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.343038+0000","last_change":"2024-09-17T15:38:06.362595+0000","last_active":"2024-09-17T15:38:27.343038+0000","last_peered":"2024-09-17T15:38:27.343038+0000","last_clean":"2024-09-17T15:38:27.343038+0000","last_became_active":"2024-09-17T15:38:06.284139+0000","last_became_peered":"2024-09-17T15:38:06.284139+0000","last_unstale":"2024-09-17T15:38:27.343038+0000","last_undegraded":"2024-09-17T15:38:27.343038+0000","last_fullsized":"2024-09-17T15:38:27.343038+0000","mapping_epoch":60,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:42:51.418256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4070,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21078,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.339083+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:38:27.339083+0000","last_peered":"2024-09-17T15:38:27.339083+0000","last_clean":"2024-09-17T15:38:27.339083+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:38:27.339083+0000","last_undegraded":"2024-09-17T15:38:27.339083+0000","last_fullsized":"2024-09-17T15:38:27.339083+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21186,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337203+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:38:27.337203+0000","last_peered":"2024-09-17T15:38:27.337203+0000","last_clean":"2024-09-17T15:38:27.337203+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:38:27.337203+0000","last_undegraded":"2024-09-17T15:38:27.337203+0000","last_fullsized":"2024-09-17T15:38:27.337203+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21083,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338986+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:38:27.338986+0000","last_peered":"2024-09-17T15:38:27.338986+0000","last_clean":"2024-09-17T15:38:27.338986+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:38:27.338986+0000","last_undegraded":"2024-09-17T15:38:27.338986+0000","last_fullsized":"2024-09-17T15:38:27.338986+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20826,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342773+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:38:27.342773+0000","last_peered":"2024-09-17T15:38:27.342773+0000","last_clean":"2024-09-17T15:38:27.342773+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:38:27.342773+0000","last_undegraded":"2024-09-17T15:38:27.342773+0000","last_fullsized":"2024-09-17T15:38:27.342773+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21577,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337752+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:38:27.337752+0000","last_peered":"2024-09-17T15:38:27.337752+0000","last_clean":"2024-09-17T15:38:27.337752+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:38:27.337752+0000","last_undegraded":"2024-09-17T15:38:27.337752+0000","last_fullsized":"2024-09-17T15:38:27.337752+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21121,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337152+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:38:27.337152+0000","last_peered":"2024-09-17T15:38:27.337152+0000","last_clean":"2024-09-17T15:38:27.337152+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:38:27.337152+0000","last_undegraded":"2024-09-17T15:38:27.337152+0000","last_fullsized":"2024-09-17T15:38:27.337152+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21088,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342372+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:38:27.342372+0000","last_peered":"2024-09-17T15:38:27.342372+0000","last_clean":"2024-09-17T15:38:27.342372+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:38:27.342372+0000","last_undegraded":"2024-09-17T15:38:27.342372+0000","last_fullsized":"2024-09-17T15:38:27.342372+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337064+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:38:27.337064+0000","last_peered":"2024-09-17T15:38:27.337064+0000","last_clean":"2024-09-17T15:38:27.337064+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:38:27.337064+0000","last_undegraded":"2024-09-17T15:38:27.337064+0000","last_fullsized":"2024-09-17T15:38:27.337064+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29143,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337570+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:38:27.337570+0000","last_peered":"2024-09-17T15:38:27.337570+0000","last_clean":"2024-09-17T15:38:27.337570+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:38:27.337570+0000","last_undegraded":"2024-09-17T15:38:27.337570+0000","last_fullsized":"2024-09-17T15:38:27.337570+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338510+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:38:27.338510+0000","last_peered":"2024-09-17T15:38:27.338510+0000","last_clean":"2024-09-17T15:38:27.338510+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:38:27.338510+0000","last_undegraded":"2024-09-17T15:38:27.338510+0000","last_fullsized":"2024-09-17T15:38:27.338510+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":25804,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:44.442697+0000","last_change":"2024-09-17T15:38:25.720175+0000","last_active":"2024-09-17T15:38:44.442697+0000","last_peered":"2024-09-17T15:38:44.442697+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:38:44.442697+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1469,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2543,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1469}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":147,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342526+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:38:27.342526+0000","last_peered":"2024-09-17T15:38:27.342526+0000","last_clean":"2024-09-17T15:38:27.342526+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:38:27.342526+0000","last_undegraded":"2024-09-17T15:38:27.342526+0000","last_fullsized":"2024-09-17T15:38:27.342526+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28848,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338595+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:38:27.338595+0000","last_peered":"2024-09-17T15:38:27.338595+0000","last_clean":"2024-09-17T15:38:27.338595+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:38:27.338595+0000","last_undegraded":"2024-09-17T15:38:27.338595+0000","last_fullsized":"2024-09-17T15:38:27.338595+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":121,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337473+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:38:27.337473+0000","last_peered":"2024-09-17T15:38:27.337473+0000","last_clean":"2024-09-17T15:38:27.337473+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:38:27.337473+0000","last_undegraded":"2024-09-17T15:38:27.337473+0000","last_fullsized":"2024-09-17T15:38:27.337473+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29185,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338248+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:38:27.338248+0000","last_peered":"2024-09-17T15:38:27.338248+0000","last_clean":"2024-09-17T15:38:27.338248+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:38:27.338248+0000","last_undegraded":"2024-09-17T15:38:27.338248+0000","last_fullsized":"2024-09-17T15:38:27.338248+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21295,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337742+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:38:27.337742+0000","last_peered":"2024-09-17T15:38:27.337742+0000","last_clean":"2024-09-17T15:38:27.337742+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:38:27.337742+0000","last_undegraded":"2024-09-17T15:38:27.337742+0000","last_fullsized":"2024-09-17T15:38:27.337742+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":111,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338212+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:38:27.338212+0000","last_peered":"2024-09-17T15:38:27.338212+0000","last_clean":"2024-09-17T15:38:27.338212+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:38:27.338212+0000","last_undegraded":"2024-09-17T15:38:27.338212+0000","last_fullsized":"2024-09-17T15:38:27.338212+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28099,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338109+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:38:27.338109+0000","last_peered":"2024-09-17T15:38:27.338109+0000","last_clean":"2024-09-17T15:38:27.338109+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:38:27.338109+0000","last_undegraded":"2024-09-17T15:38:27.338109+0000","last_fullsized":"2024-09-17T15:38:27.338109+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338779+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:38:27.338779+0000","last_peered":"2024-09-17T15:38:27.338779+0000","last_clean":"2024-09-17T15:38:27.338779+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:38:27.338779+0000","last_undegraded":"2024-09-17T15:38:27.338779+0000","last_fullsized":"2024-09-17T15:38:27.338779+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21204,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.430121+0000","last_change":"2024-09-17T15:38:27.429539+0000","last_active":"2024-09-17T15:38:27.430121+0000","last_peered":"2024-09-17T15:38:27.430121+0000","last_clean":"2024-09-17T15:38:27.430121+0000","last_became_active":"2024-09-17T15:38:27.345429+0000","last_became_peered":"2024-09-17T15:38:27.345429+0000","last_unstale":"2024-09-17T15:38:27.430121+0000","last_undegraded":"2024-09-17T15:38:27.430121+0000","last_fullsized":"2024-09-17T15:38:27.430121+0000","mapping_epoch":62,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:49:28.575667+0000","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4065,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":111,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338015+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:38:27.338015+0000","last_peered":"2024-09-17T15:38:27.338015+0000","last_clean":"2024-09-17T15:38:27.338015+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:38:27.338015+0000","last_undegraded":"2024-09-17T15:38:27.338015+0000","last_fullsized":"2024-09-17T15:38:27.338015+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":116,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337667+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:38:27.337667+0000","last_peered":"2024-09-17T15:38:27.337667+0000","last_clean":"2024-09-17T15:38:27.337667+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:38:27.337667+0000","last_undegraded":"2024-09-17T15:38:27.337667+0000","last_fullsized":"2024-09-17T15:38:27.337667+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29050,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338223+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:38:27.338223+0000","last_peered":"2024-09-17T15:38:27.338223+0000","last_clean":"2024-09-17T15:38:27.338223+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:38:27.338223+0000","last_undegraded":"2024-09-17T15:38:27.338223+0000","last_fullsized":"2024-09-17T15:38:27.338223+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1469,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47991,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":134307840,"data_stored":134228354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051748,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418560,"kb_used_data":66160,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93953280,"statfs":{"total":96636764160,"available":96208158720,"internally_reserved":0,"allocated":67747840,"data_stored":67382571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724693,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519536,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852304,"statfs":{"total":96636764160,"available":96104759296,"internally_reserved":0,"allocated":557056,"data_stored":180484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":531426987},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247976,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539472,"kb_used_data":66360,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832368,"statfs":{"total":96636764160,"available":96084344832,"internally_reserved":0,"allocated":67952640,"data_stored":67565229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":66854912,"data_stored":66836847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:38:49.499 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:38:49.798 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:49.798 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:38:49.811 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":488,"stamp":"2024-09-17T15:38:47.843922+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1469,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47992,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1477568,"kb_used_data":133064,"kb_used_omap":13,"kb_used_meta":1344434,"kb_avail":281637952,"statfs":{"total":289910292480,"available":288397262848,"internally_reserved":0,"allocated":136257536,"data_stored":135128284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1376701439},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1420,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001857"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":28918,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.343038+0000","last_change":"2024-09-17T15:38:06.362595+0000","last_active":"2024-09-17T15:38:27.343038+0000","last_peered":"2024-09-17T15:38:27.343038+0000","last_clean":"2024-09-17T15:38:27.343038+0000","last_became_active":"2024-09-17T15:38:06.284139+0000","last_became_peered":"2024-09-17T15:38:06.284139+0000","last_unstale":"2024-09-17T15:38:27.343038+0000","last_undegraded":"2024-09-17T15:38:27.343038+0000","last_fullsized":"2024-09-17T15:38:27.343038+0000","mapping_epoch":60,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:42:51.418256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4070,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21078,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.339083+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:38:27.339083+0000","last_peered":"2024-09-17T15:38:27.339083+0000","last_clean":"2024-09-17T15:38:27.339083+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:38:27.339083+0000","last_undegraded":"2024-09-17T15:38:27.339083+0000","last_fullsized":"2024-09-17T15:38:27.339083+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21186,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337203+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:38:27.337203+0000","last_peered":"2024-09-17T15:38:27.337203+0000","last_clean":"2024-09-17T15:38:27.337203+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:38:27.337203+0000","last_undegraded":"2024-09-17T15:38:27.337203+0000","last_fullsized":"2024-09-17T15:38:27.337203+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21083,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338986+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:38:27.338986+0000","last_peered":"2024-09-17T15:38:27.338986+0000","last_clean":"2024-09-17T15:38:27.338986+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:38:27.338986+0000","last_undegraded":"2024-09-17T15:38:27.338986+0000","last_fullsized":"2024-09-17T15:38:27.338986+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20826,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342773+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:38:27.342773+0000","last_peered":"2024-09-17T15:38:27.342773+0000","last_clean":"2024-09-17T15:38:27.342773+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:38:27.342773+0000","last_undegraded":"2024-09-17T15:38:27.342773+0000","last_fullsized":"2024-09-17T15:38:27.342773+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21577,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337752+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:38:27.337752+0000","last_peered":"2024-09-17T15:38:27.337752+0000","last_clean":"2024-09-17T15:38:27.337752+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:38:27.337752+0000","last_undegraded":"2024-09-17T15:38:27.337752+0000","last_fullsized":"2024-09-17T15:38:27.337752+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21121,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337152+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:38:27.337152+0000","last_peered":"2024-09-17T15:38:27.337152+0000","last_clean":"2024-09-17T15:38:27.337152+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:38:27.337152+0000","last_undegraded":"2024-09-17T15:38:27.337152+0000","last_fullsized":"2024-09-17T15:38:27.337152+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21088,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342372+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:38:27.342372+0000","last_peered":"2024-09-17T15:38:27.342372+0000","last_clean":"2024-09-17T15:38:27.342372+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:38:27.342372+0000","last_undegraded":"2024-09-17T15:38:27.342372+0000","last_fullsized":"2024-09-17T15:38:27.342372+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337064+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:38:27.337064+0000","last_peered":"2024-09-17T15:38:27.337064+0000","last_clean":"2024-09-17T15:38:27.337064+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:38:27.337064+0000","last_undegraded":"2024-09-17T15:38:27.337064+0000","last_fullsized":"2024-09-17T15:38:27.337064+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29143,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337570+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:38:27.337570+0000","last_peered":"2024-09-17T15:38:27.337570+0000","last_clean":"2024-09-17T15:38:27.337570+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:38:27.337570+0000","last_undegraded":"2024-09-17T15:38:27.337570+0000","last_fullsized":"2024-09-17T15:38:27.337570+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338510+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:38:27.338510+0000","last_peered":"2024-09-17T15:38:27.338510+0000","last_clean":"2024-09-17T15:38:27.338510+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:38:27.338510+0000","last_undegraded":"2024-09-17T15:38:27.338510+0000","last_fullsized":"2024-09-17T15:38:27.338510+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":25804,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:44.442697+0000","last_change":"2024-09-17T15:38:25.720175+0000","last_active":"2024-09-17T15:38:44.442697+0000","last_peered":"2024-09-17T15:38:44.442697+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:38:44.442697+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1469,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2543,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1469}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":147,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342526+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:38:27.342526+0000","last_peered":"2024-09-17T15:38:27.342526+0000","last_clean":"2024-09-17T15:38:27.342526+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:38:27.342526+0000","last_undegraded":"2024-09-17T15:38:27.342526+0000","last_fullsized":"2024-09-17T15:38:27.342526+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28848,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338595+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:38:27.338595+0000","last_peered":"2024-09-17T15:38:27.338595+0000","last_clean":"2024-09-17T15:38:27.338595+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:38:27.338595+0000","last_undegraded":"2024-09-17T15:38:27.338595+0000","last_fullsized":"2024-09-17T15:38:27.338595+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":121,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337473+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:38:27.337473+0000","last_peered":"2024-09-17T15:38:27.337473+0000","last_clean":"2024-09-17T15:38:27.337473+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:38:27.337473+0000","last_undegraded":"2024-09-17T15:38:27.337473+0000","last_fullsized":"2024-09-17T15:38:27.337473+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29185,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338248+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:38:27.338248+0000","last_peered":"2024-09-17T15:38:27.338248+0000","last_clean":"2024-09-17T15:38:27.338248+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:38:27.338248+0000","last_undegraded":"2024-09-17T15:38:27.338248+0000","last_fullsized":"2024-09-17T15:38:27.338248+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21295,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337742+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:38:27.337742+0000","last_peered":"2024-09-17T15:38:27.337742+0000","last_clean":"2024-09-17T15:38:27.337742+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:38:27.337742+0000","last_undegraded":"2024-09-17T15:38:27.337742+0000","last_fullsized":"2024-09-17T15:38:27.337742+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":111,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338212+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:38:27.338212+0000","last_peered":"2024-09-17T15:38:27.338212+0000","last_clean":"2024-09-17T15:38:27.338212+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:38:27.338212+0000","last_undegraded":"2024-09-17T15:38:27.338212+0000","last_fullsized":"2024-09-17T15:38:27.338212+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28099,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338109+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:38:27.338109+0000","last_peered":"2024-09-17T15:38:27.338109+0000","last_clean":"2024-09-17T15:38:27.338109+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:38:27.338109+0000","last_undegraded":"2024-09-17T15:38:27.338109+0000","last_fullsized":"2024-09-17T15:38:27.338109+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338779+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:38:27.338779+0000","last_peered":"2024-09-17T15:38:27.338779+0000","last_clean":"2024-09-17T15:38:27.338779+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:38:27.338779+0000","last_undegraded":"2024-09-17T15:38:27.338779+0000","last_fullsized":"2024-09-17T15:38:27.338779+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21204,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.430121+0000","last_change":"2024-09-17T15:38:27.429539+0000","last_active":"2024-09-17T15:38:27.430121+0000","last_peered":"2024-09-17T15:38:27.430121+0000","last_clean":"2024-09-17T15:38:27.430121+0000","last_became_active":"2024-09-17T15:38:27.345429+0000","last_became_peered":"2024-09-17T15:38:27.345429+0000","last_unstale":"2024-09-17T15:38:27.430121+0000","last_undegraded":"2024-09-17T15:38:27.430121+0000","last_fullsized":"2024-09-17T15:38:27.430121+0000","mapping_epoch":62,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:49:28.575667+0000","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4065,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":111,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338015+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:38:27.338015+0000","last_peered":"2024-09-17T15:38:27.338015+0000","last_clean":"2024-09-17T15:38:27.338015+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:38:27.338015+0000","last_undegraded":"2024-09-17T15:38:27.338015+0000","last_fullsized":"2024-09-17T15:38:27.338015+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":116,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337667+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:38:27.337667+0000","last_peered":"2024-09-17T15:38:27.337667+0000","last_clean":"2024-09-17T15:38:27.337667+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:38:27.337667+0000","last_undegraded":"2024-09-17T15:38:27.337667+0000","last_fullsized":"2024-09-17T15:38:27.337667+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29050,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338223+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:38:27.338223+0000","last_peered":"2024-09-17T15:38:27.338223+0000","last_clean":"2024-09-17T15:38:27.338223+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:38:27.338223+0000","last_undegraded":"2024-09-17T15:38:27.338223+0000","last_fullsized":"2024-09-17T15:38:27.338223+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1469,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47991,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":134307840,"data_stored":134228354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051748,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418560,"kb_used_data":66160,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93953280,"statfs":{"total":96636764160,"available":96208158720,"internally_reserved":0,"allocated":67747840,"data_stored":67382571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724693,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519536,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852304,"statfs":{"total":96636764160,"available":96104759296,"internally_reserved":0,"allocated":557056,"data_stored":180484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":531426987},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247976,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539472,"kb_used_data":66360,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832368,"statfs":{"total":96636764160,"available":96084344832,"internally_reserved":0,"allocated":67952640,"data_stored":67565229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":66854912,"data_stored":66836847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:38:49.812 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:38:50.111 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:50.112 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:38:50.124 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":489,"stamp":"2024-09-17T15:38:49.844256+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":490,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48971,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1481688,"kb_used_data":128864,"kb_used_omap":13,"kb_used_meta":1352754,"kb_avail":281633832,"statfs":{"total":289910292480,"available":288393043968,"internally_reserved":0,"allocated":131956736,"data_stored":130827484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1385221119},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2399,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2399,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001882"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":28918,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.343038+0000","last_change":"2024-09-17T15:38:06.362595+0000","last_active":"2024-09-17T15:38:27.343038+0000","last_peered":"2024-09-17T15:38:27.343038+0000","last_clean":"2024-09-17T15:38:27.343038+0000","last_became_active":"2024-09-17T15:38:06.284139+0000","last_became_peered":"2024-09-17T15:38:06.284139+0000","last_unstale":"2024-09-17T15:38:27.343038+0000","last_undegraded":"2024-09-17T15:38:27.343038+0000","last_fullsized":"2024-09-17T15:38:27.343038+0000","mapping_epoch":60,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:42:51.418256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4070,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21078,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.339083+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:38:27.339083+0000","last_peered":"2024-09-17T15:38:27.339083+0000","last_clean":"2024-09-17T15:38:27.339083+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:38:27.339083+0000","last_undegraded":"2024-09-17T15:38:27.339083+0000","last_fullsized":"2024-09-17T15:38:27.339083+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21186,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337203+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:38:27.337203+0000","last_peered":"2024-09-17T15:38:27.337203+0000","last_clean":"2024-09-17T15:38:27.337203+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:38:27.337203+0000","last_undegraded":"2024-09-17T15:38:27.337203+0000","last_fullsized":"2024-09-17T15:38:27.337203+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21083,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338986+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:38:27.338986+0000","last_peered":"2024-09-17T15:38:27.338986+0000","last_clean":"2024-09-17T15:38:27.338986+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:38:27.338986+0000","last_undegraded":"2024-09-17T15:38:27.338986+0000","last_fullsized":"2024-09-17T15:38:27.338986+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20826,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342773+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:38:27.342773+0000","last_peered":"2024-09-17T15:38:27.342773+0000","last_clean":"2024-09-17T15:38:27.342773+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:38:27.342773+0000","last_undegraded":"2024-09-17T15:38:27.342773+0000","last_fullsized":"2024-09-17T15:38:27.342773+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21577,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337752+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:38:27.337752+0000","last_peered":"2024-09-17T15:38:27.337752+0000","last_clean":"2024-09-17T15:38:27.337752+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:38:27.337752+0000","last_undegraded":"2024-09-17T15:38:27.337752+0000","last_fullsized":"2024-09-17T15:38:27.337752+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21121,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337152+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:38:27.337152+0000","last_peered":"2024-09-17T15:38:27.337152+0000","last_clean":"2024-09-17T15:38:27.337152+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:38:27.337152+0000","last_undegraded":"2024-09-17T15:38:27.337152+0000","last_fullsized":"2024-09-17T15:38:27.337152+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21088,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342372+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:38:27.342372+0000","last_peered":"2024-09-17T15:38:27.342372+0000","last_clean":"2024-09-17T15:38:27.342372+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:38:27.342372+0000","last_undegraded":"2024-09-17T15:38:27.342372+0000","last_fullsized":"2024-09-17T15:38:27.342372+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337064+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:38:27.337064+0000","last_peered":"2024-09-17T15:38:27.337064+0000","last_clean":"2024-09-17T15:38:27.337064+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:38:27.337064+0000","last_undegraded":"2024-09-17T15:38:27.337064+0000","last_fullsized":"2024-09-17T15:38:27.337064+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29143,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337570+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:38:27.337570+0000","last_peered":"2024-09-17T15:38:27.337570+0000","last_clean":"2024-09-17T15:38:27.337570+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:38:27.337570+0000","last_undegraded":"2024-09-17T15:38:27.337570+0000","last_fullsized":"2024-09-17T15:38:27.337570+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338510+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:38:27.338510+0000","last_peered":"2024-09-17T15:38:27.338510+0000","last_clean":"2024-09-17T15:38:27.338510+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:38:27.338510+0000","last_undegraded":"2024-09-17T15:38:27.338510+0000","last_fullsized":"2024-09-17T15:38:27.338510+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":27762,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:49.445400+0000","last_change":"2024-09-17T15:38:25.720175+0000","last_active":"2024-09-17T15:38:49.445400+0000","last_peered":"2024-09-17T15:38:49.445400+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:38:49.445400+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":490,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3522,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-490}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":147,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342526+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:38:27.342526+0000","last_peered":"2024-09-17T15:38:27.342526+0000","last_clean":"2024-09-17T15:38:27.342526+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:38:27.342526+0000","last_undegraded":"2024-09-17T15:38:27.342526+0000","last_fullsized":"2024-09-17T15:38:27.342526+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28848,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338595+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:38:27.338595+0000","last_peered":"2024-09-17T15:38:27.338595+0000","last_clean":"2024-09-17T15:38:27.338595+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:38:27.338595+0000","last_undegraded":"2024-09-17T15:38:27.338595+0000","last_fullsized":"2024-09-17T15:38:27.338595+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":121,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337473+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:38:27.337473+0000","last_peered":"2024-09-17T15:38:27.337473+0000","last_clean":"2024-09-17T15:38:27.337473+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:38:27.337473+0000","last_undegraded":"2024-09-17T15:38:27.337473+0000","last_fullsized":"2024-09-17T15:38:27.337473+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29185,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338248+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:38:27.338248+0000","last_peered":"2024-09-17T15:38:27.338248+0000","last_clean":"2024-09-17T15:38:27.338248+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:38:27.338248+0000","last_undegraded":"2024-09-17T15:38:27.338248+0000","last_fullsized":"2024-09-17T15:38:27.338248+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21295,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337742+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:38:27.337742+0000","last_peered":"2024-09-17T15:38:27.337742+0000","last_clean":"2024-09-17T15:38:27.337742+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:38:27.337742+0000","last_undegraded":"2024-09-17T15:38:27.337742+0000","last_fullsized":"2024-09-17T15:38:27.337742+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":111,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338212+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:38:27.338212+0000","last_peered":"2024-09-17T15:38:27.338212+0000","last_clean":"2024-09-17T15:38:27.338212+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:38:27.338212+0000","last_undegraded":"2024-09-17T15:38:27.338212+0000","last_fullsized":"2024-09-17T15:38:27.338212+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28099,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338109+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:38:27.338109+0000","last_peered":"2024-09-17T15:38:27.338109+0000","last_clean":"2024-09-17T15:38:27.338109+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:38:27.338109+0000","last_undegraded":"2024-09-17T15:38:27.338109+0000","last_fullsized":"2024-09-17T15:38:27.338109+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338779+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:38:27.338779+0000","last_peered":"2024-09-17T15:38:27.338779+0000","last_clean":"2024-09-17T15:38:27.338779+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:38:27.338779+0000","last_undegraded":"2024-09-17T15:38:27.338779+0000","last_fullsized":"2024-09-17T15:38:27.338779+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21204,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.430121+0000","last_change":"2024-09-17T15:38:27.429539+0000","last_active":"2024-09-17T15:38:27.430121+0000","last_peered":"2024-09-17T15:38:27.430121+0000","last_clean":"2024-09-17T15:38:27.430121+0000","last_became_active":"2024-09-17T15:38:27.345429+0000","last_became_peered":"2024-09-17T15:38:27.345429+0000","last_unstale":"2024-09-17T15:38:27.430121+0000","last_undegraded":"2024-09-17T15:38:27.430121+0000","last_fullsized":"2024-09-17T15:38:27.430121+0000","mapping_epoch":62,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:49:28.575667+0000","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4065,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":111,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338015+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:38:27.338015+0000","last_peered":"2024-09-17T15:38:27.338015+0000","last_clean":"2024-09-17T15:38:27.338015+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:38:27.338015+0000","last_undegraded":"2024-09-17T15:38:27.338015+0000","last_fullsized":"2024-09-17T15:38:27.338015+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":116,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337667+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:38:27.337667+0000","last_peered":"2024-09-17T15:38:27.337667+0000","last_clean":"2024-09-17T15:38:27.337667+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:38:27.337667+0000","last_undegraded":"2024-09-17T15:38:27.337667+0000","last_fullsized":"2024-09-17T15:38:27.337667+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29050,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338223+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:38:27.338223+0000","last_peered":"2024-09-17T15:38:27.338223+0000","last_clean":"2024-09-17T15:38:27.338223+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:38:27.338223+0000","last_undegraded":"2024-09-17T15:38:27.338223+0000","last_fullsized":"2024-09-17T15:38:27.338223+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":490,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48970,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":130129920,"data_stored":130050434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051749,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":414360,"kb_used_data":61960,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93957480,"statfs":{"total":96636764160,"available":96212459520,"internally_reserved":0,"allocated":63447040,"data_stored":63081771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724694,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527856,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93843984,"statfs":{"total":96636764160,"available":96096239616,"internally_reserved":0,"allocated":557056,"data_stored":180484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":539946667},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247977,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539472,"kb_used_data":66360,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832368,"statfs":{"total":96636764160,"available":96084344832,"internally_reserved":0,"allocated":67952640,"data_stored":67565229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":62676992,"data_stored":62658927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:38:50.126 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:38:50.126 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20542", 'reported_seq': 27762, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:38:49.445400+0000', 'last_change': '2024-09-17T15:38:25.720175+0000', 'last_active': '2024-09-17T15:38:49.445400+0000', 'last_peered': '2024-09-17T15:38:49.445400+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:38:49.445400+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 490, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17138, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3522, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -490}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:38:50.126 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:38:50.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-09-17T15:38:50.431+0000 7ff6a6d9d640 0 --2- 172.21.15.83:0/2537411112 >> v2:172.21.15.83:3300/0 conn(0x7ff6a8128410 0x7ff6a81300a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-17T15:38:50.720 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:50.720 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 490/24 objects degraded (2041.667%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":867,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1517248512,"bytes_avail":288393043968,"bytes_total":289910292480,"degraded_objects":490,"degraded_total":24,"degraded_ratio":20.416666666666668,"recovering_objects_per_sec":199,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2399,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:38:50.734 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:38:50.734 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:38:51.039 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:51.039 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:38:51.051 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":489,"stamp":"2024-09-17T15:38:49.844256+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":490,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48971,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1481688,"kb_used_data":128864,"kb_used_omap":13,"kb_used_meta":1352754,"kb_avail":281633832,"statfs":{"total":289910292480,"available":288393043968,"internally_reserved":0,"allocated":131956736,"data_stored":130827484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1385221119},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2399,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2399,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001882"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":28918,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.343038+0000","last_change":"2024-09-17T15:38:06.362595+0000","last_active":"2024-09-17T15:38:27.343038+0000","last_peered":"2024-09-17T15:38:27.343038+0000","last_clean":"2024-09-17T15:38:27.343038+0000","last_became_active":"2024-09-17T15:38:06.284139+0000","last_became_peered":"2024-09-17T15:38:06.284139+0000","last_unstale":"2024-09-17T15:38:27.343038+0000","last_undegraded":"2024-09-17T15:38:27.343038+0000","last_fullsized":"2024-09-17T15:38:27.343038+0000","mapping_epoch":60,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:42:51.418256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4070,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21078,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.339083+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:38:27.339083+0000","last_peered":"2024-09-17T15:38:27.339083+0000","last_clean":"2024-09-17T15:38:27.339083+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:38:27.339083+0000","last_undegraded":"2024-09-17T15:38:27.339083+0000","last_fullsized":"2024-09-17T15:38:27.339083+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21186,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337203+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:38:27.337203+0000","last_peered":"2024-09-17T15:38:27.337203+0000","last_clean":"2024-09-17T15:38:27.337203+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:38:27.337203+0000","last_undegraded":"2024-09-17T15:38:27.337203+0000","last_fullsized":"2024-09-17T15:38:27.337203+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21083,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338986+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:38:27.338986+0000","last_peered":"2024-09-17T15:38:27.338986+0000","last_clean":"2024-09-17T15:38:27.338986+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:38:27.338986+0000","last_undegraded":"2024-09-17T15:38:27.338986+0000","last_fullsized":"2024-09-17T15:38:27.338986+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20826,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342773+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:38:27.342773+0000","last_peered":"2024-09-17T15:38:27.342773+0000","last_clean":"2024-09-17T15:38:27.342773+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:38:27.342773+0000","last_undegraded":"2024-09-17T15:38:27.342773+0000","last_fullsized":"2024-09-17T15:38:27.342773+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21577,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337752+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:38:27.337752+0000","last_peered":"2024-09-17T15:38:27.337752+0000","last_clean":"2024-09-17T15:38:27.337752+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:38:27.337752+0000","last_undegraded":"2024-09-17T15:38:27.337752+0000","last_fullsized":"2024-09-17T15:38:27.337752+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21121,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337152+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:38:27.337152+0000","last_peered":"2024-09-17T15:38:27.337152+0000","last_clean":"2024-09-17T15:38:27.337152+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:38:27.337152+0000","last_undegraded":"2024-09-17T15:38:27.337152+0000","last_fullsized":"2024-09-17T15:38:27.337152+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21088,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342372+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:38:27.342372+0000","last_peered":"2024-09-17T15:38:27.342372+0000","last_clean":"2024-09-17T15:38:27.342372+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:38:27.342372+0000","last_undegraded":"2024-09-17T15:38:27.342372+0000","last_fullsized":"2024-09-17T15:38:27.342372+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337064+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:38:27.337064+0000","last_peered":"2024-09-17T15:38:27.337064+0000","last_clean":"2024-09-17T15:38:27.337064+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:38:27.337064+0000","last_undegraded":"2024-09-17T15:38:27.337064+0000","last_fullsized":"2024-09-17T15:38:27.337064+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29143,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337570+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:38:27.337570+0000","last_peered":"2024-09-17T15:38:27.337570+0000","last_clean":"2024-09-17T15:38:27.337570+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:38:27.337570+0000","last_undegraded":"2024-09-17T15:38:27.337570+0000","last_fullsized":"2024-09-17T15:38:27.337570+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338510+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:38:27.338510+0000","last_peered":"2024-09-17T15:38:27.338510+0000","last_clean":"2024-09-17T15:38:27.338510+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:38:27.338510+0000","last_undegraded":"2024-09-17T15:38:27.338510+0000","last_fullsized":"2024-09-17T15:38:27.338510+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":27762,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:49.445400+0000","last_change":"2024-09-17T15:38:25.720175+0000","last_active":"2024-09-17T15:38:49.445400+0000","last_peered":"2024-09-17T15:38:49.445400+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:38:49.445400+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":490,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3522,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-490}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":147,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342526+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:38:27.342526+0000","last_peered":"2024-09-17T15:38:27.342526+0000","last_clean":"2024-09-17T15:38:27.342526+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:38:27.342526+0000","last_undegraded":"2024-09-17T15:38:27.342526+0000","last_fullsized":"2024-09-17T15:38:27.342526+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28848,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338595+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:38:27.338595+0000","last_peered":"2024-09-17T15:38:27.338595+0000","last_clean":"2024-09-17T15:38:27.338595+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:38:27.338595+0000","last_undegraded":"2024-09-17T15:38:27.338595+0000","last_fullsized":"2024-09-17T15:38:27.338595+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":121,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337473+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:38:27.337473+0000","last_peered":"2024-09-17T15:38:27.337473+0000","last_clean":"2024-09-17T15:38:27.337473+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:38:27.337473+0000","last_undegraded":"2024-09-17T15:38:27.337473+0000","last_fullsized":"2024-09-17T15:38:27.337473+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29185,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338248+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:38:27.338248+0000","last_peered":"2024-09-17T15:38:27.338248+0000","last_clean":"2024-09-17T15:38:27.338248+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:38:27.338248+0000","last_undegraded":"2024-09-17T15:38:27.338248+0000","last_fullsized":"2024-09-17T15:38:27.338248+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21295,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337742+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:38:27.337742+0000","last_peered":"2024-09-17T15:38:27.337742+0000","last_clean":"2024-09-17T15:38:27.337742+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:38:27.337742+0000","last_undegraded":"2024-09-17T15:38:27.337742+0000","last_fullsized":"2024-09-17T15:38:27.337742+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":111,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338212+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:38:27.338212+0000","last_peered":"2024-09-17T15:38:27.338212+0000","last_clean":"2024-09-17T15:38:27.338212+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:38:27.338212+0000","last_undegraded":"2024-09-17T15:38:27.338212+0000","last_fullsized":"2024-09-17T15:38:27.338212+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28099,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338109+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:38:27.338109+0000","last_peered":"2024-09-17T15:38:27.338109+0000","last_clean":"2024-09-17T15:38:27.338109+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:38:27.338109+0000","last_undegraded":"2024-09-17T15:38:27.338109+0000","last_fullsized":"2024-09-17T15:38:27.338109+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338779+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:38:27.338779+0000","last_peered":"2024-09-17T15:38:27.338779+0000","last_clean":"2024-09-17T15:38:27.338779+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:38:27.338779+0000","last_undegraded":"2024-09-17T15:38:27.338779+0000","last_fullsized":"2024-09-17T15:38:27.338779+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21204,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.430121+0000","last_change":"2024-09-17T15:38:27.429539+0000","last_active":"2024-09-17T15:38:27.430121+0000","last_peered":"2024-09-17T15:38:27.430121+0000","last_clean":"2024-09-17T15:38:27.430121+0000","last_became_active":"2024-09-17T15:38:27.345429+0000","last_became_peered":"2024-09-17T15:38:27.345429+0000","last_unstale":"2024-09-17T15:38:27.430121+0000","last_undegraded":"2024-09-17T15:38:27.430121+0000","last_fullsized":"2024-09-17T15:38:27.430121+0000","mapping_epoch":62,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:49:28.575667+0000","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4065,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":111,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338015+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:38:27.338015+0000","last_peered":"2024-09-17T15:38:27.338015+0000","last_clean":"2024-09-17T15:38:27.338015+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:38:27.338015+0000","last_undegraded":"2024-09-17T15:38:27.338015+0000","last_fullsized":"2024-09-17T15:38:27.338015+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":116,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337667+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:38:27.337667+0000","last_peered":"2024-09-17T15:38:27.337667+0000","last_clean":"2024-09-17T15:38:27.337667+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:38:27.337667+0000","last_undegraded":"2024-09-17T15:38:27.337667+0000","last_fullsized":"2024-09-17T15:38:27.337667+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29050,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338223+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:38:27.338223+0000","last_peered":"2024-09-17T15:38:27.338223+0000","last_clean":"2024-09-17T15:38:27.338223+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:38:27.338223+0000","last_undegraded":"2024-09-17T15:38:27.338223+0000","last_fullsized":"2024-09-17T15:38:27.338223+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":490,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48970,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":130129920,"data_stored":130050434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051749,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":414360,"kb_used_data":61960,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93957480,"statfs":{"total":96636764160,"available":96212459520,"internally_reserved":0,"allocated":63447040,"data_stored":63081771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724694,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527856,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93843984,"statfs":{"total":96636764160,"available":96096239616,"internally_reserved":0,"allocated":557056,"data_stored":180484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":539946667},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247977,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539472,"kb_used_data":66360,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832368,"statfs":{"total":96636764160,"available":96084344832,"internally_reserved":0,"allocated":67952640,"data_stored":67565229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":62676992,"data_stored":62658927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:38:54.053 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:38:54.349 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:54.349 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:38:54.361 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":493,"stamp":"2024-09-17T15:38:53.844975+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":490,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48971,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1477392,"kb_used_data":124564,"kb_used_omap":13,"kb_used_meta":1352754,"kb_avail":281638128,"statfs":{"total":289910292480,"available":288397443072,"internally_reserved":0,"allocated":127553536,"data_stored":126413333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1385221119},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":979,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001365"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":28918,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.343038+0000","last_change":"2024-09-17T15:38:06.362595+0000","last_active":"2024-09-17T15:38:27.343038+0000","last_peered":"2024-09-17T15:38:27.343038+0000","last_clean":"2024-09-17T15:38:27.343038+0000","last_became_active":"2024-09-17T15:38:06.284139+0000","last_became_peered":"2024-09-17T15:38:06.284139+0000","last_unstale":"2024-09-17T15:38:27.343038+0000","last_undegraded":"2024-09-17T15:38:27.343038+0000","last_fullsized":"2024-09-17T15:38:27.343038+0000","mapping_epoch":60,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:42:51.418256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4070,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21083,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417805+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:38:52.417805+0000","last_peered":"2024-09-17T15:38:52.417805+0000","last_clean":"2024-09-17T15:38:52.417805+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:38:52.417805+0000","last_undegraded":"2024-09-17T15:38:52.417805+0000","last_fullsized":"2024-09-17T15:38:52.417805+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21186,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337203+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:38:27.337203+0000","last_peered":"2024-09-17T15:38:27.337203+0000","last_clean":"2024-09-17T15:38:27.337203+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:38:27.337203+0000","last_undegraded":"2024-09-17T15:38:27.337203+0000","last_fullsized":"2024-09-17T15:38:27.337203+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21088,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417541+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:38:52.417541+0000","last_peered":"2024-09-17T15:38:52.417541+0000","last_clean":"2024-09-17T15:38:52.417541+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:38:52.417541+0000","last_undegraded":"2024-09-17T15:38:52.417541+0000","last_fullsized":"2024-09-17T15:38:52.417541+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20826,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342773+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:38:27.342773+0000","last_peered":"2024-09-17T15:38:27.342773+0000","last_clean":"2024-09-17T15:38:27.342773+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:38:27.342773+0000","last_undegraded":"2024-09-17T15:38:27.342773+0000","last_fullsized":"2024-09-17T15:38:27.342773+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21577,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337752+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:38:27.337752+0000","last_peered":"2024-09-17T15:38:27.337752+0000","last_clean":"2024-09-17T15:38:27.337752+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:38:27.337752+0000","last_undegraded":"2024-09-17T15:38:27.337752+0000","last_fullsized":"2024-09-17T15:38:27.337752+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21121,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337152+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:38:27.337152+0000","last_peered":"2024-09-17T15:38:27.337152+0000","last_clean":"2024-09-17T15:38:27.337152+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:38:27.337152+0000","last_undegraded":"2024-09-17T15:38:27.337152+0000","last_fullsized":"2024-09-17T15:38:27.337152+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21088,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342372+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:38:27.342372+0000","last_peered":"2024-09-17T15:38:27.342372+0000","last_clean":"2024-09-17T15:38:27.342372+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:38:27.342372+0000","last_undegraded":"2024-09-17T15:38:27.342372+0000","last_fullsized":"2024-09-17T15:38:27.342372+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337064+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:38:27.337064+0000","last_peered":"2024-09-17T15:38:27.337064+0000","last_clean":"2024-09-17T15:38:27.337064+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:38:27.337064+0000","last_undegraded":"2024-09-17T15:38:27.337064+0000","last_fullsized":"2024-09-17T15:38:27.337064+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29143,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337570+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:38:27.337570+0000","last_peered":"2024-09-17T15:38:27.337570+0000","last_clean":"2024-09-17T15:38:27.337570+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:38:27.337570+0000","last_undegraded":"2024-09-17T15:38:27.337570+0000","last_fullsized":"2024-09-17T15:38:27.337570+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417181+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:38:52.417181+0000","last_peered":"2024-09-17T15:38:52.417181+0000","last_clean":"2024-09-17T15:38:52.417181+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:38:52.417181+0000","last_undegraded":"2024-09-17T15:38:52.417181+0000","last_fullsized":"2024-09-17T15:38:52.417181+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":27762,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:49.445400+0000","last_change":"2024-09-17T15:38:25.720175+0000","last_active":"2024-09-17T15:38:49.445400+0000","last_peered":"2024-09-17T15:38:49.445400+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:38:49.445400+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":490,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3522,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-490}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":147,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342526+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:38:27.342526+0000","last_peered":"2024-09-17T15:38:27.342526+0000","last_clean":"2024-09-17T15:38:27.342526+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:38:27.342526+0000","last_undegraded":"2024-09-17T15:38:27.342526+0000","last_fullsized":"2024-09-17T15:38:27.342526+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28852,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417187+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:38:52.417187+0000","last_peered":"2024-09-17T15:38:52.417187+0000","last_clean":"2024-09-17T15:38:52.417187+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:38:52.417187+0000","last_undegraded":"2024-09-17T15:38:52.417187+0000","last_fullsized":"2024-09-17T15:38:52.417187+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":121,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337473+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:38:27.337473+0000","last_peered":"2024-09-17T15:38:27.337473+0000","last_clean":"2024-09-17T15:38:27.337473+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:38:27.337473+0000","last_undegraded":"2024-09-17T15:38:27.337473+0000","last_fullsized":"2024-09-17T15:38:27.337473+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29189,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417037+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:38:52.417037+0000","last_peered":"2024-09-17T15:38:52.417037+0000","last_clean":"2024-09-17T15:38:52.417037+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:38:52.417037+0000","last_undegraded":"2024-09-17T15:38:52.417037+0000","last_fullsized":"2024-09-17T15:38:52.417037+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21295,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337742+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:38:27.337742+0000","last_peered":"2024-09-17T15:38:27.337742+0000","last_clean":"2024-09-17T15:38:27.337742+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:38:27.337742+0000","last_undegraded":"2024-09-17T15:38:27.337742+0000","last_fullsized":"2024-09-17T15:38:27.337742+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":111,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338212+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:38:27.338212+0000","last_peered":"2024-09-17T15:38:27.338212+0000","last_clean":"2024-09-17T15:38:27.338212+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:38:27.338212+0000","last_undegraded":"2024-09-17T15:38:27.338212+0000","last_fullsized":"2024-09-17T15:38:27.338212+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28099,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338109+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:38:27.338109+0000","last_peered":"2024-09-17T15:38:27.338109+0000","last_clean":"2024-09-17T15:38:27.338109+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:38:27.338109+0000","last_undegraded":"2024-09-17T15:38:27.338109+0000","last_fullsized":"2024-09-17T15:38:27.338109+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417682+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:38:52.417682+0000","last_peered":"2024-09-17T15:38:52.417682+0000","last_clean":"2024-09-17T15:38:52.417682+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:38:52.417682+0000","last_undegraded":"2024-09-17T15:38:52.417682+0000","last_fullsized":"2024-09-17T15:38:52.417682+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21204,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.430121+0000","last_change":"2024-09-17T15:38:27.429539+0000","last_active":"2024-09-17T15:38:27.430121+0000","last_peered":"2024-09-17T15:38:27.430121+0000","last_clean":"2024-09-17T15:38:27.430121+0000","last_became_active":"2024-09-17T15:38:27.345429+0000","last_became_peered":"2024-09-17T15:38:27.345429+0000","last_unstale":"2024-09-17T15:38:27.430121+0000","last_undegraded":"2024-09-17T15:38:27.430121+0000","last_fullsized":"2024-09-17T15:38:27.430121+0000","mapping_epoch":62,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:49:28.575667+0000","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4065,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":111,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338015+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:38:27.338015+0000","last_peered":"2024-09-17T15:38:27.338015+0000","last_clean":"2024-09-17T15:38:27.338015+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:38:27.338015+0000","last_undegraded":"2024-09-17T15:38:27.338015+0000","last_fullsized":"2024-09-17T15:38:27.338015+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":116,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337667+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:38:27.337667+0000","last_peered":"2024-09-17T15:38:27.337667+0000","last_clean":"2024-09-17T15:38:27.337667+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:38:27.337667+0000","last_undegraded":"2024-09-17T15:38:27.337667+0000","last_fullsized":"2024-09-17T15:38:27.337667+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29050,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338223+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:38:27.338223+0000","last_peered":"2024-09-17T15:38:27.338223+0000","last_clean":"2024-09-17T15:38:27.338223+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:38:27.338223+0000","last_undegraded":"2024-09-17T15:38:27.338223+0000","last_fullsized":"2024-09-17T15:38:27.338223+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":490,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48970,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":125710336,"data_stored":125630850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051750,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":410064,"kb_used_data":57660,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93961776,"statfs":{"total":96636764160,"available":96216858624,"internally_reserved":0,"allocated":59043840,"data_stored":58667620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724694,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527856,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93843984,"statfs":{"total":96636764160,"available":96096239616,"internally_reserved":0,"allocated":557056,"data_stored":180484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":539946667},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247977,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539472,"kb_used_data":66360,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832368,"statfs":{"total":96636764160,"available":96084344832,"internally_reserved":0,"allocated":67952640,"data_stored":67565229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":58257408,"data_stored":58239343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:38:54.363 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:38:54.645 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:54.646 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:38:54.657 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":493,"stamp":"2024-09-17T15:38:53.844975+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":490,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48971,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1477392,"kb_used_data":124564,"kb_used_omap":13,"kb_used_meta":1352754,"kb_avail":281638128,"statfs":{"total":289910292480,"available":288397443072,"internally_reserved":0,"allocated":127553536,"data_stored":126413333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1385221119},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":979,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001365"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":28918,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.343038+0000","last_change":"2024-09-17T15:38:06.362595+0000","last_active":"2024-09-17T15:38:27.343038+0000","last_peered":"2024-09-17T15:38:27.343038+0000","last_clean":"2024-09-17T15:38:27.343038+0000","last_became_active":"2024-09-17T15:38:06.284139+0000","last_became_peered":"2024-09-17T15:38:06.284139+0000","last_unstale":"2024-09-17T15:38:27.343038+0000","last_undegraded":"2024-09-17T15:38:27.343038+0000","last_fullsized":"2024-09-17T15:38:27.343038+0000","mapping_epoch":60,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:42:51.418256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4070,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21083,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417805+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:38:52.417805+0000","last_peered":"2024-09-17T15:38:52.417805+0000","last_clean":"2024-09-17T15:38:52.417805+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:38:52.417805+0000","last_undegraded":"2024-09-17T15:38:52.417805+0000","last_fullsized":"2024-09-17T15:38:52.417805+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21186,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337203+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:38:27.337203+0000","last_peered":"2024-09-17T15:38:27.337203+0000","last_clean":"2024-09-17T15:38:27.337203+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:38:27.337203+0000","last_undegraded":"2024-09-17T15:38:27.337203+0000","last_fullsized":"2024-09-17T15:38:27.337203+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21088,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417541+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:38:52.417541+0000","last_peered":"2024-09-17T15:38:52.417541+0000","last_clean":"2024-09-17T15:38:52.417541+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:38:52.417541+0000","last_undegraded":"2024-09-17T15:38:52.417541+0000","last_fullsized":"2024-09-17T15:38:52.417541+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20826,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342773+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:38:27.342773+0000","last_peered":"2024-09-17T15:38:27.342773+0000","last_clean":"2024-09-17T15:38:27.342773+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:38:27.342773+0000","last_undegraded":"2024-09-17T15:38:27.342773+0000","last_fullsized":"2024-09-17T15:38:27.342773+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21577,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337752+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:38:27.337752+0000","last_peered":"2024-09-17T15:38:27.337752+0000","last_clean":"2024-09-17T15:38:27.337752+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:38:27.337752+0000","last_undegraded":"2024-09-17T15:38:27.337752+0000","last_fullsized":"2024-09-17T15:38:27.337752+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21121,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337152+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:38:27.337152+0000","last_peered":"2024-09-17T15:38:27.337152+0000","last_clean":"2024-09-17T15:38:27.337152+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:38:27.337152+0000","last_undegraded":"2024-09-17T15:38:27.337152+0000","last_fullsized":"2024-09-17T15:38:27.337152+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21088,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342372+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:38:27.342372+0000","last_peered":"2024-09-17T15:38:27.342372+0000","last_clean":"2024-09-17T15:38:27.342372+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:38:27.342372+0000","last_undegraded":"2024-09-17T15:38:27.342372+0000","last_fullsized":"2024-09-17T15:38:27.342372+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337064+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:38:27.337064+0000","last_peered":"2024-09-17T15:38:27.337064+0000","last_clean":"2024-09-17T15:38:27.337064+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:38:27.337064+0000","last_undegraded":"2024-09-17T15:38:27.337064+0000","last_fullsized":"2024-09-17T15:38:27.337064+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29143,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337570+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:38:27.337570+0000","last_peered":"2024-09-17T15:38:27.337570+0000","last_clean":"2024-09-17T15:38:27.337570+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:38:27.337570+0000","last_undegraded":"2024-09-17T15:38:27.337570+0000","last_fullsized":"2024-09-17T15:38:27.337570+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417181+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:38:52.417181+0000","last_peered":"2024-09-17T15:38:52.417181+0000","last_clean":"2024-09-17T15:38:52.417181+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:38:52.417181+0000","last_undegraded":"2024-09-17T15:38:52.417181+0000","last_fullsized":"2024-09-17T15:38:52.417181+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":27762,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:49.445400+0000","last_change":"2024-09-17T15:38:25.720175+0000","last_active":"2024-09-17T15:38:49.445400+0000","last_peered":"2024-09-17T15:38:49.445400+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:38:49.445400+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":490,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3522,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-490}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":147,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342526+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:38:27.342526+0000","last_peered":"2024-09-17T15:38:27.342526+0000","last_clean":"2024-09-17T15:38:27.342526+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:38:27.342526+0000","last_undegraded":"2024-09-17T15:38:27.342526+0000","last_fullsized":"2024-09-17T15:38:27.342526+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28852,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417187+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:38:52.417187+0000","last_peered":"2024-09-17T15:38:52.417187+0000","last_clean":"2024-09-17T15:38:52.417187+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:38:52.417187+0000","last_undegraded":"2024-09-17T15:38:52.417187+0000","last_fullsized":"2024-09-17T15:38:52.417187+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":121,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337473+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:38:27.337473+0000","last_peered":"2024-09-17T15:38:27.337473+0000","last_clean":"2024-09-17T15:38:27.337473+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:38:27.337473+0000","last_undegraded":"2024-09-17T15:38:27.337473+0000","last_fullsized":"2024-09-17T15:38:27.337473+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29189,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417037+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:38:52.417037+0000","last_peered":"2024-09-17T15:38:52.417037+0000","last_clean":"2024-09-17T15:38:52.417037+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:38:52.417037+0000","last_undegraded":"2024-09-17T15:38:52.417037+0000","last_fullsized":"2024-09-17T15:38:52.417037+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21295,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337742+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:38:27.337742+0000","last_peered":"2024-09-17T15:38:27.337742+0000","last_clean":"2024-09-17T15:38:27.337742+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:38:27.337742+0000","last_undegraded":"2024-09-17T15:38:27.337742+0000","last_fullsized":"2024-09-17T15:38:27.337742+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":111,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338212+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:38:27.338212+0000","last_peered":"2024-09-17T15:38:27.338212+0000","last_clean":"2024-09-17T15:38:27.338212+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:38:27.338212+0000","last_undegraded":"2024-09-17T15:38:27.338212+0000","last_fullsized":"2024-09-17T15:38:27.338212+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28099,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338109+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:38:27.338109+0000","last_peered":"2024-09-17T15:38:27.338109+0000","last_clean":"2024-09-17T15:38:27.338109+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:38:27.338109+0000","last_undegraded":"2024-09-17T15:38:27.338109+0000","last_fullsized":"2024-09-17T15:38:27.338109+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417682+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:38:52.417682+0000","last_peered":"2024-09-17T15:38:52.417682+0000","last_clean":"2024-09-17T15:38:52.417682+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:38:52.417682+0000","last_undegraded":"2024-09-17T15:38:52.417682+0000","last_fullsized":"2024-09-17T15:38:52.417682+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21204,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.430121+0000","last_change":"2024-09-17T15:38:27.429539+0000","last_active":"2024-09-17T15:38:27.430121+0000","last_peered":"2024-09-17T15:38:27.430121+0000","last_clean":"2024-09-17T15:38:27.430121+0000","last_became_active":"2024-09-17T15:38:27.345429+0000","last_became_peered":"2024-09-17T15:38:27.345429+0000","last_unstale":"2024-09-17T15:38:27.430121+0000","last_undegraded":"2024-09-17T15:38:27.430121+0000","last_fullsized":"2024-09-17T15:38:27.430121+0000","mapping_epoch":62,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:49:28.575667+0000","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4065,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":111,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338015+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:38:27.338015+0000","last_peered":"2024-09-17T15:38:27.338015+0000","last_clean":"2024-09-17T15:38:27.338015+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:38:27.338015+0000","last_undegraded":"2024-09-17T15:38:27.338015+0000","last_fullsized":"2024-09-17T15:38:27.338015+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":116,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337667+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:38:27.337667+0000","last_peered":"2024-09-17T15:38:27.337667+0000","last_clean":"2024-09-17T15:38:27.337667+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:38:27.337667+0000","last_undegraded":"2024-09-17T15:38:27.337667+0000","last_fullsized":"2024-09-17T15:38:27.337667+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29050,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338223+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:38:27.338223+0000","last_peered":"2024-09-17T15:38:27.338223+0000","last_clean":"2024-09-17T15:38:27.338223+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:38:27.338223+0000","last_undegraded":"2024-09-17T15:38:27.338223+0000","last_fullsized":"2024-09-17T15:38:27.338223+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":490,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48970,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":125710336,"data_stored":125630850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051750,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":410064,"kb_used_data":57660,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93961776,"statfs":{"total":96636764160,"available":96216858624,"internally_reserved":0,"allocated":59043840,"data_stored":58667620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724694,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527856,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93843984,"statfs":{"total":96636764160,"available":96096239616,"internally_reserved":0,"allocated":557056,"data_stored":180484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":539946667},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247977,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539472,"kb_used_data":66360,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832368,"statfs":{"total":96636764160,"available":96084344832,"internally_reserved":0,"allocated":67952640,"data_stored":67565229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":58257408,"data_stored":58239343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:38:54.659 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-17T15:38:54.659 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'20542", 'reported_seq': 27762, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-17T15:38:49.445400+0000', 'last_change': '2024-09-17T15:38:25.720175+0000', 'last_active': '2024-09-17T15:38:49.445400+0000', 'last_peered': '2024-09-17T15:38:49.445400+0000', 'last_clean': '2024-09-17T15:26:20.173365+0000', 'last_became_active': '2024-09-17T15:26:22.740423+0000', 'last_became_peered': '2024-09-17T15:26:22.740423+0000', 'last_unstale': '2024-09-17T15:38:49.445400+0000', 'last_undegraded': '2024-09-17T15:26:22.271170+0000', 'last_fullsized': '2024-09-17T15:26:22.190446+0000', 'mapping_epoch': 40, 'log_start': "43'10500", 'ondisk_log_start': "43'10500", 'created': 14, 'last_epoch_clean': 37, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "34'3454", 'last_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T15:24:47.637664+0000', 'last_clean_scrub_stamp': '2024-09-17T15:26:10.852634+0000', 'objects_scrubbed': 3396, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1143, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 490, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17138, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3522, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -490}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T15:38:54.659 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T15:38:55.006 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:55.007 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 490/24 objects degraded (2041.667%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":872,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":65,"num_osds":3,"num_up_osds":3,"osd_up_since":1726586780,"num_in_osds":3,"osd_in_since":1726586773,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1512849408,"bytes_avail":288397443072,"bytes_total":289910292480,"degraded_objects":490,"degraded_total":24,"degraded_ratio":20.416666666666668,"recovering_objects_per_sec":122,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":979,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-17T15:24:23:004974+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T15:33:09.740201+0000","services":{}},"progress_events":{}} 2024-09-17T15:38:55.019 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-17T15:38:55.020 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:38:55.304 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:55.305 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:38:55.316 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":493,"stamp":"2024-09-17T15:38:53.844975+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":490,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48971,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1477392,"kb_used_data":124564,"kb_used_omap":13,"kb_used_meta":1352754,"kb_avail":281638128,"statfs":{"total":289910292480,"available":288397443072,"internally_reserved":0,"allocated":127553536,"data_stored":126413333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1385221119},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":979,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001365"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":28918,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.343038+0000","last_change":"2024-09-17T15:38:06.362595+0000","last_active":"2024-09-17T15:38:27.343038+0000","last_peered":"2024-09-17T15:38:27.343038+0000","last_clean":"2024-09-17T15:38:27.343038+0000","last_became_active":"2024-09-17T15:38:06.284139+0000","last_became_peered":"2024-09-17T15:38:06.284139+0000","last_unstale":"2024-09-17T15:38:27.343038+0000","last_undegraded":"2024-09-17T15:38:27.343038+0000","last_fullsized":"2024-09-17T15:38:27.343038+0000","mapping_epoch":60,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:42:51.418256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4070,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21083,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417805+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:38:52.417805+0000","last_peered":"2024-09-17T15:38:52.417805+0000","last_clean":"2024-09-17T15:38:52.417805+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:38:52.417805+0000","last_undegraded":"2024-09-17T15:38:52.417805+0000","last_fullsized":"2024-09-17T15:38:52.417805+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21186,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337203+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:38:27.337203+0000","last_peered":"2024-09-17T15:38:27.337203+0000","last_clean":"2024-09-17T15:38:27.337203+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:38:27.337203+0000","last_undegraded":"2024-09-17T15:38:27.337203+0000","last_fullsized":"2024-09-17T15:38:27.337203+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21088,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417541+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:38:52.417541+0000","last_peered":"2024-09-17T15:38:52.417541+0000","last_clean":"2024-09-17T15:38:52.417541+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:38:52.417541+0000","last_undegraded":"2024-09-17T15:38:52.417541+0000","last_fullsized":"2024-09-17T15:38:52.417541+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20826,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342773+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:38:27.342773+0000","last_peered":"2024-09-17T15:38:27.342773+0000","last_clean":"2024-09-17T15:38:27.342773+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:38:27.342773+0000","last_undegraded":"2024-09-17T15:38:27.342773+0000","last_fullsized":"2024-09-17T15:38:27.342773+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21577,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337752+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:38:27.337752+0000","last_peered":"2024-09-17T15:38:27.337752+0000","last_clean":"2024-09-17T15:38:27.337752+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:38:27.337752+0000","last_undegraded":"2024-09-17T15:38:27.337752+0000","last_fullsized":"2024-09-17T15:38:27.337752+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21121,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337152+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:38:27.337152+0000","last_peered":"2024-09-17T15:38:27.337152+0000","last_clean":"2024-09-17T15:38:27.337152+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:38:27.337152+0000","last_undegraded":"2024-09-17T15:38:27.337152+0000","last_fullsized":"2024-09-17T15:38:27.337152+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21088,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342372+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:38:27.342372+0000","last_peered":"2024-09-17T15:38:27.342372+0000","last_clean":"2024-09-17T15:38:27.342372+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:38:27.342372+0000","last_undegraded":"2024-09-17T15:38:27.342372+0000","last_fullsized":"2024-09-17T15:38:27.342372+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337064+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:38:27.337064+0000","last_peered":"2024-09-17T15:38:27.337064+0000","last_clean":"2024-09-17T15:38:27.337064+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:38:27.337064+0000","last_undegraded":"2024-09-17T15:38:27.337064+0000","last_fullsized":"2024-09-17T15:38:27.337064+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29143,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337570+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:38:27.337570+0000","last_peered":"2024-09-17T15:38:27.337570+0000","last_clean":"2024-09-17T15:38:27.337570+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:38:27.337570+0000","last_undegraded":"2024-09-17T15:38:27.337570+0000","last_fullsized":"2024-09-17T15:38:27.337570+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417181+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:38:52.417181+0000","last_peered":"2024-09-17T15:38:52.417181+0000","last_clean":"2024-09-17T15:38:52.417181+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:38:52.417181+0000","last_undegraded":"2024-09-17T15:38:52.417181+0000","last_fullsized":"2024-09-17T15:38:52.417181+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":27762,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-17T15:38:49.445400+0000","last_change":"2024-09-17T15:38:25.720175+0000","last_active":"2024-09-17T15:38:49.445400+0000","last_peered":"2024-09-17T15:38:49.445400+0000","last_clean":"2024-09-17T15:26:20.173365+0000","last_became_active":"2024-09-17T15:26:22.740423+0000","last_became_peered":"2024-09-17T15:26:22.740423+0000","last_unstale":"2024-09-17T15:38:49.445400+0000","last_undegraded":"2024-09-17T15:26:22.271170+0000","last_fullsized":"2024-09-17T15:26:22.190446+0000","mapping_epoch":40,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":490,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3522,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-490}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":147,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.342526+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:38:27.342526+0000","last_peered":"2024-09-17T15:38:27.342526+0000","last_clean":"2024-09-17T15:38:27.342526+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:38:27.342526+0000","last_undegraded":"2024-09-17T15:38:27.342526+0000","last_fullsized":"2024-09-17T15:38:27.342526+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28852,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417187+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:38:52.417187+0000","last_peered":"2024-09-17T15:38:52.417187+0000","last_clean":"2024-09-17T15:38:52.417187+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:38:52.417187+0000","last_undegraded":"2024-09-17T15:38:52.417187+0000","last_fullsized":"2024-09-17T15:38:52.417187+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":121,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337473+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:38:27.337473+0000","last_peered":"2024-09-17T15:38:27.337473+0000","last_clean":"2024-09-17T15:38:27.337473+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:38:27.337473+0000","last_undegraded":"2024-09-17T15:38:27.337473+0000","last_fullsized":"2024-09-17T15:38:27.337473+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29189,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417037+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:38:52.417037+0000","last_peered":"2024-09-17T15:38:52.417037+0000","last_clean":"2024-09-17T15:38:52.417037+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:38:52.417037+0000","last_undegraded":"2024-09-17T15:38:52.417037+0000","last_fullsized":"2024-09-17T15:38:52.417037+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21295,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337742+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:38:27.337742+0000","last_peered":"2024-09-17T15:38:27.337742+0000","last_clean":"2024-09-17T15:38:27.337742+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:38:27.337742+0000","last_undegraded":"2024-09-17T15:38:27.337742+0000","last_fullsized":"2024-09-17T15:38:27.337742+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":111,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338212+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:38:27.338212+0000","last_peered":"2024-09-17T15:38:27.338212+0000","last_clean":"2024-09-17T15:38:27.338212+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:38:27.338212+0000","last_undegraded":"2024-09-17T15:38:27.338212+0000","last_fullsized":"2024-09-17T15:38:27.338212+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28099,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338109+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:38:27.338109+0000","last_peered":"2024-09-17T15:38:27.338109+0000","last_clean":"2024-09-17T15:38:27.338109+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:38:27.338109+0000","last_undegraded":"2024-09-17T15:38:27.338109+0000","last_fullsized":"2024-09-17T15:38:27.338109+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417682+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:38:52.417682+0000","last_peered":"2024-09-17T15:38:52.417682+0000","last_clean":"2024-09-17T15:38:52.417682+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:38:52.417682+0000","last_undegraded":"2024-09-17T15:38:52.417682+0000","last_fullsized":"2024-09-17T15:38:52.417682+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21204,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.430121+0000","last_change":"2024-09-17T15:38:27.429539+0000","last_active":"2024-09-17T15:38:27.430121+0000","last_peered":"2024-09-17T15:38:27.430121+0000","last_clean":"2024-09-17T15:38:27.430121+0000","last_became_active":"2024-09-17T15:38:27.345429+0000","last_became_peered":"2024-09-17T15:38:27.345429+0000","last_unstale":"2024-09-17T15:38:27.430121+0000","last_undegraded":"2024-09-17T15:38:27.430121+0000","last_fullsized":"2024-09-17T15:38:27.430121+0000","mapping_epoch":62,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:49:28.575667+0000","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4065,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":111,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338015+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:38:27.338015+0000","last_peered":"2024-09-17T15:38:27.338015+0000","last_clean":"2024-09-17T15:38:27.338015+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:38:27.338015+0000","last_undegraded":"2024-09-17T15:38:27.338015+0000","last_fullsized":"2024-09-17T15:38:27.338015+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":116,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.337667+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:38:27.337667+0000","last_peered":"2024-09-17T15:38:27.337667+0000","last_clean":"2024-09-17T15:38:27.337667+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:38:27.337667+0000","last_undegraded":"2024-09-17T15:38:27.337667+0000","last_fullsized":"2024-09-17T15:38:27.337667+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29050,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T15:38:27.338223+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:38:27.338223+0000","last_peered":"2024-09-17T15:38:27.338223+0000","last_clean":"2024-09-17T15:38:27.338223+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:38:27.338223+0000","last_undegraded":"2024-09-17T15:38:27.338223+0000","last_fullsized":"2024-09-17T15:38:27.338223+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":490,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48970,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":125710336,"data_stored":125630850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051750,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":410064,"kb_used_data":57660,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93961776,"statfs":{"total":96636764160,"available":96216858624,"internally_reserved":0,"allocated":59043840,"data_stored":58667620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724694,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527856,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93843984,"statfs":{"total":96636764160,"available":96096239616,"internally_reserved":0,"allocated":557056,"data_stored":180484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":539946667},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247977,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539472,"kb_used_data":66360,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93832368,"statfs":{"total":96636764160,"available":96084344832,"internally_reserved":0,"allocated":67952640,"data_stored":67565229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67424256,"data_stored":67388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":58257408,"data_stored":58239343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:38:58.318 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:38:58.617 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:58.617 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:38:58.629 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":495,"stamp":"2024-09-17T15:38:57.845710+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49461,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1403296,"kb_used_data":58784,"kb_used_omap":13,"kb_used_meta":1344434,"kb_avail":281712224,"statfs":{"total":289910292480,"available":288473317376,"internally_reserved":0,"allocated":60194816,"data_stored":59039937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1376701439},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-490,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":490,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001453"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":28922,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.419015+0000","last_change":"2024-09-17T15:38:06.362595+0000","last_active":"2024-09-17T15:38:52.419015+0000","last_peered":"2024-09-17T15:38:52.419015+0000","last_clean":"2024-09-17T15:38:52.419015+0000","last_became_active":"2024-09-17T15:38:06.284139+0000","last_became_peered":"2024-09-17T15:38:06.284139+0000","last_unstale":"2024-09-17T15:38:52.419015+0000","last_undegraded":"2024-09-17T15:38:52.419015+0000","last_fullsized":"2024-09-17T15:38:52.419015+0000","mapping_epoch":60,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:42:51.418256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4070,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21083,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417805+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:38:52.417805+0000","last_peered":"2024-09-17T15:38:52.417805+0000","last_clean":"2024-09-17T15:38:52.417805+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:38:52.417805+0000","last_undegraded":"2024-09-17T15:38:52.417805+0000","last_fullsized":"2024-09-17T15:38:52.417805+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21190,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416151+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:38:52.416151+0000","last_peered":"2024-09-17T15:38:52.416151+0000","last_clean":"2024-09-17T15:38:52.416151+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:38:52.416151+0000","last_undegraded":"2024-09-17T15:38:52.416151+0000","last_fullsized":"2024-09-17T15:38:52.416151+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21088,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417541+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:38:52.417541+0000","last_peered":"2024-09-17T15:38:52.417541+0000","last_clean":"2024-09-17T15:38:52.417541+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:38:52.417541+0000","last_undegraded":"2024-09-17T15:38:52.417541+0000","last_fullsized":"2024-09-17T15:38:52.417541+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20830,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.418826+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:38:52.418826+0000","last_peered":"2024-09-17T15:38:52.418826+0000","last_clean":"2024-09-17T15:38:52.418826+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:38:52.418826+0000","last_undegraded":"2024-09-17T15:38:52.418826+0000","last_fullsized":"2024-09-17T15:38:52.418826+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21581,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416116+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:38:52.416116+0000","last_peered":"2024-09-17T15:38:52.416116+0000","last_clean":"2024-09-17T15:38:52.416116+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:38:52.416116+0000","last_undegraded":"2024-09-17T15:38:52.416116+0000","last_fullsized":"2024-09-17T15:38:52.416116+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21125,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416029+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:38:52.416029+0000","last_peered":"2024-09-17T15:38:52.416029+0000","last_clean":"2024-09-17T15:38:52.416029+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:38:52.416029+0000","last_undegraded":"2024-09-17T15:38:52.416029+0000","last_fullsized":"2024-09-17T15:38:52.416029+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21092,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.418808+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:38:52.418808+0000","last_peered":"2024-09-17T15:38:52.418808+0000","last_clean":"2024-09-17T15:38:52.418808+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:38:52.418808+0000","last_undegraded":"2024-09-17T15:38:52.418808+0000","last_fullsized":"2024-09-17T15:38:52.418808+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":116,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.415914+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:38:52.415914+0000","last_peered":"2024-09-17T15:38:52.415914+0000","last_clean":"2024-09-17T15:38:52.415914+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:38:52.415914+0000","last_undegraded":"2024-09-17T15:38:52.415914+0000","last_fullsized":"2024-09-17T15:38:52.415914+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416217+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:38:52.416217+0000","last_peered":"2024-09-17T15:38:52.416217+0000","last_clean":"2024-09-17T15:38:52.416217+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:38:52.416217+0000","last_undegraded":"2024-09-17T15:38:52.416217+0000","last_fullsized":"2024-09-17T15:38:52.416217+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417181+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:38:52.417181+0000","last_peered":"2024-09-17T15:38:52.417181+0000","last_clean":"2024-09-17T15:38:52.417181+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:38:52.417181+0000","last_undegraded":"2024-09-17T15:38:52.417181+0000","last_fullsized":"2024-09-17T15:38:52.417181+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":28753,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.490508+0000","last_change":"2024-09-17T15:38:52.490429+0000","last_active":"2024-09-17T15:38:52.490508+0000","last_peered":"2024-09-17T15:38:52.490508+0000","last_clean":"2024-09-17T15:38:52.490508+0000","last_became_active":"2024-09-17T15:38:52.419947+0000","last_became_peered":"2024-09-17T15:38:52.419947+0000","last_unstale":"2024-09-17T15:38:52.490508+0000","last_undegraded":"2024-09-17T15:38:52.490508+0000","last_fullsized":"2024-09-17T15:38:52.490508+0000","mapping_epoch":64,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:39:30.553247+0000","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4012,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.419246+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:38:52.419246+0000","last_peered":"2024-09-17T15:38:52.419246+0000","last_clean":"2024-09-17T15:38:52.419246+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:38:52.419246+0000","last_undegraded":"2024-09-17T15:38:52.419246+0000","last_fullsized":"2024-09-17T15:38:52.419246+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28852,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417187+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:38:52.417187+0000","last_peered":"2024-09-17T15:38:52.417187+0000","last_clean":"2024-09-17T15:38:52.417187+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:38:52.417187+0000","last_undegraded":"2024-09-17T15:38:52.417187+0000","last_fullsized":"2024-09-17T15:38:52.417187+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":126,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416392+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:38:52.416392+0000","last_peered":"2024-09-17T15:38:52.416392+0000","last_clean":"2024-09-17T15:38:52.416392+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:38:52.416392+0000","last_undegraded":"2024-09-17T15:38:52.416392+0000","last_fullsized":"2024-09-17T15:38:52.416392+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29189,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417037+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:38:52.417037+0000","last_peered":"2024-09-17T15:38:52.417037+0000","last_clean":"2024-09-17T15:38:52.417037+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:38:52.417037+0000","last_undegraded":"2024-09-17T15:38:52.417037+0000","last_fullsized":"2024-09-17T15:38:52.417037+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21299,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416490+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:38:52.416490+0000","last_peered":"2024-09-17T15:38:52.416490+0000","last_clean":"2024-09-17T15:38:52.416490+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:38:52.416490+0000","last_undegraded":"2024-09-17T15:38:52.416490+0000","last_fullsized":"2024-09-17T15:38:52.416490+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":116,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416386+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:38:52.416386+0000","last_peered":"2024-09-17T15:38:52.416386+0000","last_clean":"2024-09-17T15:38:52.416386+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:38:52.416386+0000","last_undegraded":"2024-09-17T15:38:52.416386+0000","last_fullsized":"2024-09-17T15:38:52.416386+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28103,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416283+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:38:52.416283+0000","last_peered":"2024-09-17T15:38:52.416283+0000","last_clean":"2024-09-17T15:38:52.416283+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:38:52.416283+0000","last_undegraded":"2024-09-17T15:38:52.416283+0000","last_fullsized":"2024-09-17T15:38:52.416283+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417682+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:38:52.417682+0000","last_peered":"2024-09-17T15:38:52.417682+0000","last_clean":"2024-09-17T15:38:52.417682+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:38:52.417682+0000","last_undegraded":"2024-09-17T15:38:52.417682+0000","last_fullsized":"2024-09-17T15:38:52.417682+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21208,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416197+0000","last_change":"2024-09-17T15:38:27.429539+0000","last_active":"2024-09-17T15:38:52.416197+0000","last_peered":"2024-09-17T15:38:52.416197+0000","last_clean":"2024-09-17T15:38:52.416197+0000","last_became_active":"2024-09-17T15:38:27.345429+0000","last_became_peered":"2024-09-17T15:38:27.345429+0000","last_unstale":"2024-09-17T15:38:52.416197+0000","last_undegraded":"2024-09-17T15:38:52.416197+0000","last_fullsized":"2024-09-17T15:38:52.416197+0000","mapping_epoch":62,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:49:28.575667+0000","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4065,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416251+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:38:52.416251+0000","last_peered":"2024-09-17T15:38:52.416251+0000","last_clean":"2024-09-17T15:38:52.416251+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:38:52.416251+0000","last_undegraded":"2024-09-17T15:38:52.416251+0000","last_fullsized":"2024-09-17T15:38:52.416251+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":120,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416114+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:38:52.416114+0000","last_peered":"2024-09-17T15:38:52.416114+0000","last_clean":"2024-09-17T15:38:52.416114+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:38:52.416114+0000","last_undegraded":"2024-09-17T15:38:52.416114+0000","last_fullsized":"2024-09-17T15:38:52.416114+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29054,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416154+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:38:52.416154+0000","last_peered":"2024-09-17T15:38:52.416154+0000","last_clean":"2024-09-17T15:38:52.416154+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:38:52.416154+0000","last_undegraded":"2024-09-17T15:38:52.416154+0000","last_fullsized":"2024-09-17T15:38:52.416154+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49460,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":58318848,"data_stored":58246588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051750,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":410064,"kb_used_data":57660,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93961776,"statfs":{"total":96636764160,"available":96216858624,"internally_reserved":0,"allocated":59043840,"data_stored":58667620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724695,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519560,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852280,"statfs":{"total":96636764160,"available":96104734720,"internally_reserved":0,"allocated":573440,"data_stored":185917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":531426987},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247978,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473672,"kb_used_data":564,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93898168,"statfs":{"total":96636764160,"available":96151724032,"internally_reserved":0,"allocated":577536,"data_stored":186400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":58257408,"data_stored":58239343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:38:58.631 INFO:tasks.lost_unfound.ceph_manager:clean! 2024-09-17T15:38:58.631 INFO:tasks.lost_unfound.ceph_manager:waiting for clean 2024-09-17T15:38:58.631 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:38:58.910 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:58.910 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:38:58.923 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":495,"stamp":"2024-09-17T15:38:57.845710+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49461,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1403296,"kb_used_data":58784,"kb_used_omap":13,"kb_used_meta":1344434,"kb_avail":281712224,"statfs":{"total":289910292480,"available":288473317376,"internally_reserved":0,"allocated":60194816,"data_stored":59039937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1376701439},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-490,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":490,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001453"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":28922,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.419015+0000","last_change":"2024-09-17T15:38:06.362595+0000","last_active":"2024-09-17T15:38:52.419015+0000","last_peered":"2024-09-17T15:38:52.419015+0000","last_clean":"2024-09-17T15:38:52.419015+0000","last_became_active":"2024-09-17T15:38:06.284139+0000","last_became_peered":"2024-09-17T15:38:06.284139+0000","last_unstale":"2024-09-17T15:38:52.419015+0000","last_undegraded":"2024-09-17T15:38:52.419015+0000","last_fullsized":"2024-09-17T15:38:52.419015+0000","mapping_epoch":60,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:42:51.418256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4070,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21083,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417805+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:38:52.417805+0000","last_peered":"2024-09-17T15:38:52.417805+0000","last_clean":"2024-09-17T15:38:52.417805+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:38:52.417805+0000","last_undegraded":"2024-09-17T15:38:52.417805+0000","last_fullsized":"2024-09-17T15:38:52.417805+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21190,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416151+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:38:52.416151+0000","last_peered":"2024-09-17T15:38:52.416151+0000","last_clean":"2024-09-17T15:38:52.416151+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:38:52.416151+0000","last_undegraded":"2024-09-17T15:38:52.416151+0000","last_fullsized":"2024-09-17T15:38:52.416151+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21088,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417541+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:38:52.417541+0000","last_peered":"2024-09-17T15:38:52.417541+0000","last_clean":"2024-09-17T15:38:52.417541+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:38:52.417541+0000","last_undegraded":"2024-09-17T15:38:52.417541+0000","last_fullsized":"2024-09-17T15:38:52.417541+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20830,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.418826+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:38:52.418826+0000","last_peered":"2024-09-17T15:38:52.418826+0000","last_clean":"2024-09-17T15:38:52.418826+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:38:52.418826+0000","last_undegraded":"2024-09-17T15:38:52.418826+0000","last_fullsized":"2024-09-17T15:38:52.418826+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21581,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416116+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:38:52.416116+0000","last_peered":"2024-09-17T15:38:52.416116+0000","last_clean":"2024-09-17T15:38:52.416116+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:38:52.416116+0000","last_undegraded":"2024-09-17T15:38:52.416116+0000","last_fullsized":"2024-09-17T15:38:52.416116+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21125,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416029+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:38:52.416029+0000","last_peered":"2024-09-17T15:38:52.416029+0000","last_clean":"2024-09-17T15:38:52.416029+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:38:52.416029+0000","last_undegraded":"2024-09-17T15:38:52.416029+0000","last_fullsized":"2024-09-17T15:38:52.416029+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21092,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.418808+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:38:52.418808+0000","last_peered":"2024-09-17T15:38:52.418808+0000","last_clean":"2024-09-17T15:38:52.418808+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:38:52.418808+0000","last_undegraded":"2024-09-17T15:38:52.418808+0000","last_fullsized":"2024-09-17T15:38:52.418808+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":116,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.415914+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:38:52.415914+0000","last_peered":"2024-09-17T15:38:52.415914+0000","last_clean":"2024-09-17T15:38:52.415914+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:38:52.415914+0000","last_undegraded":"2024-09-17T15:38:52.415914+0000","last_fullsized":"2024-09-17T15:38:52.415914+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416217+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:38:52.416217+0000","last_peered":"2024-09-17T15:38:52.416217+0000","last_clean":"2024-09-17T15:38:52.416217+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:38:52.416217+0000","last_undegraded":"2024-09-17T15:38:52.416217+0000","last_fullsized":"2024-09-17T15:38:52.416217+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417181+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:38:52.417181+0000","last_peered":"2024-09-17T15:38:52.417181+0000","last_clean":"2024-09-17T15:38:52.417181+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:38:52.417181+0000","last_undegraded":"2024-09-17T15:38:52.417181+0000","last_fullsized":"2024-09-17T15:38:52.417181+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":28753,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.490508+0000","last_change":"2024-09-17T15:38:52.490429+0000","last_active":"2024-09-17T15:38:52.490508+0000","last_peered":"2024-09-17T15:38:52.490508+0000","last_clean":"2024-09-17T15:38:52.490508+0000","last_became_active":"2024-09-17T15:38:52.419947+0000","last_became_peered":"2024-09-17T15:38:52.419947+0000","last_unstale":"2024-09-17T15:38:52.490508+0000","last_undegraded":"2024-09-17T15:38:52.490508+0000","last_fullsized":"2024-09-17T15:38:52.490508+0000","mapping_epoch":64,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:39:30.553247+0000","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4012,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.419246+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:38:52.419246+0000","last_peered":"2024-09-17T15:38:52.419246+0000","last_clean":"2024-09-17T15:38:52.419246+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:38:52.419246+0000","last_undegraded":"2024-09-17T15:38:52.419246+0000","last_fullsized":"2024-09-17T15:38:52.419246+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28852,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417187+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:38:52.417187+0000","last_peered":"2024-09-17T15:38:52.417187+0000","last_clean":"2024-09-17T15:38:52.417187+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:38:52.417187+0000","last_undegraded":"2024-09-17T15:38:52.417187+0000","last_fullsized":"2024-09-17T15:38:52.417187+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":126,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416392+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:38:52.416392+0000","last_peered":"2024-09-17T15:38:52.416392+0000","last_clean":"2024-09-17T15:38:52.416392+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:38:52.416392+0000","last_undegraded":"2024-09-17T15:38:52.416392+0000","last_fullsized":"2024-09-17T15:38:52.416392+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29189,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417037+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:38:52.417037+0000","last_peered":"2024-09-17T15:38:52.417037+0000","last_clean":"2024-09-17T15:38:52.417037+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:38:52.417037+0000","last_undegraded":"2024-09-17T15:38:52.417037+0000","last_fullsized":"2024-09-17T15:38:52.417037+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21299,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416490+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:38:52.416490+0000","last_peered":"2024-09-17T15:38:52.416490+0000","last_clean":"2024-09-17T15:38:52.416490+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:38:52.416490+0000","last_undegraded":"2024-09-17T15:38:52.416490+0000","last_fullsized":"2024-09-17T15:38:52.416490+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":116,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416386+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:38:52.416386+0000","last_peered":"2024-09-17T15:38:52.416386+0000","last_clean":"2024-09-17T15:38:52.416386+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:38:52.416386+0000","last_undegraded":"2024-09-17T15:38:52.416386+0000","last_fullsized":"2024-09-17T15:38:52.416386+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28103,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416283+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:38:52.416283+0000","last_peered":"2024-09-17T15:38:52.416283+0000","last_clean":"2024-09-17T15:38:52.416283+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:38:52.416283+0000","last_undegraded":"2024-09-17T15:38:52.416283+0000","last_fullsized":"2024-09-17T15:38:52.416283+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417682+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:38:52.417682+0000","last_peered":"2024-09-17T15:38:52.417682+0000","last_clean":"2024-09-17T15:38:52.417682+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:38:52.417682+0000","last_undegraded":"2024-09-17T15:38:52.417682+0000","last_fullsized":"2024-09-17T15:38:52.417682+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21208,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416197+0000","last_change":"2024-09-17T15:38:27.429539+0000","last_active":"2024-09-17T15:38:52.416197+0000","last_peered":"2024-09-17T15:38:52.416197+0000","last_clean":"2024-09-17T15:38:52.416197+0000","last_became_active":"2024-09-17T15:38:27.345429+0000","last_became_peered":"2024-09-17T15:38:27.345429+0000","last_unstale":"2024-09-17T15:38:52.416197+0000","last_undegraded":"2024-09-17T15:38:52.416197+0000","last_fullsized":"2024-09-17T15:38:52.416197+0000","mapping_epoch":62,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:49:28.575667+0000","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4065,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416251+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:38:52.416251+0000","last_peered":"2024-09-17T15:38:52.416251+0000","last_clean":"2024-09-17T15:38:52.416251+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:38:52.416251+0000","last_undegraded":"2024-09-17T15:38:52.416251+0000","last_fullsized":"2024-09-17T15:38:52.416251+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":120,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416114+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:38:52.416114+0000","last_peered":"2024-09-17T15:38:52.416114+0000","last_clean":"2024-09-17T15:38:52.416114+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:38:52.416114+0000","last_undegraded":"2024-09-17T15:38:52.416114+0000","last_fullsized":"2024-09-17T15:38:52.416114+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29054,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416154+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:38:52.416154+0000","last_peered":"2024-09-17T15:38:52.416154+0000","last_clean":"2024-09-17T15:38:52.416154+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:38:52.416154+0000","last_undegraded":"2024-09-17T15:38:52.416154+0000","last_fullsized":"2024-09-17T15:38:52.416154+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49460,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":58318848,"data_stored":58246588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051750,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":410064,"kb_used_data":57660,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93961776,"statfs":{"total":96636764160,"available":96216858624,"internally_reserved":0,"allocated":59043840,"data_stored":58667620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724695,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519560,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852280,"statfs":{"total":96636764160,"available":96104734720,"internally_reserved":0,"allocated":573440,"data_stored":185917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":531426987},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247978,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473672,"kb_used_data":564,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93898168,"statfs":{"total":96636764160,"available":96151724032,"internally_reserved":0,"allocated":577536,"data_stored":186400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":58257408,"data_stored":58239343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:38:58.924 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:38:59.205 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:59.206 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:38:59.218 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":495,"stamp":"2024-09-17T15:38:57.845710+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49461,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1403296,"kb_used_data":58784,"kb_used_omap":13,"kb_used_meta":1344434,"kb_avail":281712224,"statfs":{"total":289910292480,"available":288473317376,"internally_reserved":0,"allocated":60194816,"data_stored":59039937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1376701439},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-490,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":490,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001453"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":28922,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.419015+0000","last_change":"2024-09-17T15:38:06.362595+0000","last_active":"2024-09-17T15:38:52.419015+0000","last_peered":"2024-09-17T15:38:52.419015+0000","last_clean":"2024-09-17T15:38:52.419015+0000","last_became_active":"2024-09-17T15:38:06.284139+0000","last_became_peered":"2024-09-17T15:38:06.284139+0000","last_unstale":"2024-09-17T15:38:52.419015+0000","last_undegraded":"2024-09-17T15:38:52.419015+0000","last_fullsized":"2024-09-17T15:38:52.419015+0000","mapping_epoch":60,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:42:51.418256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4070,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21083,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417805+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:38:52.417805+0000","last_peered":"2024-09-17T15:38:52.417805+0000","last_clean":"2024-09-17T15:38:52.417805+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:38:52.417805+0000","last_undegraded":"2024-09-17T15:38:52.417805+0000","last_fullsized":"2024-09-17T15:38:52.417805+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21190,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416151+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:38:52.416151+0000","last_peered":"2024-09-17T15:38:52.416151+0000","last_clean":"2024-09-17T15:38:52.416151+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:38:52.416151+0000","last_undegraded":"2024-09-17T15:38:52.416151+0000","last_fullsized":"2024-09-17T15:38:52.416151+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21088,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417541+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:38:52.417541+0000","last_peered":"2024-09-17T15:38:52.417541+0000","last_clean":"2024-09-17T15:38:52.417541+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:38:52.417541+0000","last_undegraded":"2024-09-17T15:38:52.417541+0000","last_fullsized":"2024-09-17T15:38:52.417541+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20830,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.418826+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:38:52.418826+0000","last_peered":"2024-09-17T15:38:52.418826+0000","last_clean":"2024-09-17T15:38:52.418826+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:38:52.418826+0000","last_undegraded":"2024-09-17T15:38:52.418826+0000","last_fullsized":"2024-09-17T15:38:52.418826+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21581,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416116+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:38:52.416116+0000","last_peered":"2024-09-17T15:38:52.416116+0000","last_clean":"2024-09-17T15:38:52.416116+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:38:52.416116+0000","last_undegraded":"2024-09-17T15:38:52.416116+0000","last_fullsized":"2024-09-17T15:38:52.416116+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21125,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416029+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:38:52.416029+0000","last_peered":"2024-09-17T15:38:52.416029+0000","last_clean":"2024-09-17T15:38:52.416029+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:38:52.416029+0000","last_undegraded":"2024-09-17T15:38:52.416029+0000","last_fullsized":"2024-09-17T15:38:52.416029+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21092,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.418808+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:38:52.418808+0000","last_peered":"2024-09-17T15:38:52.418808+0000","last_clean":"2024-09-17T15:38:52.418808+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:38:52.418808+0000","last_undegraded":"2024-09-17T15:38:52.418808+0000","last_fullsized":"2024-09-17T15:38:52.418808+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":116,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.415914+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:38:52.415914+0000","last_peered":"2024-09-17T15:38:52.415914+0000","last_clean":"2024-09-17T15:38:52.415914+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:38:52.415914+0000","last_undegraded":"2024-09-17T15:38:52.415914+0000","last_fullsized":"2024-09-17T15:38:52.415914+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416217+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:38:52.416217+0000","last_peered":"2024-09-17T15:38:52.416217+0000","last_clean":"2024-09-17T15:38:52.416217+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:38:52.416217+0000","last_undegraded":"2024-09-17T15:38:52.416217+0000","last_fullsized":"2024-09-17T15:38:52.416217+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417181+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:38:52.417181+0000","last_peered":"2024-09-17T15:38:52.417181+0000","last_clean":"2024-09-17T15:38:52.417181+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:38:52.417181+0000","last_undegraded":"2024-09-17T15:38:52.417181+0000","last_fullsized":"2024-09-17T15:38:52.417181+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":28753,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.490508+0000","last_change":"2024-09-17T15:38:52.490429+0000","last_active":"2024-09-17T15:38:52.490508+0000","last_peered":"2024-09-17T15:38:52.490508+0000","last_clean":"2024-09-17T15:38:52.490508+0000","last_became_active":"2024-09-17T15:38:52.419947+0000","last_became_peered":"2024-09-17T15:38:52.419947+0000","last_unstale":"2024-09-17T15:38:52.490508+0000","last_undegraded":"2024-09-17T15:38:52.490508+0000","last_fullsized":"2024-09-17T15:38:52.490508+0000","mapping_epoch":64,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:39:30.553247+0000","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4012,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.419246+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:38:52.419246+0000","last_peered":"2024-09-17T15:38:52.419246+0000","last_clean":"2024-09-17T15:38:52.419246+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:38:52.419246+0000","last_undegraded":"2024-09-17T15:38:52.419246+0000","last_fullsized":"2024-09-17T15:38:52.419246+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28852,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417187+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:38:52.417187+0000","last_peered":"2024-09-17T15:38:52.417187+0000","last_clean":"2024-09-17T15:38:52.417187+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:38:52.417187+0000","last_undegraded":"2024-09-17T15:38:52.417187+0000","last_fullsized":"2024-09-17T15:38:52.417187+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":126,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416392+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:38:52.416392+0000","last_peered":"2024-09-17T15:38:52.416392+0000","last_clean":"2024-09-17T15:38:52.416392+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:38:52.416392+0000","last_undegraded":"2024-09-17T15:38:52.416392+0000","last_fullsized":"2024-09-17T15:38:52.416392+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29189,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417037+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:38:52.417037+0000","last_peered":"2024-09-17T15:38:52.417037+0000","last_clean":"2024-09-17T15:38:52.417037+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:38:52.417037+0000","last_undegraded":"2024-09-17T15:38:52.417037+0000","last_fullsized":"2024-09-17T15:38:52.417037+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21299,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416490+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:38:52.416490+0000","last_peered":"2024-09-17T15:38:52.416490+0000","last_clean":"2024-09-17T15:38:52.416490+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:38:52.416490+0000","last_undegraded":"2024-09-17T15:38:52.416490+0000","last_fullsized":"2024-09-17T15:38:52.416490+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":116,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416386+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:38:52.416386+0000","last_peered":"2024-09-17T15:38:52.416386+0000","last_clean":"2024-09-17T15:38:52.416386+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:38:52.416386+0000","last_undegraded":"2024-09-17T15:38:52.416386+0000","last_fullsized":"2024-09-17T15:38:52.416386+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28103,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416283+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:38:52.416283+0000","last_peered":"2024-09-17T15:38:52.416283+0000","last_clean":"2024-09-17T15:38:52.416283+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:38:52.416283+0000","last_undegraded":"2024-09-17T15:38:52.416283+0000","last_fullsized":"2024-09-17T15:38:52.416283+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417682+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:38:52.417682+0000","last_peered":"2024-09-17T15:38:52.417682+0000","last_clean":"2024-09-17T15:38:52.417682+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:38:52.417682+0000","last_undegraded":"2024-09-17T15:38:52.417682+0000","last_fullsized":"2024-09-17T15:38:52.417682+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21208,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416197+0000","last_change":"2024-09-17T15:38:27.429539+0000","last_active":"2024-09-17T15:38:52.416197+0000","last_peered":"2024-09-17T15:38:52.416197+0000","last_clean":"2024-09-17T15:38:52.416197+0000","last_became_active":"2024-09-17T15:38:27.345429+0000","last_became_peered":"2024-09-17T15:38:27.345429+0000","last_unstale":"2024-09-17T15:38:52.416197+0000","last_undegraded":"2024-09-17T15:38:52.416197+0000","last_fullsized":"2024-09-17T15:38:52.416197+0000","mapping_epoch":62,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:49:28.575667+0000","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4065,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416251+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:38:52.416251+0000","last_peered":"2024-09-17T15:38:52.416251+0000","last_clean":"2024-09-17T15:38:52.416251+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:38:52.416251+0000","last_undegraded":"2024-09-17T15:38:52.416251+0000","last_fullsized":"2024-09-17T15:38:52.416251+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":120,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416114+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:38:52.416114+0000","last_peered":"2024-09-17T15:38:52.416114+0000","last_clean":"2024-09-17T15:38:52.416114+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:38:52.416114+0000","last_undegraded":"2024-09-17T15:38:52.416114+0000","last_fullsized":"2024-09-17T15:38:52.416114+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29054,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416154+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:38:52.416154+0000","last_peered":"2024-09-17T15:38:52.416154+0000","last_clean":"2024-09-17T15:38:52.416154+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:38:52.416154+0000","last_undegraded":"2024-09-17T15:38:52.416154+0000","last_fullsized":"2024-09-17T15:38:52.416154+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49460,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":58318848,"data_stored":58246588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051750,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":410064,"kb_used_data":57660,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93961776,"statfs":{"total":96636764160,"available":96216858624,"internally_reserved":0,"allocated":59043840,"data_stored":58667620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724695,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519560,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852280,"statfs":{"total":96636764160,"available":96104734720,"internally_reserved":0,"allocated":573440,"data_stored":185917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":531426987},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247978,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473672,"kb_used_data":564,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93898168,"statfs":{"total":96636764160,"available":96151724032,"internally_reserved":0,"allocated":577536,"data_stored":186400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":58257408,"data_stored":58239343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:38:59.220 INFO:tasks.lost_unfound.ceph_manager:clean! 2024-09-17T15:38:59.220 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-17T15:38:59.231 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T15:38:59.231 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:38:59.518 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:59.519 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:38:59.531 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":495,"stamp":"2024-09-17T15:38:57.845710+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49461,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1403296,"kb_used_data":58784,"kb_used_omap":13,"kb_used_meta":1344434,"kb_avail":281712224,"statfs":{"total":289910292480,"available":288473317376,"internally_reserved":0,"allocated":60194816,"data_stored":59039937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1376701439},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-490,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":490,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001453"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":28922,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.419015+0000","last_change":"2024-09-17T15:38:06.362595+0000","last_active":"2024-09-17T15:38:52.419015+0000","last_peered":"2024-09-17T15:38:52.419015+0000","last_clean":"2024-09-17T15:38:52.419015+0000","last_became_active":"2024-09-17T15:38:06.284139+0000","last_became_peered":"2024-09-17T15:38:06.284139+0000","last_unstale":"2024-09-17T15:38:52.419015+0000","last_undegraded":"2024-09-17T15:38:52.419015+0000","last_fullsized":"2024-09-17T15:38:52.419015+0000","mapping_epoch":60,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:42:51.418256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4070,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21083,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417805+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:38:52.417805+0000","last_peered":"2024-09-17T15:38:52.417805+0000","last_clean":"2024-09-17T15:38:52.417805+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:38:52.417805+0000","last_undegraded":"2024-09-17T15:38:52.417805+0000","last_fullsized":"2024-09-17T15:38:52.417805+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21190,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416151+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:38:52.416151+0000","last_peered":"2024-09-17T15:38:52.416151+0000","last_clean":"2024-09-17T15:38:52.416151+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:38:52.416151+0000","last_undegraded":"2024-09-17T15:38:52.416151+0000","last_fullsized":"2024-09-17T15:38:52.416151+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21088,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417541+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:38:52.417541+0000","last_peered":"2024-09-17T15:38:52.417541+0000","last_clean":"2024-09-17T15:38:52.417541+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:38:52.417541+0000","last_undegraded":"2024-09-17T15:38:52.417541+0000","last_fullsized":"2024-09-17T15:38:52.417541+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20830,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.418826+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:38:52.418826+0000","last_peered":"2024-09-17T15:38:52.418826+0000","last_clean":"2024-09-17T15:38:52.418826+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:38:52.418826+0000","last_undegraded":"2024-09-17T15:38:52.418826+0000","last_fullsized":"2024-09-17T15:38:52.418826+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21581,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416116+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:38:52.416116+0000","last_peered":"2024-09-17T15:38:52.416116+0000","last_clean":"2024-09-17T15:38:52.416116+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:38:52.416116+0000","last_undegraded":"2024-09-17T15:38:52.416116+0000","last_fullsized":"2024-09-17T15:38:52.416116+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21125,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416029+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:38:52.416029+0000","last_peered":"2024-09-17T15:38:52.416029+0000","last_clean":"2024-09-17T15:38:52.416029+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:38:52.416029+0000","last_undegraded":"2024-09-17T15:38:52.416029+0000","last_fullsized":"2024-09-17T15:38:52.416029+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21092,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.418808+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:38:52.418808+0000","last_peered":"2024-09-17T15:38:52.418808+0000","last_clean":"2024-09-17T15:38:52.418808+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:38:52.418808+0000","last_undegraded":"2024-09-17T15:38:52.418808+0000","last_fullsized":"2024-09-17T15:38:52.418808+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":116,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.415914+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:38:52.415914+0000","last_peered":"2024-09-17T15:38:52.415914+0000","last_clean":"2024-09-17T15:38:52.415914+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:38:52.415914+0000","last_undegraded":"2024-09-17T15:38:52.415914+0000","last_fullsized":"2024-09-17T15:38:52.415914+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416217+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:38:52.416217+0000","last_peered":"2024-09-17T15:38:52.416217+0000","last_clean":"2024-09-17T15:38:52.416217+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:38:52.416217+0000","last_undegraded":"2024-09-17T15:38:52.416217+0000","last_fullsized":"2024-09-17T15:38:52.416217+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417181+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:38:52.417181+0000","last_peered":"2024-09-17T15:38:52.417181+0000","last_clean":"2024-09-17T15:38:52.417181+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:38:52.417181+0000","last_undegraded":"2024-09-17T15:38:52.417181+0000","last_fullsized":"2024-09-17T15:38:52.417181+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":28753,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.490508+0000","last_change":"2024-09-17T15:38:52.490429+0000","last_active":"2024-09-17T15:38:52.490508+0000","last_peered":"2024-09-17T15:38:52.490508+0000","last_clean":"2024-09-17T15:38:52.490508+0000","last_became_active":"2024-09-17T15:38:52.419947+0000","last_became_peered":"2024-09-17T15:38:52.419947+0000","last_unstale":"2024-09-17T15:38:52.490508+0000","last_undegraded":"2024-09-17T15:38:52.490508+0000","last_fullsized":"2024-09-17T15:38:52.490508+0000","mapping_epoch":64,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:39:30.553247+0000","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4012,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.419246+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:38:52.419246+0000","last_peered":"2024-09-17T15:38:52.419246+0000","last_clean":"2024-09-17T15:38:52.419246+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:38:52.419246+0000","last_undegraded":"2024-09-17T15:38:52.419246+0000","last_fullsized":"2024-09-17T15:38:52.419246+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28852,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417187+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:38:52.417187+0000","last_peered":"2024-09-17T15:38:52.417187+0000","last_clean":"2024-09-17T15:38:52.417187+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:38:52.417187+0000","last_undegraded":"2024-09-17T15:38:52.417187+0000","last_fullsized":"2024-09-17T15:38:52.417187+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":126,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416392+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:38:52.416392+0000","last_peered":"2024-09-17T15:38:52.416392+0000","last_clean":"2024-09-17T15:38:52.416392+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:38:52.416392+0000","last_undegraded":"2024-09-17T15:38:52.416392+0000","last_fullsized":"2024-09-17T15:38:52.416392+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29189,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417037+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:38:52.417037+0000","last_peered":"2024-09-17T15:38:52.417037+0000","last_clean":"2024-09-17T15:38:52.417037+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:38:52.417037+0000","last_undegraded":"2024-09-17T15:38:52.417037+0000","last_fullsized":"2024-09-17T15:38:52.417037+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21299,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416490+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:38:52.416490+0000","last_peered":"2024-09-17T15:38:52.416490+0000","last_clean":"2024-09-17T15:38:52.416490+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:38:52.416490+0000","last_undegraded":"2024-09-17T15:38:52.416490+0000","last_fullsized":"2024-09-17T15:38:52.416490+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":116,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416386+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:38:52.416386+0000","last_peered":"2024-09-17T15:38:52.416386+0000","last_clean":"2024-09-17T15:38:52.416386+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:38:52.416386+0000","last_undegraded":"2024-09-17T15:38:52.416386+0000","last_fullsized":"2024-09-17T15:38:52.416386+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28103,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416283+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:38:52.416283+0000","last_peered":"2024-09-17T15:38:52.416283+0000","last_clean":"2024-09-17T15:38:52.416283+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:38:52.416283+0000","last_undegraded":"2024-09-17T15:38:52.416283+0000","last_fullsized":"2024-09-17T15:38:52.416283+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417682+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:38:52.417682+0000","last_peered":"2024-09-17T15:38:52.417682+0000","last_clean":"2024-09-17T15:38:52.417682+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:38:52.417682+0000","last_undegraded":"2024-09-17T15:38:52.417682+0000","last_fullsized":"2024-09-17T15:38:52.417682+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21208,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416197+0000","last_change":"2024-09-17T15:38:27.429539+0000","last_active":"2024-09-17T15:38:52.416197+0000","last_peered":"2024-09-17T15:38:52.416197+0000","last_clean":"2024-09-17T15:38:52.416197+0000","last_became_active":"2024-09-17T15:38:27.345429+0000","last_became_peered":"2024-09-17T15:38:27.345429+0000","last_unstale":"2024-09-17T15:38:52.416197+0000","last_undegraded":"2024-09-17T15:38:52.416197+0000","last_fullsized":"2024-09-17T15:38:52.416197+0000","mapping_epoch":62,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:49:28.575667+0000","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4065,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416251+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:38:52.416251+0000","last_peered":"2024-09-17T15:38:52.416251+0000","last_clean":"2024-09-17T15:38:52.416251+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:38:52.416251+0000","last_undegraded":"2024-09-17T15:38:52.416251+0000","last_fullsized":"2024-09-17T15:38:52.416251+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":120,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416114+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:38:52.416114+0000","last_peered":"2024-09-17T15:38:52.416114+0000","last_clean":"2024-09-17T15:38:52.416114+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:38:52.416114+0000","last_undegraded":"2024-09-17T15:38:52.416114+0000","last_fullsized":"2024-09-17T15:38:52.416114+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29054,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416154+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:38:52.416154+0000","last_peered":"2024-09-17T15:38:52.416154+0000","last_clean":"2024-09-17T15:38:52.416154+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:38:52.416154+0000","last_undegraded":"2024-09-17T15:38:52.416154+0000","last_fullsized":"2024-09-17T15:38:52.416154+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49460,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":58318848,"data_stored":58246588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051750,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":410064,"kb_used_data":57660,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93961776,"statfs":{"total":96636764160,"available":96216858624,"internally_reserved":0,"allocated":59043840,"data_stored":58667620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724695,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519560,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852280,"statfs":{"total":96636764160,"available":96104734720,"internally_reserved":0,"allocated":573440,"data_stored":185917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":531426987},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247978,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473672,"kb_used_data":564,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93898168,"statfs":{"total":96636764160,"available":96151724032,"internally_reserved":0,"allocated":577536,"data_stored":186400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":58257408,"data_stored":58239343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:38:59.532 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:38:59.816 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:38:59.816 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:38:59.829 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":495,"stamp":"2024-09-17T15:38:57.845710+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49461,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1403296,"kb_used_data":58784,"kb_used_omap":13,"kb_used_meta":1344434,"kb_avail":281712224,"statfs":{"total":289910292480,"available":288473317376,"internally_reserved":0,"allocated":60194816,"data_stored":59039937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1376701439},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-490,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":490,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001453"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":28922,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.419015+0000","last_change":"2024-09-17T15:38:06.362595+0000","last_active":"2024-09-17T15:38:52.419015+0000","last_peered":"2024-09-17T15:38:52.419015+0000","last_clean":"2024-09-17T15:38:52.419015+0000","last_became_active":"2024-09-17T15:38:06.284139+0000","last_became_peered":"2024-09-17T15:38:06.284139+0000","last_unstale":"2024-09-17T15:38:52.419015+0000","last_undegraded":"2024-09-17T15:38:52.419015+0000","last_fullsized":"2024-09-17T15:38:52.419015+0000","mapping_epoch":60,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:42:51.418256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4070,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21083,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417805+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:38:52.417805+0000","last_peered":"2024-09-17T15:38:52.417805+0000","last_clean":"2024-09-17T15:38:52.417805+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:38:52.417805+0000","last_undegraded":"2024-09-17T15:38:52.417805+0000","last_fullsized":"2024-09-17T15:38:52.417805+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21190,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416151+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:38:52.416151+0000","last_peered":"2024-09-17T15:38:52.416151+0000","last_clean":"2024-09-17T15:38:52.416151+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:38:52.416151+0000","last_undegraded":"2024-09-17T15:38:52.416151+0000","last_fullsized":"2024-09-17T15:38:52.416151+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21088,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417541+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:38:52.417541+0000","last_peered":"2024-09-17T15:38:52.417541+0000","last_clean":"2024-09-17T15:38:52.417541+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:38:52.417541+0000","last_undegraded":"2024-09-17T15:38:52.417541+0000","last_fullsized":"2024-09-17T15:38:52.417541+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20830,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.418826+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:38:52.418826+0000","last_peered":"2024-09-17T15:38:52.418826+0000","last_clean":"2024-09-17T15:38:52.418826+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:38:52.418826+0000","last_undegraded":"2024-09-17T15:38:52.418826+0000","last_fullsized":"2024-09-17T15:38:52.418826+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21581,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416116+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:38:52.416116+0000","last_peered":"2024-09-17T15:38:52.416116+0000","last_clean":"2024-09-17T15:38:52.416116+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:38:52.416116+0000","last_undegraded":"2024-09-17T15:38:52.416116+0000","last_fullsized":"2024-09-17T15:38:52.416116+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21125,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416029+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:38:52.416029+0000","last_peered":"2024-09-17T15:38:52.416029+0000","last_clean":"2024-09-17T15:38:52.416029+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:38:52.416029+0000","last_undegraded":"2024-09-17T15:38:52.416029+0000","last_fullsized":"2024-09-17T15:38:52.416029+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21092,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.418808+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:38:52.418808+0000","last_peered":"2024-09-17T15:38:52.418808+0000","last_clean":"2024-09-17T15:38:52.418808+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:38:52.418808+0000","last_undegraded":"2024-09-17T15:38:52.418808+0000","last_fullsized":"2024-09-17T15:38:52.418808+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":116,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.415914+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:38:52.415914+0000","last_peered":"2024-09-17T15:38:52.415914+0000","last_clean":"2024-09-17T15:38:52.415914+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:38:52.415914+0000","last_undegraded":"2024-09-17T15:38:52.415914+0000","last_fullsized":"2024-09-17T15:38:52.415914+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416217+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:38:52.416217+0000","last_peered":"2024-09-17T15:38:52.416217+0000","last_clean":"2024-09-17T15:38:52.416217+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:38:52.416217+0000","last_undegraded":"2024-09-17T15:38:52.416217+0000","last_fullsized":"2024-09-17T15:38:52.416217+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417181+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:38:52.417181+0000","last_peered":"2024-09-17T15:38:52.417181+0000","last_clean":"2024-09-17T15:38:52.417181+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:38:52.417181+0000","last_undegraded":"2024-09-17T15:38:52.417181+0000","last_fullsized":"2024-09-17T15:38:52.417181+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":28753,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.490508+0000","last_change":"2024-09-17T15:38:52.490429+0000","last_active":"2024-09-17T15:38:52.490508+0000","last_peered":"2024-09-17T15:38:52.490508+0000","last_clean":"2024-09-17T15:38:52.490508+0000","last_became_active":"2024-09-17T15:38:52.419947+0000","last_became_peered":"2024-09-17T15:38:52.419947+0000","last_unstale":"2024-09-17T15:38:52.490508+0000","last_undegraded":"2024-09-17T15:38:52.490508+0000","last_fullsized":"2024-09-17T15:38:52.490508+0000","mapping_epoch":64,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:39:30.553247+0000","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4012,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.419246+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:38:52.419246+0000","last_peered":"2024-09-17T15:38:52.419246+0000","last_clean":"2024-09-17T15:38:52.419246+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:38:52.419246+0000","last_undegraded":"2024-09-17T15:38:52.419246+0000","last_fullsized":"2024-09-17T15:38:52.419246+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28852,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417187+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:38:52.417187+0000","last_peered":"2024-09-17T15:38:52.417187+0000","last_clean":"2024-09-17T15:38:52.417187+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:38:52.417187+0000","last_undegraded":"2024-09-17T15:38:52.417187+0000","last_fullsized":"2024-09-17T15:38:52.417187+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":126,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416392+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:38:52.416392+0000","last_peered":"2024-09-17T15:38:52.416392+0000","last_clean":"2024-09-17T15:38:52.416392+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:38:52.416392+0000","last_undegraded":"2024-09-17T15:38:52.416392+0000","last_fullsized":"2024-09-17T15:38:52.416392+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29189,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417037+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:38:52.417037+0000","last_peered":"2024-09-17T15:38:52.417037+0000","last_clean":"2024-09-17T15:38:52.417037+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:38:52.417037+0000","last_undegraded":"2024-09-17T15:38:52.417037+0000","last_fullsized":"2024-09-17T15:38:52.417037+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21299,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416490+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:38:52.416490+0000","last_peered":"2024-09-17T15:38:52.416490+0000","last_clean":"2024-09-17T15:38:52.416490+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:38:52.416490+0000","last_undegraded":"2024-09-17T15:38:52.416490+0000","last_fullsized":"2024-09-17T15:38:52.416490+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":116,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416386+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:38:52.416386+0000","last_peered":"2024-09-17T15:38:52.416386+0000","last_clean":"2024-09-17T15:38:52.416386+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:38:52.416386+0000","last_undegraded":"2024-09-17T15:38:52.416386+0000","last_fullsized":"2024-09-17T15:38:52.416386+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28103,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416283+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:38:52.416283+0000","last_peered":"2024-09-17T15:38:52.416283+0000","last_clean":"2024-09-17T15:38:52.416283+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:38:52.416283+0000","last_undegraded":"2024-09-17T15:38:52.416283+0000","last_fullsized":"2024-09-17T15:38:52.416283+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417682+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:38:52.417682+0000","last_peered":"2024-09-17T15:38:52.417682+0000","last_clean":"2024-09-17T15:38:52.417682+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:38:52.417682+0000","last_undegraded":"2024-09-17T15:38:52.417682+0000","last_fullsized":"2024-09-17T15:38:52.417682+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21208,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416197+0000","last_change":"2024-09-17T15:38:27.429539+0000","last_active":"2024-09-17T15:38:52.416197+0000","last_peered":"2024-09-17T15:38:52.416197+0000","last_clean":"2024-09-17T15:38:52.416197+0000","last_became_active":"2024-09-17T15:38:27.345429+0000","last_became_peered":"2024-09-17T15:38:27.345429+0000","last_unstale":"2024-09-17T15:38:52.416197+0000","last_undegraded":"2024-09-17T15:38:52.416197+0000","last_fullsized":"2024-09-17T15:38:52.416197+0000","mapping_epoch":62,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:49:28.575667+0000","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4065,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416251+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:38:52.416251+0000","last_peered":"2024-09-17T15:38:52.416251+0000","last_clean":"2024-09-17T15:38:52.416251+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:38:52.416251+0000","last_undegraded":"2024-09-17T15:38:52.416251+0000","last_fullsized":"2024-09-17T15:38:52.416251+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":120,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416114+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:38:52.416114+0000","last_peered":"2024-09-17T15:38:52.416114+0000","last_clean":"2024-09-17T15:38:52.416114+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:38:52.416114+0000","last_undegraded":"2024-09-17T15:38:52.416114+0000","last_fullsized":"2024-09-17T15:38:52.416114+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29054,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416154+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:38:52.416154+0000","last_peered":"2024-09-17T15:38:52.416154+0000","last_clean":"2024-09-17T15:38:52.416154+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:38:52.416154+0000","last_undegraded":"2024-09-17T15:38:52.416154+0000","last_fullsized":"2024-09-17T15:38:52.416154+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49460,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":58318848,"data_stored":58246588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051750,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":410064,"kb_used_data":57660,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93961776,"statfs":{"total":96636764160,"available":96216858624,"internally_reserved":0,"allocated":59043840,"data_stored":58667620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724695,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519560,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852280,"statfs":{"total":96636764160,"available":96104734720,"internally_reserved":0,"allocated":573440,"data_stored":185917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":531426987},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247978,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473672,"kb_used_data":564,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93898168,"statfs":{"total":96636764160,"available":96151724032,"internally_reserved":0,"allocated":577536,"data_stored":186400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":58257408,"data_stored":58239343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:38:59.830 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T15:38:59.831 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:39:00.115 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:39:00.115 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:39:00.127 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":496,"stamp":"2024-09-17T15:38:59.846139+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49461,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1398492,"kb_used_data":53984,"kb_used_omap":13,"kb_used_meta":1344434,"kb_avail":281717028,"statfs":{"total":289910292480,"available":288478236672,"internally_reserved":0,"allocated":55279616,"data_stored":54124737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1376701439},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-490,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":490,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.438984"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":28922,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.419015+0000","last_change":"2024-09-17T15:38:06.362595+0000","last_active":"2024-09-17T15:38:52.419015+0000","last_peered":"2024-09-17T15:38:52.419015+0000","last_clean":"2024-09-17T15:38:52.419015+0000","last_became_active":"2024-09-17T15:38:06.284139+0000","last_became_peered":"2024-09-17T15:38:06.284139+0000","last_unstale":"2024-09-17T15:38:52.419015+0000","last_undegraded":"2024-09-17T15:38:52.419015+0000","last_fullsized":"2024-09-17T15:38:52.419015+0000","mapping_epoch":60,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:42:51.418256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4070,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21083,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417805+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:38:52.417805+0000","last_peered":"2024-09-17T15:38:52.417805+0000","last_clean":"2024-09-17T15:38:52.417805+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:38:52.417805+0000","last_undegraded":"2024-09-17T15:38:52.417805+0000","last_fullsized":"2024-09-17T15:38:52.417805+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21190,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416151+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:38:52.416151+0000","last_peered":"2024-09-17T15:38:52.416151+0000","last_clean":"2024-09-17T15:38:52.416151+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:38:52.416151+0000","last_undegraded":"2024-09-17T15:38:52.416151+0000","last_fullsized":"2024-09-17T15:38:52.416151+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21088,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417541+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:38:52.417541+0000","last_peered":"2024-09-17T15:38:52.417541+0000","last_clean":"2024-09-17T15:38:52.417541+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:38:52.417541+0000","last_undegraded":"2024-09-17T15:38:52.417541+0000","last_fullsized":"2024-09-17T15:38:52.417541+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20830,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.418826+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:38:52.418826+0000","last_peered":"2024-09-17T15:38:52.418826+0000","last_clean":"2024-09-17T15:38:52.418826+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:38:52.418826+0000","last_undegraded":"2024-09-17T15:38:52.418826+0000","last_fullsized":"2024-09-17T15:38:52.418826+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21581,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416116+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:38:52.416116+0000","last_peered":"2024-09-17T15:38:52.416116+0000","last_clean":"2024-09-17T15:38:52.416116+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:38:52.416116+0000","last_undegraded":"2024-09-17T15:38:52.416116+0000","last_fullsized":"2024-09-17T15:38:52.416116+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21125,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416029+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:38:52.416029+0000","last_peered":"2024-09-17T15:38:52.416029+0000","last_clean":"2024-09-17T15:38:52.416029+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:38:52.416029+0000","last_undegraded":"2024-09-17T15:38:52.416029+0000","last_fullsized":"2024-09-17T15:38:52.416029+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21092,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.418808+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:38:52.418808+0000","last_peered":"2024-09-17T15:38:52.418808+0000","last_clean":"2024-09-17T15:38:52.418808+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:38:52.418808+0000","last_undegraded":"2024-09-17T15:38:52.418808+0000","last_fullsized":"2024-09-17T15:38:52.418808+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":116,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.415914+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:38:52.415914+0000","last_peered":"2024-09-17T15:38:52.415914+0000","last_clean":"2024-09-17T15:38:52.415914+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:38:52.415914+0000","last_undegraded":"2024-09-17T15:38:52.415914+0000","last_fullsized":"2024-09-17T15:38:52.415914+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416217+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:38:52.416217+0000","last_peered":"2024-09-17T15:38:52.416217+0000","last_clean":"2024-09-17T15:38:52.416217+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:38:52.416217+0000","last_undegraded":"2024-09-17T15:38:52.416217+0000","last_fullsized":"2024-09-17T15:38:52.416217+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417181+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:38:52.417181+0000","last_peered":"2024-09-17T15:38:52.417181+0000","last_clean":"2024-09-17T15:38:52.417181+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:38:52.417181+0000","last_undegraded":"2024-09-17T15:38:52.417181+0000","last_fullsized":"2024-09-17T15:38:52.417181+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":28753,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.490508+0000","last_change":"2024-09-17T15:38:52.490429+0000","last_active":"2024-09-17T15:38:52.490508+0000","last_peered":"2024-09-17T15:38:52.490508+0000","last_clean":"2024-09-17T15:38:52.490508+0000","last_became_active":"2024-09-17T15:38:52.419947+0000","last_became_peered":"2024-09-17T15:38:52.419947+0000","last_unstale":"2024-09-17T15:38:52.490508+0000","last_undegraded":"2024-09-17T15:38:52.490508+0000","last_fullsized":"2024-09-17T15:38:52.490508+0000","mapping_epoch":64,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:39:30.553247+0000","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4012,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.419246+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:38:52.419246+0000","last_peered":"2024-09-17T15:38:52.419246+0000","last_clean":"2024-09-17T15:38:52.419246+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:38:52.419246+0000","last_undegraded":"2024-09-17T15:38:52.419246+0000","last_fullsized":"2024-09-17T15:38:52.419246+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28852,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417187+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:38:52.417187+0000","last_peered":"2024-09-17T15:38:52.417187+0000","last_clean":"2024-09-17T15:38:52.417187+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:38:52.417187+0000","last_undegraded":"2024-09-17T15:38:52.417187+0000","last_fullsized":"2024-09-17T15:38:52.417187+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":126,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416392+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:38:52.416392+0000","last_peered":"2024-09-17T15:38:52.416392+0000","last_clean":"2024-09-17T15:38:52.416392+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:38:52.416392+0000","last_undegraded":"2024-09-17T15:38:52.416392+0000","last_fullsized":"2024-09-17T15:38:52.416392+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29189,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417037+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:38:52.417037+0000","last_peered":"2024-09-17T15:38:52.417037+0000","last_clean":"2024-09-17T15:38:52.417037+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:38:52.417037+0000","last_undegraded":"2024-09-17T15:38:52.417037+0000","last_fullsized":"2024-09-17T15:38:52.417037+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21299,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416490+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:38:52.416490+0000","last_peered":"2024-09-17T15:38:52.416490+0000","last_clean":"2024-09-17T15:38:52.416490+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:38:52.416490+0000","last_undegraded":"2024-09-17T15:38:52.416490+0000","last_fullsized":"2024-09-17T15:38:52.416490+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":116,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416386+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:38:52.416386+0000","last_peered":"2024-09-17T15:38:52.416386+0000","last_clean":"2024-09-17T15:38:52.416386+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:38:52.416386+0000","last_undegraded":"2024-09-17T15:38:52.416386+0000","last_fullsized":"2024-09-17T15:38:52.416386+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28103,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416283+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:38:52.416283+0000","last_peered":"2024-09-17T15:38:52.416283+0000","last_clean":"2024-09-17T15:38:52.416283+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:38:52.416283+0000","last_undegraded":"2024-09-17T15:38:52.416283+0000","last_fullsized":"2024-09-17T15:38:52.416283+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417682+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:38:52.417682+0000","last_peered":"2024-09-17T15:38:52.417682+0000","last_clean":"2024-09-17T15:38:52.417682+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:38:52.417682+0000","last_undegraded":"2024-09-17T15:38:52.417682+0000","last_fullsized":"2024-09-17T15:38:52.417682+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21208,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416197+0000","last_change":"2024-09-17T15:38:27.429539+0000","last_active":"2024-09-17T15:38:52.416197+0000","last_peered":"2024-09-17T15:38:52.416197+0000","last_clean":"2024-09-17T15:38:52.416197+0000","last_became_active":"2024-09-17T15:38:27.345429+0000","last_became_peered":"2024-09-17T15:38:27.345429+0000","last_unstale":"2024-09-17T15:38:52.416197+0000","last_undegraded":"2024-09-17T15:38:52.416197+0000","last_fullsized":"2024-09-17T15:38:52.416197+0000","mapping_epoch":62,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:49:28.575667+0000","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4065,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416251+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:38:52.416251+0000","last_peered":"2024-09-17T15:38:52.416251+0000","last_clean":"2024-09-17T15:38:52.416251+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:38:52.416251+0000","last_undegraded":"2024-09-17T15:38:52.416251+0000","last_fullsized":"2024-09-17T15:38:52.416251+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":120,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416114+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:38:52.416114+0000","last_peered":"2024-09-17T15:38:52.416114+0000","last_clean":"2024-09-17T15:38:52.416114+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:38:52.416114+0000","last_undegraded":"2024-09-17T15:38:52.416114+0000","last_fullsized":"2024-09-17T15:38:52.416114+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29054,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416154+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:38:52.416154+0000","last_peered":"2024-09-17T15:38:52.416154+0000","last_clean":"2024-09-17T15:38:52.416154+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:38:52.416154+0000","last_undegraded":"2024-09-17T15:38:52.416154+0000","last_fullsized":"2024-09-17T15:38:52.416154+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49460,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53403648,"data_stored":53331388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051751,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":405260,"kb_used_data":52860,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93966580,"statfs":{"total":96636764160,"available":96221777920,"internally_reserved":0,"allocated":54128640,"data_stored":53752420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724696,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519560,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852280,"statfs":{"total":96636764160,"available":96104734720,"internally_reserved":0,"allocated":573440,"data_stored":185917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":531426987},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247979,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473672,"kb_used_data":564,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93898168,"statfs":{"total":96636764160,"available":96151724032,"internally_reserved":0,"allocated":577536,"data_stored":186400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53342208,"data_stored":53324143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:39:00.128 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T15:39:00.415 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:39:00.415 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":65,"fsid":"5d1df884-ae2f-44a8-abc8-b077329edada","created":"2024-09-17T15:24:23.007214+0000","modified":"2024-09-17T15:38:52.404533+0000","last_up_change":"2024-09-17T15:26:20.158671+0000","last_in_change":"2024-09-17T15:26:13.095382+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-17T15:24:38.423278+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":13,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfound_pool","create_time":"2024-09-17T15:24:47.449679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9defe403-224a-4a6c-9184-dcb72bef6bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":19,"up_from":23,"up_thru":64,"down_at":20,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":1704571166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6805","nonce":1704571166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6807","nonce":1704571166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":1704571166}]},"public_addr":"172.21.15.83:6804/1704571166","cluster_addr":"172.21.15.83:6805/1704571166","heartbeat_back_addr":"172.21.15.83:6807/1704571166","heartbeat_front_addr":"172.21.15.83:6806/1704571166","state":["exists","up"]},{"osd":1,"uuid":"d7d78e01-f761-444b-98c4-62587b991738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":24,"up_from":39,"up_thru":62,"down_at":25,"lost_at":25,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2000475044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6809","nonce":2000475044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6811","nonce":2000475044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":2000475044}]},"public_addr":"172.21.15.83:6808/2000475044","cluster_addr":"172.21.15.83:6809/2000475044","heartbeat_back_addr":"172.21.15.83:6811/2000475044","heartbeat_front_addr":"172.21.15.83:6810/2000475044","state":["exists","up"]},{"osd":2,"uuid":"23c2275c-8731-45f3-97f9-5100b6676f99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":16,"up_from":29,"up_thru":53,"down_at":17,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3672266049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6801","nonce":3672266049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6803","nonce":3672266049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3672266049}]},"public_addr":"172.21.15.83:6800/3672266049","cluster_addr":"172.21.15.83:6801/3672266049","heartbeat_back_addr":"172.21.15.83:6803/3672266049","heartbeat_front_addr":"172.21.15.83:6802/3672266049","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T15:24:31.547535+0000","dead_epoch":20},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T15:24:30.867578+0000","dead_epoch":25},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T15:24:30.866429+0000","dead_epoch":17}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T15:39:01.429 INFO:tasks.ceph:Scrubbing osd.0 2024-09-17T15:39:01.429 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-17T15:39:01.564 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T15:39:01.564 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T15:39:01.564 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T15:39:01.575 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-17T15:39:01.860 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 0 to deep-scrub 2024-09-17T15:39:01.873 INFO:tasks.ceph:Scrubbing osd.1 2024-09-17T15:39:01.873 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-17T15:39:01.994 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T15:39:01.995 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T15:39:01.995 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T15:39:02.005 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-17T15:39:02.291 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 1 to deep-scrub 2024-09-17T15:39:02.304 INFO:tasks.ceph:Scrubbing osd.2 2024-09-17T15:39:02.304 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-17T15:39:02.426 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-09-17T15:39:02.426 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T15:39:02.426 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-09-17T15:39:02.436 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-17T15:39:02.721 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 2 to deep-scrub 2024-09-17T15:39:02.734 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:39:03.016 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:39:03.016 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:39:03.028 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":497,"stamp":"2024-09-17T15:39:01.846478+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49461,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1398492,"kb_used_data":53984,"kb_used_omap":13,"kb_used_meta":1344434,"kb_avail":281717028,"statfs":{"total":289910292480,"available":288478236672,"internally_reserved":0,"allocated":55279616,"data_stored":54124737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1376701439},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-490,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":490,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001886"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":28922,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.419015+0000","last_change":"2024-09-17T15:38:06.362595+0000","last_active":"2024-09-17T15:38:52.419015+0000","last_peered":"2024-09-17T15:38:52.419015+0000","last_clean":"2024-09-17T15:38:52.419015+0000","last_became_active":"2024-09-17T15:38:06.284139+0000","last_became_peered":"2024-09-17T15:38:06.284139+0000","last_unstale":"2024-09-17T15:38:52.419015+0000","last_undegraded":"2024-09-17T15:38:52.419015+0000","last_fullsized":"2024-09-17T15:38:52.419015+0000","mapping_epoch":60,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:42:51.418256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4070,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20932","reported_seq":21083,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417805+0000","last_change":"2024-09-17T15:25:50.846046+0000","last_active":"2024-09-17T15:38:52.417805+0000","last_peered":"2024-09-17T15:38:52.417805+0000","last_clean":"2024-09-17T15:38:52.417805+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:38:52.417805+0000","last_undegraded":"2024-09-17T15:38:52.417805+0000","last_fullsized":"2024-09-17T15:38:52.417805+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1669","last_scrub_stamp":"2024-09-17T15:25:50.845900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:50.845900+0000","objects_scrubbed":1639,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:59.642856+0000","scrub_duration":542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21190,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416151+0000","last_change":"2024-09-17T15:36:48.120216+0000","last_active":"2024-09-17T15:38:52.416151+0000","last_peered":"2024-09-17T15:38:52.416151+0000","last_clean":"2024-09-17T15:38:52.416151+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:38:52.416151+0000","last_undegraded":"2024-09-17T15:38:52.416151+0000","last_fullsized":"2024-09-17T15:38:52.416151+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'1872","last_scrub_stamp":"2024-09-17T15:25:51.885344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:51.885344+0000","objects_scrubbed":1839,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:24.633938+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21088,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417541+0000","last_change":"2024-09-17T15:25:28.787887+0000","last_active":"2024-09-17T15:38:52.417541+0000","last_peered":"2024-09-17T15:38:52.417541+0000","last_clean":"2024-09-17T15:38:52.417541+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:38:52.417541+0000","last_undegraded":"2024-09-17T15:38:52.417541+0000","last_fullsized":"2024-09-17T15:38:52.417541+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:24:47.637664+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:49:56.271697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20653","reported_seq":20830,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.418826+0000","last_change":"2024-09-17T15:26:05.725092+0000","last_active":"2024-09-17T15:38:52.418826+0000","last_peered":"2024-09-17T15:38:52.418826+0000","last_clean":"2024-09-17T15:38:52.418826+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:38:52.418826+0000","last_undegraded":"2024-09-17T15:38:52.418826+0000","last_fullsized":"2024-09-17T15:38:52.418826+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3081","last_scrub_stamp":"2024-09-17T15:26:05.724892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:05.724892+0000","objects_scrubbed":3012,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:33:12.067812+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21251","reported_seq":21581,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416116+0000","last_change":"2024-09-17T15:37:15.185363+0000","last_active":"2024-09-17T15:38:52.416116+0000","last_peered":"2024-09-17T15:38:52.416116+0000","last_clean":"2024-09-17T15:38:52.416116+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:38:52.416116+0000","last_undegraded":"2024-09-17T15:38:52.416116+0000","last_fullsized":"2024-09-17T15:38:52.416116+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3550","last_scrub_stamp":"2024-09-17T15:26:08.867394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:08.867394+0000","objects_scrubbed":3497,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:44.798629+0000","scrub_duration":1244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20768","reported_seq":21125,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416029+0000","last_change":"2024-09-17T15:36:22.047913+0000","last_active":"2024-09-17T15:38:52.416029+0000","last_peered":"2024-09-17T15:38:52.416029+0000","last_clean":"2024-09-17T15:38:52.416029+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:38:52.416029+0000","last_undegraded":"2024-09-17T15:38:52.416029+0000","last_fullsized":"2024-09-17T15:38:52.416029+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3290","last_scrub_stamp":"2024-09-17T15:26:06.806681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:06.806681+0000","objects_scrubbed":3227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:31:38.942932+0000","scrub_duration":1180,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21092,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.418808+0000","last_change":"2024-09-17T15:26:04.780547+0000","last_active":"2024-09-17T15:38:52.418808+0000","last_peered":"2024-09-17T15:38:52.418808+0000","last_clean":"2024-09-17T15:38:52.418808+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:38:52.418808+0000","last_undegraded":"2024-09-17T15:38:52.418808+0000","last_fullsized":"2024-09-17T15:38:52.418808+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3103","last_scrub_stamp":"2024-09-17T15:26:04.780403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:04.780403+0000","objects_scrubbed":3044,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:07.376220+0000","scrub_duration":1108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":116,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.415914+0000","last_change":"2024-09-17T15:26:22.196593+0000","last_active":"2024-09-17T15:38:52.415914+0000","last_peered":"2024-09-17T15:38:52.415914+0000","last_clean":"2024-09-17T15:38:52.415914+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:38:52.415914+0000","last_undegraded":"2024-09-17T15:38:52.415914+0000","last_fullsized":"2024-09-17T15:38:52.415914+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:43:58.034664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20817","reported_seq":29148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416217+0000","last_change":"2024-09-17T15:28:23.814925+0000","last_active":"2024-09-17T15:38:52.416217+0000","last_peered":"2024-09-17T15:38:52.416217+0000","last_clean":"2024-09-17T15:38:52.416217+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:38:52.416217+0000","last_undegraded":"2024-09-17T15:38:52.416217+0000","last_fullsized":"2024-09-17T15:38:52.416217+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2016","last_scrub_stamp":"2024-09-17T15:25:53.959011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:53.959011+0000","objects_scrubbed":1994,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:27:41.885915+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417181+0000","last_change":"2024-09-17T15:25:28.787554+0000","last_active":"2024-09-17T15:38:52.417181+0000","last_peered":"2024-09-17T15:38:52.417181+0000","last_clean":"2024-09-17T15:38:52.417181+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:38:52.417181+0000","last_undegraded":"2024-09-17T15:38:52.417181+0000","last_fullsized":"2024-09-17T15:38:52.417181+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:06.842672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":28753,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.490508+0000","last_change":"2024-09-17T15:38:52.490429+0000","last_active":"2024-09-17T15:38:52.490508+0000","last_peered":"2024-09-17T15:38:52.490508+0000","last_clean":"2024-09-17T15:38:52.490508+0000","last_became_active":"2024-09-17T15:38:52.419947+0000","last_became_peered":"2024-09-17T15:38:52.419947+0000","last_unstale":"2024-09-17T15:38:52.490508+0000","last_undegraded":"2024-09-17T15:38:52.490508+0000","last_fullsized":"2024-09-17T15:38:52.490508+0000","mapping_epoch":64,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3454","last_scrub_stamp":"2024-09-17T15:26:10.852634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:10.852634+0000","objects_scrubbed":3396,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:39:30.553247+0000","scrub_duration":1143,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4012,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.419246+0000","last_change":"2024-09-17T15:26:22.412449+0000","last_active":"2024-09-17T15:38:52.419246+0000","last_peered":"2024-09-17T15:38:52.419246+0000","last_clean":"2024-09-17T15:38:52.419246+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:38:52.419246+0000","last_undegraded":"2024-09-17T15:38:52.419246+0000","last_fullsized":"2024-09-17T15:38:52.419246+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:56:00.053622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20519","reported_seq":28852,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417187+0000","last_change":"2024-09-17T15:35:59.967001+0000","last_active":"2024-09-17T15:38:52.417187+0000","last_peered":"2024-09-17T15:38:52.417187+0000","last_clean":"2024-09-17T15:38:52.417187+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:38:52.417187+0000","last_undegraded":"2024-09-17T15:38:52.417187+0000","last_fullsized":"2024-09-17T15:38:52.417187+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2142","last_scrub_stamp":"2024-09-17T15:25:55.123199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:55.123199+0000","objects_scrubbed":2101,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:26.625940+0000","scrub_duration":715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":126,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416392+0000","last_change":"2024-09-17T15:26:22.249556+0000","last_active":"2024-09-17T15:38:52.416392+0000","last_peered":"2024-09-17T15:38:52.416392+0000","last_clean":"2024-09-17T15:38:52.416392+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:38:52.416392+0000","last_undegraded":"2024-09-17T15:38:52.416392+0000","last_fullsized":"2024-09-17T15:38:52.416392+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:05.323052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29189,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417037+0000","last_change":"2024-09-17T15:33:11.274338+0000","last_active":"2024-09-17T15:38:52.417037+0000","last_peered":"2024-09-17T15:38:52.417037+0000","last_clean":"2024-09-17T15:38:52.417037+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:38:52.417037+0000","last_undegraded":"2024-09-17T15:38:52.417037+0000","last_fullsized":"2024-09-17T15:38:52.417037+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'2275","last_scrub_stamp":"2024-09-17T15:25:56.130304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:25:56.130304+0000","objects_scrubbed":2232,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:43.291294+0000","scrub_duration":749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21299,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416490+0000","last_change":"2024-09-17T15:37:42.278649+0000","last_active":"2024-09-17T15:38:52.416490+0000","last_peered":"2024-09-17T15:38:52.416490+0000","last_clean":"2024-09-17T15:38:52.416490+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:38:52.416490+0000","last_undegraded":"2024-09-17T15:38:52.416490+0000","last_fullsized":"2024-09-17T15:38:52.416490+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3494","last_scrub_stamp":"2024-09-17T15:26:09.987306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:09.987306+0000","objects_scrubbed":3450,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.268461+0000","scrub_duration":1314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":116,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416386+0000","last_change":"2024-09-17T15:26:22.387501+0000","last_active":"2024-09-17T15:38:52.416386+0000","last_peered":"2024-09-17T15:38:52.416386+0000","last_clean":"2024-09-17T15:38:52.416386+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:38:52.416386+0000","last_undegraded":"2024-09-17T15:38:52.416386+0000","last_fullsized":"2024-09-17T15:38:52.416386+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:25.639633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28103,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416283+0000","last_change":"2024-09-17T15:35:38.951851+0000","last_active":"2024-09-17T15:38:52.416283+0000","last_peered":"2024-09-17T15:38:52.416283+0000","last_clean":"2024-09-17T15:38:52.416283+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:38:52.416283+0000","last_undegraded":"2024-09-17T15:38:52.416283+0000","last_fullsized":"2024-09-17T15:38:52.416283+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3750","last_scrub_stamp":"2024-09-17T15:26:12.900151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:12.900151+0000","objects_scrubbed":3705,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:53:51.803708+0000","scrub_duration":1250,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.417682+0000","last_change":"2024-09-17T15:25:28.785700+0000","last_active":"2024-09-17T15:38:52.417682+0000","last_peered":"2024-09-17T15:38:52.417682+0000","last_clean":"2024-09-17T15:38:52.417682+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:38:52.417682+0000","last_undegraded":"2024-09-17T15:38:52.417682+0000","last_fullsized":"2024-09-17T15:38:52.417682+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:57.399868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21208,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416197+0000","last_change":"2024-09-17T15:38:27.429539+0000","last_active":"2024-09-17T15:38:52.416197+0000","last_peered":"2024-09-17T15:38:52.416197+0000","last_clean":"2024-09-17T15:38:52.416197+0000","last_became_active":"2024-09-17T15:38:27.345429+0000","last_became_peered":"2024-09-17T15:38:27.345429+0000","last_unstale":"2024-09-17T15:38:52.416197+0000","last_undegraded":"2024-09-17T15:38:52.416197+0000","last_fullsized":"2024-09-17T15:38:52.416197+0000","mapping_epoch":62,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3221","last_scrub_stamp":"2024-09-17T15:26:07.177603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:07.177603+0000","objects_scrubbed":3209,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:49:28.575667+0000","scrub_duration":528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4065,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416251+0000","last_change":"2024-09-17T15:26:22.296654+0000","last_active":"2024-09-17T15:38:52.416251+0000","last_peered":"2024-09-17T15:38:52.416251+0000","last_clean":"2024-09-17T15:38:52.416251+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:38:52.416251+0000","last_undegraded":"2024-09-17T15:38:52.416251+0000","last_fullsized":"2024-09-17T15:38:52.416251+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:47.036409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":120,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416114+0000","last_change":"2024-09-17T15:26:22.192533+0000","last_active":"2024-09-17T15:38:52.416114+0000","last_peered":"2024-09-17T15:38:52.416114+0000","last_clean":"2024-09-17T15:38:52.416114+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:38:52.416114+0000","last_undegraded":"2024-09-17T15:38:52.416114+0000","last_fullsized":"2024-09-17T15:38:52.416114+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:38.602774+0000","last_clean_scrub_stamp":"2024-09-17T15:24:38.602774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:24.525269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20632","reported_seq":29054,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:38:52.416154+0000","last_change":"2024-09-17T15:30:57.127217+0000","last_active":"2024-09-17T15:38:52.416154+0000","last_peered":"2024-09-17T15:38:52.416154+0000","last_clean":"2024-09-17T15:38:52.416154+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:38:52.416154+0000","last_undegraded":"2024-09-17T15:38:52.416154+0000","last_fullsized":"2024-09-17T15:38:52.416154+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'3601","last_scrub_stamp":"2024-09-17T15:26:11.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:24:47.637664+0000","last_clean_scrub_stamp":"2024-09-17T15:26:11.852193+0000","objects_scrubbed":3548,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:16.369059+0000","scrub_duration":1181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49460,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53403648,"data_stored":53331388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051751,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":405260,"kb_used_data":52860,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93966580,"statfs":{"total":96636764160,"available":96221777920,"internally_reserved":0,"allocated":54128640,"data_stored":53752420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724696,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519560,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852280,"statfs":{"total":96636764160,"available":96104734720,"internally_reserved":0,"allocated":573440,"data_stored":185917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":531426987},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247979,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473672,"kb_used_data":564,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93898168,"statfs":{"total":96636764160,"available":96151724032,"internally_reserved":0,"allocated":577536,"data_stored":186400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53342208,"data_stored":53324143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:39:03.033 INFO:tasks.ceph:pgid 2.f last_scrub_stamp 2024-09-17T15:24:47.637664+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=24, tm_sec=47, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=39, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T15:39:03.033 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2024-09-17T15:25:50.845900+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=25, tm_sec=50, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=39, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T15:39:03.033 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2024-09-17T15:25:51.885344+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=25, tm_sec=51, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=39, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T15:39:03.034 INFO:tasks.ceph:pgid 2.c last_scrub_stamp 2024-09-17T15:24:47.637664+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=24, tm_sec=47, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=39, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T15:39:03.034 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-09-17T15:26:05.724892+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=26, tm_sec=5, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=39, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T15:39:03.034 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2024-09-17T15:26:08.867394+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=26, tm_sec=8, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=39, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T15:39:03.034 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2024-09-17T15:26:06.806681+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=26, tm_sec=6, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=39, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T15:39:03.034 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-09-17T15:26:04.780403+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=26, tm_sec=4, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=39, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T15:39:03.034 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-17T15:24:38.602774+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=24, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=39, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T15:39:03.034 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-17T15:25:53.959011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=25, tm_sec=53, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=39, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T15:39:03.034 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2024-09-17T15:24:38.602774+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=24, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=39, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T15:39:03.035 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-17T15:26:10.852634+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=26, tm_sec=10, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=39, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T15:39:03.035 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2024-09-17T15:24:38.602774+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=24, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=39, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T15:39:03.035 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-17T15:25:55.123199+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=25, tm_sec=55, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=39, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T15:39:03.035 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2024-09-17T15:24:38.602774+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=24, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=39, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T15:39:03.035 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-17T15:25:56.130304+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=25, tm_sec=56, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=39, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T15:39:03.035 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-17T15:26:09.987306+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=26, tm_sec=9, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=39, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T15:39:03.035 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2024-09-17T15:24:38.602774+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=24, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=39, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T15:39:03.036 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-17T15:26:12.900151+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=26, tm_sec=12, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=39, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T15:39:03.036 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2024-09-17T15:24:38.602774+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=24, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=39, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T15:39:03.036 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-17T15:26:07.177603+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=26, tm_sec=7, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=39, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T15:39:03.036 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2024-09-17T15:24:38.602774+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=24, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=39, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T15:39:03.036 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2024-09-17T15:24:38.602774+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=24, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=39, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T15:39:03.036 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-17T15:26:11.852193+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=26, tm_sec=11, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=15, tm_min=39, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T15:39:03.036 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-17T15:39:23.037 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T15:39:23.342 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-09-17T15:39:23.343 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-09-17T15:39:23.355 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":507,"stamp":"2024-09-17T15:39:21.850655+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292979,"num_write_kb":506187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49461,"num_bytes_recovered":41573221,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160757,"ondisk_log_size":160757,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1384224,"kb_used_data":39728,"kb_used_omap":13,"kb_used_meta":1344434,"kb_avail":281731296,"statfs":{"total":289910292480,"available":288492847104,"internally_reserved":0,"allocated":40681472,"data_stored":39526593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13313,"internal_metadata":1376701439},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002561"},"pg_stats":[{"pgid":"2.f","version":"47'20648","reported_seq":28930,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:39:04.551232+0000","last_change":"2024-09-17T15:39:04.551232+0000","last_active":"2024-09-17T15:39:04.551232+0000","last_peered":"2024-09-17T15:39:04.551232+0000","last_clean":"2024-09-17T15:39:04.551232+0000","last_became_active":"2024-09-17T15:38:06.284139+0000","last_became_peered":"2024-09-17T15:38:06.284139+0000","last_unstale":"2024-09-17T15:39:04.551232+0000","last_undegraded":"2024-09-17T15:39:04.551232+0000","last_fullsized":"2024-09-17T15:39:04.551232+0000","mapping_epoch":60,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'20648","last_scrub_stamp":"2024-09-17T15:39:04.551131+0000","last_deep_scrub":"47'20648","last_deep_scrub_stamp":"2024-09-17T15:39:04.551131+0000","last_clean_scrub_stamp":"2024-09-17T15:39:04.551131+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:19:22.824190+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20648,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4070,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"65'20934","reported_seq":21093,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:39:08.112997+0000","last_change":"2024-09-17T15:39:08.112997+0000","last_active":"2024-09-17T15:39:08.112997+0000","last_peered":"2024-09-17T15:39:08.112997+0000","last_clean":"2024-09-17T15:39:08.112997+0000","last_became_active":"2024-09-17T15:25:28.785354+0000","last_became_peered":"2024-09-17T15:25:28.785354+0000","last_unstale":"2024-09-17T15:39:08.112997+0000","last_undegraded":"2024-09-17T15:39:08.112997+0000","last_fullsized":"2024-09-17T15:39:08.112997+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'20934","last_scrub_stamp":"2024-09-17T15:39:08.112894+0000","last_deep_scrub":"65'20934","last_deep_scrub_stamp":"2024-09-17T15:39:08.112894+0000","last_clean_scrub_stamp":"2024-09-17T15:39:08.112894+0000","objects_scrubbed":2,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:23:16.842288+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":19289,"num_write_kb":35304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20934","reported_seq":21199,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:39:15.450253+0000","last_change":"2024-09-17T15:39:15.450253+0000","last_active":"2024-09-17T15:39:15.450253+0000","last_peered":"2024-09-17T15:39:15.450253+0000","last_clean":"2024-09-17T15:39:15.450253+0000","last_became_active":"2024-09-17T15:36:48.045213+0000","last_became_peered":"2024-09-17T15:36:48.045213+0000","last_unstale":"2024-09-17T15:39:15.450253+0000","last_undegraded":"2024-09-17T15:39:15.450253+0000","last_fullsized":"2024-09-17T15:39:15.450253+0000","mapping_epoch":54,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'20934","last_scrub_stamp":"2024-09-17T15:39:15.450165+0000","last_deep_scrub":"47'20934","last_deep_scrub_stamp":"2024-09-17T15:39:15.450165+0000","last_clean_scrub_stamp":"2024-09-17T15:39:15.450165+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:55:08.152316+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19091,"num_write_kb":34501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'20970","reported_seq":21096,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:39:07.144035+0000","last_change":"2024-09-17T15:39:07.144035+0000","last_active":"2024-09-17T15:39:07.144035+0000","last_peered":"2024-09-17T15:39:07.144035+0000","last_clean":"2024-09-17T15:39:07.144035+0000","last_became_active":"2024-09-17T15:25:28.787655+0000","last_became_peered":"2024-09-17T15:25:28.787655+0000","last_unstale":"2024-09-17T15:39:07.144035+0000","last_undegraded":"2024-09-17T15:39:07.144035+0000","last_fullsized":"2024-09-17T15:39:07.144035+0000","mapping_epoch":33,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'20970","last_scrub_stamp":"2024-09-17T15:39:07.143898+0000","last_deep_scrub":"47'20970","last_deep_scrub_stamp":"2024-09-17T15:39:07.143898+0000","last_clean_scrub_stamp":"2024-09-17T15:39:07.143898+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:01:37.143654+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20970,"num_write_kb":41940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"65'20654","reported_seq":20839,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:39:06.568341+0000","last_change":"2024-09-17T15:39:06.568341+0000","last_active":"2024-09-17T15:39:06.568341+0000","last_peered":"2024-09-17T15:39:06.568341+0000","last_clean":"2024-09-17T15:39:06.568341+0000","last_became_active":"2024-09-17T15:25:27.781898+0000","last_became_peered":"2024-09-17T15:25:27.781898+0000","last_unstale":"2024-09-17T15:39:06.568341+0000","last_undegraded":"2024-09-17T15:39:06.568341+0000","last_fullsized":"2024-09-17T15:39:06.568341+0000","mapping_epoch":32,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'20654","last_scrub_stamp":"2024-09-17T15:39:06.568232+0000","last_deep_scrub":"65'20654","last_deep_scrub_stamp":"2024-09-17T15:39:06.568232+0000","last_clean_scrub_stamp":"2024-09-17T15:39:06.568232+0000","objects_scrubbed":1,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:57:24.663674+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17635,"num_write_kb":29248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"65'21252","reported_seq":21590,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:39:14.413677+0000","last_change":"2024-09-17T15:39:14.413677+0000","last_active":"2024-09-17T15:39:14.413677+0000","last_peered":"2024-09-17T15:39:14.413677+0000","last_clean":"2024-09-17T15:39:14.413677+0000","last_became_active":"2024-09-17T15:37:15.114921+0000","last_became_peered":"2024-09-17T15:37:15.114921+0000","last_unstale":"2024-09-17T15:39:14.413677+0000","last_undegraded":"2024-09-17T15:39:14.413677+0000","last_fullsized":"2024-09-17T15:39:14.413677+0000","mapping_epoch":56,"log_start":"43'11200","ondisk_log_start":"43'11200","created":14,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'21252","last_scrub_stamp":"2024-09-17T15:39:14.413590+0000","last_deep_scrub":"65'21252","last_deep_scrub_stamp":"2024-09-17T15:39:14.413590+0000","last_clean_scrub_stamp":"2024-09-17T15:39:14.413590+0000","objects_scrubbed":1,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:26:22.116729+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17750,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4271,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"65'20770","reported_seq":21135,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:39:13.378134+0000","last_change":"2024-09-17T15:39:13.378134+0000","last_active":"2024-09-17T15:39:13.378134+0000","last_peered":"2024-09-17T15:39:13.378134+0000","last_clean":"2024-09-17T15:39:13.378134+0000","last_became_active":"2024-09-17T15:36:21.962690+0000","last_became_peered":"2024-09-17T15:36:21.962690+0000","last_unstale":"2024-09-17T15:39:13.378134+0000","last_undegraded":"2024-09-17T15:39:13.378134+0000","last_fullsized":"2024-09-17T15:39:13.378134+0000","mapping_epoch":52,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'20770","last_scrub_stamp":"2024-09-17T15:39:13.378023+0000","last_deep_scrub":"65'20770","last_deep_scrub_stamp":"2024-09-17T15:39:13.378023+0000","last_clean_scrub_stamp":"2024-09-17T15:39:13.378023+0000","objects_scrubbed":2,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:58:46.673308+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17537,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4174,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20916","reported_seq":21100,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:39:03.583883+0000","last_change":"2024-09-17T15:39:03.583883+0000","last_active":"2024-09-17T15:39:03.583883+0000","last_peered":"2024-09-17T15:39:03.583883+0000","last_clean":"2024-09-17T15:39:03.583883+0000","last_became_active":"2024-09-17T15:25:27.781355+0000","last_became_peered":"2024-09-17T15:25:27.781355+0000","last_unstale":"2024-09-17T15:39:03.583883+0000","last_undegraded":"2024-09-17T15:39:03.583883+0000","last_fullsized":"2024-09-17T15:39:03.583883+0000","mapping_epoch":32,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'20916","last_scrub_stamp":"2024-09-17T15:39:03.583808+0000","last_deep_scrub":"47'20916","last_deep_scrub_stamp":"2024-09-17T15:39:03.583808+0000","last_clean_scrub_stamp":"2024-09-17T15:39:03.583808+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:16:55.882434+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17870,"num_write_kb":29652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:39:03.300162+0000","last_change":"2024-09-17T15:39:03.300162+0000","last_active":"2024-09-17T15:39:03.300162+0000","last_peered":"2024-09-17T15:39:03.300162+0000","last_clean":"2024-09-17T15:39:03.300162+0000","last_became_active":"2024-09-17T15:26:22.196263+0000","last_became_peered":"2024-09-17T15:26:22.196263+0000","last_unstale":"2024-09-17T15:39:03.300162+0000","last_undegraded":"2024-09-17T15:39:03.300162+0000","last_fullsized":"2024-09-17T15:39:03.300162+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:39:03.300083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:39:03.300083+0000","last_clean_scrub_stamp":"2024-09-17T15:39:03.300083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:24:39.631657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053127700000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"65'20818","reported_seq":29157,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:39:08.391314+0000","last_change":"2024-09-17T15:39:08.391314+0000","last_active":"2024-09-17T15:39:08.391314+0000","last_peered":"2024-09-17T15:39:08.391314+0000","last_clean":"2024-09-17T15:39:08.391314+0000","last_became_active":"2024-09-17T15:28:23.807029+0000","last_became_peered":"2024-09-17T15:28:23.807029+0000","last_unstale":"2024-09-17T15:39:08.391314+0000","last_undegraded":"2024-09-17T15:39:08.391314+0000","last_fullsized":"2024-09-17T15:39:08.391314+0000","mapping_epoch":42,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'20818","last_scrub_stamp":"2024-09-17T15:39:08.391228+0000","last_deep_scrub":"65'20818","last_deep_scrub_stamp":"2024-09-17T15:39:08.391228+0000","last_clean_scrub_stamp":"2024-09-17T15:39:08.391228+0000","objects_scrubbed":1,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:05:36.449801+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18817,"num_write_kb":33648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":16671203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":126,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:39:03.109830+0000","last_change":"2024-09-17T15:39:03.109830+0000","last_active":"2024-09-17T15:39:03.109830+0000","last_peered":"2024-09-17T15:39:03.109830+0000","last_clean":"2024-09-17T15:39:03.109830+0000","last_became_active":"2024-09-17T15:25:28.787022+0000","last_became_peered":"2024-09-17T15:25:28.787022+0000","last_unstale":"2024-09-17T15:39:03.109830+0000","last_undegraded":"2024-09-17T15:39:03.109830+0000","last_fullsized":"2024-09-17T15:39:03.109830+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:39:03.109759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:39:03.109759+0000","last_clean_scrub_stamp":"2024-09-17T15:39:03.109759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:39:02.062738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20542","reported_seq":28761,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:39:05.523155+0000","last_change":"2024-09-17T15:39:05.523155+0000","last_active":"2024-09-17T15:39:05.523155+0000","last_peered":"2024-09-17T15:39:05.523155+0000","last_clean":"2024-09-17T15:39:05.523155+0000","last_became_active":"2024-09-17T15:38:52.419947+0000","last_became_peered":"2024-09-17T15:38:52.419947+0000","last_unstale":"2024-09-17T15:39:05.523155+0000","last_undegraded":"2024-09-17T15:39:05.523155+0000","last_fullsized":"2024-09-17T15:39:05.523155+0000","mapping_epoch":64,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'20542","last_scrub_stamp":"2024-09-17T15:39:05.523039+0000","last_deep_scrub":"47'20542","last_deep_scrub_stamp":"2024-09-17T15:39:05.523039+0000","last_clean_scrub_stamp":"2024-09-17T15:39:05.523039+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:49:40.588792+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17138,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4012,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"65'3","reported_seq":160,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:39:02.577180+0000","last_change":"2024-09-17T15:39:02.577180+0000","last_active":"2024-09-17T15:39:02.577180+0000","last_peered":"2024-09-17T15:39:02.577180+0000","last_clean":"2024-09-17T15:39:02.577180+0000","last_became_active":"2024-09-17T15:26:22.411915+0000","last_became_peered":"2024-09-17T15:26:22.411915+0000","last_unstale":"2024-09-17T15:39:02.577180+0000","last_undegraded":"2024-09-17T15:39:02.577180+0000","last_fullsized":"2024-09-17T15:39:02.577180+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'3","last_scrub_stamp":"2024-09-17T15:39:02.577125+0000","last_deep_scrub":"65'3","last_deep_scrub_stamp":"2024-09-17T15:39:02.577125+0000","last_clean_scrub_stamp":"2024-09-17T15:39:02.577125+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:54:18.802181+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00069172500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"65'20520","reported_seq":28861,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:39:06.188726+0000","last_change":"2024-09-17T15:39:06.188726+0000","last_active":"2024-09-17T15:39:06.188726+0000","last_peered":"2024-09-17T15:39:06.188726+0000","last_clean":"2024-09-17T15:39:06.188726+0000","last_became_active":"2024-09-17T15:35:59.901497+0000","last_became_peered":"2024-09-17T15:35:59.901497+0000","last_unstale":"2024-09-17T15:39:06.188726+0000","last_undegraded":"2024-09-17T15:39:06.188726+0000","last_fullsized":"2024-09-17T15:39:06.188726+0000","mapping_epoch":50,"log_start":"43'10500","ondisk_log_start":"43'10500","created":14,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'20520","last_scrub_stamp":"2024-09-17T15:39:06.188635+0000","last_deep_scrub":"65'20520","last_deep_scrub_stamp":"2024-09-17T15:39:06.188635+0000","last_clean_scrub_stamp":"2024-09-17T15:39:06.188635+0000","objects_scrubbed":1,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:25:33.065296+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18414,"num_write_kb":32628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4078,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"65'2","reported_seq":135,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:39:05.357983+0000","last_change":"2024-09-17T15:39:05.357983+0000","last_active":"2024-09-17T15:39:05.357983+0000","last_peered":"2024-09-17T15:39:05.357983+0000","last_clean":"2024-09-17T15:39:05.357983+0000","last_became_active":"2024-09-17T15:26:22.249201+0000","last_became_peered":"2024-09-17T15:26:22.249201+0000","last_unstale":"2024-09-17T15:39:05.357983+0000","last_undegraded":"2024-09-17T15:39:05.357983+0000","last_fullsized":"2024-09-17T15:39:05.357983+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'2","last_scrub_stamp":"2024-09-17T15:39:05.357873+0000","last_deep_scrub":"65'2","last_deep_scrub_stamp":"2024-09-17T15:39:05.357873+0000","last_clean_scrub_stamp":"2024-09-17T15:39:05.357873+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:52:34.381521+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00054981400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20736","reported_seq":29197,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:39:05.170645+0000","last_change":"2024-09-17T15:39:05.170645+0000","last_active":"2024-09-17T15:39:05.170645+0000","last_peered":"2024-09-17T15:39:05.170645+0000","last_clean":"2024-09-17T15:39:05.170645+0000","last_became_active":"2024-09-17T15:33:11.227102+0000","last_became_peered":"2024-09-17T15:33:11.227102+0000","last_unstale":"2024-09-17T15:39:05.170645+0000","last_undegraded":"2024-09-17T15:39:05.170645+0000","last_fullsized":"2024-09-17T15:39:05.170645+0000","mapping_epoch":46,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'20736","last_scrub_stamp":"2024-09-17T15:39:05.170540+0000","last_deep_scrub":"47'20736","last_deep_scrub_stamp":"2024-09-17T15:39:05.170540+0000","last_clean_scrub_stamp":"2024-09-17T15:39:05.170540+0000","objects_scrubbed":0,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:40:24.557894+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18498,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":8073216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20966","reported_seq":21307,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:39:12.397424+0000","last_change":"2024-09-17T15:39:12.397424+0000","last_active":"2024-09-17T15:39:12.397424+0000","last_peered":"2024-09-17T15:39:12.397424+0000","last_clean":"2024-09-17T15:39:12.397424+0000","last_became_active":"2024-09-17T15:37:42.211896+0000","last_became_peered":"2024-09-17T15:37:42.211896+0000","last_unstale":"2024-09-17T15:39:12.397424+0000","last_undegraded":"2024-09-17T15:39:12.397424+0000","last_fullsized":"2024-09-17T15:39:12.397424+0000","mapping_epoch":58,"log_start":"43'10900","ondisk_log_start":"43'10900","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'20966","last_scrub_stamp":"2024-09-17T15:39:12.397329+0000","last_deep_scrub":"47'20966","last_deep_scrub_stamp":"2024-09-17T15:39:12.397329+0000","last_clean_scrub_stamp":"2024-09-17T15:39:12.397329+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:37.503179+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17514,"num_write_kb":28128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:39:04.334324+0000","last_change":"2024-09-17T15:39:04.334324+0000","last_active":"2024-09-17T15:39:04.334324+0000","last_peered":"2024-09-17T15:39:04.334324+0000","last_clean":"2024-09-17T15:39:04.334324+0000","last_became_active":"2024-09-17T15:26:22.384928+0000","last_became_peered":"2024-09-17T15:26:22.384928+0000","last_unstale":"2024-09-17T15:39:04.334324+0000","last_undegraded":"2024-09-17T15:39:04.334324+0000","last_fullsized":"2024-09-17T15:39:04.334324+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:39:04.334253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:39:04.334253+0000","last_clean_scrub_stamp":"2024-09-17T15:39:04.334253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:48:24.435718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087700899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20772","reported_seq":28111,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:39:11.423108+0000","last_change":"2024-09-17T15:39:11.423108+0000","last_active":"2024-09-17T15:39:11.423108+0000","last_peered":"2024-09-17T15:39:11.423108+0000","last_clean":"2024-09-17T15:39:11.423108+0000","last_became_active":"2024-09-17T15:35:38.873753+0000","last_became_peered":"2024-09-17T15:35:38.873753+0000","last_unstale":"2024-09-17T15:39:11.423108+0000","last_undegraded":"2024-09-17T15:39:11.423108+0000","last_fullsized":"2024-09-17T15:39:11.423108+0000","mapping_epoch":48,"log_start":"43'10700","ondisk_log_start":"43'10700","created":14,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'20772","last_scrub_stamp":"2024-09-17T15:39:11.423007+0000","last_deep_scrub":"47'20772","last_deep_scrub_stamp":"2024-09-17T15:39:11.423007+0000","last_clean_scrub_stamp":"2024-09-17T15:39:11.423007+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:41:30.285947+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17059,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":126,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:39:04.157080+0000","last_change":"2024-09-17T15:39:04.157080+0000","last_active":"2024-09-17T15:39:04.157080+0000","last_peered":"2024-09-17T15:39:04.157080+0000","last_clean":"2024-09-17T15:39:04.157080+0000","last_became_active":"2024-09-17T15:25:28.785182+0000","last_became_peered":"2024-09-17T15:25:28.785182+0000","last_unstale":"2024-09-17T15:39:04.157080+0000","last_undegraded":"2024-09-17T15:39:04.157080+0000","last_fullsized":"2024-09-17T15:39:04.157080+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:39:04.157007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:39:04.157007+0000","last_clean_scrub_stamp":"2024-09-17T15:39:04.157007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:53:31.565816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20886","reported_seq":21216,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:39:09.433093+0000","last_change":"2024-09-17T15:39:09.433093+0000","last_active":"2024-09-17T15:39:09.433093+0000","last_peered":"2024-09-17T15:39:09.433093+0000","last_clean":"2024-09-17T15:39:09.433093+0000","last_became_active":"2024-09-17T15:38:27.345429+0000","last_became_peered":"2024-09-17T15:38:27.345429+0000","last_unstale":"2024-09-17T15:39:09.433093+0000","last_undegraded":"2024-09-17T15:39:09.433093+0000","last_fullsized":"2024-09-17T15:39:09.433093+0000","mapping_epoch":62,"log_start":"43'10800","ondisk_log_start":"43'10800","created":14,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'20886","last_scrub_stamp":"2024-09-17T15:39:09.432962+0000","last_deep_scrub":"47'20886","last_deep_scrub_stamp":"2024-09-17T15:39:09.432962+0000","last_clean_scrub_stamp":"2024-09-17T15:39:09.432962+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:38:32.991678+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4065,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:39:06.364333+0000","last_change":"2024-09-17T15:39:06.364238+0000","last_active":"2024-09-17T15:39:06.364333+0000","last_peered":"2024-09-17T15:39:06.364333+0000","last_clean":"2024-09-17T15:39:06.364333+0000","last_became_active":"2024-09-17T15:26:22.296311+0000","last_became_peered":"2024-09-17T15:26:22.296311+0000","last_unstale":"2024-09-17T15:39:06.364333+0000","last_undegraded":"2024-09-17T15:39:06.364333+0000","last_fullsized":"2024-09-17T15:39:06.364333+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:39:06.364153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:39:06.364153+0000","last_clean_scrub_stamp":"2024-09-17T15:39:06.364153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:46:53.722271+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00098595900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":128,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:39:07.351443+0000","last_change":"2024-09-17T15:39:07.351443+0000","last_active":"2024-09-17T15:39:07.351443+0000","last_peered":"2024-09-17T15:39:07.351443+0000","last_clean":"2024-09-17T15:39:07.351443+0000","last_became_active":"2024-09-17T15:26:22.191802+0000","last_became_peered":"2024-09-17T15:26:22.191802+0000","last_unstale":"2024-09-17T15:39:07.351443+0000","last_undegraded":"2024-09-17T15:39:07.351443+0000","last_fullsized":"2024-09-17T15:39:07.351443+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T15:39:07.351360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T15:39:07.351360+0000","last_clean_scrub_stamp":"2024-09-17T15:39:07.351360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:59:41.022898+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00076898899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"65'20634","reported_seq":29065,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T15:39:10.420148+0000","last_change":"2024-09-17T15:39:10.420090+0000","last_active":"2024-09-17T15:39:10.420148+0000","last_peered":"2024-09-17T15:39:10.420148+0000","last_clean":"2024-09-17T15:39:10.420148+0000","last_became_active":"2024-09-17T15:30:57.098943+0000","last_became_peered":"2024-09-17T15:30:57.098943+0000","last_unstale":"2024-09-17T15:39:10.420148+0000","last_undegraded":"2024-09-17T15:39:10.420148+0000","last_fullsized":"2024-09-17T15:39:10.420148+0000","mapping_epoch":44,"log_start":"43'10600","ondisk_log_start":"43'10600","created":14,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'20634","last_scrub_stamp":"2024-09-17T15:39:10.419990+0000","last_deep_scrub":"65'20634","last_deep_scrub_stamp":"2024-09-17T15:39:10.419990+0000","last_clean_scrub_stamp":"2024-09-17T15:39:10.419990+0000","objects_scrubbed":2,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:37:34.287738+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4111,"num_bytes_recovered":16826851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":292977,"num_write_kb":506185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49460,"num_bytes_recovered":41573202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":38928384,"data_stored":38856124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160752,"ondisk_log_size":160752,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":29,"seq":124554051755,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":391004,"kb_used_data":38604,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93980836,"statfs":{"total":96636764160,"available":96236376064,"internally_reserved":0,"allocated":39530496,"data_stored":39154276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":360836780},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":39,"seq":167503724700,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519552,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852288,"statfs":{"total":96636764160,"available":96104742912,"internally_reserved":0,"allocated":573440,"data_stored":185917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":531426987},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247983,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473668,"kb_used_data":564,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93898172,"statfs":{"total":96636764160,"available":96151728128,"internally_reserved":0,"allocated":577536,"data_stored":186400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":484437672},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":38866944,"data_stored":38848879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T15:39:23.357 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-17T15:39:23.662 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-17T15:39:23.663 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-17T15:39:23.663 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-17T15:39:23.663 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T15:39:24.729 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-09-17T15:39:29.765 INFO:tasks.ceph.osd.0:Stopped 2024-09-17T15:39:29.765 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-17T15:39:29.766 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T15:39:29.838 INFO:tasks.ceph.osd.1:Stopped 2024-09-17T15:39:29.838 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-17T15:39:29.838 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T15:39:30.334 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~6s 2024-09-17T15:39:35.738 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-09-17T15:39:35.739 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-09-17T15:39:35.940 INFO:tasks.ceph.osd.2:Stopped 2024-09-17T15:39:35.941 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-17T15:39:35.941 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-17T15:39:35.941 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T15:39:35.971 INFO:tasks.ceph.mgr.x:Stopped 2024-09-17T15:39:35.971 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-17T15:39:35.971 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-17T15:39:35.971 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T15:39:36.026 INFO:tasks.ceph.mon.a:Stopped 2024-09-17T15:39:36.026 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-17T15:39:36.027 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T15:39:36.081 INFO:tasks.ceph.mon.b:Stopped 2024-09-17T15:39:36.081 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-17T15:39:36.081 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T15:39:36.135 INFO:tasks.ceph.mon.c:Stopped 2024-09-17T15:39:36.135 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-17T15:39:36.136 DEBUG:teuthology.orchestra.run.smithi083:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v '\(MON_DOWN\)' | head -n 1 2024-09-17T15:39:36.446 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi083.front.sepia.ceph.com 2024-09-17T15:39:36.446 DEBUG:teuthology.orchestra.run.smithi083:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-17T15:39:37.323 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi083.front.sepia.ceph.com 2024-09-17T15:39:37.323 DEBUG:teuthology.orchestra.run.smithi083:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-17T15:39:37.449 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi083.front.sepia.ceph.com 2024-09-17T15:39:37.450 DEBUG:teuthology.orchestra.run.smithi083:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-17T15:39:37.514 INFO:tasks.ceph:Archiving mon data... 2024-09-17T15:39:37.515 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7908934/data/mon.a.tgz 2024-09-17T15:39:37.515 DEBUG:teuthology.orchestra.run.smithi083:> mktemp 2024-09-17T15:39:37.541 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/tmp.N4n01Z4byy 2024-09-17T15:39:37.542 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.N4n01Z4byy 2024-09-17T15:39:37.669 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0666 /tmp/tmp.N4n01Z4byy 2024-09-17T15:39:37.769 DEBUG:teuthology.orchestra.remote:smithi083:/tmp/tmp.N4n01Z4byy is 299KB 2024-09-17T15:39:37.864 DEBUG:teuthology.orchestra.run.smithi083:> rm -fr /tmp/tmp.N4n01Z4byy 2024-09-17T15:39:37.879 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7908934/data/mon.b.tgz 2024-09-17T15:39:37.879 DEBUG:teuthology.orchestra.run.smithi083:> mktemp 2024-09-17T15:39:37.933 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/tmp.jfP1gTJLIo 2024-09-17T15:39:37.933 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.jfP1gTJLIo 2024-09-17T15:39:38.048 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0666 /tmp/tmp.jfP1gTJLIo 2024-09-17T15:39:38.131 DEBUG:teuthology.orchestra.remote:smithi083:/tmp/tmp.jfP1gTJLIo is 299KB 2024-09-17T15:39:38.225 DEBUG:teuthology.orchestra.run.smithi083:> rm -fr /tmp/tmp.jfP1gTJLIo 2024-09-17T15:39:38.240 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7908934/data/mon.c.tgz 2024-09-17T15:39:38.240 DEBUG:teuthology.orchestra.run.smithi083:> mktemp 2024-09-17T15:39:38.294 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/tmp.4c67mctfJz 2024-09-17T15:39:38.295 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.4c67mctfJz 2024-09-17T15:39:38.409 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0666 /tmp/tmp.4c67mctfJz 2024-09-17T15:39:38.493 DEBUG:teuthology.orchestra.remote:smithi083:/tmp/tmp.4c67mctfJz is 299KB 2024-09-17T15:39:38.588 DEBUG:teuthology.orchestra.run.smithi083:> rm -fr /tmp/tmp.4c67mctfJz 2024-09-17T15:39:38.603 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-17T15:39:38.603 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-17T15:39:38.736 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-17T15:39:38.736 INFO:tasks.ceph:Archiving crash dumps... 2024-09-17T15:39:38.738 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/lib/ceph/crash to /home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7908934/remote/smithi083/crash 2024-09-17T15:39:38.739 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-17T15:39:38.800 INFO:tasks.ceph:Compressing logs... 2024-09-17T15:39:38.800 DEBUG:teuthology.orchestra.run.smithi083:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-17T15:39:38.869 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-17T15:39:38.869 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51876.log 2024-09-17T15:39:38.871 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52128.log 2024-09-17T15:39:38.871 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43365.log 2024-09-17T15:39:38.871 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.52128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70451.log 2024-09-17T15:39:38.871 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35683.log 2024-09-17T15:39:38.871 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63310.log 2024-09-17T15:39:38.871 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57507.log 2024-09-17T15:39:38.871 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35263.log 2024-09-17T15:39:38.871 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70577.log 2024-09-17T15:39:38.873 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44106.log 2024-09-17T15:39:38.873 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64528.log 2024-09-17T15:39:38.874 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66880.log 2024-09-17T15:39:38.874 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43555.log 2024-09-17T15:39:38.874 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36756.log 2024-09-17T15:39:38.874 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62134.log 2024-09-17T15:39:38.874 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67468.log 2024-09-17T15:39:38.874 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61839.log 2024-09-17T15:39:38.874 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35389.log 2024-09-17T15:39:38.874 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41333.log 2024-09-17T15:39:38.874 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55869.log 2024-09-17T15:39:38.874 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54817.log 2024-09-17T15:39:38.877 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55911.log 2024-09-17T15:39:38.877 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50994.log 2024-09-17T15:39:38.877 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58140.log 2024-09-17T15:39:38.877 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68980.log 2024-09-17T15:39:38.878 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67720.log 2024-09-17T15:39:38.878 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54439.log 2024-09-17T15:39:38.878 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58896.log 2024-09-17T15:39:38.878 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49020.log 2024-09-17T15:39:38.878 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52465.log 2024-09-17T15:39:38.878 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.70451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69526.log 2024-09-17T15:39:38.878 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69442.log 2024-09-17T15:39:38.878 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45618.log 2024-09-17T15:39:38.878 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54103.log 2024-09-17T15:39:38.878 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67762.log 2024-09-17T15:39:38.878 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45996.log 2024-09-17T15:39:38.878 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45786.log 2024-09-17T15:39:38.878 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34918.log 2024-09-17T15:39:39.138 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67594.log 2024-09-17T15:39:39.138 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37728.log 2024-09-17T15:39:39.138 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38246.log 2024-09-17T15:39:39.138 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53221.log 2024-09-17T15:39:39.138 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35683.log: /var/log/ceph/ceph-client.admin.63310.log: /var/log/ceph/ceph-client.admin.57507.log: /var/log/ceph/ceph-client.admin.35263.log: /var/log/ceph/ceph-client.admin.70577.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63310.log.gz 2024-09-17T15:39:39.138 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57507.log.gz 2024-09-17T15:39:39.139 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38246.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35263.log.gz 2024-09-17T15:39:39.139 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70577.log.gz 2024-09-17T15:39:39.139 INFO:teuthology.orchestra.run.smithi083.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.62218.log 2024-09-17T15:39:39.139 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44106.log: /var/log/ceph/ceph-client.admin.64528.log: /var/log/ceph/ceph-client.admin.66880.log: /var/log/ceph/ceph-client.admin.43555.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44106.log.gz 2024-09-17T15:39:39.139 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36756.log: /var/log/ceph/ceph-client.admin.62134.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64528.log.gz 2024-09-17T15:39:39.139 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66880.log.gz 2024-09-17T15:39:39.139 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43555.log.gz 2024-09-17T15:39:39.139 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.67468.log: /var/log/ceph/ceph-client.admin.61839.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62134.log.gz 2024-09-17T15:39:39.139 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35389.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61839.log.gz 2024-09-17T15:39:39.139 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51498.log 2024-09-17T15:39:39.139 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55869.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67468.log.gz 2024-09-17T15:39:39.139 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54817.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41333.log.gz 2024-09-17T15:39:39.139 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55869.log.gz 2024-09-17T15:39:39.140 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55911.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54817.log.gz 2024-09-17T15:39:39.140 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50994.log: /var/log/ceph/ceph-client.admin.58140.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55911.log.gz 2024-09-17T15:39:39.140 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.62218.log: /var/log/ceph/ceph-client.admin.68980.log: /var/log/ceph/ceph-client.admin.67720.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50994.log.gz 0.0% 2024-09-17T15:39:39.140 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58140.log.gz 2024-09-17T15:39:39.140 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62218.log.gz 2024-09-17T15:39:39.140 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68980.log.gz 2024-09-17T15:39:39.140 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67720.log.gz 2024-09-17T15:39:39.140 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54439.log: /var/log/ceph/ceph-client.admin.58896.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43365.log.gz 2024-09-17T15:39:39.140 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51876.log.gz 2024-09-17T15:39:39.140 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52128.log.gz 2024-09-17T15:39:39.140 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49020.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54439.log.gz 2024-09-17T15:39:39.140 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58896.log.gz 2024-09-17T15:39:39.140 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70451.log.gz 2024-09-17T15:39:39.141 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.52465.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49020.log.gz 2024-09-17T15:39:39.141 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69526.log: /var/log/ceph/ceph-client.admin.69442.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52465.log.gz 2024-09-17T15:39:39.141 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45618.log: /var/log/ceph/ceph-client.admin.54103.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69526.log.gz 2024-09-17T15:39:39.141 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66250.log 2024-09-17T15:39:39.141 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.67762.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69442.log.gz 2024-09-17T15:39:39.141 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45618.log.gz 2024-09-17T15:39:39.141 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45996.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67762.log.gz 2024-09-17T15:39:39.141 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45786.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54103.log.gz 2024-09-17T15:39:39.141 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34918.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45996.log.gz 2024-09-17T15:39:39.141 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.67594.log: /var/log/ceph/ceph-client.admin.51498.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45786.log.gz 2024-09-17T15:39:39.141 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34918.log.gz 2024-09-17T15:39:39.141 INFO:teuthology.orchestra.run.smithi083.stderr:gzip/var/log/ceph/ceph-client.admin.37728.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51498.log.gz 2024-09-17T15:39:39.141 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67594.log.gz 2024-09-17T15:39:39.141 INFO:teuthology.orchestra.run.smithi083.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.68140.log 2024-09-17T15:39:39.142 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53221.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53221.log.gz 2024-09-17T15:39:39.142 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.66250.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66250.log.gz 2024-09-17T15:39:39.142 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48684.log 2024-09-17T15:39:39.142 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.68140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39701.log 2024-09-17T15:39:39.142 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68140.log.gz 2024-09-17T15:39:39.142 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58308.log 2024-09-17T15:39:39.142 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48684.log.gz 2024-09-17T15:39:39.142 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48810.log 2024-09-17T15:39:39.142 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.58308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47046.log 2024-09-17T15:39:39.142 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58308.log.gz 2024-09-17T15:39:39.142 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70325.log 2024-09-17T15:39:39.142 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48810.log.gz 2024-09-17T15:39:39.142 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55743.log 2024-09-17T15:39:39.143 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47046.log.gz 2024-09-17T15:39:39.143 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.70325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53641.log 2024-09-17T15:39:39.143 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70325.log.gz 2024-09-17T15:39:39.143 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59400.log 2024-09-17T15:39:39.143 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55743.log.gz 2024-09-17T15:39:39.143 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54271.log 2024-09-17T15:39:39.143 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53641.log.gz 2024-09-17T15:39:39.143 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.59400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46542.log 2024-09-17T15:39:39.143 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59400.log.gz 2024-09-17T15:39:39.143 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59946.log 2024-09-17T15:39:39.143 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54271.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54271.log.gz 2024-09-17T15:39:39.143 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51918.log 2024-09-17T15:39:39.143 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46542.log.gz 2024-09-17T15:39:39.144 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.59946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54523.log 2024-09-17T15:39:39.144 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59946.log.gz 2024-09-17T15:39:39.144 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38135.log 2024-09-17T15:39:39.144 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51918.log.gz 2024-09-17T15:39:39.144 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49104.log 2024-09-17T15:39:39.144 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54523.log.gz 2024-09-17T15:39:39.144 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67216.log 2024-09-17T15:39:39.144 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38431.log 2024-09-17T15:39:39.144 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49104.log.gz 2024-09-17T15:39:39.144 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.67216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47004.log 2024-09-17T15:39:39.144 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67216.log.gz 2024-09-17T15:39:39.144 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49650.log 2024-09-17T15:39:39.144 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66754.log 2024-09-17T15:39:39.145 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47004.log.gz 2024-09-17T15:39:39.145 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69611.log 2024-09-17T15:39:39.145 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49650.log.gz 2024-09-17T15:39:39.145 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58686.log 2024-09-17T15:39:39.145 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.66754.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66754.log.gz 2024-09-17T15:39:39.145 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51372.log 2024-09-17T15:39:39.145 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69611.log.gz 2024-09-17T15:39:39.145 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.58686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34837.log 2024-09-17T15:39:39.145 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58686.log.gz 2024-09-17T15:39:39.145 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61713.log 2024-09-17T15:39:39.145 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51372.log.gz 2024-09-17T15:39:39.145 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37654.log 2024-09-17T15:39:39.145 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34837.log.gz 2024-09-17T15:39:39.146 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.61713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59358.log 2024-09-17T15:39:39.146 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61713.log.gz 2024-09-17T15:39:39.146 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59064.log 2024-09-17T15:39:39.146 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.59358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52717.log 2024-09-17T15:39:39.146 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59358.log.gz 2024-09-17T15:39:39.146 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.59064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38172.log 2024-09-17T15:39:39.146 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59064.log.gz 2024-09-17T15:39:39.146 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.52717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57969.log 2024-09-17T15:39:39.146 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52717.log.gz 2024-09-17T15:39:39.146 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50490.log 2024-09-17T15:39:39.146 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.57969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59610.log 2024-09-17T15:39:39.146 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57969.log.gz 2024-09-17T15:39:39.146 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62554.log 2024-09-17T15:39:39.146 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50490.log.gz 2024-09-17T15:39:39.147 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.59610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64864.log 2024-09-17T15:39:39.147 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59610.log.gz 2024-09-17T15:39:39.147 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.62554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70199.log 2024-09-17T15:39:39.147 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62554.log.gz 2024-09-17T15:39:39.147 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.64864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69905.log 2024-09-17T15:39:39.147 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64864.log.gz 2024-09-17T15:39:39.147 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.70199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47298.log 2024-09-17T15:39:39.147 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70199.log.gz 2024-09-17T15:39:39.147 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56961.log 2024-09-17T15:39:39.147 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69905.log.gz 2024-09-17T15:39:39.147 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37913.log 2024-09-17T15:39:39.147 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47298.log.gz 2024-09-17T15:39:39.147 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56961.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35389.log.gz 2024-09-17T15:39:39.148 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39785.log 2024-09-17T15:39:39.148 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56961.log.gz 2024-09-17T15:39:39.148 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43847.log 2024-09-17T15:39:39.148 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68098.log 2024-09-17T15:39:39.148 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70283.log 2024-09-17T15:39:39.148 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43847.log.gz 2024-09-17T15:39:39.148 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49188.log 2024-09-17T15:39:39.148 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.68098.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68098.log.gz 2024-09-17T15:39:39.148 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35683.log.gz 2024-09-17T15:39:39.148 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.70283.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36756.log.gz 2024-09-17T15:39:39.148 INFO:teuthology.orchestra.run.smithi083.stderr:gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39785.log.gz -5 2024-09-17T15:39:39.148 INFO:teuthology.orchestra.run.smithi083.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48264.log 2024-09-17T15:39:39.148 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38246.log.gz 2024-09-17T15:39:39.148 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37728.log.gz 2024-09-17T15:39:39.149 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39701.log.gz 2024-09-17T15:39:39.149 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37654.log.gz 2024-09-17T15:39:39.149 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38172.log.gz -- replaced with /var/log/ceph/ceph-client.admin.37913.log.gz 2024-09-17T15:39:39.149 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-09-17T15:39:39.149 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38135.log.gz 2024-09-17T15:39:39.149 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38431.log.gz 2024-09-17T15:39:39.149 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70283.log.gz 2024-09-17T15:39:39.149 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67846.log 2024-09-17T15:39:39.149 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49188.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49188.log.gz 2024-09-17T15:39:39.150 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56499.log 2024-09-17T15:39:39.150 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48264.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48264.log.gz 2024-09-17T15:39:39.150 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.67846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63436.log 2024-09-17T15:39:39.150 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67846.log.gz 2024-09-17T15:39:39.150 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66796.log 2024-09-17T15:39:39.150 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56499.log.gz 2024-09-17T15:39:39.150 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.63436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44587.log 2024-09-17T15:39:39.150 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63436.log.gz 2024-09-17T15:39:39.150 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.66796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67804.log 2024-09-17T15:39:39.150 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66796.log.gz 2024-09-17T15:39:39.150 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63940.log 2024-09-17T15:39:39.150 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44587.log.gz 2024-09-17T15:39:39.150 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.67804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36534.log 2024-09-17T15:39:39.150 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67804.log.gz 2024-09-17T15:39:39.151 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.63940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63100.log 2024-09-17T15:39:39.151 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63940.log.gz 2024-09-17T15:39:39.151 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65326.log 2024-09-17T15:39:39.151 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36534.log.gz 2024-09-17T15:39:39.151 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.63100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60494.log 2024-09-17T15:39:39.151 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63100.log.gz 2024-09-17T15:39:39.151 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.65326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37200.log 2024-09-17T15:39:39.151 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65326.log.gz 2024-09-17T15:39:39.151 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.60494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65872.log 2024-09-17T15:39:39.151 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60494.log.gz 2024-09-17T15:39:39.151 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61587.log 2024-09-17T15:39:39.151 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37200.log.gz 2024-09-17T15:39:39.151 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.65872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60115.log 2024-09-17T15:39:39.151 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65872.log.gz 2024-09-17T15:39:39.152 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.61587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43884.log 2024-09-17T15:39:39.152 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61587.log.gz 2024-09-17T15:39:39.152 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.60115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50154.log 2024-09-17T15:39:39.152 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60115.log.gz 2024-09-17T15:39:39.152 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62302.log 2024-09-17T15:39:39.152 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43884.log.gz 2024-09-17T15:39:39.152 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70829.log 2024-09-17T15:39:39.152 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50154.log.gz 2024-09-17T15:39:39.152 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.62302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42883.log 2024-09-17T15:39:39.152 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62302.log.gz 2024-09-17T15:39:39.152 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.70829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46794.log 2024-09-17T15:39:39.152 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70829.log.gz 2024-09-17T15:39:39.152 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42883.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-17T15:39:39.153 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42883.log.gz 2024-09-17T15:39:39.153 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56877.log 2024-09-17T15:39:39.153 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46794.log.gz 2024-09-17T15:39:39.153 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64444.log 2024-09-17T15:39:39.153 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42337.log 2024-09-17T15:39:39.153 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56877.log.gz 2024-09-17T15:39:39.153 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.64444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32211.log 2024-09-17T15:39:39.153 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64444.log.gz 2024-09-17T15:39:39.153 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52044.log 2024-09-17T15:39:39.153 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42337.log.gz 2024-09-17T15:39:39.153 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.32211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46920.log 2024-09-17T15:39:39.153 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32211.log.gz 2024-09-17T15:39:39.153 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.52044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32145.log 2024-09-17T15:39:39.154 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52044.log.gz 2024-09-17T15:39:39.154 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39616.log 2024-09-17T15:39:39.154 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46920.log.gz 2024-09-17T15:39:39.154 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.32145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53137.log 2024-09-17T15:39:39.154 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32145.log.gz 2024-09-17T15:39:39.154 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42505.log 2024-09-17T15:39:39.154 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39616.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39616.log.gz 2024-09-17T15:39:39.154 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70703.log 2024-09-17T15:39:39.154 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53137.log.gz 2024-09-17T15:39:39.154 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47760.log 2024-09-17T15:39:39.154 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42505.log.gz 2024-09-17T15:39:39.154 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.70703.log: /var/log/ceph/ceph-client.admin.63814.log 2024-09-17T15:39:39.154 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70703.log.gz 2024-09-17T15:39:39.154 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35302.log 2024-09-17T15:39:39.155 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47760.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47760.log.gz 2024-09-17T15:39:39.155 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.63814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53389.log 2024-09-17T15:39:39.155 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63814.log.gz 2024-09-17T15:39:39.155 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38024.log 2024-09-17T15:39:39.155 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35302.log.gz 2024-09-17T15:39:39.155 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50700.log 2024-09-17T15:39:39.155 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53389.log.gz 2024-09-17T15:39:39.155 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44680.log 2024-09-17T15:39:39.155 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38024.log.gz 2024-09-17T15:39:39.155 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63268.log 2024-09-17T15:39:39.155 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50700.log.gz 2024-09-17T15:39:39.155 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42253.log 2024-09-17T15:39:39.155 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44680.log.gz 2024-09-17T15:39:39.155 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.63268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61209.log 2024-09-17T15:39:39.156 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63268.log.gz 2024-09-17T15:39:39.156 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49230.log 2024-09-17T15:39:39.156 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42253.log.gz 2024-09-17T15:39:39.156 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.61209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60073.log 2024-09-17T15:39:39.156 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61209.log.gz 2024-09-17T15:39:39.156 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68434.log 2024-09-17T15:39:39.156 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49230.log.gz 2024-09-17T15:39:39.156 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.60073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67972.log 2024-09-17T15:39:39.156 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60073.log.gz 2024-09-17T15:39:39.156 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.68434.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68434.log.gz 2024-09-17T15:39:39.156 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54355.log 2024-09-17T15:39:39.156 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.67972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69947.log 2024-09-17T15:39:39.156 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67972.log.gz 2024-09-17T15:39:39.157 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46836.log 2024-09-17T15:39:39.157 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54355.log.gz 2024-09-17T15:39:39.157 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56625.log 2024-09-17T15:39:39.157 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69947.log.gz 2024-09-17T15:39:39.157 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46836.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46836.log.gz 2024-09-17T15:39:39.157 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41917.log 2024-09-17T15:39:39.157 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41501.log 2024-09-17T15:39:39.157 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56625.log.gz 2024-09-17T15:39:39.157 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44069.log 2024-09-17T15:39:39.157 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41917.log.gz 2024-09-17T15:39:39.157 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36163.log 2024-09-17T15:39:39.157 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41501.log.gz 2024-09-17T15:39:39.157 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64696.log 2024-09-17T15:39:39.157 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44069.log.gz 2024-09-17T15:39:39.158 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43051.log 2024-09-17T15:39:39.158 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36163.log.gz 2024-09-17T15:39:39.158 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.64696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51162.log 2024-09-17T15:39:39.158 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64696.log.gz 2024-09-17T15:39:39.158 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52170.log 2024-09-17T15:39:39.158 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43051.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43051.log.gz 2024-09-17T15:39:39.158 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42799.log 2024-09-17T15:39:39.158 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51162.log.gz 2024-09-17T15:39:39.158 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.52170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47088.log 2024-09-17T15:39:39.158 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52170.log.gz 2024-09-17T15:39:39.158 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41818.log 2024-09-17T15:39:39.158 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42799.log.gz 2024-09-17T15:39:39.158 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52423.log 2024-09-17T15:39:39.158 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47088.log.gz 2024-09-17T15:39:39.159 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54229.log 2024-09-17T15:39:39.159 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.52423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65410.log 2024-09-17T15:39:39.159 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52423.log.gz 2024-09-17T15:39:39.159 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68014.log 2024-09-17T15:39:39.159 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54229.log.gz 2024-09-17T15:39:39.159 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.65410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50070.log 2024-09-17T15:39:39.159 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65410.log.gz 2024-09-17T15:39:39.159 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.68014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60284.log 2024-09-17T15:39:39.159 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68014.log.gz 2024-09-17T15:39:39.159 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63982.log 2024-09-17T15:39:39.159 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50070.log.gz 2024-09-17T15:39:39.159 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.60284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55449.log 2024-09-17T15:39:39.159 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60284.log.gz 2024-09-17T15:39:39.160 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.63982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51960.log 2024-09-17T15:39:39.160 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63982.log.gz 2024-09-17T15:39:39.160 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55449.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.55449.log.gz 2024-09-17T15:39:39.160 INFO:teuthology.orchestra.run.smithi083.stderr: /var/log/ceph/ceph-client.admin.58854.log 2024-09-17T15:39:39.160 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46626.log 2024-09-17T15:39:39.160 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51960.log.gz 2024-09-17T15:39:39.160 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.58854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55575.log 2024-09-17T15:39:39.160 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58854.log.gz 2024-09-17T15:39:39.160 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69779.log 2024-09-17T15:39:39.160 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46626.log.gz 2024-09-17T15:39:39.160 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69148.log 2024-09-17T15:39:39.160 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55575.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55575.log.gz 2024-09-17T15:39:39.160 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63352.log 2024-09-17T15:39:39.160 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69779.log.gz 2024-09-17T15:39:39.161 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68896.log 2024-09-17T15:39:39.161 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69148.log.gz 2024-09-17T15:39:39.161 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.63352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67426.log 2024-09-17T15:39:39.161 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63352.log.gz 2024-09-17T15:39:39.161 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.68896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48390.log 2024-09-17T15:39:39.161 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68896.log.gz 2024-09-17T15:39:39.161 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64024.log 2024-09-17T15:39:39.161 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.67426.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67426.log.gz 2024-09-17T15:39:39.161 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66502.log 2024-09-17T15:39:39.161 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48390.log.gz 2024-09-17T15:39:39.161 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.64024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59988.log 2024-09-17T15:39:39.161 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64024.log.gz 2024-09-17T15:39:39.161 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.66502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43921.log 2024-09-17T15:39:39.162 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66502.log.gz 2024-09-17T15:39:39.162 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.59988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46500.log 2024-09-17T15:39:39.162 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59988.log.gz 2024-09-17T15:39:39.162 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57465.log 2024-09-17T15:39:39.162 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43921.log.gz 2024-09-17T15:39:39.162 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46500.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.69358.log 2024-09-17T15:39:39.162 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46500.log.gz 2024-09-17T15:39:39.162 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.57465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31847.log 2024-09-17T15:39:39.162 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57465.log.gz 2024-09-17T15:39:39.162 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37052.log 2024-09-17T15:39:39.162 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69358.log.gz 2024-09-17T15:39:39.162 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.31847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62176.log 2024-09-17T15:39:39.162 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31847.log.gz 2024-09-17T15:39:39.162 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67552.log 2024-09-17T15:39:39.163 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37052.log.gz 2024-09-17T15:39:39.163 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.62176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38357.log 2024-09-17T15:39:39.163 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62176.log.gz 2024-09-17T15:39:39.163 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.67552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67342.log 2024-09-17T15:39:39.163 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67552.log.gz 2024-09-17T15:39:39.163 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61377.log 2024-09-17T15:39:39.163 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38357.log.gz 2024-09-17T15:39:39.163 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.67342.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67342.log.gz 2024-09-17T15:39:39.163 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31966.log 2024-09-17T15:39:39.163 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.61377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54649.log 2024-09-17T15:39:39.163 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61377.log.gz 2024-09-17T15:39:39.163 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.31966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55785.log 2024-09-17T15:39:39.163 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31966.log.gz 2024-09-17T15:39:39.163 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58350.log 2024-09-17T15:39:39.164 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54649.log.gz 2024-09-17T15:39:39.164 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32006.log 2024-09-17T15:39:39.164 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55785.log.gz 2024-09-17T15:39:39.164 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.58350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64990.log 2024-09-17T15:39:39.164 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58350.log.gz 2024-09-17T15:39:39.164 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.32006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55281.log 2024-09-17T15:39:39.164 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32006.log.gz 2024-09-17T15:39:39.164 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.64990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56289.log 2024-09-17T15:39:39.164 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64990.log.gz 2024-09-17T15:39:39.164 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48474.log 2024-09-17T15:39:39.164 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55281.log.gz 2024-09-17T15:39:39.164 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36497.log 2024-09-17T15:39:39.164 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56289.log.gz 2024-09-17T15:39:39.165 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54019.log 2024-09-17T15:39:39.165 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48474.log.gz 2024-09-17T15:39:39.165 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60873.log 2024-09-17T15:39:39.165 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36497.log.gz 2024-09-17T15:39:39.165 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48936.log 2024-09-17T15:39:39.165 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54019.log.gz 2024-09-17T15:39:39.165 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.60873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34969.log 2024-09-17T15:39:39.165 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60873.log.gz 2024-09-17T15:39:39.165 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52507.log 2024-09-17T15:39:39.165 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48936.log.gz 2024-09-17T15:39:39.165 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47676.log 2024-09-17T15:39:39.165 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34969.log.gz 2024-09-17T15:39:39.165 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.52507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39513.log 2024-09-17T15:39:39.165 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52507.log.gz 2024-09-17T15:39:39.166 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45702.log 2024-09-17T15:39:39.166 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47676.log.gz 2024-09-17T15:39:39.166 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49860.log 2024-09-17T15:39:39.166 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39513.log.gz 2024-09-17T15:39:39.166 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62512.log 2024-09-17T15:39:39.166 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45702.log.gz 2024-09-17T15:39:39.166 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60158.log 2024-09-17T15:39:39.166 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49860.log.gz 2024-09-17T15:39:39.166 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.62512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58560.log 2024-09-17T15:39:39.166 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62512.log.gz 2024-09-17T15:39:39.166 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.60158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35641.log 2024-09-17T15:39:39.166 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60158.log.gz 2024-09-17T15:39:39.166 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.58560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57423.log 2024-09-17T15:39:39.167 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58560.log.gz 2024-09-17T15:39:39.167 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70745.log 2024-09-17T15:39:39.167 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35641.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35641.log.gz 2024-09-17T15:39:39.167 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.57423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49902.log 2024-09-17T15:39:39.167 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57423.log.gz 2024-09-17T15:39:39.167 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.70745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53767.log 2024-09-17T15:39:39.167 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70745.log.gz 2024-09-17T15:39:39.167 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66586.log 2024-09-17T15:39:39.167 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49902.log.gz 2024-09-17T15:39:39.167 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67258.log 2024-09-17T15:39:39.167 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53767.log.gz 2024-09-17T15:39:39.167 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.66586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63772.log 2024-09-17T15:39:39.167 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66586.log.gz 2024-09-17T15:39:39.168 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.67258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51708.log 2024-09-17T15:39:39.168 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67258.log.gz 2024-09-17T15:39:39.168 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.63772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45744.log 2024-09-17T15:39:39.168 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63772.log.gz 2024-09-17T15:39:39.168 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51708.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51708.log.gz 2024-09-17T15:39:39.168 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66376.log 2024-09-17T15:39:39.168 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39784.log 2024-09-17T15:39:39.168 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45744.log.gz 2024-09-17T15:39:39.168 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.66376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58182.log 2024-09-17T15:39:39.168 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66376.log.gz 2024-09-17T15:39:39.168 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47130.log 2024-09-17T15:39:39.168 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39784.log.gz 2024-09-17T15:39:39.169 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.58182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51750.log 2024-09-17T15:39:39.169 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58182.log.gz 2024-09-17T15:39:39.169 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49944.log 2024-09-17T15:39:39.169 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47130.log.gz 2024-09-17T15:39:39.169 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54691.log 2024-09-17T15:39:39.169 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51750.log.gz 2024-09-17T15:39:39.169 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50826.log 2024-09-17T15:39:39.169 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49944.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49944.log.gz 2024-09-17T15:39:39.169 INFO:teuthology.orchestra.run.smithi083.stderr:gzip/var/log/ceph/ceph-client.admin.54691.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.54145.log 2024-09-17T15:39:39.169 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54691.log.gz 2024-09-17T15:39:39.169 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38505.log 2024-09-17T15:39:39.169 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50826.log.gz 2024-09-17T15:39:39.169 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63058.log 2024-09-17T15:39:39.169 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54145.log.gz 2024-09-17T15:39:39.170 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41562.log 2024-09-17T15:39:39.170 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38505.log.gz 2024-09-17T15:39:39.170 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.63058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46668.log 2024-09-17T15:39:39.170 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63058.log.gz 2024-09-17T15:39:39.170 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61797.log 2024-09-17T15:39:39.170 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41562.log.gz 2024-09-17T15:39:39.170 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47172.log 2024-09-17T15:39:39.170 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46668.log.gz 2024-09-17T15:39:39.170 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.61797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48054.log 2024-09-17T15:39:39.170 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61797.log.gz 2024-09-17T15:39:39.170 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69316.log 2024-09-17T15:39:39.170 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47172.log.gz 2024-09-17T15:39:39.170 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37274.log 2024-09-17T15:39:39.170 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48054.log.gz 2024-09-17T15:39:39.171 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65116.log 2024-09-17T15:39:39.171 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69316.log.gz 2024-09-17T15:39:39.171 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45660.log 2024-09-17T15:39:39.171 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37274.log.gz 2024-09-17T15:39:39.171 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.65116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41143.log 2024-09-17T15:39:39.171 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65116.log.gz 2024-09-17T15:39:39.171 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53725.log 2024-09-17T15:39:39.171 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45660.log.gz 2024-09-17T15:39:39.171 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46038.log 2024-09-17T15:39:39.171 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41143.log.gz 2024-09-17T15:39:39.171 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46458.log 2024-09-17T15:39:39.171 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53725.log.gz 2024-09-17T15:39:39.171 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52885.log 2024-09-17T15:39:39.172 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46038.log.gz 2024-09-17T15:39:39.172 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69274.log 2024-09-17T15:39:39.172 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46458.log.gz 2024-09-17T15:39:39.172 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.52885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59274.log 0.0% 2024-09-17T15:39:39.172 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52885.log.gz 2024-09-17T15:39:39.172 INFO:teuthology.orchestra.run.smithi083.stderr: 95.9% -- replaced with /var/log/ceph/ceph-client.admin.41818.log.gz 2024-09-17T15:39:39.172 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69274.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.54187.log -- replaced with /var/log/ceph/ceph-client.admin.69274.log.gz 2024-09-17T15:39:39.172 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-09-17T15:39:39.172 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.59274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36079.log 2024-09-17T15:39:39.172 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59274.log.gz 2024-09-17T15:39:39.172 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36571.log 2024-09-17T15:39:39.172 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54187.log.gz 2024-09-17T15:39:39.172 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48894.log 2024-09-17T15:39:39.172 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36079.log.gz 2024-09-17T15:39:39.173 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36003.log 2024-09-17T15:39:39.173 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36571.log.gz 2024-09-17T15:39:39.173 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47340.log 2024-09-17T15:39:39.173 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48894.log.gz 2024-09-17T15:39:39.173 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49146.log 2024-09-17T15:39:39.173 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36003.log.gz 2024-09-17T15:39:39.173 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67048.log 2024-09-17T15:39:39.173 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47340.log.gz 2024-09-17T15:39:39.173 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53515.log 2024-09-17T15:39:39.173 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49146.log.gz 2024-09-17T15:39:39.173 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.67048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50784.log 2024-09-17T15:39:39.173 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67048.log.gz 2024-09-17T15:39:39.173 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65662.log 2024-09-17T15:39:39.174 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53515.log.gz 2024-09-17T15:39:39.174 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.50784.log: /var/log/ceph/ceph-client.admin.44291.log 2024-09-17T15:39:39.174 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50784.log.gz 2024-09-17T15:39:39.174 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.65662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49608.log 2024-09-17T15:39:39.174 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65662.log.gz 2024-09-17T15:39:39.174 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35725.log 2024-09-17T15:39:39.174 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44291.log.gz 2024-09-17T15:39:39.174 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49692.log 2024-09-17T15:39:39.174 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49608.log.gz 2024-09-17T15:39:39.174 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68392.log 2024-09-17T15:39:39.174 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35725.log.gz 2024-09-17T15:39:39.174 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50280.log 2024-09-17T15:39:39.174 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49692.log.gz 2024-09-17T15:39:39.175 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.68392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62680.log 2024-09-17T15:39:39.175 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68392.log.gz 2024-09-17T15:39:39.175 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44513.log 2024-09-17T15:39:39.175 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50280.log.gz 2024-09-17T15:39:39.175 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.62680.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62680.log.gz 2024-09-17T15:39:39.175 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58602.log 2024-09-17T15:39:39.175 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37520.log 2024-09-17T15:39:39.175 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44513.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44513.log.gz 2024-09-17T15:39:39.175 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.58602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65032.log 2024-09-17T15:39:39.175 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58602.log.gz 2024-09-17T15:39:39.175 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35011.log 2024-09-17T15:39:39.175 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37520.log.gz 2024-09-17T15:39:39.175 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.65032.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.65032.log.gz 2024-09-17T15:39:39.175 INFO:teuthology.orchestra.run.smithi083.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.60789.log 2024-09-17T15:39:39.176 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59820.log 2024-09-17T15:39:39.176 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35011.log.gz 2024-09-17T15:39:39.176 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.60789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49776.log 2024-09-17T15:39:39.176 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60789.log.gz 2024-09-17T15:39:39.176 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.59820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51456.log 2024-09-17T15:39:39.176 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59820.log.gz 2024-09-17T15:39:39.176 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39933.log 2024-09-17T15:39:39.176 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49776.log.gz 2024-09-17T15:39:39.176 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48558.log 2024-09-17T15:39:39.176 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51456.log.gz 2024-09-17T15:39:39.176 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37422.log 2024-09-17T15:39:39.176 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39933.log.gz 2024-09-17T15:39:39.176 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46374.log 2024-09-17T15:39:39.177 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48558.log.gz 2024-09-17T15:39:39.177 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64402.log 2024-09-17T15:39:39.177 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37422.log.gz 2024-09-17T15:39:39.177 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60831.log 2024-09-17T15:39:39.177 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46374.log.gz 2024-09-17T15:39:39.177 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.64402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49524.log 2024-09-17T15:39:39.177 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64402.log.gz 2024-09-17T15:39:39.177 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48096.log 2024-09-17T15:39:39.177 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.60831.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60831.log.gz 2024-09-17T15:39:39.177 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53347.log 2024-09-17T15:39:39.177 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49524.log.gz 2024-09-17T15:39:39.177 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68560.log 2024-09-17T15:39:39.177 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48096.log.gz 2024-09-17T15:39:39.177 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65200.log 2024-09-17T15:39:39.178 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53347.log.gz 2024-09-17T15:39:39.178 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.68560.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.68560.log.gz 2024-09-17T15:39:39.178 INFO:teuthology.orchestra.run.smithi083.stderr: /var/log/ceph/ceph-client.admin.37987.log 2024-09-17T15:39:39.178 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.65200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63520.log 2024-09-17T15:39:39.178 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65200.log.gz 2024-09-17T15:39:39.178 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53851.log 2024-09-17T15:39:39.178 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37987.log.gz 2024-09-17T15:39:39.178 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.63520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69569.log 2024-09-17T15:39:39.178 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63520.log.gz 2024-09-17T15:39:39.178 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53851.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53851.log.gzgzip 2024-09-17T15:39:39.178 INFO:teuthology.orchestra.run.smithi083.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.57087.log 2024-09-17T15:39:39.178 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47214.log 2024-09-17T15:39:39.178 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69569.log.gz 2024-09-17T15:39:39.179 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.57087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57003.log 2024-09-17T15:39:39.179 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57087.log.gz 2024-09-17T15:39:39.179 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47214.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-17T15:39:39.179 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47214.log.gz 2024-09-17T15:39:39.179 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.57003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36941.log 2024-09-17T15:39:39.179 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57003.log.gz 2024-09-17T15:39:39.179 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62722.log 2024-09-17T15:39:39.179 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66460.log 2024-09-17T15:39:39.179 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36941.log.gz 2024-09-17T15:39:39.179 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50868.log 2024-09-17T15:39:39.179 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.62722.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62722.log.gz 2024-09-17T15:39:39.179 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.66460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55701.log 2024-09-17T15:39:39.179 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66460.log.gz 2024-09-17T15:39:39.179 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67636.log 2024-09-17T15:39:39.180 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50868.log.gz 2024-09-17T15:39:39.180 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52843.log 2024-09-17T15:39:39.180 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55701.log.gz 2024-09-17T15:39:39.180 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.67636.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.67636.log.gz 2024-09-17T15:39:39.180 INFO:teuthology.orchestra.run.smithi083.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.51078.log 2024-09-17T15:39:39.180 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.52843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60957.log 2024-09-17T15:39:39.180 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52843.log.gz 2024-09-17T15:39:39.180 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35303.log 2024-09-17T15:39:39.180 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51078.log.gz 2024-09-17T15:39:39.180 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.60957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68476.log 2024-09-17T15:39:39.180 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60957.log.gz 2024-09-17T15:39:39.180 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41227.log 2024-09-17T15:39:39.180 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35303.log.gz 2024-09-17T15:39:39.181 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.68476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67510.log 2024-09-17T15:39:39.181 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68476.log.gz 2024-09-17T15:39:39.181 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53179.log 2024-09-17T15:39:39.181 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41227.log.gz 2024-09-17T15:39:39.181 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.67510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39975.log 2024-09-17T15:39:39.181 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67510.log.gz 2024-09-17T15:39:39.181 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69400.log 2024-09-17T15:39:39.181 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53179.log.gz 2024-09-17T15:39:39.181 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38394.log 2024-09-17T15:39:39.181 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39975.log.gz 2024-09-17T15:39:39.181 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.69400.log: /var/log/ceph/ceph-client.admin.58224.log 2024-09-17T15:39:39.181 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69400.log.gz 2024-09-17T15:39:39.181 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68056.log 2024-09-17T15:39:39.182 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38394.log.gz 2024-09-17T15:39:39.182 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.58224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58938.log 2024-09-17T15:39:39.182 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58224.log.gz 2024-09-17T15:39:39.182 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.68056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68308.log 2024-09-17T15:39:39.182 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68056.log.gz 2024-09-17T15:39:39.182 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.58938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61755.log 2024-09-17T15:39:39.182 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58938.log.gz 2024-09-17T15:39:39.182 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.68308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53599.log 2024-09-17T15:39:39.182 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68308.log.gz 2024-09-17T15:39:39.182 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.61755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47970.log 2024-09-17T15:39:39.182 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61755.log.gz 2024-09-17T15:39:39.182 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53599.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53599.log.gz 2024-09-17T15:39:39.182 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52254.log 2024-09-17T15:39:39.182 INFO:teuthology.orchestra.run.smithi083.stderr:gzip/var/log/ceph/ceph-client.admin.47970.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.56751.log 2024-09-17T15:39:39.183 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47970.log.gz 2024-09-17T15:39:39.183 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.52254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61293.log 2024-09-17T15:39:39.183 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52254.log.gz 2024-09-17T15:39:39.183 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59022.log 2024-09-17T15:39:39.183 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56751.log.gz 2024-09-17T15:39:39.183 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.61293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51036.log 2024-09-17T15:39:39.183 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61293.log.gz 2024-09-17T15:39:39.183 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.59022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57633.log 2024-09-17T15:39:39.183 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59022.log.gz 2024-09-17T15:39:39.183 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49314.log 2024-09-17T15:39:39.183 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51036.log.gz 2024-09-17T15:39:39.183 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.57633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63646.log 2024-09-17T15:39:39.183 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57633.log.gz 2024-09-17T15:39:39.183 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56667.log 2024-09-17T15:39:39.184 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49314.log.gz 2024-09-17T15:39:39.184 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.63646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55069.log 2024-09-17T15:39:39.184 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63646.log.gz 2024-09-17T15:39:39.184 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70031.log 2024-09-17T15:39:39.184 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56667.log.gz 2024-09-17T15:39:39.184 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42421.log 2024-09-17T15:39:39.184 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55069.log.gz 2024-09-17T15:39:39.184 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.70031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61503.log 2024-09-17T15:39:39.184 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70031.log.gz 2024-09-17T15:39:39.184 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41563.log 2024-09-17T15:39:39.184 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42421.log.gz 2024-09-17T15:39:39.184 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.61503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64066.log 2024-09-17T15:39:39.184 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61503.log.gz 2024-09-17T15:39:39.184 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66964.log 2024-09-17T15:39:39.185 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41563.log.gz 2024-09-17T15:39:39.185 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.64066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56541.log 2024-09-17T15:39:39.185 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64066.log.gz 2024-09-17T15:39:39.185 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.66964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61083.log 2024-09-17T15:39:39.185 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66964.log.gz 2024-09-17T15:39:39.185 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53893.log 2024-09-17T15:39:39.185 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56541.log.gz 2024-09-17T15:39:39.185 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.61083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65704.log 2024-09-17T15:39:39.185 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61083.log.gz 2024-09-17T15:39:39.185 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50364.log 2024-09-17T15:39:39.185 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53893.log.gz 2024-09-17T15:39:39.185 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.65704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62386.log 2024-09-17T15:39:39.185 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65704.log.gz 2024-09-17T15:39:39.186 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54901.log 2024-09-17T15:39:39.186 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50364.log.gz 2024-09-17T15:39:39.186 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.62386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56457.log 2024-09-17T15:39:39.186 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62386.log.gz 2024-09-17T15:39:39.186 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61671.log 2024-09-17T15:39:39.186 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54901.log.gz 2024-09-17T15:39:39.186 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56205.log 2024-09-17T15:39:39.186 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56457.log.gz 2024-09-17T15:39:39.186 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.61671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58392.log 2024-09-17T15:39:39.186 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61671.log.gz 2024-09-17T15:39:39.186 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57591.log 2024-09-17T15:39:39.186 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56205.log.gz 2024-09-17T15:39:39.186 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.58392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42169.log 2024-09-17T15:39:39.187 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58392.log.gz 2024-09-17T15:39:39.187 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.57591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58011.log 2024-09-17T15:39:39.187 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57591.log.gz 2024-09-17T15:39:39.187 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49734.log 2024-09-17T15:39:39.187 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42169.log.gz 2024-09-17T15:39:39.187 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.58011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65578.log 2024-09-17T15:39:39.187 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58011.log.gz 2024-09-17T15:39:39.187 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57381.log 2024-09-17T15:39:39.187 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49734.log.gz 2024-09-17T15:39:39.187 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.65578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37839.log 2024-09-17T15:39:39.187 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65578.log.gz 2024-09-17T15:39:39.187 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.57381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55659.log 2024-09-17T15:39:39.187 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57381.log.gz 2024-09-17T15:39:39.187 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36867.log 2024-09-17T15:39:39.188 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37839.log.gz 2024-09-17T15:39:39.188 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37543.log 2024-09-17T15:39:39.188 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55659.log.gz 2024-09-17T15:39:39.188 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58980.log 2024-09-17T15:39:39.188 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36867.log.gz 2024-09-17T15:39:39.188 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43281.log 2024-09-17T15:39:39.188 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37543.log.gz 2024-09-17T15:39:39.188 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.58980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53473.log 2024-09-17T15:39:39.188 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58980.log.gz 2024-09-17T15:39:39.188 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42841.log 2024-09-17T15:39:39.188 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43281.log.gz 2024-09-17T15:39:39.188 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68686.log 2024-09-17T15:39:39.188 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53473.log.gz 2024-09-17T15:39:39.189 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51834.log 2024-09-17T15:39:39.189 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42841.log.gz 2024-09-17T15:39:39.189 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.68686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48642.log 2024-09-17T15:39:39.189 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68686.log.gz 2024-09-17T15:39:39.189 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51834.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51834.log.gz 2024-09-17T15:39:39.189 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42631.log 2024-09-17T15:39:39.189 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48138.log 2024-09-17T15:39:39.189 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48642.log.gz 2024-09-17T15:39:39.189 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46206.log 2024-09-17T15:39:39.189 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42631.log.gz 2024-09-17T15:39:39.189 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36078.log 2024-09-17T15:39:39.189 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48138.log.gz 2024-09-17T15:39:39.189 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46206.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46206.log.gz 2024-09-17T15:39:39.189 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68518.log 2024-09-17T15:39:39.190 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60200.log 2024-09-17T15:39:39.190 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36078.log.gz 2024-09-17T15:39:39.190 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.68518.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68518.log.gz 2024-09-17T15:39:39.190 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47844.log 2024-09-17T15:39:39.190 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.60200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54565.log 2024-09-17T15:39:39.190 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60200.log.gz 2024-09-17T15:39:39.190 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69821.log 2024-09-17T15:39:39.190 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47844.log.gz 2024-09-17T15:39:39.190 INFO:teuthology.orchestra.run.smithi083.stderr: 96.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-17T15:39:39.190 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42967.log 2024-09-17T15:39:39.190 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54565.log.gz 2024-09-17T15:39:39.190 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63184.log 2024-09-17T15:39:39.190 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69821.log.gz 2024-09-17T15:39:39.190 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57129.log 2024-09-17T15:39:39.191 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42967.log.gz 2024-09-17T15:39:39.191 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.63184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68854.log 2024-09-17T15:39:39.191 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63184.log.gz 2024-09-17T15:39:39.191 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.57129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58266.log 2024-09-17T15:39:39.191 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57129.log.gz 2024-09-17T15:39:39.191 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.68854.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68854.log.gz 2024-09-17T15:39:39.191 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43639.log 2024-09-17T15:39:39.191 INFO:teuthology.orchestra.run.smithi083.stderr:gzip/var/log/ceph/ceph-client.admin.58266.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.43093.log 2024-09-17T15:39:39.191 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58266.log.gz 2024-09-17T15:39:39.191 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66040.log 2024-09-17T15:39:39.191 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43639.log.gz 2024-09-17T15:39:39.191 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62008.log 2024-09-17T15:39:39.191 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43093.log.gz 2024-09-17T15:39:39.191 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.66040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47886.log 2024-09-17T15:39:39.192 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66040.log.gz 2024-09-17T15:39:39.192 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.62008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62344.log 2024-09-17T15:39:39.192 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62008.log.gz 2024-09-17T15:39:39.192 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48852.log 2024-09-17T15:39:39.192 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47886.log.gz 2024-09-17T15:39:39.192 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.62344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32046.log 2024-09-17T15:39:39.192 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62344.log.gz 2024-09-17T15:39:39.192 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52212.log 2024-09-17T15:39:39.192 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48852.log.gz 2024-09-17T15:39:39.192 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.32046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69064.log 2024-09-17T15:39:39.192 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32046.log.gz 2024-09-17T15:39:39.192 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.52212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61167.log 2024-09-17T15:39:39.192 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52212.log.gz 2024-09-17T15:39:39.193 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49440.log 2024-09-17T15:39:39.193 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69064.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69064.log.gz 2024-09-17T15:39:39.193 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.61167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35175.log 2024-09-17T15:39:39.193 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61167.log.gz 2024-09-17T15:39:39.193 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59568.log 2024-09-17T15:39:39.193 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49440.log.gz 2024-09-17T15:39:39.193 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65914.log 2024-09-17T15:39:39.193 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35175.log.gz 2024-09-17T15:39:39.193 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.59568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60578.log 2024-09-17T15:39:39.193 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59568.log.gz 2024-09-17T15:39:39.193 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.65914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62428.log 2024-09-17T15:39:39.193 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65914.log.gz 2024-09-17T15:39:39.193 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43699.log 2024-09-17T15:39:39.194 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.60578.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60578.log.gz 2024-09-17T15:39:39.194 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.62428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50532.log 2024-09-17T15:39:39.194 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62428.log.gz 2024-09-17T15:39:39.194 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37348.log 2024-09-17T15:39:39.194 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43699.log.gz 2024-09-17T15:39:39.194 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54481.log 2024-09-17T15:39:39.194 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50532.log.gz 2024-09-17T15:39:39.194 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52086.log 2024-09-17T15:39:39.194 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37348.log.gz 2024-09-17T15:39:39.194 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66670.log 2024-09-17T15:39:39.194 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54481.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54481.log.gz 2024-09-17T15:39:39.194 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.52086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37015.log 2024-09-17T15:39:39.194 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52086.log.gz 2024-09-17T15:39:39.194 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.66670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48516.log 2024-09-17T15:39:39.195 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66670.log.gz 2024-09-17T15:39:39.195 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35557.log 2024-09-17T15:39:39.195 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37015.log.gz 2024-09-17T15:39:39.195 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57297.log 2024-09-17T15:39:39.195 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48516.log.gz 2024-09-17T15:39:39.195 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48012.log 2024-09-17T15:39:39.195 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35557.log.gz 2024-09-17T15:39:39.195 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.57297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34758.log 2024-09-17T15:39:39.195 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57297.log.gz 2024-09-17T15:39:39.195 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52591.log 2024-09-17T15:39:39.195 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48012.log.gz 2024-09-17T15:39:39.195 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58476.log 2024-09-17T15:39:39.195 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34758.log.gz 2024-09-17T15:39:39.196 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.52591.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-17T15:39:39.196 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52591.log.gz 2024-09-17T15:39:39.196 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54943.log 2024-09-17T15:39:39.196 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.58476.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58476.log.gz 2024-09-17T15:39:39.196 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69022.log 2024-09-17T15:39:39.196 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58518.log 2024-09-17T15:39:39.196 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54943.log.gz 2024-09-17T15:39:39.196 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55533.log 2024-09-17T15:39:39.196 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69022.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69022.log.gz 2024-09-17T15:39:39.196 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43009.log 2024-09-17T15:39:39.196 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.58518.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58518.log.gz 2024-09-17T15:39:39.196 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34893.log 2024-09-17T15:39:39.196 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55533.log.gz 2024-09-17T15:39:39.197 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41059.log 2024-09-17T15:39:39.197 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43009.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43009.log.gz 2024-09-17T15:39:39.197 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54733.log 2024-09-17T15:39:39.197 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34893.log.gz 2024-09-17T15:39:39.197 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41059.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41059.log.gzgzip 2024-09-17T15:39:39.197 INFO:teuthology.orchestra.run.smithi083.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.60031.log 2024-09-17T15:39:39.197 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35851.log 2024-09-17T15:39:39.197 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54733.log.gz 2024-09-17T15:39:39.197 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44180.log 2024-09-17T15:39:39.197 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.60031.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60031.log.gz 2024-09-17T15:39:39.197 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65368.log 2024-09-17T15:39:39.197 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35851.log.gz 2024-09-17T15:39:39.197 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47256.log 2024-09-17T15:39:39.198 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44180.log.gz 2024-09-17T15:39:39.198 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.65368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69989.log 2024-09-17T15:39:39.198 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65368.log.gz 2024-09-17T15:39:39.198 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62932.log 2024-09-17T15:39:39.198 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47256.log.gz 2024-09-17T15:39:39.198 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64612.log 2024-09-17T15:39:39.198 INFO:teuthology.orchestra.run.smithi083.stderr: 26.7% -- replaced with /var/log/ceph/ceph-client.admin.69989.log.gz 2024-09-17T15:39:39.198 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.62932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66292.log 2024-09-17T15:39:39.198 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62932.log.gz 2024-09-17T15:39:39.198 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.64612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46878.log 2024-09-17T15:39:39.198 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64612.log.gz 2024-09-17T15:39:39.198 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.66292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43958.log 2024-09-17T15:39:39.198 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66292.log.gz 2024-09-17T15:39:39.198 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35809.log 2024-09-17T15:39:39.199 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46878.log.gz 2024-09-17T15:39:39.199 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50574.log 2024-09-17T15:39:39.199 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43958.log.gz 2024-09-17T15:39:39.199 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50238.log 2024-09-17T15:39:39.199 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35809.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35809.log.gz 2024-09-17T15:39:39.199 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55407.log 2024-09-17T15:39:39.199 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50574.log.gz 2024-09-17T15:39:39.199 INFO:teuthology.orchestra.run.smithi083.stderr:gzip/var/log/ceph/ceph-client.admin.50238.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.44217.log 2024-09-17T15:39:39.199 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50238.log.gz 2024-09-17T15:39:39.199 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44328.log 2024-09-17T15:39:39.199 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55407.log.gz 2024-09-17T15:39:39.199 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57255.log 2024-09-17T15:39:39.199 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44217.log.gz 2024-09-17T15:39:39.199 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60999.log 2024-09-17T15:39:39.200 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44328.log.gz 2024-09-17T15:39:39.200 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.57255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42085.log 2024-09-17T15:39:39.200 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57255.log.gz 2024-09-17T15:39:39.200 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.60999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65158.log 2024-09-17T15:39:39.200 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60999.log.gz 2024-09-17T15:39:39.200 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64192.log 2024-09-17T15:39:39.200 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42085.log.gz 2024-09-17T15:39:39.200 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.65158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54397.log 2024-09-17T15:39:39.200 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65158.log.gz 2024-09-17T15:39:39.200 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.64192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60326.log 2024-09-17T15:39:39.200 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64192.log.gz 2024-09-17T15:39:39.200 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36162.log 2024-09-17T15:39:39.200 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54397.log.gz 2024-09-17T15:39:39.201 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.60326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45828.log 2024-09-17T15:39:39.201 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60326.log.gz 2024-09-17T15:39:39.201 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36162.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.36162.log.gz --verbose 2024-09-17T15:39:39.201 INFO:teuthology.orchestra.run.smithi083.stderr: -- /var/log/ceph/ceph-client.admin.52759.log 2024-09-17T15:39:39.201 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53557.log 2024-09-17T15:39:39.201 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45828.log.gz 2024-09-17T15:39:39.201 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58095.log 2024-09-17T15:39:39.201 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.52759.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52759.log.gz 2024-09-17T15:39:39.201 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53011.log 2024-09-17T15:39:39.201 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53557.log.gz 2024-09-17T15:39:39.201 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.58095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47382.log 2024-09-17T15:39:39.201 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58095.log.gz 2024-09-17T15:39:39.201 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50910.log 2024-09-17T15:39:39.201 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53011.log.gz 2024-09-17T15:39:39.202 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46752.log 2024-09-17T15:39:39.202 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47382.log.gz 2024-09-17T15:39:39.202 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68182.log 2024-09-17T15:39:39.202 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50910.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50910.log.gz 2024-09-17T15:39:39.202 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52675.log 2024-09-17T15:39:39.202 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46752.log.gz 2024-09-17T15:39:39.202 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41459.log 2024-09-17T15:39:39.202 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.68182.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68182.log.gz 2024-09-17T15:39:39.202 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.52675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56121.log 2024-09-17T15:39:39.202 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52675.log.gz 2024-09-17T15:39:39.202 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41459.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41459.log.gz 2024-09-17T15:39:39.202 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58728.log 2024-09-17T15:39:39.202 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52338.log 2024-09-17T15:39:39.202 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56121.log.gz 2024-09-17T15:39:39.203 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.58728.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58728.log.gz 2024-09-17T15:39:39.203 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70073.log 2024-09-17T15:39:39.203 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.52338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56709.log 2024-09-17T15:39:39.203 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52338.log.gz 2024-09-17T15:39:39.203 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.70073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61966.log 2024-09-17T15:39:39.203 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70073.log.gz 2024-09-17T15:39:39.203 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67132.log 2024-09-17T15:39:39.203 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56709.log.gz 2024-09-17T15:39:39.203 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.61966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47592.log 2024-09-17T15:39:39.203 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61966.log.gz 2024-09-17T15:39:39.203 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.67132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36682.log 2024-09-17T15:39:39.203 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67132.log.gz 2024-09-17T15:39:39.203 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62890.log 2024-09-17T15:39:39.203 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47592.log.gz 2024-09-17T15:39:39.204 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49482.log 2024-09-17T15:39:39.204 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36682.log.gz 2024-09-17T15:39:39.204 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.62890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37802.log 2024-09-17T15:39:39.204 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62890.log.gz 2024-09-17T15:39:39.204 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69737.log 2024-09-17T15:39:39.204 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49482.log.gz 2024-09-17T15:39:39.204 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37802.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.37802.log.gz -- 2024-09-17T15:39:39.204 INFO:teuthology.orchestra.run.smithi083.stderr: /var/log/ceph/ceph-client.admin.49986.log 2024-09-17T15:39:39.204 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39555.log 2024-09-17T15:39:39.204 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69737.log.gz 2024-09-17T15:39:39.204 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67384.log 2024-09-17T15:39:39.204 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49986.log.gz 2024-09-17T15:39:39.204 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54061.log 2024-09-17T15:39:39.205 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39555.log.gz 2024-09-17T15:39:39.205 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.67384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46584.log 2024-09-17T15:39:39.205 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67384.log.gz 2024-09-17T15:39:39.205 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54859.log 2024-09-17T15:39:39.205 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54061.log.gz 2024-09-17T15:39:39.205 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42589.log 2024-09-17T15:39:39.205 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46584.log.gz 2024-09-17T15:39:39.205 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54859.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54859.log.gz 2024-09-17T15:39:39.205 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52633.log 2024-09-17T15:39:39.205 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56919.log 2024-09-17T15:39:39.205 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42589.log.gz 2024-09-17T15:39:39.205 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.52633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36719.log 2024-09-17T15:39:39.205 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52633.log.gz 2024-09-17T15:39:39.205 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46290.log 2024-09-17T15:39:39.206 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56919.log.gz 2024-09-17T15:39:39.206 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64654.log 2024-09-17T15:39:39.206 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36719.log.gz 2024-09-17T15:39:39.206 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41058.log 2024-09-17T15:39:39.206 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46290.log.gz 2024-09-17T15:39:39.206 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.64654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70619.log 2024-09-17T15:39:39.206 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64654.log.gz 2024-09-17T15:39:39.206 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36247.log 2024-09-17T15:39:39.206 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41058.log.gz 2024-09-17T15:39:39.206 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.70619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57045.log 2024-09-17T15:39:39.206 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70619.log.gz 2024-09-17T15:39:39.206 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64906.log 2024-09-17T15:39:39.206 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36247.log.gz 2024-09-17T15:39:39.206 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.57045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51540.log 2024-09-17T15:39:39.207 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57045.log.gz 2024-09-17T15:39:39.207 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.64906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42001.log 2024-09-17T15:39:39.207 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64906.log.gz 2024-09-17T15:39:39.207 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56331.log 2024-09-17T15:39:39.207 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51540.log.gz 2024-09-17T15:39:39.207 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42001.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42001.log.gz 2024-09-17T15:39:39.207 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-17T15:39:39.207 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60620.log 2024-09-17T15:39:39.207 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56331.log.gz 2024-09-17T15:39:39.207 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55027.log 2024-09-17T15:39:39.207 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.60620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59442.log 2024-09-17T15:39:39.207 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60620.log.gz 2024-09-17T15:39:39.207 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57801.log 2024-09-17T15:39:39.207 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55027.log.gz 2024-09-17T15:39:39.208 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.59442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65452.log 2024-09-17T15:39:39.208 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59442.log.gz 2024-09-17T15:39:39.208 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.57801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48978.log 2024-09-17T15:39:39.208 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57801.log.gz 2024-09-17T15:39:39.208 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.65452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37311.log 2024-09-17T15:39:39.208 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65452.log.gz 2024-09-17T15:39:39.208 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61881.log 2024-09-17T15:39:39.208 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48978.log.gz 2024-09-17T15:39:39.208 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60536.log 2024-09-17T15:39:39.208 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37311.log.gz 2024-09-17T15:39:39.208 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.61881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42043.log 2024-09-17T15:39:39.208 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61881.log.gz 2024-09-17T15:39:39.208 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.60536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44476.log 2024-09-17T15:39:39.209 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60536.log.gz 2024-09-17T15:39:39.209 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49062.log 2024-09-17T15:39:39.209 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42043.log.gz 2024-09-17T15:39:39.209 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60705.log 2024-09-17T15:39:39.209 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44476.log.gz 2024-09-17T15:39:39.209 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57213.log 2024-09-17T15:39:39.209 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49062.log.gz 2024-09-17T15:39:39.209 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.60705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46164.log 2024-09-17T15:39:39.209 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60705.log.gz 2024-09-17T15:39:39.209 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.57213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64822.log 2024-09-17T15:39:39.209 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57213.log.gz 2024-09-17T15:39:39.209 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70493.log 2024-09-17T15:39:39.209 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46164.log.gz 2024-09-17T15:39:39.209 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.64822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58770.log 2024-09-17T15:39:39.210 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64822.log.gz 2024-09-17T15:39:39.210 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.70493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59652.log 2024-09-17T15:39:39.210 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70493.log.gz 2024-09-17T15:39:39.210 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.58770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43995.log 2024-09-17T15:39:39.210 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58770.log.gz 2024-09-17T15:39:39.210 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.59652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63016.log 2024-09-17T15:39:39.210 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59652.log.gz 2024-09-17T15:39:39.210 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66334.log 2024-09-17T15:39:39.212 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43995.log.gz 2024-09-17T15:39:39.212 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.63016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35176.log 2024-09-17T15:39:39.212 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63016.log.gz 2024-09-17T15:39:39.212 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.66334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66124.log 2024-09-17T15:39:39.212 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66334.log.gz 2024-09-17T15:39:39.212 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37950.log 2024-09-17T15:39:39.213 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35176.log.gz 2024-09-17T15:39:39.213 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.66124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60747.log 2024-09-17T15:39:39.213 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66124.log.gz 2024-09-17T15:39:39.213 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36645.log 2024-09-17T15:39:39.213 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37950.log.gz 2024-09-17T15:39:39.213 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.60747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60663.log 2024-09-17T15:39:39.213 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60747.log.gz 2024-09-17T15:39:39.213 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51204.log 2024-09-17T15:39:39.213 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36645.log.gz 2024-09-17T15:39:39.213 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.60663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67930.log 2024-09-17T15:39:39.213 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60663.log.gz 2024-09-17T15:39:39.213 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41711.log 2024-09-17T15:39:39.213 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51204.log.gz 2024-09-17T15:39:39.213 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.67930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35053.log 2024-09-17T15:39:39.214 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67930.log.gz 2024-09-17T15:39:39.214 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67678.log 2024-09-17T15:39:39.214 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41711.log.gz 2024-09-17T15:39:39.214 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46332.log 2024-09-17T15:39:39.214 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35053.log.gz 2024-09-17T15:39:39.214 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.67678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35095.log 2024-09-17T15:39:39.214 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67678.log.gz 2024-09-17T15:39:39.214 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46248.log 2024-09-17T15:39:39.214 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46332.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46332.log.gz 2024-09-17T15:39:39.214 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47718.log 2024-09-17T15:39:39.214 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35095.log.gz 2024-09-17T15:39:39.214 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46248.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.46248.log.gz 2024-09-17T15:39:39.214 INFO:teuthology.orchestra.run.smithi083.stderr: /var/log/ceph/ceph-client.admin.38061.log 2024-09-17T15:39:39.214 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52296.log 2024-09-17T15:39:39.215 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47718.log.gz 2024-09-17T15:39:39.215 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36904.log 2024-09-17T15:39:39.215 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38061.log.gz 2024-09-17T15:39:39.215 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.52296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60410.log 2024-09-17T15:39:39.215 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52296.log.gz 2024-09-17T15:39:39.215 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44032.log 2024-09-17T15:39:39.215 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36904.log.gz 2024-09-17T15:39:39.215 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.60410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62260.log 2024-09-17T15:39:39.215 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60410.log.gz 2024-09-17T15:39:39.215 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56415.log 2024-09-17T15:39:39.215 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44032.log.gz 2024-09-17T15:39:39.215 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.62260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46710.log 2024-09-17T15:39:39.215 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62260.log.gz 2024-09-17T15:39:39.215 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43429.log 2024-09-17T15:39:39.216 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56415.log.gz 2024-09-17T15:39:39.216 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64486.log 2024-09-17T15:39:39.216 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46710.log.gz 2024-09-17T15:39:39.216 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43429.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43429.log.gz 2024-09-17T15:39:39.216 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45870.log 2024-09-17T15:39:39.216 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.64486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48600.log 2024-09-17T15:39:39.216 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64486.log.gz 2024-09-17T15:39:39.216 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45870.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-17T15:39:39.216 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45870.log.gz 2024-09-17T15:39:39.216 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53683.log 2024-09-17T15:39:39.216 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48600.log.gz 2024-09-17T15:39:39.216 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52969.log 2024-09-17T15:39:39.216 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66922.log 2024-09-17T15:39:39.217 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53683.log.gz 2024-09-17T15:39:39.217 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.52969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43135.log 2024-09-17T15:39:39.217 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52969.log.gz 2024-09-17T15:39:39.217 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57885.log 2024-09-17T15:39:39.217 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.66922.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66922.log.gz 2024-09-17T15:39:39.217 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60452.log 2024-09-17T15:39:39.217 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43135.log.gz 2024-09-17T15:39:39.217 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.57885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59526.log 2024-09-17T15:39:39.217 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57885.log.gz 2024-09-17T15:39:39.217 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.60452.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.60452.log.gz -5 2024-09-17T15:39:39.217 INFO:teuthology.orchestra.run.smithi083.stderr: --verbose -- /var/log/ceph/ceph-client.admin.56835.log 2024-09-17T15:39:39.217 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.59526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40955.log 2024-09-17T15:39:39.217 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59526.log.gz 2024-09-17T15:39:39.218 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56163.log 2024-09-17T15:39:39.218 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56835.log.gz 2024-09-17T15:39:39.218 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65536.log 2024-09-17T15:39:39.218 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40955.log.gz 2024-09-17T15:39:39.218 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56163.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.56163.log.gz --verbose 2024-09-17T15:39:39.218 INFO:teuthology.orchestra.run.smithi083.stderr: -- /var/log/ceph/ceph-client.admin.55112.log 2024-09-17T15:39:39.218 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.65536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37089.log 2024-09-17T15:39:39.218 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65536.log.gz 2024-09-17T15:39:39.218 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48222.log 2024-09-17T15:39:39.218 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55112.log.gz 2024-09-17T15:39:39.218 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53305.log 2024-09-17T15:39:39.218 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37089.log.gz 2024-09-17T15:39:39.218 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57717.log 2024-09-17T15:39:39.218 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48222.log.gz 2024-09-17T15:39:39.219 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65746.log 2024-09-17T15:39:39.219 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53305.log.gz 2024-09-17T15:39:39.219 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.57717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62596.log 2024-09-17T15:39:39.219 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57717.log.gz 2024-09-17T15:39:39.219 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.65746.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.65746.log.gz -5 2024-09-17T15:39:39.219 INFO:teuthology.orchestra.run.smithi083.stderr: --verbose -- /var/log/ceph/ceph-client.admin.62974.log 2024-09-17T15:39:39.219 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50952.log 2024-09-17T15:39:39.219 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.62596.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62596.log.gz 2024-09-17T15:39:39.219 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.62974.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62974.log.gz -5 2024-09-17T15:39:39.219 INFO:teuthology.orchestra.run.smithi083.stderr: --verbose -- /var/log/ceph/ceph-client.admin.61041.log 2024-09-17T15:39:39.219 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50952.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.50952.log.gz 2024-09-17T15:39:39.219 INFO:teuthology.orchestra.run.smithi083.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47508.log 2024-09-17T15:39:39.219 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.61041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43597.log 2024-09-17T15:39:39.219 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61041.log.gz 2024-09-17T15:39:39.220 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54775.log 2024-09-17T15:39:39.220 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47508.log.gz 2024-09-17T15:39:39.220 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53431.log 2024-09-17T15:39:39.220 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43597.log.gz 2024-09-17T15:39:39.220 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64234.log 2024-09-17T15:39:39.220 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54775.log.gz 2024-09-17T15:39:39.220 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59190.log 2024-09-17T15:39:39.220 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53431.log.gz 2024-09-17T15:39:39.220 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49356.log 2024-09-17T15:39:39.220 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.64234.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64234.log.gz 2024-09-17T15:39:39.220 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.59190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38468.log 2024-09-17T15:39:39.220 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59190.log.gz 2024-09-17T15:39:39.220 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61545.log 2024-09-17T15:39:39.220 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49356.log.gz 2024-09-17T15:39:39.221 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50112.log 2024-09-17T15:39:39.221 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38468.log.gz 2024-09-17T15:39:39.221 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.61545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59694.log 2024-09-17T15:39:39.221 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61545.log.gz 2024-09-17T15:39:39.221 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50112.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50112.log.gz -5 2024-09-17T15:39:39.221 INFO:teuthology.orchestra.run.smithi083.stderr: --verbose -- /var/log/ceph/ceph-client.admin.41417.log 2024-09-17T15:39:39.221 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.59694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41959.log 2024-09-17T15:39:39.221 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59694.log.gz 2024-09-17T15:39:39.221 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56583.log 2024-09-17T15:39:39.221 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41417.log.gz 2024-09-17T15:39:39.221 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59106.log 2024-09-17T15:39:39.221 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41959.log.gz 2024-09-17T15:39:39.221 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5/var/log/ceph/ceph-client.admin.56583.log: --verbose -- /var/log/ceph/ceph-client.admin.64360.log 2024-09-17T15:39:39.221 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56583.log.gz 2024-09-17T15:39:39.222 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.59106.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59106.log.gz 2024-09-17T15:39:39.222 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41875.log 2024-09-17T15:39:39.222 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.64360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43264.log 2024-09-17T15:39:39.222 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64360.log.gz 2024-09-17T15:39:39.222 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47550.log 2024-09-17T15:39:39.222 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41875.log.gz 2024-09-17T15:39:39.222 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56037.log 2024-09-17T15:39:39.222 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43264.log.gz 2024-09-17T15:39:39.222 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35515.log 2024-09-17T15:39:39.222 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47550.log.gz 2024-09-17T15:39:39.222 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68812.log 2024-09-17T15:39:39.222 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56037.log.gz 2024-09-17T15:39:39.222 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57549.log 2024-09-17T15:39:39.222 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35515.log.gz 2024-09-17T15:39:39.223 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.68812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63394.log 2024-09-17T15:39:39.223 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68812.log.gz 2024-09-17T15:39:39.223 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.57549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43196.log 2024-09-17T15:39:39.223 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57549.log.gz 2024-09-17T15:39:39.223 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.63394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43662.log 2024-09-17T15:39:39.223 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63394.log.gz 2024-09-17T15:39:39.223 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39891.log 2024-09-17T15:39:39.223 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43196.log.gz 2024-09-17T15:39:39.223 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43773.log 2024-09-17T15:39:39.223 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43662.log.gz 2024-09-17T15:39:39.223 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66544.log 2024-09-17T15:39:39.223 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39891.log.gz 2024-09-17T15:39:39.223 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43773.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43773.log.gz 2024-09-17T15:39:39.224 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58812.log 2024-09-17T15:39:39.224 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.66544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47928.log 2024-09-17T15:39:39.224 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66544.log.gz 2024-09-17T15:39:39.224 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.58812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68266.log 2024-09-17T15:39:39.224 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58812.log.gz 2024-09-17T15:39:39.224 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47928.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47928.log.gz 2024-09-17T15:39:39.224 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-17T15:39:39.224 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.68266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37765.log 2024-09-17T15:39:39.224 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68266.log.gz 2024-09-17T15:39:39.224 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42757.log 2024-09-17T15:39:39.224 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64738.log 2024-09-17T15:39:39.224 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37765.log.gz 2024-09-17T15:39:39.224 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42757.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42757.log.gz 2024-09-17T15:39:39.225 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44143.log 2024-09-17T15:39:39.225 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.64738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65620.log 2024-09-17T15:39:39.225 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64738.log.gz 2024-09-17T15:39:39.225 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51624.log 2024-09-17T15:39:39.225 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44143.log.gz 2024-09-17T15:39:39.225 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.65620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41646.log 2024-09-17T15:39:39.225 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65620.log.gz 2024-09-17T15:39:39.225 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48432.log 2024-09-17T15:39:39.225 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51624.log.gz 2024-09-17T15:39:39.225 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62764.log 2024-09-17T15:39:39.225 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41646.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41646.log.gz 2024-09-17T15:39:39.225 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60368.log 2024-09-17T15:39:39.225 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48432.log.gz 2024-09-17T15:39:39.225 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.62764.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62764.log.gz 2024-09-17T15:39:39.226 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70661.log 2024-09-17T15:39:39.226 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.60368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66712.log 2024-09-17T15:39:39.226 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60368.log.gz 2024-09-17T15:39:39.226 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.70661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39849.log 2024-09-17T15:39:39.226 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70661.log.gz 2024-09-17T15:39:39.226 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.66712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39700.log 2024-09-17T15:39:39.226 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66712.log.gz 2024-09-17T15:39:39.226 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50322.log 2024-09-17T15:39:39.226 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39849.log.gz 2024-09-17T15:39:39.226 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62470.log 2024-09-17T15:39:39.226 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39700.log.gz 2024-09-17T15:39:39.226 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65788.log 2024-09-17T15:39:39.226 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50322.log.gz 2024-09-17T15:39:39.226 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.62470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53809.log 2024-09-17T15:39:39.227 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62470.log.gz 2024-09-17T15:39:39.227 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.65788.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65788.log.gz 2024-09-17T15:39:39.227 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61125.log 2024-09-17T15:39:39.227 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48180.log 2024-09-17T15:39:39.227 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53809.log.gz 2024-09-17T15:39:39.227 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.61125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42211.log 2024-09-17T15:39:39.227 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61125.log.gz 2024-09-17T15:39:39.227 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54607.log 2024-09-17T15:39:39.227 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48180.log.gz 2024-09-17T15:39:39.227 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69653.log 2024-09-17T15:39:39.227 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42211.log.gz 2024-09-17T15:39:39.227 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47424.log 2024-09-17T15:39:39.227 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54607.log.gz 2024-09-17T15:39:39.227 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65074.log 2024-09-17T15:39:39.228 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69653.log.gz 2024-09-17T15:39:39.228 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52801.log 2024-09-17T15:39:39.228 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47424.log.gz 2024-09-17T15:39:39.228 INFO:teuthology.orchestra.run.smithi083.stderr: 95.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-17T15:39:39.228 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.65074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50196.log 2024-09-17T15:39:39.228 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65074.log.gz 2024-09-17T15:39:39.228 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.52801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42379.log 2024-09-17T15:39:39.228 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52801.log.gz 2024-09-17T15:39:39.228 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50196.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-17T15:39:39.228 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50196.log.gz 2024-09-17T15:39:39.228 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51120.log 2024-09-17T15:39:39.228 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42379.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42379.log.gz 2024-09-17T15:39:39.228 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44439.log 2024-09-17T15:39:39.228 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64108.log 2024-09-17T15:39:39.229 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51120.log.gz 2024-09-17T15:39:39.229 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42463.log 2024-09-17T15:39:39.229 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44439.log.gz 2024-09-17T15:39:39.229 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.64108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58644.log 2024-09-17T15:39:39.229 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64108.log.gz 2024-09-17T15:39:39.229 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35177.log 2024-09-17T15:39:39.229 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42463.log.gz 2024-09-17T15:39:39.229 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.58644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56247.log 2024-09-17T15:39:39.229 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58644.log.gz 2024-09-17T15:39:39.229 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53935.log 2024-09-17T15:39:39.229 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35177.log.gz 2024-09-17T15:39:39.229 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49818.log 2024-09-17T15:39:39.229 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56247.log.gz 2024-09-17T15:39:39.230 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35599.log 2024-09-17T15:39:39.230 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53935.log.gz 2024-09-17T15:39:39.230 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57843.log 2024-09-17T15:39:39.230 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49818.log.gz 2024-09-17T15:39:39.230 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62050.log 2024-09-17T15:39:39.230 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35599.log.gz 2024-09-17T15:39:39.230 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.57843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43810.log 2024-09-17T15:39:39.230 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57843.log.gz 2024-09-17T15:39:39.230 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.62050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68350.log 2024-09-17T15:39:39.230 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62050.log.gz 2024-09-17T15:39:39.230 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64570.log 2024-09-17T15:39:39.230 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43810.log.gz 2024-09-17T15:39:39.230 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.68350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50448.log 2024-09-17T15:39:39.230 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68350.log.gz 2024-09-17T15:39:39.231 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.64570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62092.log 2024-09-17T15:39:39.231 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64570.log.gz 2024-09-17T15:39:39.231 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53095.log 2024-09-17T15:39:39.231 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50448.log.gz 2024-09-17T15:39:39.231 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.62092.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.62092.log.gz 2024-09-17T15:39:39.231 INFO:teuthology.orchestra.run.smithi083.stderr: -- /var/log/ceph/ceph-client.admin.36331.log 2024-09-17T15:39:39.231 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47802.log 2024-09-17T15:39:39.231 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53095.log.gz 2024-09-17T15:39:39.231 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60915.log 2024-09-17T15:39:39.231 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36331.log.gz 2024-09-17T15:39:39.231 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50616.log 2024-09-17T15:39:39.231 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47802.log.gz 2024-09-17T15:39:39.231 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.60915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51582.log 2024-09-17T15:39:39.232 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60915.log.gz 2024-09-17T15:39:39.232 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66208.log 2024-09-17T15:39:39.232 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50616.log.gz 2024-09-17T15:39:39.232 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69695.log 2024-09-17T15:39:39.232 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51582.log.gz 2024-09-17T15:39:39.232 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.66208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51246.log 2024-09-17T15:39:39.232 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66208.log.gz 2024-09-17T15:39:39.232 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49272.log 2024-09-17T15:39:39.232 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69695.log.gz 2024-09-17T15:39:39.232 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65242.log 2024-09-17T15:39:39.232 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51246.log.gz 2024-09-17T15:39:39.232 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36246.log 2024-09-17T15:39:39.232 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49272.log.gz 2024-09-17T15:39:39.232 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.65242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44550.log 2024-09-17T15:39:39.233 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65242.log.gz 2024-09-17T15:39:39.233 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36460.log 2024-09-17T15:39:39.233 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36246.log.gz 2024-09-17T15:39:39.233 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66838.log 2024-09-17T15:39:39.233 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44550.log.gz 2024-09-17T15:39:39.233 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56079.log 2024-09-17T15:39:39.233 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36460.log.gz 2024-09-17T15:39:39.233 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.66838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49398.log 2024-09-17T15:39:39.233 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66838.log.gz 2024-09-17T15:39:39.233 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68224.log 2024-09-17T15:39:39.233 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56079.log.gz 2024-09-17T15:39:39.233 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67006.log 2024-09-17T15:39:39.233 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49398.log.gz 2024-09-17T15:39:39.234 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.68224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43345.log 2024-09-17T15:39:39.234 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68224.log.gz 2024-09-17T15:39:39.234 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.67006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59316.log 2024-09-17T15:39:39.234 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67006.log.gz 2024-09-17T15:39:39.234 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43345.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43345.log.gz 2024-09-17T15:39:39.234 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52381.log 2024-09-17T15:39:39.234 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.59316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57675.log 2024-09-17T15:39:39.234 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59316.log.gz 2024-09-17T15:39:39.234 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.52381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36437.log 2024-09-17T15:39:39.234 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52381.log.gz 2024-09-17T15:39:39.234 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.57675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46080.log 2024-09-17T15:39:39.234 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57675.log.gz 2024-09-17T15:39:39.234 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36437.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36437.log.gz 2024-09-17T15:39:39.234 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69190.log 2024-09-17T15:39:39.235 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37163.log 2024-09-17T15:39:39.235 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46080.log.gz 2024-09-17T15:39:39.235 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46122.log 2024-09-17T15:39:39.235 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69190.log.gz 2024-09-17T15:39:39.235 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64780.log 2024-09-17T15:39:39.235 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37163.log.gz 2024-09-17T15:39:39.235 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70241.log 2024-09-17T15:39:39.235 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46122.log.gz 2024-09-17T15:39:39.235 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.64780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57927.log 2024-09-17T15:39:39.235 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64780.log.gz 2024-09-17T15:39:39.235 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.70241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60242.log 2024-09-17T15:39:39.235 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70241.log.gz 2024-09-17T15:39:39.235 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.57927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36793.log 2024-09-17T15:39:39.235 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57927.log.gz 2024-09-17T15:39:39.236 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.60242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49566.log 2024-09-17T15:39:39.236 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60242.log.gz 2024-09-17T15:39:39.236 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53053.log 2024-09-17T15:39:39.236 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36793.log.gz 2024-09-17T15:39:39.236 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42295.log 2024-09-17T15:39:39.236 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49566.log.gz 2024-09-17T15:39:39.236 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53053.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-17T15:39:39.236 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53053.log.gz 2024-09-17T15:39:39.236 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37691.log 2024-09-17T15:39:39.236 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42295.log.gz 2024-09-17T15:39:39.236 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55323.log 2024-09-17T15:39:39.236 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68728.log 2024-09-17T15:39:39.236 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37691.log.gz 2024-09-17T15:39:39.236 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67174.log 2024-09-17T15:39:39.237 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55323.log.gz 2024-09-17T15:39:39.237 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.68728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31671.log 2024-09-17T15:39:39.237 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68728.log.gz 2024-09-17T15:39:39.237 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.67174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42925.log 2024-09-17T15:39:39.237 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67174.log.gz 2024-09-17T15:39:39.237 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.31671.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27250.log 2024-09-17T15:39:39.237 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31671.log.gz 2024-09-17T15:39:39.237 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42925.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.57339.log 2024-09-17T15:39:39.237 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42925.log.gz 2024-09-17T15:39:39.237 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph.tmp-client.admin.27250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35938.log 2024-09-17T15:39:39.237 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27250.log.gz 2024-09-17T15:39:39.237 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.57339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66082.log 2024-09-17T15:39:39.237 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57339.log.gz 2024-09-17T15:39:39.238 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62638.log 2024-09-17T15:39:39.238 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35938.log.gz 2024-09-17T15:39:39.238 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.66082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67090.log 2024-09-17T15:39:39.238 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66082.log.gz 2024-09-17T15:39:39.238 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.62638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41226.log 2024-09-17T15:39:39.238 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62638.log.gz 2024-09-17T15:39:39.238 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.67090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44624.log 2024-09-17T15:39:39.238 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67090.log.gz 2024-09-17T15:39:39.238 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38283.log 2024-09-17T15:39:39.238 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41226.log.gz 2024-09-17T15:39:39.238 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61335.log 2024-09-17T15:39:39.238 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44624.log.gz 2024-09-17T15:39:39.238 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59778.log 2024-09-17T15:39:39.238 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38283.log.gz 2024-09-17T15:39:39.239 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.61335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41142.log 2024-09-17T15:39:39.239 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61335.log.gz 2024-09-17T15:39:39.239 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.59778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70157.log 2024-09-17T15:39:39.239 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59778.log.gz 2024-09-17T15:39:39.239 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70367.log 2024-09-17T15:39:39.239 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41142.log.gz 2024-09-17T15:39:39.239 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.70157.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70157.log.gz 2024-09-17T15:39:39.239 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36608.log 2024-09-17T15:39:39.239 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.70367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44402.log 2024-09-17T15:39:39.239 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70367.log.gz 2024-09-17T15:39:39.239 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46962.log 2024-09-17T15:39:39.239 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36608.log.gz 2024-09-17T15:39:39.239 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63478.log 2024-09-17T15:39:39.239 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44402.log.gz 2024-09-17T15:39:39.240 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46962.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46962.log.gz 2024-09-17T15:39:39.240 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42127.log 2024-09-17T15:39:39.240 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.63478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54985.log 2024-09-17T15:39:39.240 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63478.log.gz 2024-09-17T15:39:39.240 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63856.log 2024-09-17T15:39:39.240 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42127.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42127.log.gz 2024-09-17T15:39:39.240 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40997.log 2024-09-17T15:39:39.240 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54985.log.gz 2024-09-17T15:39:39.240 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.63856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63730.log 2024-09-17T15:39:39.240 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63856.log.gz 2024-09-17T15:39:39.240 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70409.log 2024-09-17T15:39:39.240 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40997.log.gz 2024-09-17T15:39:39.240 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.63730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54313.log 2024-09-17T15:39:39.240 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63730.log.gz 2024-09-17T15:39:39.241 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.70409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43513.log 2024-09-17T15:39:39.241 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70409.log.gz 2024-09-17T15:39:39.241 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.54313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39617.log 2024-09-17T15:39:39.241 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54313.log.gz 2024-09-17T15:39:39.241 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43513.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.43513.log.gz 2024-09-17T15:39:39.241 INFO:teuthology.orchestra.run.smithi083.stderr: /var/log/ceph/ceph-client.admin.61419.log 2024-09-17T15:39:39.241 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53977.log 2024-09-17T15:39:39.241 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39617.log.gz 2024-09-17T15:39:39.241 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.61419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58434.log 2024-09-17T15:39:39.241 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61419.log.gz 2024-09-17T15:39:39.241 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63142.log 2024-09-17T15:39:39.241 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53977.log.gz 2024-09-17T15:39:39.241 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.58434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47466.log 2024-09-17T15:39:39.241 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58434.log.gz 2024-09-17T15:39:39.242 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.63142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61461.log 2024-09-17T15:39:39.242 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63142.log.gz 2024-09-17T15:39:39.242 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41291.log 2024-09-17T15:39:39.242 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47466.log.gz 2024-09-17T15:39:39.242 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.61461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61629.log 2024-09-17T15:39:39.242 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61461.log.gz 2024-09-17T15:39:39.242 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63226.log 2024-09-17T15:39:39.242 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41291.log.gz 2024-09-17T15:39:39.242 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.61629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51330.log 2024-09-17T15:39:39.242 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61629.log.gz 2024-09-17T15:39:39.242 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.63226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68644.log 2024-09-17T15:39:39.242 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63226.log.gz 2024-09-17T15:39:39.242 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68938.log 2024-09-17T15:39:39.242 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51330.log.gz 2024-09-17T15:39:39.243 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.68644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35429.log 2024-09-17T15:39:39.243 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68644.log.gz 2024-09-17T15:39:39.243 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.68938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66628.log 2024-09-17T15:39:39.243 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68938.log.gz 2024-09-17T15:39:39.243 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51666.log 2024-09-17T15:39:39.243 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35429.log.gz 2024-09-17T15:39:39.243 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.66628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59862.log 2024-09-17T15:39:39.243 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66628.log.gz 2024-09-17T15:39:39.243 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68602.log 2024-09-17T15:39:39.243 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51666.log.gz 2024-09-17T15:39:39.243 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.59862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37580.log 2024-09-17T15:39:39.243 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59862.log.gz 2024-09-17T15:39:39.243 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.68602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67888.log 2024-09-17T15:39:39.244 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68602.log.gz 2024-09-17T15:39:39.244 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63688.log 2024-09-17T15:39:39.244 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37580.log.gz 2024-09-17T15:39:39.244 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.67888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62806.log 2024-09-17T15:39:39.244 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67888.log.gz 2024-09-17T15:39:39.244 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.63688.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.63688.log.gz 2024-09-17T15:39:39.244 INFO:teuthology.orchestra.run.smithi083.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.37126.log 2024-09-17T15:39:39.244 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.62806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55995.log 2024-09-17T15:39:39.244 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62806.log.gz 2024-09-17T15:39:39.244 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41795.log 2024-09-17T15:39:39.244 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37126.log.gz 2024-09-17T15:39:39.244 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50742.log 2024-09-17T15:39:39.244 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55995.log.gz 2024-09-17T15:39:39.244 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55953.log 2024-09-17T15:39:39.245 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41795.log.gz 2024-09-17T15:39:39.245 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55617.log 2024-09-17T15:39:39.245 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50742.log.gz 2024-09-17T15:39:39.245 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41731.log 2024-09-17T15:39:39.245 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55953.log.gz 2024-09-17T15:39:39.245 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48306.log 2024-09-17T15:39:39.245 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55617.log.gz 2024-09-17T15:39:39.245 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41731.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.41731.log.gz --verbose 2024-09-17T15:39:39.245 INFO:teuthology.orchestra.run.smithi083.stderr: -- /var/log/ceph/ceph-client.admin.38098.log 2024-09-17T15:39:39.245 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44365.log 2024-09-17T15:39:39.245 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48306.log.gz 2024-09-17T15:39:39.245 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42673.log 2024-09-17T15:39:39.245 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38098.log.gz 2024-09-17T15:39:39.246 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36311.log 2024-09-17T15:39:39.246 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44365.log.gz 2024-09-17T15:39:39.246 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64948.log 2024-09-17T15:39:39.246 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42673.log.gz 2024-09-17T15:39:39.246 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44254.log 2024-09-17T15:39:39.246 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36311.log.gz 2024-09-17T15:39:39.246 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.64948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63562.log 2024-09-17T15:39:39.246 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64948.log.gz 2024-09-17T15:39:39.246 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61251.log 2024-09-17T15:39:39.246 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44254.log.gz 2024-09-17T15:39:39.246 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.63562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51792.log 2024-09-17T15:39:39.246 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63562.log.gz 2024-09-17T15:39:39.246 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.61251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57171.log 2024-09-17T15:39:39.246 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61251.log.gz 2024-09-17T15:39:39.247 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43736.log 2024-09-17T15:39:39.247 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51792.log.gz 2024-09-17T15:39:39.247 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.57171.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.57171.log.gz 2024-09-17T15:39:39.247 INFO:teuthology.orchestra.run.smithi083.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.69484.log 2024-09-17T15:39:39.247 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35896.log 2024-09-17T15:39:39.247 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43736.log.gz 2024-09-17T15:39:39.247 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59904.log 2024-09-17T15:39:39.247 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69484.log.gz 2024-09-17T15:39:39.247 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56373.log 2024-09-17T15:39:39.247 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35896.log.gz 2024-09-17T15:39:39.247 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.59904.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.57759.log 2024-09-17T15:39:39.247 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59904.log.gz 2024-09-17T15:39:39.247 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69106.log 2024-09-17T15:39:39.247 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56373.log.gz 2024-09-17T15:39:39.248 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.57759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36978.log 2024-09-17T15:39:39.248 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57759.log.gz 2024-09-17T15:39:39.248 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45912.log 2024-09-17T15:39:39.248 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69106.log.gz 2024-09-17T15:39:39.248 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47634.log 2024-09-17T15:39:39.248 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36978.log.gz 2024-09-17T15:39:39.248 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37876.log 2024-09-17T15:39:39.248 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45912.log.gz 2024-09-17T15:39:39.248 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48348.log 2024-09-17T15:39:39.248 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47634.log.gz 2024-09-17T15:39:39.248 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35767.log 2024-09-17T15:39:39.248 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37876.log.gz 2024-09-17T15:39:39.248 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.48348.log: -- /var/log/ceph/ceph-client.admin.66166.log 2024-09-17T15:39:39.248 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48348.log.gz 2024-09-17T15:39:39.249 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59484.log 2024-09-17T15:39:39.249 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35767.log.gz 2024-09-17T15:39:39.249 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.66166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41375.log 2024-09-17T15:39:39.249 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66166.log.gz 2024-09-17T15:39:39.249 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.59484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67300.log 2024-09-17T15:39:39.249 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59484.log.gz 2024-09-17T15:39:39.249 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41375.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41375.log.gz 2024-09-17T15:39:39.249 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37237.log 2024-09-17T15:39:39.249 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.67300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42547.log 2024-09-17T15:39:39.250 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67300.log.gz 2024-09-17T15:39:39.250 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53263.log 2024-09-17T15:39:39.250 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37237.log.gz 2024-09-17T15:39:39.250 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68770.log 2024-09-17T15:39:39.250 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42547.log.gz 2024-09-17T15:39:39.250 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.53263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42715.log 2024-09-17T15:39:39.250 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53263.log.gz 2024-09-17T15:39:39.250 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.68770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37478.log 2024-09-17T15:39:39.250 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68770.log.gz 2024-09-17T15:39:39.250 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65956.log 2024-09-17T15:39:39.250 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42715.log.gz 2024-09-17T15:39:39.250 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55239.log 2024-09-17T15:39:39.250 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37478.log.gz 2024-09-17T15:39:39.250 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.65956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59232.log 2024-09-17T15:39:39.251 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65956.log.gz 2024-09-17T15:39:39.251 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65494.log 2024-09-17T15:39:39.251 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55239.log.gz 2024-09-17T15:39:39.251 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.59232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35980.log 2024-09-17T15:39:39.251 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59232.log.gz 2024-09-17T15:39:39.251 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.65494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55827.log 2024-09-17T15:39:39.251 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65494.log.gz 2024-09-17T15:39:39.251 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50406.log 2024-09-17T15:39:39.251 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35980.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35980.log.gz 2024-09-17T15:39:39.251 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70787.log 2024-09-17T15:39:39.251 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55827.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55827.log.gz 2024-09-17T15:39:39.251 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64276.log 2024-09-17T15:39:39.251 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50406.log.gz 2024-09-17T15:39:39.252 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.70787.log: /var/log/ceph/ceph-client.admin.45954.log 2024-09-17T15:39:39.252 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70787.log.gz 2024-09-17T15:39:39.252 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.64276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40017.log 2024-09-17T15:39:39.252 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64276.log.gz 2024-09-17T15:39:39.252 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52549.log 2024-09-17T15:39:39.252 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45954.log.gz 2024-09-17T15:39:39.252 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35428.log 2024-09-17T15:39:39.252 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40017.log.gz 2024-09-17T15:39:39.252 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.52549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59736.log 2024-09-17T15:39:39.252 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52549.log.gz 2024-09-17T15:39:39.252 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55197.log 2024-09-17T15:39:39.252 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35428.log.gz 2024-09-17T15:39:39.252 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.59736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62848.log 2024-09-17T15:39:39.253 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59736.log.gz 2024-09-17T15:39:39.253 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51414.log 2024-09-17T15:39:39.253 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55197.log.gz 2024-09-17T15:39:39.253 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.62848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50658.log 2024-09-17T15:39:39.253 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62848.log.gz 2024-09-17T15:39:39.253 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65284.log 2024-09-17T15:39:39.253 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51414.log.gz 2024-09-17T15:39:39.253 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65998.log 2024-09-17T15:39:39.253 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50658.log.gz 2024-09-17T15:39:39.253 INFO:teuthology.orchestra.run.smithi083.stderr:gzip/var/log/ceph/ceph-client.admin.65284.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.55155.log 2024-09-17T15:39:39.253 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65284.log.gz 2024-09-17T15:39:39.253 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.65998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65830.log 2024-09-17T15:39:39.253 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65998.log.gz 2024-09-17T15:39:39.254 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70869.log 2024-09-17T15:39:39.254 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55155.log.gz 2024-09-17T15:39:39.254 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.65830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64150.log 2024-09-17T15:39:39.254 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65830.log.gz 2024-09-17T15:39:39.254 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.70869.log: /var/log/ceph/ceph-client.admin.48726.log 2024-09-17T15:39:39.254 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70869.log.gz 2024-09-17T15:39:39.254 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.64150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52002.log 2024-09-17T15:39:39.254 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64150.log.gz 2024-09-17T15:39:39.254 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70115.log 2024-09-17T15:39:39.254 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48726.log.gz 2024-09-17T15:39:39.254 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.52002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55491.log 2024-09-17T15:39:39.254 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52002.log.gz 2024-09-17T15:39:39.254 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.70115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43471.log 2024-09-17T15:39:39.254 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70115.log.gz 2024-09-17T15:39:39.255 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55491.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55491.log.gz 2024-09-17T15:39:39.255 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38320.log 2024-09-17T15:39:39.255 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66418.log 2024-09-17T15:39:39.255 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43471.log.gz 2024-09-17T15:39:39.255 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56793.log 2024-09-17T15:39:39.255 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38320.log.gz 2024-09-17T15:39:39.255 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.66418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43197.log 2024-09-17T15:39:39.255 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66418.log.gz 2024-09-17T15:39:39.255 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.56793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63604.log 2024-09-17T15:39:39.255 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56793.log.gz 2024-09-17T15:39:39.255 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36830.log 2024-09-17T15:39:39.255 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43197.log.gz 2024-09-17T15:39:39.255 INFO:teuthology.orchestra.run.smithi083.stderr:gzip/var/log/ceph/ceph-client.admin.63604.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.46416.log 2024-09-17T15:39:39.255 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63604.log.gz 2024-09-17T15:39:39.256 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52927.log 2024-09-17T15:39:39.256 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36830.log.gz 2024-09-17T15:39:39.256 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63898.log 2024-09-17T15:39:39.256 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46416.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46416.log.gz 2024-09-17T15:39:39.256 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.52927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64318.log 2024-09-17T15:39:39.256 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52927.log.gz 2024-09-17T15:39:39.256 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70535.log 2024-09-17T15:39:39.256 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.63898.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63898.log.gz 2024-09-17T15:39:39.256 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.64318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55365.log 2024-09-17T15:39:39.256 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64318.log.gz 2024-09-17T15:39:39.256 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.70535.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.70535.log.gz --verbose 2024-09-17T15:39:39.256 INFO:teuthology.orchestra.run.smithi083.stderr: -- /var/log/ceph/ceph-client.admin.69232.log 2024-09-17T15:39:39.256 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.55365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48768.log 2024-09-17T15:39:39.257 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55365.log.gz 2024-09-17T15:39:39.257 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51288.log 2024-09-17T15:39:39.257 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69232.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69232.log.gz 2024-09-17T15:39:39.257 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50028.log 2024-09-17T15:39:39.257 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48768.log.gz 2024-09-17T15:39:39.257 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.51288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38209.log 2024-09-17T15:39:39.257 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51288.log.gz 2024-09-17T15:39:39.257 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69863.log 2024-09-17T15:39:39.257 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50028.log.gz 2024-09-17T15:39:39.257 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61924.log 2024-09-17T15:39:39.257 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38209.log.gz 2024-09-17T15:39:39.257 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.69863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41647.log 2024-09-17T15:39:39.257 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69863.log.gz 2024-09-17T15:39:39.257 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.61924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58053.log 2024-09-17T15:39:39.258 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61924.log.gz 2024-09-17T15:39:39.258 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59148.log 2024-09-17T15:39:39.258 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41647.log.gz 2024-09-17T15:39:39.258 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.58053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37617.log 2024-09-17T15:39:39.258 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58053.log.gz 2024-09-17T15:39:39.258 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.59148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37385.log 2024-09-17T15:39:39.258 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59148.log.gz 2024-09-17T15:39:39.258 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36395.log 2024-09-17T15:39:39.258 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37617.log.gz 2024-09-17T15:39:39.258 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37385.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37385.log.gz 2024-09-17T15:39:39.258 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36395.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36395.log.gz 2024-09-17T15:39:39.967 INFO:teuthology.orchestra.run.smithi083.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-17T15:39:40.727 INFO:teuthology.orchestra.run.smithi083.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-17T15:39:40.740 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-09-17T15:39:40.799 INFO:teuthology.orchestra.run.smithi083.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-17T15:39:43.002 INFO:teuthology.orchestra.run.smithi083.stderr: 89.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-17T15:47:42.856 INFO:teuthology.orchestra.run.smithi083.stderr: 91.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-17T15:51:45.942 INFO:teuthology.orchestra.run.smithi083.stderr: 90.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-17T15:53:07.965 INFO:teuthology.orchestra.run.smithi083.stderr: 91.0% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-17T15:53:07.967 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-09-17T15:53:07.967 INFO:teuthology.orchestra.run.smithi083.stderr:real 13m29.106s 2024-09-17T15:53:07.967 INFO:teuthology.orchestra.run.smithi083.stderr:user 5m53.651s 2024-09-17T15:53:07.967 INFO:teuthology.orchestra.run.smithi083.stderr:sys 0m15.792s 2024-09-17T15:53:07.967 INFO:tasks.ceph:Archiving logs... 2024-09-17T15:53:07.968 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/log/ceph to /home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7908934/remote/smithi083/log 2024-09-17T15:53:07.969 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T15:54:13.107 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-17T15:54:13.118 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-17T15:54:13.119 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T15:54:13.246 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-17T15:54:13.254 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-17T15:54:13.254 DEBUG:teuthology.orchestra.run.smithi083:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T15:54:13.272 INFO:teuthology.orchestra.run.smithi083.stderr:bash: line 1: ntpq: command not found 2024-09-17T15:54:13.317 INFO:teuthology.orchestra.run.smithi083.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T15:54:13.317 INFO:teuthology.orchestra.run.smithi083.stdout:=============================================================================== 2024-09-17T15:54:13.317 INFO:teuthology.orchestra.run.smithi083.stdout:^* hv01.front.sepia.ceph.com 3 6 377 23 -151us[ -173us] +/- 35ms 2024-09-17T15:54:13.318 INFO:teuthology.orchestra.run.smithi083.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 21 +542us[ +542us] +/- 60ms 2024-09-17T15:54:13.318 INFO:teuthology.orchestra.run.smithi083.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 23 -299us[ -299us] +/- 61ms 2024-09-17T15:54:13.318 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T15:54:13.319 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-17T15:54:13.327 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-17T15:54:13.328 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-17T15:54:13.335 DEBUG:teuthology.orchestra.run.smithi083:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-17T15:54:13.488 DEBUG:teuthology.orchestra.run.smithi083:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T15:54:13.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T15:54:13.522 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-17T15:54:13.531 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-17T15:54:13.539 INFO:teuthology.task.internal:Duration was 2234.356613 seconds 2024-09-17T15:54:13.539 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-17T15:54:13.547 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-17T15:54:13.547 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T15:54:13.640 INFO:teuthology.orchestra.run.smithi083.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T15:54:13.793 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-17T15:54:13.793 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi083.front.sepia.ceph.com 2024-09-17T15:54:13.794 DEBUG:teuthology.orchestra.run.smithi083:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T15:54:13.816 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-17T15:54:13.816 DEBUG:teuthology.orchestra.run.smithi083:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T15:54:13.885 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-17T15:54:13.886 DEBUG:teuthology.orchestra.run.smithi083:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T15:54:14.058 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-17T15:54:14.068 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-17T15:54:14.068 DEBUG:teuthology.orchestra.run.smithi083:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T15:54:14.111 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-17T15:54:14.121 DEBUG:teuthology.orchestra.run.smithi083:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T15:54:14.183 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern = core 2024-09-17T15:54:14.202 DEBUG:teuthology.orchestra.run.smithi083:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T15:54:14.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T15:54:14.258 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-17T15:54:14.269 INFO:teuthology.task.internal:Transferring archived files... 2024-09-17T15:54:14.270 DEBUG:teuthology.misc:Transferring archived files from smithi083:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7908934/remote/smithi083 2024-09-17T15:54:14.270 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T15:54:14.332 INFO:teuthology.task.internal:Removing archive directory... 2024-09-17T15:54:14.332 DEBUG:teuthology.orchestra.run.smithi083:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T15:54:14.382 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-17T15:54:14.392 INFO:teuthology.task.internal:Not uploading archives. 2024-09-17T15:54:14.392 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-17T15:54:14.401 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-17T15:54:14.401 DEBUG:teuthology.orchestra.run.smithi083:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T15:54:14.438 INFO:teuthology.orchestra.run.smithi083.stdout: 266063 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 15:54 /home/ubuntu/cephtest 2024-09-17T15:54:14.439 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-17T15:54:14.449 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-17T15:54:14.471 INFO:teuthology.run:Summary data: description: rados/singleton/{all/lost-unfound mon_election/classic msgr-failures/few msgr/async-v2only objectstore/{bluestore-options/write$/{write_random} bluestore/bluestore-comp-lz4} rados supported-random-distro$/{centos_latest}} duration: 2234.356612920761 flavor: default owner: scheduled_skanta@teuthology success: true 2024-09-17T15:54:14.471 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T15:54:14.556 INFO:teuthology.run:pass